IBM Informix - The Ideal Database for Internet of Things
Exclusive luncheon at IBM World of Watson 2016. Informix is the best fit for IoT sensor data analytics at the edge and in the cloud.
Learn the current state of the NoSQL landscape and discover the different data models within it. From document stores and key value databases to graph and Wide Column. Then you’ll learn why wide column databases are the most appropriate for scalable high performance use cases, including capabilities for massive scale-out architecture, peer-to-peer clustering to avoid bottlenecking and built-in multi-datacenter replication.
This document provides an overview of NoSQL databases and summarizes key information about several NoSQL databases, including HBase, Redis, Cassandra, MongoDB, and Memcached. It discusses concepts like horizontal scalability, the CAP theorem, eventual consistency, and data models used by different NoSQL databases like key-value, document, columnar, and graph structures.
This document provides an overview of NoSQL data architecture patterns, including key-value stores, graph stores, and column family stores. It describes key aspects of each pattern such as how keys and values are structured. Key-value stores use a simple key-value approach with no query language, while graph stores are optimized for relationships between objects. Column family stores use row and column identifiers as keys and scale well for large volumes of data.
In this lecture we analyze document oriented databases. In particular we consider why there are the first approach to nosql and what are the main features. Then, we analyze as example MongoDB. We consider the data model, CRUD operations, write concerns, scaling (replication and sharding).
Finally we presents other document oriented database and when to use or not document oriented databases.
This document provides an overview of a SQL-on-Hadoop tutorial. It introduces the presenters and discusses why SQL is important for Hadoop, as MapReduce is not optimal for all use cases. It also notes that while the database community knows how to efficiently process data, SQL-on-Hadoop systems face challenges due to the limitations of running on top of HDFS and Hadoop ecosystems. The tutorial outline covers SQL-on-Hadoop technologies like storage formats, runtime engines, and query optimization.
This document provides an introduction to NoSQL databases. It discusses that NoSQL is a non-relational approach to data storage that does not rely on fixed schemas and provides better scalability than traditional relational databases. Specific NoSQL examples mentioned include document databases like CouchDB and MongoDB, as well as key-value stores like Redis and Cassandra. The document outlines some of the characteristics and usage of these NoSQL solutions.
This document compares SQL and NoSQL databases. It defines databases, describes different types including relational and NoSQL, and explains key differences between SQL and NoSQL in areas like scaling, modeling, and query syntax. SQL databases are better suited for projects with logical related discrete data requirements and data integrity needs, while NoSQL is more ideal for projects with unrelated, evolving data where speed and scalability are important. MongoDB is provided as an example of a NoSQL database, and the CAP theorem is introduced to explain tradeoffs in distributed systems.
This document provides an introduction to big data and NoSQL databases. It begins with an introduction of the presenter. It then discusses how the era of big data came to be due to limitations of traditional relational databases and scaling approaches. The document introduces different NoSQL data models including document, key-value, graph and column-oriented databases. It provides examples of NoSQL databases that use each data model. The document discusses how NoSQL databases are better suited than relational databases for big data problems and provides a real-world example of Twitter's use of FlockDB. It concludes by discussing approaches for working with big data using MapReduce and provides examples of using MongoDB and Azure for big data.
This document provides an overview and introduction to MongoDB. It discusses how new types of applications, data, volumes, development methods and architectures necessitated new database technologies like NoSQL. It then defines MongoDB and describes its features, including using documents to store data, dynamic schemas, querying capabilities, indexing, auto-sharding for scalability, replication for availability, and using memory for performance. Use cases are presented for companies like Foursquare and Craigslist that have migrated large volumes of data and traffic to MongoDB to gain benefits like flexibility, scalability, availability and ease of use over traditional relational database systems.
This presentation explains the major differences between SQL and NoSQL databases in terms of Scalability, Flexibility and Performance. It also talks about MongoDB which is a document-based NoSQL database and explains the database strutre for my mouse-human research classifier project.
Cassandra is a decentralized structured storage system that was initially developed at Facebook to power their inbox search. It is based on Amazon's Dynamo and Google's BigTable data models. Cassandra provides tunable consistency, high availability with no single points of failure, horizontal scalability and elasticity. It allows data to be distributed across multiple data centers and easily handles the addition or removal of nodes.
Architecture web aujourd'hui, besoin de scalabilité des bases de données relationnelles, découverte des bases de données NoSQL et des différents types de celles-ci. La vidéo de présentation peut être consultée à l'adresse suivante : http://youtu.be/oIpjcqHyx2M
The document provides information about Hadoop, its core components, and MapReduce programming model. It defines Hadoop as an open source software framework used for distributed storage and processing of large datasets. It describes the main Hadoop components like HDFS, NameNode, DataNode, JobTracker and Secondary NameNode. It also explains MapReduce as a programming model used for distributed processing of big data across clusters.
This document provides an overview of SQL and NoSQL databases. It defines SQL as a language used to communicate with relational databases, allowing users to query, manipulate, and retrieve data. NoSQL databases are defined as non-relational and allow for flexible schemas. The document compares key aspects of SQL and NoSQL such as data structure, querying, scalability and provides examples of popular SQL and NoSQL database systems. It concludes that both SQL and NoSQL databases will continue to be important with polyglot persistence, using the best database for each storage need.
Hive Tutorial | Hive Architecture | Hive Tutorial For Beginners | Hive In Had...Simplilearn
This presentation about Hive will help you understand the history of Hive, what is Hive, Hive architecture, data flow in Hive, Hive data modeling, Hive data types, different modes in which Hive can run on, differences between Hive and RDBMS, features of Hive and a demo on HiveQL commands. Hive is a data warehouse system which is used for querying and analyzing large datasets stored in HDFS. Hive uses a query language called HiveQL which is similar to SQL. Hive issues SQL abstraction to integrate SQL queries (like HiveQL) into Java without the necessity to implement queries in the low-level Java API. Now, let us get started and understand Hadoop Hive in detail
Below topics are explained in this Hive presetntation:
1. History of Hive
2. What is Hive?
3. Architecture of Hive
4. Data flow in Hive
5. Hive data modeling
6. Hive data types
7. Different modes of Hive
8. Difference between Hive and RDBMS
9. Features of Hive
10. Demo on HiveQL
What is this Big Data Hadoop training course about?
The Big Data Hadoop and Spark developer course have been designed to impart in-depth knowledge of Big Data processing using Hadoop and Spark. The course is packed with real-life projects and case studies to be executed in the CloudLab.
What are the course objectives?
This course will enable you to:
1. Understand the different components of the Hadoop ecosystem such as Hadoop 2.7, Yarn, MapReduce, Pig, Hive, Impala, HBase, Sqoop, Flume, and Apache Spark
2. Understand Hadoop Distributed File System (HDFS) and YARN as well as their architecture, and learn how to work with them for storage and resource management
3. Understand MapReduce and its characteristics, and assimilate some advanced MapReduce concepts
4. Get an overview of Sqoop and Flume and describe how to ingest data using them
5. Create database and tables in Hive and Impala, understand HBase, and use Hive and Impala for partitioning
6. Understand different types of file formats, Avro Schema, using Arvo with Hive, and Sqoop and Schema evolution
7. Understand Flume, Flume architecture, sources, flume sinks, channels, and flume configurations
8. Understand HBase, its architecture, data storage, and working with HBase. You will also understand the difference between HBase and RDBMS
9. Gain a working knowledge of Pig and its components
10. Do functional programming in Spark
11. Understand resilient distribution datasets (RDD) in detail
12. Implement and build Spark applications
13. Gain an in-depth understanding of parallel processing in Spark and Spark RDD optimization techniques
14. Understand the common use-cases of Spark and the various interactive algorithms
15. Learn Spark SQL, creating, transforming, and querying Data frames
Learn more at https://www.simplilearn.com/big-data-and-analytics/big-data-and-hadoop-training
The document discusses data modeling for MongoDB. It begins by recognizing the differences between modeling for a document database versus a relational database. It then outlines a flexible methodology for MongoDB modeling including defining the workload, identifying relationships between entities, and applying schema design patterns. Finally, it recognizes the need to apply patterns like schema versioning, subset, computed, bucket, and external reference when modeling for MongoDB.
The document is a slide presentation on MongoDB that introduces the topic and provides an overview. It defines MongoDB as a document-oriented, open source database that provides high performance, high availability, and easy scalability. It also discusses MongoDB's use for big data applications, how it is non-relational and stores data as JSON-like documents in collections without a defined schema. The presentation provides steps for installing MongoDB and describes some basic concepts like databases, collections, documents and commands.
This document provides an introduction to NoSQL and MongoDB. It discusses that NoSQL is a non-relational database management system that avoids joins and is easy to scale. It then summarizes the different flavors of NoSQL including key-value stores, graphs, BigTable, and document stores. The remainder of the document focuses on MongoDB, describing its structure, how to perform inserts and searches, features like map-reduce and replication. It concludes by encouraging the reader to try MongoDB themselves.
Cassandra & puppet, scaling data at $15 per monthdaveconnors
Constant Contact shares lessons learned from DevOps approach to implementing Cassandra to manage social media data for over 400k small business customers. Puppet is the critical in our tool chain. Single most important factor was the willingness of Development and Operations to stretch beyond traditional roles and responsibilities.
This is a presentation of the popular NoSQL database Apache Cassandra which was created by our team in the context of the module "Business Intelligence and Big Data Analysis".
This document provides an overview of Apache Phoenix, including:
- A brief history of how it originated as an internal project at Salesforce before becoming a top-level Apache project.
- An architectural overview explaining that Phoenix provides a SQL interface for Apache HBase and runs on top of HDFS to enable next-generation data applications on HBase.
- Descriptions of Phoenix's key capabilities like SQL support, transactions, user-defined functions, and secondary indexes to boost query performance.
- Examples of how Phoenix can be used for common scenarios like analyzing server metrics data.
Introduction to Hadoop and Hadoop component rebeccatho
This document provides an introduction to Apache Hadoop, which is an open-source software framework for distributed storage and processing of large datasets. It discusses Hadoop's main components of MapReduce and HDFS. MapReduce is a programming model for processing large datasets in a distributed manner, while HDFS provides distributed, fault-tolerant storage. Hadoop runs on commodity computer clusters and can scale to thousands of nodes.
Mysql is an open source relational database management system that can be downloaded for free from mysql.com. It allows users to define, construct, manipulate and access databases through SQL queries. The document provides an overview of mysql and databases, instructions for downloading and starting mysql, descriptions of basic SQL queries like SELECT, INSERT, UPDATE and DELETE, and examples of creating a sample employee table and running queries on it.
This document provides an introduction and overview of Apache Hive. It discusses how Hive originated at Facebook to manage large amounts of data stored in Oracle databases. It then defines what Hive is, how it works by compiling SQL queries into MapReduce jobs, and its architecture. Key components of Hive like its data model, metastore, and commands for creating tables and loading data are summarized.
Informix Spark Streaming is an extension of Informix that allows data to be streamed out of the database as soon as it is inserted, updated, or deleted.
The protocol currently used to stream the changes is MQTT v3.1.1 (older versions not supported!). This extension is able to stream data to any MQTT broker where it can be processed or passed on to subscribing clients for processing.
IOT Paris Seminar 2015 - Storage Challenges in IOTMongoDB
Joe Drumgoole gave a presentation on the Internet of Things storage challenges. The number of sensors deployed is rapidly increasing from billions to trillions. These sensors generate massive amounts of real-time data that needs to be stored, filtered, and distributed. Relational databases are not well-suited for this type of unstructured IoT data. MongoDB is presented as a solution because it offers dynamic schemas, automatic scaling, text search, and other features that address the demands of storing and analyzing massive amounts of real-time IoT data.
This document provides an introduction to big data and NoSQL databases. It begins with an introduction of the presenter. It then discusses how the era of big data came to be due to limitations of traditional relational databases and scaling approaches. The document introduces different NoSQL data models including document, key-value, graph and column-oriented databases. It provides examples of NoSQL databases that use each data model. The document discusses how NoSQL databases are better suited than relational databases for big data problems and provides a real-world example of Twitter's use of FlockDB. It concludes by discussing approaches for working with big data using MapReduce and provides examples of using MongoDB and Azure for big data.
This document provides an overview and introduction to MongoDB. It discusses how new types of applications, data, volumes, development methods and architectures necessitated new database technologies like NoSQL. It then defines MongoDB and describes its features, including using documents to store data, dynamic schemas, querying capabilities, indexing, auto-sharding for scalability, replication for availability, and using memory for performance. Use cases are presented for companies like Foursquare and Craigslist that have migrated large volumes of data and traffic to MongoDB to gain benefits like flexibility, scalability, availability and ease of use over traditional relational database systems.
This presentation explains the major differences between SQL and NoSQL databases in terms of Scalability, Flexibility and Performance. It also talks about MongoDB which is a document-based NoSQL database and explains the database strutre for my mouse-human research classifier project.
Cassandra is a decentralized structured storage system that was initially developed at Facebook to power their inbox search. It is based on Amazon's Dynamo and Google's BigTable data models. Cassandra provides tunable consistency, high availability with no single points of failure, horizontal scalability and elasticity. It allows data to be distributed across multiple data centers and easily handles the addition or removal of nodes.
Architecture web aujourd'hui, besoin de scalabilité des bases de données relationnelles, découverte des bases de données NoSQL et des différents types de celles-ci. La vidéo de présentation peut être consultée à l'adresse suivante : http://youtu.be/oIpjcqHyx2M
The document provides information about Hadoop, its core components, and MapReduce programming model. It defines Hadoop as an open source software framework used for distributed storage and processing of large datasets. It describes the main Hadoop components like HDFS, NameNode, DataNode, JobTracker and Secondary NameNode. It also explains MapReduce as a programming model used for distributed processing of big data across clusters.
This document provides an overview of SQL and NoSQL databases. It defines SQL as a language used to communicate with relational databases, allowing users to query, manipulate, and retrieve data. NoSQL databases are defined as non-relational and allow for flexible schemas. The document compares key aspects of SQL and NoSQL such as data structure, querying, scalability and provides examples of popular SQL and NoSQL database systems. It concludes that both SQL and NoSQL databases will continue to be important with polyglot persistence, using the best database for each storage need.
Hive Tutorial | Hive Architecture | Hive Tutorial For Beginners | Hive In Had...Simplilearn
This presentation about Hive will help you understand the history of Hive, what is Hive, Hive architecture, data flow in Hive, Hive data modeling, Hive data types, different modes in which Hive can run on, differences between Hive and RDBMS, features of Hive and a demo on HiveQL commands. Hive is a data warehouse system which is used for querying and analyzing large datasets stored in HDFS. Hive uses a query language called HiveQL which is similar to SQL. Hive issues SQL abstraction to integrate SQL queries (like HiveQL) into Java without the necessity to implement queries in the low-level Java API. Now, let us get started and understand Hadoop Hive in detail
Below topics are explained in this Hive presetntation:
1. History of Hive
2. What is Hive?
3. Architecture of Hive
4. Data flow in Hive
5. Hive data modeling
6. Hive data types
7. Different modes of Hive
8. Difference between Hive and RDBMS
9. Features of Hive
10. Demo on HiveQL
What is this Big Data Hadoop training course about?
The Big Data Hadoop and Spark developer course have been designed to impart in-depth knowledge of Big Data processing using Hadoop and Spark. The course is packed with real-life projects and case studies to be executed in the CloudLab.
What are the course objectives?
This course will enable you to:
1. Understand the different components of the Hadoop ecosystem such as Hadoop 2.7, Yarn, MapReduce, Pig, Hive, Impala, HBase, Sqoop, Flume, and Apache Spark
2. Understand Hadoop Distributed File System (HDFS) and YARN as well as their architecture, and learn how to work with them for storage and resource management
3. Understand MapReduce and its characteristics, and assimilate some advanced MapReduce concepts
4. Get an overview of Sqoop and Flume and describe how to ingest data using them
5. Create database and tables in Hive and Impala, understand HBase, and use Hive and Impala for partitioning
6. Understand different types of file formats, Avro Schema, using Arvo with Hive, and Sqoop and Schema evolution
7. Understand Flume, Flume architecture, sources, flume sinks, channels, and flume configurations
8. Understand HBase, its architecture, data storage, and working with HBase. You will also understand the difference between HBase and RDBMS
9. Gain a working knowledge of Pig and its components
10. Do functional programming in Spark
11. Understand resilient distribution datasets (RDD) in detail
12. Implement and build Spark applications
13. Gain an in-depth understanding of parallel processing in Spark and Spark RDD optimization techniques
14. Understand the common use-cases of Spark and the various interactive algorithms
15. Learn Spark SQL, creating, transforming, and querying Data frames
Learn more at https://www.simplilearn.com/big-data-and-analytics/big-data-and-hadoop-training
The document discusses data modeling for MongoDB. It begins by recognizing the differences between modeling for a document database versus a relational database. It then outlines a flexible methodology for MongoDB modeling including defining the workload, identifying relationships between entities, and applying schema design patterns. Finally, it recognizes the need to apply patterns like schema versioning, subset, computed, bucket, and external reference when modeling for MongoDB.
The document is a slide presentation on MongoDB that introduces the topic and provides an overview. It defines MongoDB as a document-oriented, open source database that provides high performance, high availability, and easy scalability. It also discusses MongoDB's use for big data applications, how it is non-relational and stores data as JSON-like documents in collections without a defined schema. The presentation provides steps for installing MongoDB and describes some basic concepts like databases, collections, documents and commands.
This document provides an introduction to NoSQL and MongoDB. It discusses that NoSQL is a non-relational database management system that avoids joins and is easy to scale. It then summarizes the different flavors of NoSQL including key-value stores, graphs, BigTable, and document stores. The remainder of the document focuses on MongoDB, describing its structure, how to perform inserts and searches, features like map-reduce and replication. It concludes by encouraging the reader to try MongoDB themselves.
Cassandra & puppet, scaling data at $15 per monthdaveconnors
Constant Contact shares lessons learned from DevOps approach to implementing Cassandra to manage social media data for over 400k small business customers. Puppet is the critical in our tool chain. Single most important factor was the willingness of Development and Operations to stretch beyond traditional roles and responsibilities.
This is a presentation of the popular NoSQL database Apache Cassandra which was created by our team in the context of the module "Business Intelligence and Big Data Analysis".
This document provides an overview of Apache Phoenix, including:
- A brief history of how it originated as an internal project at Salesforce before becoming a top-level Apache project.
- An architectural overview explaining that Phoenix provides a SQL interface for Apache HBase and runs on top of HDFS to enable next-generation data applications on HBase.
- Descriptions of Phoenix's key capabilities like SQL support, transactions, user-defined functions, and secondary indexes to boost query performance.
- Examples of how Phoenix can be used for common scenarios like analyzing server metrics data.
Introduction to Hadoop and Hadoop component rebeccatho
This document provides an introduction to Apache Hadoop, which is an open-source software framework for distributed storage and processing of large datasets. It discusses Hadoop's main components of MapReduce and HDFS. MapReduce is a programming model for processing large datasets in a distributed manner, while HDFS provides distributed, fault-tolerant storage. Hadoop runs on commodity computer clusters and can scale to thousands of nodes.
Mysql is an open source relational database management system that can be downloaded for free from mysql.com. It allows users to define, construct, manipulate and access databases through SQL queries. The document provides an overview of mysql and databases, instructions for downloading and starting mysql, descriptions of basic SQL queries like SELECT, INSERT, UPDATE and DELETE, and examples of creating a sample employee table and running queries on it.
This document provides an introduction and overview of Apache Hive. It discusses how Hive originated at Facebook to manage large amounts of data stored in Oracle databases. It then defines what Hive is, how it works by compiling SQL queries into MapReduce jobs, and its architecture. Key components of Hive like its data model, metastore, and commands for creating tables and loading data are summarized.
Informix Spark Streaming is an extension of Informix that allows data to be streamed out of the database as soon as it is inserted, updated, or deleted.
The protocol currently used to stream the changes is MQTT v3.1.1 (older versions not supported!). This extension is able to stream data to any MQTT broker where it can be processed or passed on to subscribing clients for processing.
IOT Paris Seminar 2015 - Storage Challenges in IOTMongoDB
Joe Drumgoole gave a presentation on the Internet of Things storage challenges. The number of sensors deployed is rapidly increasing from billions to trillions. These sensors generate massive amounts of real-time data that needs to be stored, filtered, and distributed. Relational databases are not well-suited for this type of unstructured IoT data. MongoDB is presented as a solution because it offers dynamic schemas, automatic scaling, text search, and other features that address the demands of storing and analyzing massive amounts of real-time IoT data.
High-performance database technology for rock-solid IoT solutionsClusterpoint
Clusterpoint is a privately held database software company founded in 2006 with 32 employees. Their product is a hybrid operational database, analytics, and search platform that provides secure, high-performance distributed data management at scale. It reduces total cost of ownership by 80% over traditional relational databases by providing blazing fast performance, unlimited scalability, and bulletproof transactions with instant text search and security. Clusterpoint also offers their database software as a cloud database as a service to instantly scale databases on demand.
Mastering the IoT With JavaScript and C++ - Günter ObiltschnigWithTheBest
Mastering The IoT With JavaScript And C++ discusses macchina.io, an open source toolkit for building embedded IoT applications. It uses C++ for performance and efficiency and JavaScript for parts of the web interface. It combines the POCO C++ Libraries, Google V8 JavaScript engine, and other open source projects. The toolkit allows connecting sensors and devices to cloud services via protocols like MQTT and REST. It also features a service platform, automatic JavaScript wrappers for C++ objects, and examples of using sensors, databases, and SMS notifications with JavaScript.
Writing native bindings to node.js in C++nsm.nikhil
The document provides an overview of how to build a C/C++ link to the V8 JavaScript engine and Node.js in order to use C/C++ libraries and functions in Node.js. It discusses topics like initializing V8, handling handles, injecting primitives, defining functions and objects, adding methods, asynchronous I/O, and linking external libraries. Code examples are provided for basic functions, objects, inheritance using ObjectWrap, and asynchronous functions.
Internet of Things Cologne 2015: Why Your Dad’s Database won’t Work for IoT a...MongoDB
IoT is the next big paradigm shift in computing. The move to super-dense sensor networks creates a completely new set of opportunities and challenges for developers, designers and end-users. The databases we designed for the computing environments of the early 90s can no longer support modern, mobile super-scale web applications. In this talk, Joe discussed some of these changes and how they impact the requirements for a modern database.
Fineo Technical Overview - NextSQL for IoTJesse Yates
Fineo is a turn-key data management platform for enterprise IoT that provides a NoSQL time-series database integrated with an analytics warehouse. It offers insights with 10x lower cost and the ability to scale to 100x more data. Fineo provides a "simple" big data deployment through its web scale architecture, security/compliance features, and one-click ETL tools to enable faster adoption and lower complexity.
OSGi Community Event 2014
Abstract:
This presentation tells how OSGi can help developing a distributed and cloud ready Internet of Things platform.
IoT brings unprecedented complexity both in terms of technological variety and new development paradigms. Modularity offered by OSGi is the key concept to build maintainable and robust IoT platforms. OSGi declarative services and dependency injection mechanism allow service producers and service consumers to interact with full respect of mutual component boundaries: this is the fundamental requirement to enable important aspects of an IoT platform like multi-tenancy, separation of concerns between M2M protocols management and application development and dynamic services management.
Plat.One IoT platform revolves around the OSGi technology: this presentation describes our lesson learnt during several years of “hands-on OSGi activities” and development.
Speaker Bio:
After graduating in Physics with specialisation in High Energy Physics, he started working in industrial automation and machine to machine applications. Since 2006 he joined Abo Data and he started the development of PLAT.ONE IoT and M2M platform. Currently, he is leading the PLAT.ONE development team. PLAT.ONE has already been adopted by major telco operators and system integrators to enable a new breed of cloud-based IoT applications and services
MongoDB IoT City Tour LONDON: Managing the Database Complexity, by Arthur Vie...MongoDB
Arthur Viegers, Senior Solutions Architect, MongoDB.
The value of the fast growing class of NoSQL databases is the ability to handle high velocity and volumes of data while enabling greater agility with dynamic schemas. MongoDB gives you those benefits while also providing a rich querying capability and a document model for developer productivity. Arthur Viegers outlines the reasons for MongoDB's popularity in IoT applications and how you can leverage the core concepts of NoSQL to build robust and highly scalable IoT applications.
Authorization Aspects of the Distributed Dataflow-oriented IoT Framework CalvinTomas Nilsson
This document discusses authorization aspects of the distributed dataflow-oriented IoT framework Calvin. It presents an implementation of authorization for applications and actors in Calvin using attribute-based access control. The implementation uses JSON-based message and policy formats to enable fine-grained access control decisions while being lightweight and adaptable to constrained devices. It also describes how authorization can be used to guide smart migration of actors between runtimes in the distributed system.
Getting to know oracle database objects iot, mviews, clusters and more…Aaron Shilo
This document provides an overview of various Oracle database objects and storage structures including:
- Index-organized tables store data within the index based on key values for faster access times and reduced storage.
- Materialized views store the results of a query for faster access instead of re-executing joins and aggregations.
- Virtual indexes allow testing whether a potential new index would be used by the optimizer before implementing.
The presenter discusses how different segment types like index-organized tables, materialized views, and clusters can reduce I/O and improve query performance by organizing data to reduce physical reads and consistent gets. Experienced Oracle DBAs use these features to minimize disk I/O, the greatest factor in
Developing io t applications in the fog a distributed dataflow approachNam Giang
In this paper we examine the development of IoT applications from the perspective of the Fog Computing paradigm, where computing infrastructure at the network edge in devices and gateways is leverage for efficiency and timeliness. Due to the intrinsic nature of the IoT: heterogeneous devices/resources, a tightly coupled perception-action cycle and widely distributed devices and processing, application development in the Fog can be challenging. To address these challenges, we propose a Distributed Dataflow (DDF) programming model for the IoT that utilises computing infrastructures across the Fog and the Cloud. We evaluate our proposal by implementing a DDF framework based on Node-RED (Distributed Node-RED or D-NR), a visual programming tool that uses a flow-based model for building IoT applications. Via demonstrations, we show that our approach eases the development process and can be used to build a variety of IoT applications that work efficiently in the Fog.
Understanding the Operational Database Infrastructure for IoT and Fast DataVoltDB
Join this webinar as Ryan Betts, CTO of VoltDB, describes several data-as-a-service reference architectures for IoT and discuss a real use case highlighting how an in-memory operational database simplified a large-scale enterprise architecture to handle real-time data for IoT -- faster and smarter. View the webinar in its entirety here: http://learn.voltdb.com/WRIoT.html
This document discusses using Apache Spark and Cassandra for IoT applications. Cassandra is a distributed database that is highly available, horizontally scalable, and supports multiple datacenters with no single point of failure. It is well-suited for storing time series sensor data. Spark can be used for both batch and stream processing of data in Cassandra. The Spark Cassandra Connector allows Cassandra tables to be accessed as Spark RDDs. Real-time sensor data can be ingested using Spark Streaming and stored in Cassandra. Common use cases with this architecture include real-time analytics on streaming data and batch analytics on historical sensor data.
Choosing the right platform for your Internet -of-Things solutionIBM_Info_Management
Deploying a solution within the context of the Internet of Things (IoT) typically requires involves many considerations, ranging from the hardware involved to the architecture of the whole environment, and from the decisions about where processing and analytics is to take place to the software choices that allow you to exploit the Internet of Things. This presentation will focus on the need to support a homogeneous processing environment. That is, it will be preferable if processing in all tiers of the IoT is consistent and compatible. This joint presentation will go on to discuss the implications of this consistency for database selection.
The Internet of Things (IoT) is one of the hottest mega-trends in technology – and for good reason , IoT deals with all the components of what we consider web 3.0 including Big Data Analytics, Cloud Computing and Mobile Computing .
This document summarizes a presentation about Internet of Things (IoT) protocols. It discusses how the IoT is projected to connect 100 billion objects by 2020 and defines the IoT according to different companies. It then analyzes common IoT protocols like AMQP, MQTT, XMPP, and DDS, explaining what types of applications each is best suited for. Finally, it discusses how to choose a protocol based on requirements like performance, connectivity, and use cases like smart grids, vehicles, healthcare and more.
How to build a Distributed Serverless Polyglot Microservices IoT Platform us...Animesh Singh
When people aren't talking about VMs and containers, they're talking about serverless architecture. Serverless is about no maintenance. It means you are not worried about low-level infrastructural and operational details. An event-driven serverless platform is a great use case for IoT.
In this session at @ThingsExpo, Animesh Singh, an STSM and Lead for IBM Cloud Platform and Infrastructure, detailed how to build a distributed serverless, polyglot, microservices framework using open source technologies like:
OpenWhisk: Open source distributed compute service to execute application logic in response to events
Docker: To run event driven actions 6. Ansible and BOSH: to deploy the serverless platform
MQTT: Messaging protocol for IoT
Node-RED: Tool to wire IoT together
Consul: Tool for service discovery and configuration. Consul is distributed, highly available, and extremely scalable.
Kafka: A high-throughput distributed messaging system.
StatsD/ELK/Graphite: For statistics, monitoring and logging
Reactive Data Centric Architectures with Vortex, Spark and ReactiveXAngelo Corsaro
An increasing number of Software Architects are realising that data is the most important asset of a system and are staring to embrace the Data-Centric revolution (datacentricmanifesto.org) — setting data at the center of their architecture and modelling applications as “visitors” to the data. At the same time, architect have also realised that reactive architectures (reactivemanifesto.org) facilitates the design of scalable, fault-tolerant and high performance systems.
Yet, few architects have realised that reactive and data-centric architectures are the two sides to the same coin and should always go hand in hand.
This presentation shows how reactive data-centric systems can be designed and built taking advantage of Vortex data sharing capabilities along with its integration with reactive and data-centric processing technologies such as Apache Spark and ReactiveX.
Presentation at IoT World, May 2016 in Santa Clara, CA. Session "Manage your IoT Sensor Data at the Edge! Control your IoT sensor data at the most appropriate spot" (Thursday, 12 May 2016. IoT & the Cloud Track)
IBM IoT Architecture and Capabilities at the Edge and Cloud Pradeep Natarajan
IBM Informix is presented as the ideal database solution for IoT architectures due to its small footprint, low memory requirements, support for time series and spatial data, and driverless operation requiring no administration. It can run on gateways to filter and analyze sensor data locally before transmitting to the cloud. In the cloud, Informix can ingest streaming data in real-time, perform operational analytics, and scale out across servers. Benchmarks show Informix outperforming SQLite for IoT workloads in areas like data loading speed, storage requirements, and analytic query speeds.
This document provides an overview of IBM's Internet of Things architecture and capabilities. It discusses how IBM's Informix database can be used in intelligent gateways and the cloud for IoT solutions. Specifically, it outlines how Informix is well-suited for gateway and cloud environments due to its small footprint, support for time series and spatial data, and ability to handle both structured and unstructured data. The document also provides examples of how Informix can be used with Node-RED and Docker to develop IoT applications and deploy databases in the cloud.
This document provides an overview of IBM's Internet of Things (IoT) architecture and capabilities. It discusses the key components of an IoT architecture including intelligent gateways, sensor analytics zones, and the deep analytics zone in the cloud. It describes how gateways can help IoT solutions by reducing cloud costs and latency through local analytics and filtering of sensor data. The document then outlines the requirements for databases in gateways, and explains how IBM's Informix database is well-suited to meet these requirements through its small footprint, low memory usage, support for time series and spatial data, and ability to ingest and analyze sensor data in real-time. Finally, it discusses how Informix can be used both in gateways and
How Crosser Built a Modern Industrial Data Historian with InfluxDB and GrafanaInfluxData
Crosser are the creators of Crosser Node, a streaming analytics platform. This real-time analytics engine is installed at the edge and pulls data from any sensor, PLC, DCS, MES, SCADA system or historian. Their drag-and-drop tool enables Industry 4.0 data collection and integration. Discover how Crosser’s easy-to-use IIoT monitoring platform empowers non-developers to connect IIoT machine and sensor data with cloud services.
In this webinar, Dr. Göran Appelquist will dive into:
Crosser’s approach to enabling better IIoT data analysis and anomaly detection
Their methodology to equipping their clients with ML models by supporting all Python-based frameworks
How Crosser uses InfluxDB time series platform for storage
WSO2 Data Analytics Server is a comprehensive enterprise data analytics platform; it fuses batch and real-time analytics of any source of data with predictive analytics via machine learning.
How to scale your PaaS with OVH infrastructure?OVHcloud
ForePaaS provides a platform for data infrastructure automation that allows customers to collect, store, transform and analyze data across multiple cloud providers or on-premise in a unified manner. Key features of the ForePaaS platform include being end-to-end, multi-cloud, providing a marketplace for sharing elements of work, and offering automated infrastructure that scales based on customer needs. ForePaaS has partnered with OVH to leverage their public cloud, private cloud, and bare metal server offerings to power ForePaaS infrastructure globally.
Data Pipelines with Spark & DataStax EnterpriseDataStax
This document discusses building data pipelines for both static and streaming data using Apache Spark and DataStax Enterprise (DSE). For static data, it recommends using optimized data storage formats, distributed and scalable technologies like Spark, interactive analysis tools like notebooks, and DSE for persistent storage. For streaming data, it recommends using scalable distributed technologies, Kafka to decouple producers and consumers, and DSE for real-time analytics and persistent storage across datacenters.
Serverless SQL provides a serverless analytics platform that allows users to analyze data stored in object storage without having to manage infrastructure. Key features include seamless elasticity, pay-per-query consumption, and the ability to analyze data directly in object storage without having to move it. The platform includes serverless storage, data ingest, data transformation, analytics, and automation capabilities. It aims to create a sharing economy for analytics by allowing various users like developers, data engineers, and analysts flexible access to data and analytics.
Extensible and Standard-based XaaS Platform To Manage Everything in The Cloud...OCCIware
Who uses multi cloud today ? Everybody. Docker AND VMs, scaling internally AND bursting to Amazon, storing on a public cloud except for data legally required to stay within the country: different solutions for different needs, but more often than not used at the same time. Alas, this leads to a "noodle plate" architecture where a lot of "technical glue" with the various, incompatible clouds creeps in and makes it impossible to evolve.
To solve this problem, the OCCIware project builds on the Open Cloud Computing (OCCI) standard's unified, uniform architectural approach and provides a platform to manage all layers and domains of the Cloud (XaaS), with two main components: the OCCIware Studio Factory and Runtime. The talk includes a demonstration of the Docker connector and of how to use the OCCIware Cloud Designer to configure a real life, SmartCity-themed Cloud application (a Java API server on top of a MongoDB cluster)'s business, platform and infrastructure layers seamlessly on both VirtualBox and OW2's OpenStack infrastructure.
OCCIware@CloudExpoLondon2017 - an extensible, standard XaaS Cloud consumer pl...Marc Dutoo
This document introduces OCCIware, an extensible, standard-based cloud platform that manages resources across clouds. It discusses how OCCIware addresses issues with multi-cloud management by providing a unified interface. The document then provides an overview of OCCIware's features and use cases, demonstrating how it can manage infrastructure, platform and software services. It also briefly describes a demo of OCCIware's Docker Studio tool and Linked Data as a Service capabilities.
Ultralight data movement for IoT with SDC Edge. Guglielmo Iozzia - OptumData Driven Innovation
This document provides an overview and demonstration of Streamsets Data Collector (SDC) and SDC Edge for ingesting data from IoT devices and the edge. It discusses the challenges of ingesting data from distributed edge locations. It then describes the key features of SDC for designing flexible data flows with minimal coding. It also introduces SDC Edge, a lightweight agent for running SDC pipelines on edge devices. The presentation includes demonstrations of using SDC with Kafka and using SDC Edge to ingest and analyze data from Android devices and send it to Elasticsearch. It concludes with discussing additional topics and providing useful links.
ADV Slides: When and How Data Lakes Fit into a Modern Data ArchitectureDATAVERSITY
Whether to take data ingestion cycles off the ETL tool and the data warehouse or to facilitate competitive Data Science and building algorithms in the organization, the data lake – a place for unmodeled and vast data – will be provisioned widely in 2020.
Though it doesn’t have to be complicated, the data lake has a few key design points that are critical, and it does need to follow some principles for success. Avoid building the data swamp, but not the data lake! The tool ecosystem is building up around the data lake and soon many will have a robust lake and data warehouse. We will discuss policy to keep them straight, send data to its best platform, and keep users’ confidence up in their data platforms.
Data lakes will be built in cloud object storage. We’ll discuss the options there as well.
Get this data point for your data lake journey.
By 2020, 50% of all new software will process machine-generated data of some sort (Gartner). Historically, machine data use cases have required non-SQL data stores like Splunk, Elasticsearch, or InfluxDB.
Today, new SQL DB architectures rival the non-SQL solutions in ease of use, scalability, cost, and performance. Please join this webinar for a detailed comparison of machine data management approaches.
SpringPeople - Introduction to Cloud ComputingSpringPeople
Cloud computing is no longer a fad that is going around. It is for real and is perhaps the most talked about subject. Various players in the cloud eco-system have provided a definition that is closely aligned to their sweet spot –let it be infrastructure, platforms or applications.
This presentation will provide an exposure of a variety of cloud computing techniques, architecture, technology options to the participants and in general will familiarize cloud fundamentals in a holistic manner spanning all dimensions such as cost, operations, technology etc
Dopo una breve introduzione dei concetti di base legati all'Internet of Things, durante questa sessione si fornirà una panoramica degli strumenti che Microsoft mette a diposizione degli sviluppatori per creare le proprie soluzioni IoT: Windows 10 for IoT e alcuni servizi di Azure quali Event Hubs e Stream Analytics. Si utilizzerà un semplice esempio di telemetria per mostrare la realizzazione pratica di uno scenario end-to-end per la trasformazione dei dati provenienti da un sensore in informazioni utili per effettuare analisi e/o prendere decisioni.
This document discusses Microsoft's perspective on the Internet of Things (IoT). It outlines an end-to-end IoT scenario from sensors and devices to analytics on Microsoft Azure. The agenda includes discussing how to gain value from connected devices through connectivity, data collection, and analytics. Predictive maintenance and remote monitoring are highlighted as examples of IoT applications. The document then demonstrates connecting sensors to a Raspberry Pi gateway and sending the data to Azure Event Hubs for analysis using Stream Analytics.
How to get Real-Time Value from your IoT Data - DatastaxDataStax
This document discusses DataStax Enterprise (DSE), a distributed database platform for IoT applications. DSE provides a fully integrated technology stack including Apache Cassandra, real-time analytics with Spark, search with Solr, file storage with DSEFS, and management tools. It allows ingesting large volumes of IoT data, performing real-time and batch analytics, and powering low-latency applications at global scale. The document highlights several DSE customer use cases handling trillions of transactions daily from millions of devices.
Google's Infrastructure and Specific IoT ServicesIntel® Software
This document discusses Google Cloud Platform's Internet of Things (IoT) solutions. It describes IoT Core, which handles device management and communication, including the Device Manager for registering devices and MQTT Broker for bidirectional messaging. It explains how IoT Core collects analog sensor data from devices and transforms it into useful business insights and intelligence through data processing and analytics services like Cloud Dataflow, BigQuery, and Cloud ML.
Artificial Intelligence is providing benefits in many areas of work within the heritage sector, from image analysis, to ideas generation, and new research tools. However, it is more critical than ever for people, with analogue intelligence, to ensure the integrity and ethical use of AI. Including real people can improve the use of AI by identifying potential biases, cross-checking results, refining workflows, and providing contextual relevance to AI-driven results.
News about the impact of AI often paints a rosy picture. In practice, there are many potential pitfalls. This presentation discusses these issues and looks at the role of analogue intelligence and analogue interfaces in providing the best results to our audiences. How do we deal with factually incorrect results? How do we get content generated that better reflects the diversity of our communities? What roles are there for physical, in-person experiences in the digital world?
Procurement Insights Cost To Value Guide.pptxJon Hansen
Procurement Insights integrated Historic Procurement Industry Archives, serves as a powerful complement — not a competitor — to other procurement industry firms. It fills critical gaps in depth, agility, and contextual insight that most traditional analyst and association models overlook.
Learn more about this value- driven proprietary service offering here.
Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan
This is a Quick Research Guide (QRG).
QRGs include the following:
- A brief, high-level overview of the QRG topic.
- A milestone timeline for the QRG topic.
- Links to various free online resource materials to provide a deeper dive into the QRG topic.
- Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic.
QRGs planned for the series:
- Artificial Intelligence QRG
- Quantum Computing QRG
- Big Data Analytics QRG
- Spacecraft Guidance, Navigation & Control QRG (coming 2026)
- UK Home Computing & The Birth of ARM QRG (coming 2027)
Any questions or comments?
- Please contact Arthur Morgan at [email protected].
100% human made.
Linux Support for SMARC: How Toradex Empowers Embedded DevelopersToradex
Toradex brings robust Linux support to SMARC (Smart Mobility Architecture), ensuring high performance and long-term reliability for embedded applications. Here’s how:
• Optimized Torizon OS & Yocto Support – Toradex provides Torizon OS, a Debian-based easy-to-use platform, and Yocto BSPs for customized Linux images on SMARC modules.
• Seamless Integration with i.MX 8M Plus and i.MX 95 – Toradex SMARC solutions leverage NXP’s i.MX 8 M Plus and i.MX 95 SoCs, delivering power efficiency and AI-ready performance.
• Secure and Reliable – With Secure Boot, over-the-air (OTA) updates, and LTS kernel support, Toradex ensures industrial-grade security and longevity.
• Containerized Workflows for AI & IoT – Support for Docker, ROS, and real-time Linux enables scalable AI, ML, and IoT applications.
• Strong Ecosystem & Developer Support – Toradex offers comprehensive documentation, developer tools, and dedicated support, accelerating time-to-market.
With Toradex’s Linux support for SMARC, developers get a scalable, secure, and high-performance solution for industrial, medical, and AI-driven applications.
Do you have a specific project or application in mind where you're considering SMARC? We can help with Free Compatibility Check and help you with quick time-to-market
For more information: https://www.toradex.com/computer-on-modules/smarc-arm-family
Spark is a powerhouse for large datasets, but when it comes to smaller data workloads, its overhead can sometimes slow things down. What if you could achieve high performance and efficiency without the need for Spark?
At S&P Global Commodity Insights, having a complete view of global energy and commodities markets enables customers to make data-driven decisions with confidence and create long-term, sustainable value. 🌍
Explore delta-rs + CDC and how these open-source innovations power lightweight, high-performance data applications beyond Spark! 🚀
Complete Guide to Advanced Logistics Management Software in Riyadh.pdfSoftware Company
Explore the benefits and features of advanced logistics management software for businesses in Riyadh. This guide delves into the latest technologies, from real-time tracking and route optimization to warehouse management and inventory control, helping businesses streamline their logistics operations and reduce costs. Learn how implementing the right software solution can enhance efficiency, improve customer satisfaction, and provide a competitive edge in the growing logistics sector of Riyadh.
HCL Nomad Web – Best Practices and Managing Multiuser Environmentspanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-nomad-web-best-practices-and-managing-multiuser-environments/
HCL Nomad Web is heralded as the next generation of the HCL Notes client, offering numerous advantages such as eliminating the need for packaging, distribution, and installation. Nomad Web client upgrades will be installed “automatically” in the background. This significantly reduces the administrative footprint compared to traditional HCL Notes clients. However, troubleshooting issues in Nomad Web present unique challenges compared to the Notes client.
Join Christoph and Marc as they demonstrate how to simplify the troubleshooting process in HCL Nomad Web, ensuring a smoother and more efficient user experience.
In this webinar, we will explore effective strategies for diagnosing and resolving common problems in HCL Nomad Web, including
- Accessing the console
- Locating and interpreting log files
- Accessing the data folder within the browser’s cache (using OPFS)
- Understand the difference between single- and multi-user scenarios
- Utilizing Client Clocking
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfAbi john
Analyze the growth of meme coins from mere online jokes to potential assets in the digital economy. Explore the community, culture, and utility as they elevate themselves to a new era in cryptocurrency.
How Can I use the AI Hype in my Business Context?Daniel Lehner
𝙄𝙨 𝘼𝙄 𝙟𝙪𝙨𝙩 𝙝𝙮𝙥𝙚? 𝙊𝙧 𝙞𝙨 𝙞𝙩 𝙩𝙝𝙚 𝙜𝙖𝙢𝙚 𝙘𝙝𝙖𝙣𝙜𝙚𝙧 𝙮𝙤𝙪𝙧 𝙗𝙪𝙨𝙞𝙣𝙚𝙨𝙨 𝙣𝙚𝙚𝙙𝙨?
Everyone’s talking about AI but is anyone really using it to create real value?
Most companies want to leverage AI. Few know 𝗵𝗼𝘄.
✅ What exactly should you ask to find real AI opportunities?
✅ Which AI techniques actually fit your business?
✅ Is your data even ready for AI?
If you’re not sure, you’re not alone. This is a condensed version of the slides I presented at a Linkedin webinar for Tecnovy on 28.04.2025.
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies
In late April 2025, a significant portion of Europe, particularly Spain, Portugal, and parts of southern France, experienced widespread, rolling power outages that continue to affect millions of residents, businesses, and infrastructure systems.
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025BookNet Canada
Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next.
Link to recording, transcript, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/
Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB
Want to learn practical tips for designing systems that can scale efficiently without compromising speed?
Join us for a workshop where we’ll address these challenges head-on and explore how to architect low-latency systems using Rust. During this free interactive workshop oriented for developers, engineers, and architects, we’ll cover how Rust’s unique language features and the Tokio async runtime enable high-performance application development.
As you explore key principles of designing low-latency systems with Rust, you will learn how to:
- Create and compile a real-world app with Rust
- Connect the application to ScyllaDB (NoSQL data store)
- Negotiate tradeoffs related to data modeling and querying
- Manage and monitor the database for consistently low latencies
TrsLabs - Fintech Product & Business ConsultingTrs Labs
Hybrid Growth Mandate Model with TrsLabs
Strategic Investments, Inorganic Growth, Business Model Pivoting are critical activities that business don't do/change everyday. In cases like this, it may benefit your business to choose a temporary external consultant.
An unbiased plan driven by clearcut deliverables, market dynamics and without the influence of your internal office equations empower business leaders to make right choices.
Getting things done within a budget within a timeframe is key to Growing Business - No matter whether you are a start-up or a big company
Talk to us & Unlock the competitive advantage
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...Noah Loul
Artificial intelligence is changing how businesses operate. Companies are using AI agents to automate tasks, reduce time spent on repetitive work, and focus more on high-value activities. Noah Loul, an AI strategist and entrepreneur, has helped dozens of companies streamline their operations using smart automation. He believes AI agents aren't just tools—they're workers that take on repeatable tasks so your human team can focus on what matters. If you want to reduce time waste and increase output, AI agents are the next move.
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity
Join this UiPath Community Berlin meetup to explore the Orchestrator API, Swagger interface, and the Test Manager API. Learn how to leverage these tools to streamline automation, enhance testing, and integrate more efficiently with UiPath. Perfect for developers, testers, and automation enthusiasts!
📕 Agenda
Welcome & Introductions
Orchestrator API Overview
Exploring the Swagger Interface
Test Manager API Highlights
Streamlining Automation & Testing with APIs (Demo)
Q&A and Open Discussion
Perfect for developers, testers, and automation enthusiasts!
👉 Join our UiPath Community Berlin chapter: https://community.uipath.com/berlin/
This session streamed live on April 29, 2025, 18:00 CET.
Check out all our upcoming UiPath Community sessions at https://community.uipath.com/events/.
3. World of Watson 2016
Internet of Things Architecture – Analytics End-to-End
Deep Analytics Zone
Analytics Zone
Smart Gateways
Devices/Sensors
Uses
Gateway Direct to
Cloud
Insights from BigData
Message Broker
Flexible Hybrid
Data Management
3
4. World of Watson 2016
IoT applications have a common set of requirements
Opportunities for innovation
Quickly and easily provision new sensors
Create a real-time communication channel with the
sensor
Capture data from the sensor and store it in a time
series database
Provide Secure access to the collected data –
analytics at the Edge and Cloud, in real-time & on
historical data
Trigger events based on specific data conditions
Interact with the sensor from business/enterprise
applications and/or from mobile devices
Monetize the service based on usage
4
5. World of Watson 2016
• Gateways can reduce the cost of the backend cloud
• Reduces cloud storage by filtering/aggregating/analyzing data locally
• Reduces cloud CPU requirements by precomputing values
• Reduces latency since actions can be taken immediately
• Intelligent gateways can detect and respond to local events as they happen rather
than waiting for transfer to the cloud
• Some users are not comfortable putting all their data in the cloud
• Gateways allow customers to capture and get value from their sensors without
sending data to the cloud
• Protocol Consolidation
• Cloud does not need to implement the 100’s of IoT protocols
Over time more and more of the processing will move from the cloud to gateway
devices
How Do Gateways Help IoT Solutions?
5
6. World of Watson 2016
What are the Requirements for a Gateway Database?
• The database management system must:
Have a small install footprint, less than 100 MB
Run with low memory requirements – less than 256 MB
Use lossless compression techniques to minimize storage space
Have built-in support for common types of IoT data like time series,
spatial, and JSON data
Simple application development supporting both NoSQL and SQL
Driverless, easy access to the data
Require little or no administration
Ideally should be able to network multiple gateways together to create a
single distributed database
6
The database must be powerful enough to ingest, process and
analyze data in real-time
7. World of Watson 2016
IBM Informix: The Ideal Database for Gateways
Simple to use
Hands-Free operation – No administration
Supports popular interfaces such as MQTT, REST, and Mongo as
well as ODBC/JDBC
Handles SQL and NoSQL data in the same database
Performance
One of a kind support for TimeSeries and Spatial data
Stream data continuously into the database
Run analytics as data arrives
Dynamically add and update analytics when needed
Storage is typically 1/3 the size compared to other vendors
Invisible
Agile
7
Informix is the only database management system perfectly suited
to run in Gateways
8. World of Watson 2016
Sensor Data is TimeSeries Data
• What is a Time Series?
A logically connected set of records ordered by time
• What are the Key Strengths of Informix TimeSeries?
Much less space required
• Typically about 1/3 the space required by other vendors
Queries run orders of magnitude faster
• Unique optimized storage means codes paths are shorter and more data fits in
memory
Purpose built streaming data loader for sensor data
• Automatically run analytic and/or aggregate functions on new data
Can store structured (SQL) or unstructured (JSON) data for quick application
development
• REST/ODBC/JDBC/MongoDB/MQTT interfaces available
100’s of functions predefined
• Programming APIs available to create your own analytics
8
9. World of Watson 2016
Traditional Table Approach
Informix TimeSeries Approach
Device_ID Time Sensor1 Sensor2 ColN
1 1-1-11 12:00 Value 1 Value 2 ……… Value N
2 1-1-11 12:00 Value 1 Value 2 ……… Value N
3 1-1-11 12:00 Value 1 Value 2 ……… Value N
… … … … ……… …
1 1-1-11 12:15 Value 1 Value 2 ……… Value N
2 1-1-11 12:15 Value 1 Value 2 ……… Value N
3 1-1-11 12:15 Value 1 Value 2 ……… Value N
… … … … ……… …
Device_ID Series
1 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
2 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
3 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
4 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
…
Traditional Sensor data storage vs Informix TimeSeries Storage
9
10. World of Watson 2016
IoT Requirements for SpatioTemporal Data
• Many IoT applications have a spatial component
to them
Vehicles, cell phones, even pets… tracking is
common
• In these cases both location and time is
important
Show me the vehicles that have passed by location
X in the last hour
Where has my car been over the last few hours?
• Informix allows you to combine Time series and
Spatial data in the same query
10
11. World of Watson 2016
Ability to Recognize Patterns and Predict Events
an abnormal or a critical pattern
Similar patterns found
Patterns are significant – heart rate, ECG, blood glucose, respiratory rate,
physical activity, …
• Now looking for anomalies and deviations, and symptomatic patterns
12. World of Watson 201612
Both Structured and Unstructured Data is Common in IoT
JSON
Collection
SQL Driver
NoSQL Driver
SQL Data
Join Data
• Informix can store SQL and JSON data in the same database
• Write programs using SQL drivers or Mongo/NoSQL drivers
• SQL data automatically transformed into JSON documents when needed
• NoSQL data automatically transformed into SQL when needed
Embedded
Device or
Database server
Horizontal
Scale-out
with
Shards
13. World of Watson 2016
Informix Data Access Options
13
MongoDB
Client
REST Client
SQLI Client
DRDA Client
Informix
DBMS
Informix NoSQL
Listener
Informix
• NoSQL ↔ SQL Translation
• MQTT, REST, MongoDB
Protocol Support
• SQLI, DRDA Protocol Support
• Relational, Collection, Time
Series, and Spatial Data
Support
Spatial
Time Series
JSON Collection
Relational Table
A REST client is any
program capable of
making a HTTP request
14. World of Watson 2016
Informix Data Access Options
14
MongoDB
Client
REST Client
SQLI Client
DRDA Client
Informix
DBMS
Informix NoSQL
Listener
Informix
• NoSQL ↔ SQL Translation
• MQTT, REST, MongoDB
Protocol Support
• SQLI, DRDA Protocol
Support
• Relational, Collection, Time
Series, and Spatial Data
Support
Spatial
Time Series
JSON Collection
Relational Table
You can use all the
client drivers that are
available for
MongoDB with the
NoSQL Listener
MQTT Client
15. World of Watson 2016
IBM IoT Smart Gateway Kit
• git clone https://github.com/ibm-iot/iot-gateway-kit.git
• The iot-gateway-kit will install the following:
▪ Node.js
▪ Node-red
▪ TimeSeries nodes
▪ Bluetooth node.js application sample
15
16. World of Watson 2016
IoT Developers - Get Started!
• Smart Gateway kit - https://ibm.biz/BdXr2W
• Code samples - https://ibm.biz/BdX4QV
• Github - https://github.com/IBM-IoT/
16
17. World of Watson 2016
Informix on Docker Hub
https://registry.hub.docker.com/u/ibmcom/informix-innovator-c/
• IBM Informix Innovator-C
• 12.10.FC7W1
https://registry.hub.docker.com/r/ibmcom/informix-rpi/
• IBM Informix Developer Edition for Raspberry Pi (32bit)
17
Docker Hub
$docker pull ibmcom/informix-innovator-c
18. World of Watson 2016 18
Informix for the Cloud and
Operational Zone
19. World of Watson 2016 19
What are the IoT Requirements for the Cloud?
• Requirements - similar to gateways (but for different reasons):
• Potentially 1000’s of servers means zero administration is a must
• Data volume adds up very quickly. Low storage overhead is required
• Data flows into the cloud continuously and must be processed in real-time
• Must be able to handle time series, spatial, and NoSQL data natively
• Additional requirements
• Must be able to scale-out
• Must be available as a service
The database must be able to ingest, process and analyze
data in real-time
20. World of Watson 2016 20
Why use Informix in the “Operational Zone”?
Simple to use
• Hands-Free operation
• Supports REST and Mongo APIs as well as ODBC/JDBC
• Stores SQL and JSON database in the same database
Highly Available
• Close to zero down time
• Partition or Hash your data across servers in the cloud
• Dynamically add/remove additional servers
Performance
• Continuous High Performance Analytics
• Specialized support for Time Series and Spatial data
Invisible
Agile
Resilient
21. World of Watson 2016 21
Shards: Scale-out your Database across Servers or Gateways
• Distribute data among servers by
range or hash partitioning
• Each shard can have an associated
secondary server for high availability
• Run queries across all shards or a
subset of the shards
• Only shards that could qualify are
searched
• Shards are searched in parallel
• Ignores shards that are offline
Shards in a
Cloud
22. World of Watson 2016 22
TCP/IP
Bulk Loader
SQL Queries (from apps)
Informix Warehouse Accelerator
Compressed
DB partition
Query
Processor
Data Warehouse
Informix
SQL
Query Router
Results
Informix Warehouse Accelerator:
• Connects to Informix via TCP/IP & DRDA
• Analyzes, compresses, and loads to memory
• Copy of (portion of) warehouse
• Processes routed SQL query and
• returns answer to Informix
Use Informix Warehouse Accelerator for
Mixed Operational/Analytic Workloads
Informix:
• Routes SQL queries to accelerator
• User need not change SQL or apps.
• Can always run query in Informix
• Too short an est. execution time
23. World of Watson 2016 23
Every IoT deployment will need to store time series data and
possibly spatial data and/or NoSQL data
Bluemix Cloud Service
Informix on Cloud – Hosted Service
• Quickly and simply deploy Informix
• Pre installed and pre configured instance
• Multiple size options (S, M, L, XL)
25. World of Watson 2016
Changing Business Model – Health care & Assisted Living
Informix Historian
Operational Analytics
Notification to Assisted
Living Central Monitoring
Station
Change patients medication, closer
monitoring, prevent stroke
1
2
3
Patient/Care giver
Hundreds of patients
Thousands of
devices
Locally Act Upon
Insights
Data
Consolidation
Gateway
Sensor Data
Input
Display Alerts and
Recommended Actions
4
5
Collection and analysis of data
for all devices across assisted
living facilities
Assisted Living Corporation
changes food sodium usage based
on trend of high blood pressure
Filter critical and life-saving
data
Blood pressure threshold
exceeded
• Embedded at
device/gateway
• Local decision making at
Facility
• Leverage all data:
NoSQL/SQL & Timeseries
data
Automatic sensors to
monitor well being
Pendants, shower &
bath buttons
Activity sensors – rising
in the morning, taking
meds, using the fridge
Bed & Chair sensors for
inactivity monitoring
Outside alarms to alert
neighbors
26. World of Watson 2016 26
And Many More…
See for yourself and talk to us @
Analytics Demo Room – 530
DMT09, DMT13
27. World of Watson 2016
Summary
• IBM Informix - best fit for IoT architecture
• IoT gateway
• IoT cloud analytics
• Supported on a wide array of platforms
• Best in class embeddability
• Native support for sensor data - TimeSeries & Spatial data
• Native support for unstructured (JSON) data
• Ease of application development - REST access
• Support to receive IoT data via MQTT protocol
• High availability and dynamic scaling
27