Choosing the right platform for your Internet -of-Things solution

The Internet of Things:
choosing a database
Philip Howard
Research Director

telling the right storyConfidential © Bloor Research 2016
Generalisations
Use cases
Edge processing
Deployment options
Necessary database features
Agenda

IoT use cases like Olympic sports

Many use cases similar

Some use cases wildly different

Typical use cases

Where the money is going

What’s the difference?
Direct and indirect connectivity

Sensors
Sensor readings are only interesting if:
They are combined with other readings of
the same type
They are combined with readings of a
different type
Or both
And
Changes and/or trends can be identified

How smart is the Edge?

Edge Gateway

Data flow in IoT
Sensors
Aggregation
Point(s)
Aggregation
Point(s)
Centre

Deploying smarts

What do you need?
You may need streaming analytics
You will need a database at each
aggregation point and in the centre

What you need from your database
Fire and forget
No or self-tuning
Autonomics and self-healing
Performance
Small footprint
Specific feature support
Deployable in all locations

Specific features

Deployable in all locations
Aggregation points and the centre –
operational, HTAP and warehousing

The advantage of homogeneity

Conclusion
IoT use cases vary widely
How and where data is aggregated
and/or processed varies similarly
It will be useful to deploy the same
database across the whole IoT
infrastructure
Such a database must provide a number
of specific features

Thank you

IBM Informix 12.1: The Intelligent Database
Simply Powerful
IBM Informix for Your IoT Data Management
Solution at the Edge & Cloud
August 25, 2016
Shawn Moe – Informix Engineering Lab

© 2016 IBM Corporation22
The Internet of Things Landscape
Data Management at the IoT Edge
What about the Cloud?
Why Informix 12.1 for both?
Informix in the field – IoT applications
Wrap it Up….
Agenda

Changing Business Model – IoT for Health Care & Assisted
Living
Informix Historian
Operational
Analytics
Notification to Assisted
Living Central Monitoring
Station
Change patients medication,
closer monitoring, prevent stroke
1
2
3
Patient/Care giver
Hundreds of
patients
Thousands of
devices
Locally Act Upon
Insights
Data
Consolidation
Gateway
Sensor Data
Input
Display Alerts and
Recommended Actions
4
5
Collection and analysis of
data for all devices across
assisted living facilities
Assisted Living Corporation
changes food sodium usage
based on trend of high blood
pressure
Filter critical and life-
saving data
Blood pressure threshold
exceeded
• Embedded at
device/gateway
• Local decision making
at Facility
• Leverage all data:
NoSQL/SQL &
Timeseries data
 Automatic sensors
to monitor patient
well-being
 New devices:
Pendants, shower &
bath buttons
 Activity sensors –
rising in the
morning, taking
meds, using the
fridge
 Bed & Chair sensors
for inactivity
monitoring
 Outside alarms to
alert neighbors

IoT applications have a common set of requirements
Opportunities for innovation
 Quickly and easily provision new sensors
 Create a real-time communication channel
with the sensor
 Capture data from the sensor and store it
in a time series database
 Provide secure access to the collected data
– analytics at the Edge and Cloud, in real-
time & on historical data
 Trigger events based on specific data
conditions
 Interact with the sensor from
business/enterprise applications and/or
from mobile devices
 Monetize the service based on usage

Simplified IoT Data Flow
Sensor Data History
Sensors
In-memory Analytics
Predictive Analytics
Publish /
Subscribe
Cloud Infrastructure
Real-time Analytics
Real-time Analytics Operational Analytics Big Data Analytics
(no gateway)
(Gateways)
HDFS / Hadoop
Big Data Analytics
MessageSight /
MQTT
Gateways for local
analytics
InfoSphere
Streams
Informix / Cloudant / Watson
IOT Platform Service /
Informix on Cloud Service
Informix Warehouse
Accelerator / DashDB
PMQ / SPSS /
Cognos
Softlayer / Bluemix
Watson / DashDB /
BigInsights
Informix / Node-
Red

IoT requires Analytics and Data Management at each level!
Streams
Deep Analytics Zone
Analytics Zone
Smart Gateways
Devices/Sensors

Wrap it Up….
Agenda

• Reduce Cost
• Reduces cloud storage by filtering/aggregating/analyzing data locally
• Reduces cloud CPU requirements by precomputing values
• Reduce Latency
• Intelligent gateways can detect and respond to local events as they
happen rather than waiting for transfer to the cloud
• Performs 80% simple operations locally
• Security
• Gateways allow customers to capture and get value from their sensors
without sending data to the cloud
• Protocol Consolidation
• Cloud does not need to deal with the hundreds of IoT protocols.
Gateway can “normalize” data before sending to cloud
More and more IoT processing will move from the cloud to gateway devices
How Do Smart Gateways Help IoT Solutions?

 The database management system must:
 Have a small install footprint, less than 100 MB
 Run with low memory requirements – less than 256 MB
 Use lossless compression to minimize storage space
 Have built-in support for common types of IoT data like time series and spatial/GIS data
 Simple application development environment supporting NoSQL, SQL, and REST
 Require absolutely no administration
 Be able to network multiple gateways together to create a single distributed logical
database
What are the Requirements for a Gateway Database?
The database must be powerful enough to ingest, process and analyze data in real-time

 Many IoT applications have a spatial component to them
• Vehicles, cell phones, even pets…
 In these cases both location and time is important
• Show me the vehicles that have passed by location X in the last hour
• Where has my car been over the last few hours?
IoT Requires Highly Optimized Spatial & Temporal Data Processing

• Industrial IoT is reaching point of stabilization and standardization
• Now looking towards predictive analytics for anomalies and deviations
IoT requires the Ability to recognize Patterns and Predict Events
an abnormal power usage pattern
Similar patterns found

Simple to use
• Millions of embedded-use installations
• Hands-Free operation – No administration
• Supports popular interfaces such as REST & MongoDB as well as
traditional SQL interfaces (ODBC/JDBC)
• Handles SQL and JSON data in the same database
• Seamless data replication and HA solutions to move or copy data
where needed
Performance
• Unique support for TimeSeries and Spatial data
• Stream data continuously into the database
• Run analytics operations as data arrives
• Dynamically add and update analytics when needed
• Storage is typically 1/3 the size compared to other vendors
Invisible
Agile
Informix is the only database management system perfectly suited to run in Gateways
IBM Informix: The Ideal Database for Gateways

 Informix is proven technology as an
embedded database. Providing a small
footprint with a fully featured enterprise
database server.
 Built in autonomics with self healing, self
configuration and automation with the DB
scheduler
 Automated space provisioning
 Automated Memory Management
 Dynamically tuned Engine parameters
 Embedding Informix in consolidation devices
on the IoT edge allows for:
– Complex store and forward capabilities
with transformation and aggregation of
data
– Business decisions made on the edge,
closer to the producer of the data
– Security policies to determine which
sensor data gets captured, processed
and sent to the cloud
Why embed Informix 12.1 at the Edge?

 What is a Time Series?
• A logically connected set of records ordered by time
 What are the key strengths of Informix TimeSeries?
• Native data type resulting in significantly less space requirements
• Typically about 1/3 the space required by other vendors
• Queries run orders of magnitude faster
• Unique optimized storage means more data fits in memory
• Purpose built streaming data loader for sensor data
• Automatically run analytic and/or aggregate functions on new data
• Integrates structured (SQL) or unstructured (JSON) data to store metadata for
each time value
• REST/ODBC/JDBC/JSON interfaces available to work with this data
• API contains hundreds of manipulation & analytics functions + APIs to create your
own analytics
Sensor Data is TimeSeries Data

Traditional (relational) table storage
Informix TimeSeries storage
Meter_ID Time KWH Voltage ColN
1 1-1-11 12:00 Value 1 Value 2 ……… Value N
… … … … ……… …
… … … … ……… …
Meter_ID Series
1 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
…
Traditional Sensor data storage vs. Informix TimeSeries storage

Metric Competitor Informix
Daily processing time
Maximum number of cores used
11 hours
62
5 hour 50 min
32
Maximum amount of memory used 192GB 192GB
Size of database per month of data 15TB 5TB
# Records processed each day 2.88 Billion 2.88 Billion
Billing determinants creation (1/21 of the total
meter population)
51,322 ~2 million reads per
second
TimeSeries Meter Data Management Benchmark
- 30 million smart meters sending data every 15 minutes
- 2.88 billion records inserted each day
- Workload: data ingestion, data cleanup, and a daily billing cycle

Wrap it Up….
Agenda

What are the IoT Requirements for a Cloud data storage solution?
• Requirements are similar to gateways, but for different
reasons:
• Potentially 1000’s of servers means zero administration is a must
• Data volume adds up very quickly so low storage overhead is required
• Data flows into the cloud continuously and must be processed in real-time
• Must be able to handle time series, spatial, SQL and NoSQL data natively
• Additional requirements
• Must be able to scale-out & scale-up
• Must be available as a service

Why Informix 12.1 for both Edge & Cloud?
Enterprise class database for the Edge and the Cloud
 Enterprise database embedded in gateways and
consolidators based on ARM or Intel/Quark
processors – maximizing availability in this space
 Horizontal scaling with sharded data across ER –
delivering elasticity for the Cloud
 Softlayer and Bluemix support – delivering
Informix on Cloud Service for SaaS offering
 Multi-tenancy support – allowing hosting of
multiple logically independent server instances
within on single physical instance
 Delivering cost benefits on hardware resources
and software licenses
 Simplified administration for backup of multiple
database servers in Cloud
 Accelerate all types of data with in-memory
Informix Warehouse Accelerator
Informix Server Instance
DB
Tenant
A
DB
Tenant
B
DB
Tenant
C

Informix Hybrid Cloud enables seamless control and
flexibility
• Utilize any machine from anywhere
• Single platform for both on & off-premise
• Scalability for peak periods
• Interconnectivity with IoT data
• Maximize existing resources
• Reduce costs and IT footprint
• The same hybrid apps (using both structured
and unstructured data) can run against any
machine in your hybrid cloud (both on & off
premise machines)
• Tremendous Flexibility!
Put your data where you need it, when you need it

 Informix is available on Bluemix – Coming in September!
IBM Informix on Cloud
• Cloud hosted service includes Informix license and cloud
“hardware”
• T-shirt sizing: S, M, L, XL instances match Informix license and
hardware capacities to provide optimal value at each size
• Informix instance hosted in IBM SoftLayer data centers with
world wide deployment options
• IBM provisions, configures, and tests the instance and then
passes the credentials on to the customer
• Full Informix functionality to support all kinds of work loads:
• OLTP
• Hybrid NoSQL, SQL, TimeSeries and Spatial
• IoT
• IWA
• Rapid application development with support for SQL, MongoDB,
REST or MQTT themed applications
• It’s Informix!
Rationale: deliver high-quality cloud service with low cost of operations

Wrap it Up….
Agenda

1. High Availability Data Replication (HDR)
2. Enterprise Replication (ER)
3. Flexible Grid
Remote Standalone Secondaries (RSS)
Shared Disk Secondaries (SDS)
4. Sharding
Benefits of an Informix data availability solution:
 Scale globally
 Manage easily
 Balance workloads
 Rolling version upgrades
 Heterogeneous platform support
 Ideal for the Cloud
Why Informix 12.1? – Flexible Set of Data Availability Options
Informix has the most complete set of
Data Availability options in the industry!
Mix and match these technologies

Shards: Scale-out your Database across Servers or Gateways
• Distribute data among servers by
range or hash partitioning
• Each shard can have an associated
secondary server for high availability
• Run queries across all shards or a
subset of the shards
• Only shards that could qualify are
searched
• Shards are searched in parallel
• Ignores shards that are offline
Shards in a Cloud

Shard Key
Country = “FR”
Shard Key
Country = “UK”
Shard Key
Country =“DE”
Sharding + Informix HA means your data is always available!
Shared Disk
Secondary Remote
Standalone
Secondary
HDR pair

Informix Flexible Grid is ideal for the cloud environment
Informix Connection Manager
Query Sharded
Data
Sales Database
Shards• Vertical & horizontal scaling
– Adding physical resources in the cloud & with the
Informix Grid easily add a node to the Informix cluster
• Quick response to business needs
– Allocate or de-allocate resources in the public or
private cloud as needed (matter of minutes)
• Data Flexibility
– Balance data across the nodes in a cluster
– Query sharded data, data spread out across the
nodes in the cluster
– Layer of abstraction between user and where the data
is actually located

 Flexibility in schema – or schema-less
applications
• Ease of application development,
reducing cost
• Faster to market
 Native support for JSON and binary JSON
(BSON) data types
 Data Access is not restricted by Data
Models
• TimeSeries and JSON can co-exist with
traditional SQL in the same DB
– Rapid application & services development
support
• Enhanced API support
• REST (REpresentational State
Transfer) API support enables
developers to use any programming
paradigm that supports HTTP
Why Informix 12.1? – Flexibility for App Developers
Relational Table JSON
Collections
SQL API Standard ODBC,
JDBC, .NET,
OData, etc.
Language SQL.
Direct SQL Access.
Dynamic Views
Row types
MongoDB
API
(NoSQL)
Mongo APIs for
Java, Javascript,
C++, C#, etc.
Mongo APIs for
Java, Javascript,
C++, C#, etc.
Access to Relational Tables & JSON Collections

48
Why Informix 12.1? - All Clients can access all Data Models
• NoSQL ↔ SQL
Translation
• Wire Listeners for
MongoDB, REST &
MQTT protocols
• SQLI, DRDA Protocol
Support
• Relational,
Collection, Time
Series, & Spatial
Data Support
Mobile
Desktop
Web
REST Client
MongoDB
Client
SQLI Client
DRDA Client
MQTT Client
Informix
DBMS
Informix Wire
Listener
Spatial
Time Series
JSON Collection
Relational Table

 IBM BLU (In-memory) technology runs on the highly
compressed Data Marts
 No indexes required
 Query predicates are evaluated in compressed
format – no need to uncompress to evaluate
 Storage and compression techniques allow memory
requirements to be 1/3 to 1/5 of underlying OLTP
storage
 Accelerates all types of data directly or using views
 Support for OLAP SQL functions which enhance
performance and integration with BI tools
Warehouse Accelerator
Data Marts
Why Informix 12.1? – Query Acceleration with Informix
Warehouse Accelerator (IWA)
Intelligent Database for Business Intelligence

You can use IWA’s In-Memory Analytics to Speed Up queries on…
Synonyms
to local or remote tables
NoSQL Data
ex: JSON collections
SQL Data
from local Informix tables
External Tables
ASCII/binary files in file
system or network pipes
Views
to Informix tables or
NoSQL data
Sensor data
TimeSeries data,
time-stamped data
Why Informix 12.1? – Query & Warehouse Acceleration

Why Informix 12.1? – Query & Warehouse Acceleration
IWA Use Case – Pharmacy Chain in Mexico
In largest Fact Table
23x: 90min (IDS) -> 4min (IWA)
75x: 10min (IDS) -> 8sec (IWA)
IDS’ longest queries took 40-90min
IWA’s longest queries take 2-6min
Number of Rows
1.2B Rows
75x Faster
Data Volume
Compressed IWA data mart
800Gigabyte
6Months of data
Over 2 TB of data in Informix/IDS
IWA has latest 6 months of data
IDS has 18 months of data
10 Fact Tables are for end user queries
IWA’ s 6 last months data kept
through: Month-Cyclic Roll-in/out
Informix/IDS + IWA
12.10.FC2
Tables in Data Mart
59Tables
12.10 Ver.
12 Fact Tables in biggest datamart
3.6 Billion rows in IDS
Production Roll-Out Plan: Nov 2013
No more need to summarize data
Removed some indexes in Informix/IDS

Wrap it Up….
Agenda

IoT in Retail – Sample Informix Deployment
Enabling a better consumer experience
Collection of data for all
devices across the
enterprise
Hints/Suggestions
Correlation/Comparison
Detection/Predictions
Store
Manager/Supervisor
Data Input – Sensors in
Retail Store
Device data
consolidation
Gateway
Embedded
Informix Cluster in the Cloud
NoSQL/Relational &
Timeseries data
Informix Warehouse Accelerator
Real-Time Analytics
Streaming data
SPSS/Cognos
MessageSight
Infosphere Streams
Device protocols
including CoAP,
6LoWPAN, ZigBee
etc..
SoftLayer /
BlueMix
BigInsights
Sensors monitoring people
traffic, smart shelves, PoS,
Vending machines, carts..
Real-time Response
on Alerts and
Analytics
Smart
Shopper
Deals/Promotions
/Coupons

Fan & sensor (wifi)
Temperature Sensors (wifi)
Inside and outside the bin
Gateway
Web
Server
Wifi
module
via EOP
Humidity sensor (wifi)
Inside and outside the bin
3G/4G cell
communication
module
OnSite Weather Data
• Temperature
• Pressure
• Humidity
• Rainfall
• Wind
Direction
• Wind Speed
Every1minute
Every1minute
Mobile App for
bin operators
Console Dashboard
Data
Center /
Cloud
IoT in Agriculture – Sample Informix Deployment
Enabling a smarter grain management system – emphasizing food safety & quality

 Analyze
 Suspension, wheels, alignment, noise & vibration
 Wear on mechanical components: bearings, gears, belts
 Human responses
 Insights from data helps with:
 Safety improvements
 Preventive maintenance scheduling
 Testing & simulation scenarios
 Capacity & route planning
 Much more ….
 Thousands of different types of sensors involved
 Possible IoT solution architecture could include:
 Informix at the “mobile Edge” to support immediate
actions and near real-time analytics
 Informix in the Data Center/Cloud for operational
analytics
55
IoT in Transportation - Smarter Vehicle Maintenance
Enabling a smarter transportation system – emphasizing safety & cost savings

Informix offers Enterprise Class Embedded Data Management
Call Management
A Call Management Solution that provides call features, such as call forwarding,
automated voice, etc. in a geographically distributed, highly available
environment
Benefits:
 Flexible solution to today’s call features
 Flexible architecture allows solution to fit into a wide variety of environments,
without changes
 Cost performance allows competitive solution in a battleground technology
Why do Customers choose Informix based solutions?
Low Cost – Low Administration- High Performance – High Availability – 99.999% Uptime- Accelerate time to market

Wrap it Up….
Agenda

 IBM Informix - best fit for IoT architecture
• IoT Smart Gateway
• IoT Cloud analytics
 Supported on a wide array of platforms, including SoC
computers
 Best in class embeddability
 Native support for sensor data - TimeSeries & Spatial data
 Native support for unstructured (JSON) data
 Ease of application development - REST access
 Support to receive IoT data via MQTT protocol
 High availability and dynamic scaling
 In-memory query acceleration
Summary

 Bloor White Paper - IBM Informix and the Internet of Things- http://ibm.co/2bITDyU
 IBM Informix - http://www-01.ibm.com/software/data/informix/
• IBM Informix Support - http://www-
947.ibm.com/support/entry/portal/overview/software/information_management/informix_servers
• IBM developerWorks pages for Informix -
http://www.ibm.com/developerworks/data/products/informix/
• Informix International User Group (IIUG) - http://www.iiug.org/index.php
• Planet IDS - http://planetids.com/
• IBM Informix on LinkedIn -
http://www.linkedin.com/groups?home=&gid=4029470&trk=anet_ug_hm
• IBM Informix on Facebook - https://www.facebook.com/IBM.Informix
• IBM Informix on Twitter - https://twitter.com/IBM_Informix
• IBM Informix Blogs (a few of them):
• https://www.ibm.com/developerworks/community/blogs/smoe/?lang=en
• https://www.ibm.com/developerworks/community/blogs/idsteam/?lang=en
• http://www.ibmnosql.com/author/jmiller/
• https://www.ibm.com/developerworks/community/blogs/fredho66/?lang=en_us
• https://www.ibm.com/developerworks/community/blogs/idsdoc/?lang=en_us
Some Useful Information

 IBM Smart Gateway kit - https://ibm.biz/BdXr2W
 Code samples - https://ibm.biz/BdX4QV
 Github - https://github.com/IBM-IoT/
 Free Informix Developer Edition - https://ibm.biz/BdXp2g
 Informix on Docker Hub https://registry.hub.docker.com/u/ibmcom/informix-innovator-c/
 Informix Developer Edition for Raspberry Pi (32bit)
https://registry.hub.docker.com/r/ibmcom/informix-rpi/
IoT Developers - Get Started!
Docker Hub

Shawn Moe – smoe@us.ibm.com

Informix 12.10: Simply Powerful
28

Choosing the right platform for your Internet -of-Things solution

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Choosing the right platform for your Internet -of-Things solution (20)

Recently uploaded (20)

Choosing the right platform for your Internet -of-Things solution