MongoDB IoT City Tour STUTTGART: Managing the Database Complexity, by Arthur Viegers

Sep 30, 2014Download as PPTX, PDF0 likes1,090 views

Arthur Viegers, Senior Solutions Architect, MongoDB. The value of the fast growing class of NoSQL databases is the ability to handle high velocity and volumes of data while enabling greater agility with dynamic schemas. MongoDB gives you those benefits while also providing a rich querying capability and a document model for developer productivity. Arthur Viegers outlines the reasons for MongoDB's popularity in IoT applications and how you can leverage the core concepts of NoSQL to build robust and highly scalable IoT applications.

I o T E U R O P E A N C I T Y T O U R
S T U T T G A R T

MongoDB and
The Internet of Things
Arthur Viegers
Senior Solutions Architect, MongoDB
MongoDB IoT City Tour 2014

MongoDB
*
Document
Database
Open-
Source
General
Purpose

$Documents Are Core Relational MongoDB * { first_name: "Paul", surname: "Miller", city: "London", location: [45.123,47.232], cars: [ { model: "Bentley", year: 1973, value: 100000, … }, { model: "Rolls Royce", year: 1965, value: 330000, … } ] }$

Time series schema design goal
*
• Store event data
• Support Analytical Queries
• Find best compromise of:
- Memory utilization
- Write performance
- Read/Analytical Query Performance
• Accomplish with realistic amount of hardware

Modelling time series data
*
• Document per event
• Document per minute (average)
• Document per minute (second)
• Document per hour

Document per event
*
{
deviceId: "Test123",
timestamp: ISODate("2014-07-03T22:07:38.000Z"),
temperature: 21
}
• Relational-centric approach
• Insert-driven workload

$Document per minute (average) { * deviceId: "Test123", timestamp: ISODate("2014-07-03T22:07:00.000Z"), temperature_num: 18, temperature_sum: 357 } • Pre-aggregate to compute average per minute more easily • Update-driven workload • Resolution at the minute level$

Document per minute (by second)
{
*
deviceId: "Test123",
timestamp: ISODate("2014-07-03T22:07:00.000Z"),
temperature: { 0: 18, 1: 18, …, 58: 21, 59: 21 }
}
• Store per-second data at the minute level
• Update-driven workload
• Pre-allocate structure to avoid document moves

Document per hour (by second)
{
*
deviceId: "Test123",
timestamp: ISODate("2014-07-03T22:00:00.000Z"),
temperature: { 0: 18, 1: 18, …, 3598: 20, 3599: 20 }
}
• Store per-second data at the hourly level
• Update-driven workload
• Pre-allocate structure to avoid document moves
• Updating last second requires 3599 steps

Document per hour (by second)
{
*
deviceId: "Test123",
timestamp: ISODate("2014-07-03T22:00:00.000Z"),
temperature: {
0: { 0: 18, …, 59: 18 },
…,
59: { 0: 21, …, 59: 20 }
}
}
• Store per-second data at the hourly level with nesting
• Update-driven workload
• Pre-allocate structure to avoid document moves
• Updating last second requires 59 + 59 steps

$Rexroth NEXO schema { * _id: ObjectID("52ecf3d6bf1e623a52000001"), assetId: "NEXO 109", hour: ISODate("2014-07-03T22:00:00.000Z"), status: "Online", type: "Nutrunner", serialNo : "100-210-ABC", ip: "127.0.0.1", positions: { 0: { 0: { x: "10", y:"40", zone: "itc-1", accuracy: "20” }, …, 59: { x: "15", y: "30", zone: "itc-1", accuracy: "25” } }, …, 59: { 0: { x: "22", y: "27", zone: "itc-1", accuracy: "22” }, …, 59: { x: "18", y: "23", zone: "itc-1", accuracy: "24” } } } }$

Working Set Exceeds Physical Memory
*
Working Set
Indexes Data
Working Set
Indexes Data

Scaling Out
First Edition (1771)
*
3 Volumes
Fifteenth Edition (2010)
32 Volumes

Shards and Shard Keys
*
Shard
Shard key
range

Shard Key Considerations
*
• Cardinality
• Write distribution
• Query isolation
• Reliability
• Index locality

Shard Key Selection Rexroth NEXO
*
Cardinality
Write
Distribution
Query Isolation Reliability
Index
Locality
_id Doc level One shard Scatter/gather
All users
affected
Good
hash(_id) Hash level All Shards Scatter/gather
All users
affected
Poor
assetId Many docs All Shards Targeted
Some assets
affected
Good
assetId, hour Doc level All Shards Targeted
Some assets
affected
Good

Scaling Data - Summarized
*
• MongoDB scales horizontally (sharding)
• Each shard is an independent database, and
collectively, the shards make up a single logical
database
• MMS makes it easy and reliable to run
MongoDB at scale
• Sharding requires minimal effort from the
application code: same interface as single
mongod

Why is MongoDB a good fit for IoT?
*
• IoT processes are real-time
• Relational technologies can simply not compete on cost,
performance, scalability, and manageability
• IoT data can come in any format, structured or
unstructured, ranging from text and numbers to
audio, picture and video
• Time series data is a natural fit
• IoT applications often require geographically
distributed systems

The value of the fast growing class of NoSQL databases is the ability to handle high velocity and volumes of data while enabling greater agility with dynamic schemas. MongoDB gives you those benefits while also providing a rich querying capability and a document model for developer productivity. Arthur Viegers will outline the reasons for MongoDB's popularity in IoT applications and how you can leverage the core concepts of NoSQL to build robust and highly scalable IoT applications.

MongoDBAlbin John

MongoDB is an open-source document-oriented database that provides horizontal scalability, high performance, and flexibility. It stores data as JSON-like documents which allows for flexible, schemaless structures. MongoDB can scale horizontally by sharding data across multiple servers and provides replication for redundancy and high availability. It is a popular NoSQL database choice for applications that need to handle large volumes of both structured and unstructured data.

Introduction to Azure DocumentDBRadenko Zec

This document provides an introduction to Microsoft Azure DocumentDB. It discusses how DocumentDB is a non-relational or NoSQL database that stores data in JSON documents. It also overview how DocumentDB provides scalability, high availability, and fast performance for large document workloads. Key features of DocumentDB discussed include its resource and interaction models, indexing, consistency options, querying capabilities, and support for JavaScript transactions.

Securededuplicationschemeforcloudstorage 141128075306-conversion-gate01shobhiya kumar

This document proposes a system for secure client-side data deduplication for cloud storage. It suggests using convergent encryption, where keys are derived from the hash of the plaintext, along with a security model. The solution includes a metadata manager that handles deduplication on encrypted blocks and key management separately from data storage. A prototype is being tested for performance analysis of typical operations like edit, append and delete with promising initial results. The proposed system provides confidentiality while allowing for block-level deduplication with negligible performance impact.

Logging for ContainersEduardo Silva Pereira

Leveraging AWSForum One

This document discusses leveraging AWS to host various websites and applications. It describes the hosting history of Forum One Hosting and then provides details on how AWS was used to host three specific websites - www.globalchange.gov, www.countyhealthrankings.org, and www.mcc.gov. For each website, it outlines the challenges, key AWS components used (EC2, S3, RDS, CloudWatch, etc.), and results. The document concludes with some final notes on best practices for using AWS like using one account, configuration management, and external monitoring.

Data Lessons Learned at ScaleCharlie Reverte

Charlie Reverte, VP of Engineering at AddThis, discusses lessons learned from processing large-scale web data. AddThis processes data from 14 million domains, including 100 billion monthly page views and 50,000 events per second. Reverte outlines challenges around distributed ID generation, counting unique values, joining distributed data, sampling large datasets, and deploying systems that invalidate over 1.4 billion browser caches. He advocates for loose coupling between systems using approaches like Kafka for asynchronous event logging. Reverte also discusses techniques for columnar compression, tunable quality of service, and open sourcing Hydra, AddThis' custom processing system optimized for real-time data.

Monitoring your shiny new docker environmentSamuel Vandamme

You’ve spent considerable time picking your orchestrator, choosing the right cloud provider and configuring all the intricate details of your new Docker environment, but what about monitoring? In this talk we will cover the tools available on the market: upsides, downsides and upcoming changes. We’ll open the floor to questions, comments and feedback for each tool, so you have a complete view on the monitoring landscape.

Quick overview on mongo dbEman Mohamed

This document provides a quick overview of MongoDB, including: - MongoDB trades off ACID compliance for availability and scalability. - CRUD operations allow creating, reading, updating, and deleting documents. Indexes improve query performance. - Embedded documents and references model one-to-one and one-to-many relationships. - Aggregation operations group and transform data from multiple documents. Map-reduce is an alternative but less efficient.

Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...Miguel Pérez Colino

Going Serverless Schezarnie Racip

A simple traditional web application consists of a web server, web client and an optional database. A traditional software developer needs to be aware of the server in which his code runs, because it has to be created, scaled, and maintained. If it goes down at the most critical time, it could result in drastic consequences. When a single application is deployed in a server, there will be lot of unused resources. For example, if a developer deploys a Storm cluster which uses CPU heavily for computation, there will be less use of the hard disk. These servers are not custom made for our applications so there will be resource wastages. The concept of a Serverless Architecture came up as a solution to overcome this problem and a lot of other issues coupled with physical servers. In this Innovation Session, Danula Eranjith, Malinga Perera and Shehan Perera from the Data and Analytics team discuss the Serverless Architecture concept, and how it can be implemented using AWS technologies.

Cassandra meetup slides - Oct 15 Santa Monica ColoftJon Haddad

This document summarizes Shift.com's migration from MongoDB to Cassandra. Shift is a platform that enables marketers to communicate across organizations. The initial database stack included MongoDB, but it was replaced with Cassandra for better operational benefits like easier node management, better control of data storage, and improved long-term scalability. The migration goals were zero downtime and no loss of performance. The strategy involved carefully structuring the Cassandra data model and schema to match MongoDB's performance. Benefits of Cassandra included its familiar CQL query language and improved support for features like time series data storage.

Fluentd Intro for OpenShift Commons BriefingEduardo Silva Pereira

This document discusses logging in cloud native environments and introduces Fluentd and Fluent Bit as solutions. Fluentd is well-suited for log aggregation due to its pluggable architecture, reliability, and integration with Docker and Kubernetes. Fluent Bit is designed as a lightweight log forwarder that can send logs from edge nodes to Fluentd for aggregation. Using Fluent Bit for forwarding reduces resource usage on edge nodes. Both support features important for cloud environments like backpressure handling.

Node.js by Alex and NalinTagged Social

Containers and LoggingEduardo Silva Pereira

HNSciCloud Info Day, 7 Sept 2016, Functional Requirements by Helge MeinhardHelix Nebula The Science Cloud

This document provides an overview of the technical aspects of the 2Helix Nebula – The Science Cloud project. It discusses research and development goals around transparent data access and workload portability. It describes planned prototype and pilot installations involving 3,500 and 10,000 cores with 350TB and 1PB of shared storage respectively. Details are provided around virtual machines, shared storage requirements, and networking connectivity. The document aims to highlight key questions for discussion rather than provide a fully comprehensive technical overview.

Eagle6 Enterprise Situational AwarenessMongoDB

Eagle6 is a product that use system artifacts to create a replica model that represents a near real-time view of system architecture. Eagle6 was built to collect system data (log files, application source code, etc.) and to link system behaviors in such a way that the user is able to quickly identify risks associated with unknown or unwanted behavioral events that may result in unknown impacts to seemingly unrelated down-stream systems. This session is designed to present the capabilities of the Eagle6 modeling product and how we are using MongoDB to support near-real-time analysis of large disparate datasets.

Zentrales logging mit dem Elastic StackSimonSchneider24

The document discusses centralized logging with the Elastic Stack. It describes the components of the Elastic Stack including Beats, Logstash, Elasticsearch, and Kibana. It explains how Beats collects log data and sends it to Logstash for ingestion into Elasticsearch for storage, search, and analysis. Kibana is used for visualizing and managing the logged data. The document provides an example of using the Elastic Stack with Filebeat, Logstash, and Elasticsearch for log ingestion and analysis of access logs from a web server. It discusses benefits like reduced costs using a hot-warm architecture with the Elastic Stack.

Timur Shemsedinov "Эволюция архитектуры ИС"OdessaJS Conf

The document traces the evolution of application architecture from early data files to modern web and mobile applications. It outlines key stages including file servers, local databases, client-server, and three-tier architectures. Emerging trends are seen moving to cloud-based, distributed, high-load, interactive, and flexible applications. The future may see more database-centric, peer-to-peer, offline, low-latency, and big data capable architectures that synchronize data across high volumes of interconnected users and servers. The full article provides additional details on these stages and directions for application evolution.

Alexander Sibiryakov- FronteraPyData

This document summarizes a presentation given by Alexander Sibiryakov about Frontera, an open source web crawling framework. Frontera allows building large-scale web crawlers that can crawl billions of pages per month in a distributed manner. It provides abstractions for crawling strategies, message buses, and backend storage. The document describes example uses of Frontera including focused crawls, news analysis, and due diligence. It also outlines the software and hardware requirements and discusses future plans for Frontera.

Http BasicsMushfekur Rahman

This document provides an overview of key HTTP concepts including developer tools, security, sessions, cookies, caching, APIs, and resources for learning more. It discusses Google Chrome developer tools and their panels for debugging websites. It also covers how HTTP and HTTPS work, including how HTTPS uses encryption and SSL certificates. Additionally, it explains how cookies are used to make HTTP stateful through client-side storage of values and how they can be used for authentication, personalization, and tracking.

Draft slide of Demystifying DHT in GlusterFSAnkit Raj

This document discusses distributed hash tables (DHT) in GlusterFS. It introduces key terminology like bricks, volumes, and nodes. It explains that GlusterFS uses a distributed hash model to store and access files across multiple servers, organizing and displaying files as if they were stored locally. This allows for centralized storage and easier distribution of documents to multiple clients without using the clients' local storage resources. The document then outlines how DHT solves problems and lists some common file operations like mkdir, create, lookup, and read that DHT facilitates. It also addresses managing scalability through operations like expanding volumes, rebalancing, and replacing bricks.

MinIO January 2020 BriefingJonathan Symonds

This document provides an overview of MinIO object storage. It discusses how MinIO is focused on performance and simplicity, and is cloud native and open source. It highlights MinIO's growth, traction with developers, and deployments across industries and configurations. The document also includes benchmark results demonstrating MinIO's high performance, as well as descriptions of how MinIO can be deployed on Kubernetes and with other technologies.

Cloudoc supermicro mini_svr_appliance_Engsang yoo

Cloudoc Supermicro Mini Server Appliance from NetID. Basic Functions - Document management & Document collaboration - SW installation files management - Community function - Windows Explorer UI support Mobile Support - Provide iOS, Android, Smart phone UI - Device registration, location tracking, remote deletion - Viewing Personal / Department / Public Documents - Internal connection via external Proxy server Disk Lock - Ransomware blocking function - Prohibit local saving - Application-specific management - Export procedure Disk Lock Plus - Prohibit screen capture - Prohibit print - Prohibit print log info - Prohibit Clipboard Copy - Print watermark

Open stack @ iiit hyderabad openstackindia

This document summarizes the work done by OpenStack@IIIT-H, which uses OpenStack to run an Indian language search engine, conduct research in information extraction/retrieval/access and virtualization/cloud computing, and provide users with OpenStack, Hadoop, and other open-source software. Before OpenStack, provisioning resources was ad hoc and unmanaged, user management lacked controls, and storage had reliability and duplication issues. After implementing OpenStack, resources can be quickly and reliably provisioned on demand, usage is monitored and restricted with quotas, and storage uses Swift for reliability without data fragmentation. OpenStack also supports research and teaching over 350 students through hands-on projects.

WiredTiger OverviewWiredTiger

WiredTiger is a new open source database engine designed for modern hardware and big data workloads. It provides high performance, low latency access to data stored either in RAM or on disk through its row-store, column-store, and log-structured merge tree storage engines. WiredTiger supports ACID transactions, standard isolation levels, and flexible storage and configuration options to optimize for different workloads and data access patterns. Initial benchmarks show WiredTiger provides up to 50% cost savings compared to other databases for the same workload.

MongoDB NoSQL DBMSVlad Vega

This document provides an overview of NoSQL document stores. It discusses the structure of document stores, including collections, indexing, and map reduce functions. It also describes the distributed process architecture using shards, chunks, replication for failover, and a configuration server. Several use cases are highlighted, including archiving, e-commerce, healthcare, and science applications like particle physics research.

MongoDB IoT City Tour STUTTGART: Analysing the Internet of Things. By, PentahoMongoDB

Dominik Claßen, Sales Engineering Team Laed at Pentaho Drawing on Pentaho's wide experience in solving customers' big data issues, Dominik positions the importance of analytics in the IoT. [-] Understanding the challenges behind data integration & analytics for IoT [-] Future proofing your information architecture for IoT [-] Delivering IoT analytics, now and tomorrow [-] Real customer examples of where Pentaho can help

IOT Paris Seminar 2015 - MAXXING PresentationMongoDB

Cas client Maxxing : Utilisation de MongoDB dans un contexte Retail Transformation du business model : comment passer d’une architecture magasin à une architecture qui intègre le parcours clients connecté en temps réel sur l’ensemble des canaux ? Face à la multitude de données à collecter issues des nouveaux comportements clients hyper-connectés, comment MongoDB s’interface avec la solution Maxxing pour réinjecter en temps réel les données qualifiées ? Le choix MongoDB vs Postgresql (free-schema and high availability)

IoT – The Machine Guys Meet the Internet GuysMongoDB

More Related Content

What's hot (19)

Quick overview on mongo dbEman Mohamed

Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...Miguel Pérez Colino

Going Serverless Schezarnie Racip

Cassandra meetup slides - Oct 15 Santa Monica ColoftJon Haddad

Fluentd Intro for OpenShift Commons BriefingEduardo Silva Pereira

Node.js by Alex and NalinTagged Social

Containers and LoggingEduardo Silva Pereira

HNSciCloud Info Day, 7 Sept 2016, Functional Requirements by Helge MeinhardHelix Nebula The Science Cloud

Eagle6 Enterprise Situational AwarenessMongoDB

Zentrales logging mit dem Elastic StackSimonSchneider24

Timur Shemsedinov "Эволюция архитектуры ИС"OdessaJS Conf

Alexander Sibiryakov- FronteraPyData

Http BasicsMushfekur Rahman

Draft slide of Demystifying DHT in GlusterFSAnkit Raj

MinIO January 2020 BriefingJonathan Symonds

Cloudoc supermicro mini_svr_appliance_Engsang yoo

Open stack @ iiit hyderabad openstackindia

WiredTiger OverviewWiredTiger

MongoDB NoSQL DBMSVlad Vega

Quick overview on mongo dbEman Mohamed

Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...Miguel Pérez Colino

Going Serverless Schezarnie Racip

Cassandra meetup slides - Oct 15 Santa Monica ColoftJon Haddad

Fluentd Intro for OpenShift Commons BriefingEduardo Silva Pereira

Node.js by Alex and NalinTagged Social

Containers and LoggingEduardo Silva Pereira

HNSciCloud Info Day, 7 Sept 2016, Functional Requirements by Helge MeinhardHelix Nebula The Science Cloud

Eagle6 Enterprise Situational AwarenessMongoDB

Zentrales logging mit dem Elastic StackSimonSchneider24

Timur Shemsedinov "Эволюция архитектуры ИС"OdessaJS Conf

Alexander Sibiryakov- FronteraPyData

Http BasicsMushfekur Rahman

Draft slide of Demystifying DHT in GlusterFSAnkit Raj

MinIO January 2020 BriefingJonathan Symonds

Cloudoc supermicro mini_svr_appliance_Engsang yoo

Open stack @ iiit hyderabad openstackindia

WiredTiger OverviewWiredTiger

MongoDB NoSQL DBMSVlad Vega

Viewers also liked (20)

MongoDB IoT City Tour STUTTGART: Analysing the Internet of Things. By, PentahoMongoDB

IOT Paris Seminar 2015 - MAXXING PresentationMongoDB

IoT – The Machine Guys Meet the Internet GuysMongoDB

MongoDB IoT CITY Tour STUTTGART: The IoT Market Landscape, Machina ResearchMongoDB

MongoDB IoT City Tour STUTTGART: Industry 4.0 and the Internet of Things: Inm...MongoDB

The document discusses using MongoDB as a universal datastore for industrial Internet of Things applications. It presents the benefits of MongoDB, including its flexibility to store different data types in the same field, its scalability for high-frequency industrial data, and its simplicity as a schema-less database that can interconnect data from various sources without restrictions. MongoDB is seen as a perfect fit for industrial big data as it can handle the performance, storage volumes, and variant data structures required for real-time IoT applications in manufacturing.

IOT Paris Seminar 2015 - Storage Challenges in IOTMongoDB

Joe Drumgoole gave a presentation on the Internet of Things storage challenges. The number of sensors deployed is rapidly increasing from billions to trillions. These sensors generate massive amounts of real-time data that needs to be stored, filtered, and distributed. Relational databases are not well-suited for this type of unstructured IoT data. MongoDB is presented as a solution because it offers dynamic schemas, automatic scaling, text search, and other features that address the demands of storing and analyzing massive amounts of real-time IoT data.

IOT Seminar Paris 2015 - AXA France PresentationMongoDB

Intégration de MongoDB dans le projet IOT d’AXA France: Dans le cadre de la mise en œuvre de sa stratégie sur les objets connectés et suite à la réalisation d’un premier POC en 2014, AXA France souhaite proposer rapidement à ses clients des premiers services exploitant les objets connectés pour prévenir et réduire le risque ou les assister en cas d’incident. Lors de ce retour d’expérience, vous apprendrez pourquoi et comment la DSI AXA France a décidé d’utiliser MongoDB pour assurer la persistance des données de son API de pilotage des objets connectés.

MongoDB IoT City Tour STUTTGART: Why your Dad's database won't work for IoT. ...MongoDB

Dmc cuscogchaman

Cusco is considered the belly button of the world by the Incas. It contains many important Inca archaeological sites, such as Sacsayhuaman, Qenko, Tipon, Pisac, Ollantaytambo, Machu Picchu, and Choquequirao. It is also home to beautiful colonial architecture like the Cathedral and churches of San Blas and Santo Domingo. Cusco was the historic capital of the Inca Empire and remains an important cultural center of Peru.

MongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, ClouderaMongoDB

Bernard Doering, Senior Slaes Director DACH, Cloudera. Hadoop and the Future of Data Management. As Hadoop takes the data management market by storm, organisations are evolving the role it plays in the modern data centre. Explore how this disruptive technology is quickly transforming an industry and how you can leverage it today, in combination with MongoDB, to drive meaningful change in your business.

Why Your Dad’s Database Won’t Work for IoTSam_Francis

This document discusses how relational databases are not suitable for IoT applications due to assumptions they make that do not align with IoT realities. Relational databases assume expensive storage, cheap programmers, static schemas and requirements, and specific programming languages which do not apply in an IoT context where data is dynamic, distributed and generated by many low-cost sensors. Instead, the document recommends a database like MongoDB that supports dynamic schemas, automatic scaling, text search, aggregation, replication and other features needed to handle IoT data characteristics and requirements.

MongoDB IoT CITY Tour LONDON: How M2M and IoT are changing the playing field ...MongoDB

IOT Paris Seminar 2015 - Enabling Transformation to the IOTMongoDB

Jim Duffy, Director of Enterprise Architecture at MongoDB, presented on how MongoDB enables transformation to the Internet of Things (IoT). He discussed how the IoT market is set for massive growth, with device shipments reaching 6.7 billion and $1.7 trillion added to the global economy by 2019. However, the challenges of constant data changes, security, and scaling pose barriers. MongoDB addresses these challenges through its flexible document model, ability to scale out data distribution, and providing in-database analytics and security features. Customer testimonies affirmed how MongoDB supports IoT use cases that require agility, flexibility, and handling big data.

IOT Paris Seminar 2015 - intro by Yann AubryMongoDB

The document discusses an event about using MongoDB for Internet of Things (IoT) applications. It notes that with 40 billion sensors and $19 trillion in potential revenue, larger databases will be needed to handle the big data generated by IoT. The event agenda outlines presentations on how MongoDB allows for flexible schemas, horizontal scaling, and a query language to enable organizations like Axa and Orange to build IoT solutions. Adoption of MongoDB is growing among Fortune 500 companies for applications like real-time analytics, mobile, and content management that require managing high volumes and varieties of dynamic data.

Internet of Things Cologne 2015: Rethinking Global Real-Time Data Integration...MongoDB

The document discusses rethinking global real-time data integration. It covers topics like the Internet of Things, Industry 4.0, big data, fast data, and the convergence of operational technology (OT) and information technology (IT). The presentation argues that companies need to remove data silos, adopt new technologies like OPC Unified Architecture, merge OT and IT departments, and get ready for Industry 4.0 in order to make better and faster decisions through increasing the reach and speed of information sharing. It then demonstrates the enterprise:inmation software, which is described as a distributed, real-time in-memory database that can serve as a true corporate middleware to support business and operations.

IOT Paris Seminar 2015 - Connected Objects makers, How to deal with Data?MongoDB

Connected Objects makers, How to deal with Data? Whether you are a connected objects manufacturer or a service provider, Datavenue can collect, store, secure all your connected objects data and allows service providers to benefit from the access to these data and mix them in order to develop new services. This is the basis of tomorrow’s services. All this is possible taking into account the 3 major Orange prerequisites that are conformance to the user’s privacy, data security and data empowerment. Datavenue is an open innovation platform where Internet of Things meets data services. Datavenue offers a range of solutions among which: [-] A scalable and secured solution to collect and store connected objects data addressing IoT manufacturers and developers to help them build their new services [-] A privacy respectful secured service to support consumers in managing their connected objects and data [-] A data powered ecosystem to foster the development of innovative services based on data cross fertilization [-] With Datavenue connect your objects to the next digital era and give sense to data.

Internet of Things Cologne 2015: The Contribution of New Data Storage and Ana...MongoDB

The document summarizes a presentation on how new data storage and analytics strategies can contribute to smart manufacturing and economic growth. It discusses how industrial data constitutes big data and new approaches are needed to handle the velocity, volume, and variety of this data. Predictive analytics and text analytics on structured and unstructured data from IoT sensors and devices can provide insights. Case studies show benefits from asset monitoring, production optimization, and quality improvements. The presentation argues manufacturers should target radical efficiency gains through innovation and establish goals and roadmaps to realize benefits.

MongoDB and the Internet of ThingsMongoDB

Hype, buzzword, threat; however you want to characterize it, the Internet of Things (IoT) is here. IoT scenarios that were hypothetical only a few years ago are real today. Still thinking along the line of fleet management and temperature measurements? You’re out. Endless possibilities of IoT applications are surfacing every day, from the connected cow (huh?) to things that monitor and analyze your daily life (really?). In this webinar, we will discuss architecture of IoT data management solutions and the challenges that arise. We will explore how MongoDB features provide solutions to those problems. Time permitting, we will demonstrate an IoT Cloud service built on top of MongoDB.

MongoDB Solution for Internet of Things and Big DataStefano Dindo

Internet of Things è uno degli scenari di mercato più importanti su cui investire entro il 2020. L'Internet of Things permette di trasferire sul Web la vita reale delle persone grazie all'interazione con oggetti e spazi fisici scambiando un grande volume di dati. Durante il Lab è stata fornita una descrizione di architettura necessaria a supportare progetti di Internet of Things con un focus sull'organizzazione dei dati all'interno di MongoDB, database NoSQL Leader di mercato, per raccogliere ed analizzare grandi volumi di dati in tempo reale ed in modo efficiente.

MongoDB IoT City Tour LONDON: Why your Dad's database won't work for IoT. Joe...MongoDB

This document discusses why relational databases are not suitable for Internet of Things applications and proposes MongoDB as a better alternative. The key points are: 1) Relational databases make assumptions that do not apply to IoT such as expensive storage, centralized architecture, and static schemas, which IoT applications require distributed, flexible schemas to handle massive amounts of real-time sensor data. 2) MongoDB is a more suitable database as it uses a document model with dynamic schemas, auto-sharding for horizontal scalability, text search, aggregation capabilities, and replication for high availability, which are better suited to the demands of storing, filtering, and distributing millions of events per minute from IoT devices. 3) MongoDB

MongoDB IoT City Tour STUTTGART: Analysing the Internet of Things. By, PentahoMongoDB

IOT Paris Seminar 2015 - MAXXING PresentationMongoDB

IoT – The Machine Guys Meet the Internet GuysMongoDB

MongoDB IoT CITY Tour STUTTGART: The IoT Market Landscape, Machina ResearchMongoDB

MongoDB IoT City Tour STUTTGART: Industry 4.0 and the Internet of Things: Inm...MongoDB

IOT Paris Seminar 2015 - Storage Challenges in IOTMongoDB

IOT Seminar Paris 2015 - AXA France PresentationMongoDB

MongoDB IoT City Tour STUTTGART: Why your Dad's database won't work for IoT. ...MongoDB

Dmc cuscogchaman

MongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, ClouderaMongoDB

Why Your Dad’s Database Won’t Work for IoTSam_Francis

MongoDB IoT CITY Tour LONDON: How M2M and IoT are changing the playing field ...MongoDB

IOT Paris Seminar 2015 - Enabling Transformation to the IOTMongoDB

IOT Paris Seminar 2015 - intro by Yann AubryMongoDB

Internet of Things Cologne 2015: Rethinking Global Real-Time Data Integration...MongoDB

IOT Paris Seminar 2015 - Connected Objects makers, How to deal with Data?MongoDB

Internet of Things Cologne 2015: The Contribution of New Data Storage and Ana...MongoDB

MongoDB and the Internet of ThingsMongoDB

MongoDB Solution for Internet of Things and Big DataStefano Dindo

MongoDB IoT City Tour LONDON: Why your Dad's database won't work for IoT. Joe...MongoDB

Similar to MongoDB IoT City Tour STUTTGART: Managing the Database Complexity, by Arthur Viegers (20)

MongoDB IoT City Tour LONDON: Managing the Database Complexity, by Arthur Vie...MongoDB

Codemotion Milano 2014 - MongoDB and the Internet of ThingsMassimo Brignoli

Webinar: Best Practices for Getting Started with MongoDBMongoDB

MongoDB adoption continues to grow at a record pace due to the significant enhancements in developer productivity and scalability that the database provides. Occasionally, however, organizations new to the technology make mistakes that limit their ability to leverage the significant advantages MongoDB provides. This webinar will discuss some of the common mistakes made by users when they first start working with MongoDB, how to identify when you've made those mistakes, and how to resolve them.

MongoDB Best PracticesLewis Lin 🦊

This document provides MongoDB best practices for schema design and data modeling. It discusses when to embed documents versus reference them, and how that impacts performance. It also uses an example of medical device data to illustrate how aggregating data at regular intervals (e.g. hourly instead of per minute) can significantly reduce storage requirements and improve query performance. Proper schema design is important for determining the required hardware resources.

MongoDB and the Internet of ThingsSam_Francis

This document discusses using MongoDB for Internet of Things (IoT) applications. It notes that IoT requires flexibility as there are billions of heterogeneous objects interacting in unpredictable ways, making a rigid relational database a poor fit. MongoDB is presented as a good solution with its document model instead of fixed tables. Time series data, which is common in IoT, can be modeled effectively in MongoDB at different granularities like per event, per minute, or per hour. Scaling in MongoDB can be done both vertically by increasing hardware and horizontally by sharding data across servers. The document concludes that MongoDB is well-suited for IoT due to its real-time capabilities, flexibility to handle diverse data formats, support for time series data

MongoDB and the Internet of ThingsMongoDB

- MongoDB is well-suited for IoT applications due to its ability to handle large volumes of variable data from sensors, perform analytics on both real-time and historical data, and scale horizontally to support growing workloads. - Its flexible document model accommodates changing sensor schemas and nested/complex data structures from devices, while secondary indexes enable expressive queries. - Time series data from sensors can be optimized in MongoDB using bucketing which improves write performance, storage usage, and analytics capabilities.

MongoDB for Time Series DataMongoDB

Imagine that self-driving cars now exist and are becoming widespread around the world. To facilitate the transition, it's necessary to set up central service to monitor traffic conditions nationwide, deploy sensors throughout the interstate system that monitor traffic conditions including car speeds, pavement and weather conditions, as well as accidents, construction, and other sources of traffic tie ups. MongoDB has been selected as the database for this application. In this webinar, we will walk through designing the application’s schema that will both support the high update and read volumes as well as the data aggregation and analytics queries.

Io t world_2016_iot_smart_gateways_moeShawn Moe

Agility and Scalability with MongoDBMongoDB

MongoDB has taken a clear lead in adoption among the new generation of databases, including the enormous variety of NoSQL offerings. A key reason for this lead has been a unique combination of agility and scalability. Agility provides business units with a quick start and flexibility to maintain development velocity, despite changing data and requirements. Scalability maintains that flexibility while providing fast, interactive performance as data volume and usage increase. We'll address the key organizational, operational, and engineering considerations to ensure that agility and scalability stay aligned at increasing scale, from small development instances to web-scale applications. We will also survey some key examples of highly-scaled customer applications of MongoDB.

Webinar: MongoDB Use Cases within the Oil, Gas, and Energy IndustriesMongoDB

In this session we will dive into some of the use-cases companies are currently deploying MongoDB for in the energy space. It is becoming more important for companies to make data driven decisions, and MongoDB can often be the right tool for analyzing the massive amounts of data coming in. Whether tracking oil well site statistics, power meter data, or feeds from sensors, MongoDB can be a great fit for tracking and analyzing that data, using it to make smart, informed business decisions.

Why Gateways are Important in Your IoT ArchitectureIBM Analytics

MongoDB Tick Data PresentationMongoDB

The document discusses using MongoDB as a tick store for financial data. It provides an overview of MongoDB and its benefits for handling tick data, including its flexible data model, rich querying capabilities, native aggregation framework, ability to do pre-aggregation for continuous data snapshots, language drivers and Hadoop connector. It also presents a case study of AHL, a quantitative hedge fund, using MongoDB and Python as their market data platform to easily onboard large volumes of financial data in different formats and provide low-latency access for backtesting and research applications.

IBM IoT Architecture and Capabilities at the Edge and Cloud Pradeep Natarajan

IBM Informix is presented as the ideal database solution for IoT architectures due to its small footprint, low memory requirements, support for time series and spatial data, and driverless operation requiring no administration. It can run on gateways to filter and analyze sensor data locally before transmitting to the cloud. In the cloud, Informix can ingest streaming data in real-time, perform operational analytics, and scale out across servers. Benchmarks show Informix outperforming SQLite for IoT workloads in areas like data loading speed, storage requirements, and analytic query speeds.

MongoDB .local Houston 2019: Best Practices for Working with IoT and Time-ser...MongoDB

Time series data is increasingly at the heart of modern applications - think IoT, stock trading, clickstreams, social media, and more. With the move from batch to real time systems, the efficient capture and analysis of time series data can enable organizations to better detect and respond to events ahead of their competitors or to improve operational efficiency to reduce cost and risk. Working with time series data is often different from regular application data, and there are best practices you should observe. This talk covers: Common components of an IoT solution The challenges involved with managing time-series data in IoT applications Different schema designs, and how these affect memory and disk utilization – two critical factors in application performance. How to query, analyze and present IoT time-series data using MongoDB Compass and MongoDB Charts At the end of the session, you will have a better understanding of key best practices in managing IoT time-series data with MongoDB.

Ibm_IoT_Architecture_and_CapabilitiesIBM_Info_Management

This document provides an overview of IBM's Internet of Things architecture and capabilities. It discusses how IBM's Informix database can be used in intelligent gateways and the cloud for IoT solutions. Specifically, it outlines how Informix is well-suited for gateway and cloud environments due to its small footprint, support for time series and spatial data, and ability to handle both structured and unstructured data. The document also provides examples of how Informix can be used with Node-RED and Docker to develop IoT applications and deploy databases in the cloud.

MongoDB.pdfKuldeepKumar778733

This document provides a technical overview of MongoDB presented by Learnizo Global LLP. It begins with an introduction and agenda. It then discusses key aspects of MongoDB like its document-oriented data model, flexible schema, and horizontal scalability. It covers MongoDB's core functions like CRUD operations, indexing, aggregation, and deployment options like replica sets and sharding. It also introduces key MongoDB concepts like its support for dynamic schemas, embedded documents, and ACID versus BASE properties. Finally, it demonstrates sample queries and uses MongoDB to highlight its capabilities.

Sizing Your MongoDB ClusterMongoDB

Jay Runkel presented a methodology for sizing MongoDB clusters to meet the requirements of an application. The key steps are: 1) Analyze data size and index size, 2) Estimate the working set based on frequently accessed data, 3) Use a simplified model to estimate IOPS and adjust for real-world factors, 4) Calculate the number of shards needed based on storage, memory and IOPS requirements. He demonstrated this process for an application that collects mobile events, requiring a cluster that can store over 200 billion documents with 50,000 IOPS.

High Performance, Scalable MongoDB in a Bare Metal CloudMongoDB

This document summarizes the results of performance testing MongoDB deployments on bare metal cloud instances compared to public cloud instances. Small, medium, and large tests were conducted using different hardware configurations and data set sizes. The bare metal cloud instances consistently outperformed the public cloud instances, achieving higher operations per second, especially at higher concurrency levels. The document attributes the performance differences to the dedicated, tuned hardware resources of the bare metal instances compared to the shared resources of public cloud virtual instances.

MongoDB .local London 2019: Best Practices for Working with IoT and Time-seri...MongoDB

Time series data is increasingly at the heart of modern applications - think IoT, stock trading, clickstreams, social media, and more. With the move from batch to real time systems, the efficient capture and analysis of time series data can enable organizations to better detect and respond to events ahead of their competitors or to improve operational efficiency to reduce cost and risk. Working with time series data is often different from regular application data, and there are best practices you should observe. This talk covers: Common components of an IoT solution • The challenges involved with managing time-series data in IoT applications • Different schema designs, and how these affect memory and disk utilization – two critical factors in application performance. • How to query, analyze and present IoT time-series data using MongoDB Compass and MongoDB Charts • At the end of the session, you will have a better understanding of key best practices in managing IoT time-series data with MongoDB.

MongoDB World 2018: Building a New Transactional ModelMongoDB

The document discusses MongoDB's development of ACID transactions over multiple releases. Key points include: - MongoDB added initial transaction support in 2.6 using MMAPv1 storage engine and document-level locking. - Version 3.0 introduced a new storage engine, WiredTiger, that supports multi-version concurrency control. - Later versions added features like causal consistency, prepared transactions, and secondary reads to enhance consistency. - MongoDB 4.0 introduced multi-document transactions across replica sets using the storage engine's prepared transaction support. - Future work may integrate oplog and journaling further and use timestamps for cross-shard transactions.

MongoDB IoT City Tour LONDON: Managing the Database Complexity, by Arthur Vie...MongoDB

Codemotion Milano 2014 - MongoDB and the Internet of ThingsMassimo Brignoli

Webinar: Best Practices for Getting Started with MongoDBMongoDB

MongoDB Best PracticesLewis Lin 🦊

MongoDB and the Internet of ThingsSam_Francis

MongoDB and the Internet of ThingsMongoDB

MongoDB for Time Series DataMongoDB

Io t world_2016_iot_smart_gateways_moeShawn Moe

Agility and Scalability with MongoDBMongoDB

Webinar: MongoDB Use Cases within the Oil, Gas, and Energy IndustriesMongoDB

Why Gateways are Important in Your IoT ArchitectureIBM Analytics

MongoDB Tick Data PresentationMongoDB

IBM IoT Architecture and Capabilities at the Edge and Cloud Pradeep Natarajan

MongoDB .local Houston 2019: Best Practices for Working with IoT and Time-ser...MongoDB

Ibm_IoT_Architecture_and_CapabilitiesIBM_Info_Management

MongoDB.pdfKuldeepKumar778733

Sizing Your MongoDB ClusterMongoDB

High Performance, Scalable MongoDB in a Bare Metal CloudMongoDB

MongoDB .local London 2019: Best Practices for Working with IoT and Time-seri...MongoDB

MongoDB World 2018: Building a New Transactional ModelMongoDB

More from MongoDB (20)

MongoDB SoCal 2020: Migrate Anything* to MongoDB AtlasMongoDB

This presentation discusses migrating data from other data stores to MongoDB Atlas. It begins by explaining why MongoDB and Atlas are good choices for data management. Several preparation steps are covered, including sizing the target Atlas cluster, increasing the source oplog, and testing connectivity. Live migration, mongomirror, and dump/restore options are presented for migrating between replicasets or sharded clusters. Post-migration steps like monitoring and backups are also discussed. Finally, migrating from other data stores like AWS DocumentDB, Azure CosmosDB, DynamoDB, and relational databases are briefly covered.

MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!MongoDB

MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...MongoDB

MongoDB Kubernetes operator and MongoDB Open Service Broker are ready for production operations. Learn about how MongoDB can be used with the most popular container orchestration platform, Kubernetes, and bring self-service, persistent storage to your containerized applications. A demo will show you how easy it is to enable MongoDB clusters as an External Service using the Open Service Broker API for MongoDB

MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDBMongoDB

MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...MongoDB

MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series DataMongoDB

MongoDB SoCal 2020: MongoDB Atlas Jump StartMongoDB

MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]MongoDB

Our clients have unique use cases and data patterns that mandate the choice of a particular strategy. To implement these strategies, it is mandatory that we unlearn a lot of relational concepts while designing and rapidly developing efficient applications on NoSQL. In this session, we will talk about some of our client use cases, the strategies we have adopted, and the features of MongoDB that assisted in implementing these strategies.

MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2MongoDB

Encryption is not a new concept to MongoDB. Encryption may occur in-transit (with TLS) and at-rest (with the encrypted storage engine). But MongoDB 4.2 introduces support for Client Side Encryption, ensuring the most sensitive data is encrypted before ever leaving the client application. Even full access to your MongoDB servers is not enough to decrypt this data. And better yet, Client Side Encryption can be enabled at the "flick of a switch". This session covers using Client Side Encryption in your applications. This includes the necessary setup, how to encrypt data without sacrificing queryability, and what trade-offs to expect.

MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...MongoDB

MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!MongoDB

MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your MindsetMongoDB

When you need to model data, is your first instinct to start breaking it down into rows and columns? Mine used to be too. When you want to develop apps in a modern, agile way, NoSQL databases can be the best option. Come to this talk to learn how to take advantage of all that NoSQL databases have to offer and discover the benefits of changing your mindset from the legacy, tabular way of modeling data. We’ll compare and contrast the terms and concepts in SQL databases and MongoDB, explain the benefits of using MongoDB compared to SQL databases, and walk through data modeling basics so you feel confident as you begin using MongoDB.

MongoDB .local San Francisco 2020: MongoDB Atlas JumpstartMongoDB

MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...MongoDB

The document discusses guidelines for ordering fields in compound indexes to optimize query performance. It recommends the E-S-R approach: placing equality fields first, followed by sort fields, and range fields last. This allows indexes to leverage equality matches, provide non-blocking sorts, and minimize scanning. Examples show how indexes ordered by these guidelines can support queries more efficiently by narrowing the search bounds.

MongoDB .local San Francisco 2020: Aggregation Pipeline Power++MongoDB

MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...MongoDB

The document describes a methodology for data modeling with MongoDB. It begins by recognizing the differences between document and tabular databases, then outlines a three step methodology: 1) describe the workload by listing queries, 2) identify and model relationships between entities, and 3) apply relevant patterns when modeling for MongoDB. The document uses examples around modeling a coffee shop franchise to illustrate modeling approaches and techniques.

MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep DiveMongoDB

MongoDB Atlas Data Lake is a new service offered by MongoDB Atlas. Many organizations store long term, archival data in cost-effective storage like S3, GCP, and Azure Blobs. However, many of them do not have robust systems or tools to effectively utilize large amounts of data to inform decision making. MongoDB Atlas Data Lake is a service allowing organizations to analyze their long-term data to discover a wealth of information about their business. This session will take a deep dive into the features that are currently available in MongoDB Atlas Data Lake and how they are implemented. In addition, we'll discuss future plans and opportunities and offer ample Q&A time with the engineers on the project.

MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & GolangMongoDB

Virtual assistants are becoming the new norm when it comes to daily life, with Amazon’s Alexa being the leader in the space. As a developer, not only do you need to make web and mobile compliant applications, but you need to be able to support virtual assistants like Alexa. However, the process isn’t quite the same between the platforms. How do you handle requests? Where do you store your data and work with it to create meaningful responses with little delay? How much of your code needs to change between platforms? In this session we’ll see how to design and develop applications known as Skills for Amazon Alexa powered devices using the Go programming language and MongoDB.

MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...MongoDB

MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...MongoDB

Il n’a jamais été aussi facile de commander en ligne et de se faire livrer en moins de 48h très souvent gratuitement. Cette simplicité d’usage cache un marché complexe de plus de 8000 milliards de $. La data est bien connu du monde de la Supply Chain (itinéraires, informations sur les marchandises, douanes,…), mais la valeur de ces données opérationnelles reste peu exploitée. En alliant expertise métier et Data Science, Upply redéfinit les fondamentaux de la Supply Chain en proposant à chacun des acteurs de surmonter la volatilité et l’inefficacité du marché.