SlideShare a Scribd company logo

Building your first java application with MongoDB

Download as PPTX, PDF
MongoDB
MongoDB

This document provides a detailed overview of building a location-based surf reporting application using MongoDB, highlighting its high performance, availability, and scalability. It outlines the data model, including the concepts of documents, collections, and replica sets, and also details the implementation of queries to retrieve surf conditions. Additionally, it discusses sharding for scaling and features like the aggregation framework for analyzing surf data.

TechnologyNews & Politics
1 of 40
Downloaded 23 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Building Your First Application in Java Bryan Reinero Bryan.reinero@10gen.com September 2012 1
 High performance  Highly available  Easily scalable  Easy to use  Feature rich Document store ©2012 Jaspersoft Corporation. Proprietary and Confidential 2
Data Model  A Mongo system holds a set of databases  A database holds a set of collections  A collection holds a set of documents  A document is a set of fields  A field is a key-value pair  A key is a name (string)  A value is a basic type like string, integer, float, timestamp, binary, etc., a document, or an array of values ©2012 Jaspersoft Corporation. Proprietary and Confidential 3
High Availability: Replica Sets  Initialize -> Election  Primary + data replication from primary to secondary Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary Replication Replication ©2012 Jaspersoft Corporation. Proprietary and Confidential 4
High Availability: Failure  Primary down/network failure  Automatic election of new primary if majority exists Primary Election Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary ©2012 Jaspersoft Corporation. Proprietary and Confidential 5
High Availability: Failover  New primary elected  Replication established from new primary Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary ©2012 Jaspersoft Corporation. Proprietary and Confidential 6
Durability  Fire and forget  Wait for error  Wait for journal sync  Wait for fsync  Wait for replication ©2012 Jaspersoft Corporation. Proprietary and Confidential 7
Read Preferences  PRIMARY  PRIMARY PREFERRED  SECONDARY  SECONDARY PREFERRED  NEAREST ©2012 Jaspersoft Corporation. Proprietary and Confidential 8
Let’s build a location based surf reporting app! ©2012 Jaspersoft Corporation. Proprietary and Confidential 9
Let’s build a location based surf reporting app! • Report current conditions
Let’s build a location based surf reporting app! • Report current conditions • Get current local conditions
Let’s build a location based surf reporting app! • Report current conditions • Get current local conditions • Determine best conditions per beach
Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), "reporter" : "test", "location" : { "coordinates" : [ -122.477222, 37.810556 ], "name" : "Fort Point" }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Auto-indexed -122.477222, 37.810556 ], "name" : "Fort Point" }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Autoindexed -122.477222, 37.810556 ], Compound Index, "name" : "Fort Point" Geospacial }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Autoindexed -122.477222, 37.810556 ], Compound Index, "name" : "Fort Point" Geospacial }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, Indexed for "date" : ISODate("2011-11-16T20:17:17.277Z") Time-To-Live }
Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1})
Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports
Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports
Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports • Return only the relevant info
Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports • Return only the relevant info • Show me the best surf first
Results { "location" : { "name" : "Montara" }, "conditions" : { "height" : 6, "period" : 20, "rating" : 5 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 5, "period" : 13, "rating" : 3 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 3, "period" : 15, "rating" : 3 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 3, "period" : 16, "rating" : 2 } } { "location" : { "name" : "Montara" }, "conditions" : { "height" : 0, "period" : 8, "rating" : 1 } } { "location" : { "name" : "Linda Mar" }, "conditions" : { "height" : 3, "period" : 10, "rating" : 1 } } { "location" : { "name" : "Sharp Park" }, "conditions" : { "height" : 1, "period" : 15, "rating" : 1 } } { "location" : { "name" : "Sharp Park" }, "conditions" : { "height" : 5, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 1, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 0, "period" : 10, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 4, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 0, "period" : 14, "rating" : 1 } }
Scaling  Sharding is the partitioning of data among multiple machines  Balancing occurs when the load on any one node grows out of proportion ©2012 Jaspersoft Corporation. Proprietary and Confidential 23
Scaling MongoDB Sharded cluster MongoDB Single Instance Or Replica Set Client Application
The Mechanism of Sharding Complete Data Set Define Shard Key on Location Name Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
The Mechanism of Sharding Chunk Chunk Define Shard Key on Location Name Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
The Mechanism of Sharding Chunk Chunk Chunk Chunk Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
The Mechanism of Sharding Chunk Chunk Chunk Chunk Fort Point Linda Mar Maverick’s Ocean BeachRockaway Shard 1 Shard 2 Shard 3 Shard 4
The Mechanism of Sharding Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 ©2012 Jaspersoft Corporation. Proprietary and Confidential 29
The Mechanism of Sharding Client Query: Linda Mar Application Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 ©2012 Jaspersoft Corporation. Proprietary and Confidential 30
The Mechanism of Sharding Client Query: Maverick’s Application Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 ©2012 Jaspersoft Corporation. Proprietary and Confidential 31
Analysis Features: Aggregation Framework What are the best conditions for my local beach?
Pipelining Operations $match Match “Linda Mar” $project Only interested in conditions $group Group by rating, averaging wave height and wave period $sort Order by best conditions
Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] }
Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Match “Linda Mar”
Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Only interested in conditions
Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Group by rating & average conditions
Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Show me best conditions first
Other Features…  Native MapReduce  Hadoop Connector  Tagging  Drivers for all major languages ©2012 Jaspersoft Corporation. Proprietary and Confidential 39
Thanks! Office Hours Thursdays 4-6 pm 555 University Ave. Palo Alto We’re Hiring ! Bryan.reinero@10gen.com

More Related Content

Viewers also liked (7)

PPTX
MongoDB Roadmap
MongoDB
 
PPT
Giftivo mongodb
MongoDB
 
PPT
A Morning with MongoDB - Helsinki
MongoDB
 
PPTX
Indexing and Query Optimisation
MongoDB
 
PPTX
Webinar: Replication and Replica Sets
MongoDB
 
PPTX
Branf final bringing mongodb into your organization - mongo db-boston2012
MongoDB
 
KEY
Discover MongoDB - Israel
Michael Fiedler
 
MongoDB Roadmap
MongoDB
 
Giftivo mongodb
MongoDB
 
A Morning with MongoDB - Helsinki
MongoDB
 
Indexing and Query Optimisation
MongoDB
 
Webinar: Replication and Replica Sets
MongoDB
 
Branf final bringing mongodb into your organization - mongo db-boston2012
MongoDB
 
Discover MongoDB - Israel
Michael Fiedler
 

Similar to Building your first java application with MongoDB (20)

PDF
Ensuring High Availability for Real-time Analytics featuring Boxed Ice / Serv...
MongoDB
 
PDF
Improving Operational Space Responsiveness
Pat Cappelaere
 
PPTX
Operational Intelligence with MongoDB Webinar
MongoDB
 
PPTX
MongoDB for Time Series Data Part 3: Sharding
MongoDB
 
PDF
Webinar: Building Your First App with MongoDB and Java
MongoDB
 
PPTX
High-Volume Data Collection and Real Time Analytics Using Redis
cacois
 
PPT
Meetup#1: 10 reasons to fall in love with MongoDB
Minsk MongoDB User Group
 
PDF
Getting started with Spark & Cassandra by Jon Haddad of Datastax
Data Con LA
 
PDF
10 Key MongoDB Performance Indicators
iammutex
 
PDF
Composable Data Processing with Apache Spark
Databricks
 
PDF
2013 london advanced-replication
Marc Schwering
 
PDF
Mongodb workshop
Harun Yardımcı
 
PDF
A Century Of Weather Data - Midwest.io
Randall Hunt
 
PPTX
Average- An android project
Ipsit Dash
 
PDF
HMS: Scalable Configuration Management System for Hadoop
DataWorks Summit
 
PDF
Spring Data MongoDB 介紹
Kuo-Chun Su
 
PDF
NoSQL Infrastructure
Server Density
 
PDF
Maintenance for MongoDB Replica Sets
Igor Donchovski
 
PPTX
Data-centric Invocable Services
Rick Warren
 
KEY
OpenStack Folsom Summit: Melange overview
troytoman
 
Ensuring High Availability for Real-time Analytics featuring Boxed Ice / Serv...
MongoDB
 
Improving Operational Space Responsiveness
Pat Cappelaere
 
Operational Intelligence with MongoDB Webinar
MongoDB
 
MongoDB for Time Series Data Part 3: Sharding
MongoDB
 
Webinar: Building Your First App with MongoDB and Java
MongoDB
 
High-Volume Data Collection and Real Time Analytics Using Redis
cacois
 
Meetup#1: 10 reasons to fall in love with MongoDB
Minsk MongoDB User Group
 
Getting started with Spark & Cassandra by Jon Haddad of Datastax
Data Con LA
 
10 Key MongoDB Performance Indicators
iammutex
 
Composable Data Processing with Apache Spark
Databricks
 
2013 london advanced-replication
Marc Schwering
 
Mongodb workshop
Harun Yardımcı
 
A Century Of Weather Data - Midwest.io
Randall Hunt
 
Average- An android project
Ipsit Dash
 
HMS: Scalable Configuration Management System for Hadoop
DataWorks Summit
 
Spring Data MongoDB 介紹
Kuo-Chun Su
 
NoSQL Infrastructure
Server Density
 
Maintenance for MongoDB Replica Sets
Igor Donchovski
 
Data-centric Invocable Services
Rick Warren
 
OpenStack Folsom Summit: Melange overview
troytoman
 
Ad

More from MongoDB (20)

PDF
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB
 
PDF
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
PDF
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB
 
PDF
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB
 
PDF
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB
 
PDF
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB
 
PDF
MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
PDF
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB
 
PDF
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB
 
PDF
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB
 
PDF
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB
 
PDF
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB
 
PDF
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB
 
PDF
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB
 
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB
 
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB
 
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB
 
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB
 
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB
 
MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB
 
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB
 
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB
 
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB
 
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB
 
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB
 
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB
 
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB
 
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB
 
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB
 
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB
 
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB
 
Ad

Recently uploaded (20)

PPTX
Webinar: Introduction to LF Energy EVerest
DanBrown980551
 
PDF
CIFDAQ Market Insights for July 7th 2025
CIFDAQ
 
PDF
Jak MŚP w Europie Środkowo-Wschodniej odnajdują się w świecie AI
dominikamizerska1
 
PPTX
COMPARISON OF RASTER ANALYSIS TOOLS OF QGIS AND ARCGIS
Sharanya Sarkar
 
PDF
"AI Transformation: Directions and Challenges", Pavlo Shaternik
Fwdays
 
PDF
Automating Feature Enrichment and Station Creation in Natural Gas Utility Net...
Safe Software
 
PPTX
AUTOMATION AND ROBOTICS IN PHARMA INDUSTRY.pptx
sameeraaabegumm
 
PDF
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
 
PDF
Bitcoin for Millennials podcast with Bram, Power Laws of Bitcoin
Stephen Perrenod
 
PDF
Achieving Consistent and Reliable AI Code Generation - Medusa AI
medusaaico
 
PDF
Peak of Data & AI Encore AI-Enhanced Workflows for the Real World
Safe Software
 
PPTX
Designing_the_Future_AI_Driven_Product_Experiences_Across_Devices.pptx
presentifyai
 
PDF
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
Edge AI and Vision Alliance
 
PPTX
MuleSoft MCP Support (Model Context Protocol) and Use Case Demo
shyamraj55
 
PDF
LOOPS in C Programming Language - Technology
RishabhDwivedi43
 
PDF
Transcript: New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
PPTX
The Project Compass - GDG on Campus MSIT
dscmsitkol
 
PPTX
Seamless Tech Experiences Showcasing Cross-Platform App Design.pptx
presentifyai
 
PDF
What’s my job again? Slides from Mark Simos talk at 2025 Tampa BSides
Mark Simos
 
PDF
New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
Webinar: Introduction to LF Energy EVerest
DanBrown980551
 
CIFDAQ Market Insights for July 7th 2025
CIFDAQ
 
Jak MŚP w Europie Środkowo-Wschodniej odnajdują się w świecie AI
dominikamizerska1
 
COMPARISON OF RASTER ANALYSIS TOOLS OF QGIS AND ARCGIS
Sharanya Sarkar
 
"AI Transformation: Directions and Challenges", Pavlo Shaternik
Fwdays
 
Automating Feature Enrichment and Station Creation in Natural Gas Utility Net...
Safe Software
 
AUTOMATION AND ROBOTICS IN PHARMA INDUSTRY.pptx
sameeraaabegumm
 
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
 
Bitcoin for Millennials podcast with Bram, Power Laws of Bitcoin
Stephen Perrenod
 
Achieving Consistent and Reliable AI Code Generation - Medusa AI
medusaaico
 
Peak of Data & AI Encore AI-Enhanced Workflows for the Real World
Safe Software
 
Designing_the_Future_AI_Driven_Product_Experiences_Across_Devices.pptx
presentifyai
 
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
Edge AI and Vision Alliance
 
MuleSoft MCP Support (Model Context Protocol) and Use Case Demo
shyamraj55
 
LOOPS in C Programming Language - Technology
RishabhDwivedi43
 
Transcript: New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
The Project Compass - GDG on Campus MSIT
dscmsitkol
 
Seamless Tech Experiences Showcasing Cross-Platform App Design.pptx
presentifyai
 
What’s my job again? Slides from Mark Simos talk at 2025 Tampa BSides
Mark Simos
 
New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 

Building your first java application with MongoDB

  • 1. Building Your First Application in Java Bryan Reinero [email protected] September 2012 1
  • 2. High performance  Highly available  Easily scalable  Easy to use  Feature rich Document store ©2012 Jaspersoft Corporation. Proprietary and Confidential 2
  • 3. Data Model  A Mongo system holds a set of databases  A database holds a set of collections  A collection holds a set of documents  A document is a set of fields  A field is a key-value pair  A key is a name (string)  A value is a basic type like string, integer, float, timestamp, binary, etc., a document, or an array of values ©2012 Jaspersoft Corporation. Proprietary and Confidential 3
  • 4. High Availability: Replica Sets  Initialize -> Election  Primary + data replication from primary to secondary Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary Replication Replication ©2012 Jaspersoft Corporation. Proprietary and Confidential 4
  • 5. High Availability: Failure  Primary down/network failure  Automatic election of new primary if majority exists Primary Election Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary ©2012 Jaspersoft Corporation. Proprietary and Confidential 5
  • 6. High Availability: Failover  New primary elected  Replication established from new primary Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary ©2012 Jaspersoft Corporation. Proprietary and Confidential 6
  • 7. Durability  Fire and forget  Wait for error  Wait for journal sync  Wait for fsync  Wait for replication ©2012 Jaspersoft Corporation. Proprietary and Confidential 7
  • 8. Read Preferences  PRIMARY  PRIMARY PREFERRED  SECONDARY  SECONDARY PREFERRED  NEAREST ©2012 Jaspersoft Corporation. Proprietary and Confidential 8
  • 9. Let’s build a location based surf reporting app! ©2012 Jaspersoft Corporation. Proprietary and Confidential 9
  • 10. Let’s build a location based surf reporting app! • Report current conditions
  • 11. Let’s build a location based surf reporting app! • Report current conditions • Get current local conditions
  • 12. Let’s build a location based surf reporting app! • Report current conditions • Get current local conditions • Determine best conditions per beach
  • 13. Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), "reporter" : "test", "location" : { "coordinates" : [ -122.477222, 37.810556 ], "name" : "Fort Point" }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
  • 14. Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Auto-indexed -122.477222, 37.810556 ], "name" : "Fort Point" }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
  • 15. Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Autoindexed -122.477222, 37.810556 ], Compound Index, "name" : "Fort Point" Geospacial }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
  • 16. Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Autoindexed -122.477222, 37.810556 ], Compound Index, "name" : "Fort Point" Geospacial }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, Indexed for "date" : ISODate("2011-11-16T20:17:17.277Z") Time-To-Live }
  • 17. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1})
  • 18. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports
  • 19. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports
  • 20. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports • Return only the relevant info
  • 21. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports • Return only the relevant info • Show me the best surf first
  • 22. Results { "location" : { "name" : "Montara" }, "conditions" : { "height" : 6, "period" : 20, "rating" : 5 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 5, "period" : 13, "rating" : 3 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 3, "period" : 15, "rating" : 3 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 3, "period" : 16, "rating" : 2 } } { "location" : { "name" : "Montara" }, "conditions" : { "height" : 0, "period" : 8, "rating" : 1 } } { "location" : { "name" : "Linda Mar" }, "conditions" : { "height" : 3, "period" : 10, "rating" : 1 } } { "location" : { "name" : "Sharp Park" }, "conditions" : { "height" : 1, "period" : 15, "rating" : 1 } } { "location" : { "name" : "Sharp Park" }, "conditions" : { "height" : 5, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 1, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 0, "period" : 10, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 4, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 0, "period" : 14, "rating" : 1 } }
  • 23. Scaling  Sharding is the partitioning of data among multiple machines  Balancing occurs when the load on any one node grows out of proportion ©2012 Jaspersoft Corporation. Proprietary and Confidential 23
  • 24. Scaling MongoDB Sharded cluster MongoDB Single Instance Or Replica Set Client Application
  • 25. The Mechanism of Sharding Complete Data Set Define Shard Key on Location Name Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
  • 26. The Mechanism of Sharding Chunk Chunk Define Shard Key on Location Name Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
  • 27. The Mechanism of Sharding Chunk Chunk Chunk Chunk Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
  • 28. The Mechanism of Sharding Chunk Chunk Chunk Chunk Fort Point Linda Mar Maverick’s Ocean BeachRockaway Shard 1 Shard 2 Shard 3 Shard 4
  • 29. The Mechanism of Sharding Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 ©2012 Jaspersoft Corporation. Proprietary and Confidential 29
  • 30. The Mechanism of Sharding Client Query: Linda Mar Application Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 ©2012 Jaspersoft Corporation. Proprietary and Confidential 30
  • 31. The Mechanism of Sharding Client Query: Maverick’s Application Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 ©2012 Jaspersoft Corporation. Proprietary and Confidential 31
  • 32. Analysis Features: Aggregation Framework What are the best conditions for my local beach?
  • 33. Pipelining Operations $match Match “Linda Mar” $project Only interested in conditions $group Group by rating, averaging wave height and wave period $sort Order by best conditions
  • 34. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] }
  • 35. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Match “Linda Mar”
  • 36. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Only interested in conditions
  • 37. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Group by rating & average conditions
  • 38. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Show me best conditions first
  • 39. Other Features…  Native MapReduce  Hadoop Connector  Tagging  Drivers for all major languages ©2012 Jaspersoft Corporation. Proprietary and Confidential 39
  • 40. Thanks! Office Hours Thursdays 4-6 pm 555 University Ave. Palo Alto We’re Hiring ! [email protected]