Building your first Java Application with MongoDB
Download as PPTX, PDF1 like1,285 views
The document provides a guide on building a location-based surf reporting app using Java and MongoDB, detailing data structures for reports, including weather conditions and location-specific information. It discusses essential features of MongoDB such as high availability with replica sets, failure handling, and the use of sharding for scaling the application. It also includes examples of database queries to retrieve and aggregate surf conditions based on specified criteria.
![Document Structure
{
"_id" : ObjectId("504ceb3d30042d707af96fef"),
"reporter" : "test",
"location" : {
"coordinates" : [
-122.477222,
37.810556
],
"name" : "Fort Point"
},
"conditions" : {
"height" : 0,
"period" : 9,
"rating" : 1
},
"date" : ISODate("2011-11-16T20:17:17.277Z")
}](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-13-320.jpg)
![Document Structure
{
"_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key,
"reporter" : "test",
Unique,
"location" : {
"coordinates" : [ Auto-indexed
-122.477222,
37.810556
],
"name" : "Fort Point"
},
"conditions" : {
"height" : 0,
"period" : 9,
"rating" : 1
},
"date" : ISODate("2011-11-16T20:17:17.277Z")
}](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-14-320.jpg)
![Document Structure
{
"_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key,
"reporter" : "test",
Unique,
"location" : {
"coordinates" : [ Autoindexed
-122.477222,
37.810556
], Compound Index,
"name" : "Fort Point" Geospacial
},
"conditions" : {
"height" : 0,
"period" : 9,
"rating" : 1
},
"date" : ISODate("2011-11-16T20:17:17.277Z")
}](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-15-320.jpg)
![Document Structure
{
"_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key,
"reporter" : "test",
Unique,
"location" : {
"coordinates" : [ Autoindexed
-122.477222,
37.810556
], Compound Index,
"name" : "Fort Point" Geospacial
},
"conditions" : {
"height" : 0,
"period" : 9,
"rating" : 1
}, Indexed for
"date" : ISODate("2011-11-16T20:17:17.277Z") Time-To-Live
}](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-16-320.jpg)
![Get local surf conditions
db.reports.find(
{
"location.coordinates" : { $near : [-122, 37] ,
$maxDistance : 0.9},
date : { $gte : new Date(2012, 8, 9)}
},
{"date" : 1, "location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-17-320.jpg)
![Get local surf conditions
db.reports.find(
{
"location.coordinates" : { $near : [-122, 37] ,
$maxDistance : 0.9},
date : { $gte : new Date(2012, 8, 9)}
},
{"date" : 1, "location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
• Get local reports](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-18-320.jpg)
![Get local surf conditions
db.reports.find(
{
"location.coordinates" : { $near : [-122, 37] ,
$maxDistance : 0.9},
date : { $gte : new Date(2012, 8, 9)}
},
{"date" : 1, "location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
• Get local reports
• Get today’s reports](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-19-320.jpg)
![Get local surf conditions
db.reports.find(
{
"location.coordinates" : { $near : [-122, 37] ,
$maxDistance : 0.9},
date : { $gte : new Date(2012, 8, 9)}
},
{"location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
• Get local reports
• Get today’s reports
• Return only the relevant info](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-20-320.jpg)
![Get local surf conditions
db.reports.find(
{
"location.coordinates" : { $near : [-122, 37] ,
$maxDistance : 0.9},
date : { $gte : new Date(2012, 8, 9)}
},
{"location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
• Get local reports
• Get today’s reports
• Return only the relevant info
• Show me the best surf first](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-21-320.jpg)
![Aggregation Framework
{ "aggregate" : "reports" ,
"pipeline" : [
{ "$match" : { "location.name" : "Linda Mar"}} ,
{ "$project" : { "conditions" : 1}} ,
{ "$group" : {
"_id" : "$conditions.rating" ,
"average height" : { "$avg" : "$conditions.height"} ,
"average period" : { "$avg" : "$conditions.period"}}} ,
{ "$sort" : { "_id" : -1}}
]
}](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-28-320.jpg)
![Aggregation Framework
{ "aggregate" : "reports" ,
"pipeline" : [
{ "$match" : { "location.name" : "Linda Mar"}} ,
{ "$project" : { "conditions" : 1}} ,
{ "$group" : {
"_id" : "$conditions.rating" ,
"average height" : { "$avg" : "$conditions.height"} ,
"average period" : { "$avg" : "$conditions.period"}}} ,
{ "$sort" : { "_id" : -1}}
]
}
Match “Linda Mar”](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-29-320.jpg)
![Aggregation Framework
{ "aggregate" : "reports" ,
"pipeline" : [
{ "$match" : { "location.name" : "Linda Mar"}} ,
{ "$project" : { "conditions" : 1}} ,
{ "$group" : {
"_id" : "$conditions.rating" ,
"average height" : { "$avg" : "$conditions.height"} ,
"average period" : { "$avg" : "$conditions.period"}}} ,
{ "$sort" : { "_id" : -1}}
]
}
Only interested in conditions](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-30-320.jpg)
![Aggregation Framework
{ "aggregate" : "reports" ,
"pipeline" : [
{ "$match" : { "location.name" : "Linda Mar"}} ,
{ "$project" : { "conditions" : 1}} ,
{ "$group" : {
"_id" : "$conditions.rating" ,
"average height" : { "$avg" : "$conditions.height"} ,
"average period" : { "$avg" : "$conditions.period"}}} ,
{ "$sort" : { "_id" : -1}}
]
}
Group by rating & average conditions](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-31-320.jpg)
![Aggregation Framework
{ "aggregate" : "reports" ,
"pipeline" : [
{ "$match" : { "location.name" : "Linda Mar"}} ,
{ "$project" : { "conditions" : 1}} ,
{ "$group" : {
"_id" : "$conditions.rating" ,
"average height" : { "$avg" : "$conditions.height"} ,
"average period" : { "$avg" : "$conditions.period"}}} ,
{ "$sort" : { "_id" : -1}}
]
}
Show me best conditions first](https://image.slidesharecdn.com/buildingyourfirstjavaapplicationwithmongodb-121018125422-phpapp01/85/Building-your-first-Java-Application-with-MongoDB-32-320.jpg)
Building your first Java Application with MongoDB
- 1. Building Your First Application in Java 1
- 2. is a… • High performance • Highly available • Easily scalable • Easy to use • Feature rich Document store
- 3. Data Model • A Mongo system holds a set of databases • A database holds a set of collections • A collection holds a set of documents • A document is a set of fields • A field is a key-value pair • A key is a name (string) • A value is a basic type like string, integer, float, timestamp, binary, etc., a document, or an array of values
- 4. High Availability: Replica Sets • Initialize -> Election • Primary + data replication from primary to secondary Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary Replication Replication
- 5. Replica Set - Failure • Primary down/network failure • Automatic election of new primary if majority exists Primary Election Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary
- 6. Replica Set - Failover • New primary elected • Replication established from new primary Node 1 Node 2 Secondary Heartbeat Primary Node 3 Primary
- 7. Durability • Fire and forget • Wait for error • Wait for journal sync • Wait for flush to disk • Wait for replication
- 8. Read Preferences • PRIMARY • PRIMARY PREFERRED • SECONDARY • SECONDARY PREFERRED • NEAREST
- 9. Let’s build a location based surf reporting app!
- 10. Let’s build a location based surf reporting app! • Report current conditions
- 11. Let’s build a location based surf reporting app! • Report current conditions • Get current local conditions
- 12. Let’s build a location based surf reporting app! • Report current conditions • Get current local conditions • Determine best conditions per beach
- 13. Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), "reporter" : "test", "location" : { "coordinates" : [ -122.477222, 37.810556 ], "name" : "Fort Point" }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
- 14. Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Auto-indexed -122.477222, 37.810556 ], "name" : "Fort Point" }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
- 15. Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Autoindexed -122.477222, 37.810556 ], Compound Index, "name" : "Fort Point" Geospacial }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, "date" : ISODate("2011-11-16T20:17:17.277Z") }
- 16. Document Structure { "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "test", Unique, "location" : { "coordinates" : [ Autoindexed -122.477222, 37.810556 ], Compound Index, "name" : "Fort Point" Geospacial }, "conditions" : { "height" : 0, "period" : 9, "rating" : 1 }, Indexed for "date" : ISODate("2011-11-16T20:17:17.277Z") Time-To-Live }
- 17. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1})
- 18. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports
- 19. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports
- 20. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports • Return only the relevant info
- 21. Get local surf conditions db.reports.find( { "location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"location.name" :1, _id : 0, "conditions" :1} ).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports • Return only the relevant info • Show me the best surf first
- 22. Get local surf conditions: Connecting
- 23. DBObjects Output: { "name" : "test"} parsed
- 25. Results { "location" : { "name" : "Montara" }, "conditions" : { "height" : 6, "period" : 20, "rating" : 5 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 5, "period" : 13, "rating" : 3 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 3, "period" : 15, "rating" : 3 } } { "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 3, "period" : 16, "rating" : 2 } } { "location" : { "name" : "Montara" }, "conditions" : { "height" : 0, "period" : 8, "rating" : 1 } } { "location" : { "name" : "Linda Mar" }, "conditions" : { "height" : 3, "period" : 10, "rating" : 1 } } { "location" : { "name" : "Sharp Park" }, "conditions" : { "height" : 1, "period" : 15, "rating" : 1 } } { "location" : { "name" : "Sharp Park" }, "conditions" : { "height" : 5, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 1, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 0, "period" : 10, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 4, "period" : 6, "rating" : 1 } } { "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 0, "period" : 14, "rating" : 1 } }
- 26. Analysis Features: Aggregation Framework What are the best conditions for my local beach?
- 27. Pipelining Operations $match Match “Linda Mar” $project Only interested in conditions Group by rating, averaging $group wave height and wave period $sort Order by best conditions
- 28. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] }
- 29. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Match “Linda Mar”
- 30. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Only interested in conditions
- 31. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Group by rating & average conditions
- 32. Aggregation Framework { "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ] } Show me best conditions first
- 34. Scaling • Sharding is the partitioning of data among multiple machines • Balancing occurs when the load on any one node grows out of proportion
- 35. Scaling MongoDB Sharded cluster MongoDB Single Instance Or Replica Set Client Application
- 36. The Mechanism of Sharding Complete Data Set Define Shard Key on Location Name Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
- 37. The Mechanism of Sharding Chunk Chunk Define Shard Key on Location Name Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
- 38. The Mechanism of Sharding Chunk Chunk Chunk Chunk Fort Point Linda Mar Maverick’s Ocean Beach Rockaway
- 39. The Mechanism of Sharding Chunk Chunk Chunk Chunk Fort Point Linda Mar Maverick’s Ocean Beach Rockaway Shard 1 Shard 2 Shard 3 Shard 4
- 40. The Mechanism of Sharding Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 40
- 41. The Mechanism of Sharding Client Query: Linda Mar Application Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 41
- 42. The Mechanism of Sharding Client Query: Maverick’s Application Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 42
- 43. Thanks! Office Hours Thursdays 4-6 pm 555 University Ave. Palo Alto We’re Hiring ! [email protected]
Editor's Notes
- #9: Read from any of the fastest responding nodes.