Building a Social Network with MongoDB
9 likes9,240 views
This document discusses how Eventbrite built a social network using MongoDB to provide event recommendations. It stores user, event, and order data in MongoDB collections with indexes. It generates recommendations by querying neighbors' attended events and scoring events based on neighbor attendance. It also discusses challenges around dynamic neighbor relationships and performance needs that led it to choose MongoDB.
![User Data in MongoDB
{ "_id": 4558992,
"events" : {
"all_ids": [ 116706, 179487, 16389, 827496 ],
"curr_ids": [ 827496 ],
}
},
Past and current
attendance
13
Friday, December 3, 2010](https://image.slidesharecdn.com/eventbrite-mongosv-social-graph-120822011050-phpapp02/85/Building-a-Social-Network-with-MongoDB-13-320.jpg)
![User Data in MongoDB
{ "_id": 4558992,
"events" : {
"all_ids": [ 116706, 179487, 16389, 827496 ],
"curr_ids": [ 827496 ],
},
"nns" : [
[ 2816442, 0.2 ],
[ 1615962, 0.047619047619047616 ],
],
} Nearest neighbors
(user_id, score)
14
Friday, December 3, 2010](https://image.slidesharecdn.com/eventbrite-mongosv-social-graph-120822011050-phpapp02/85/Building-a-Social-Network-with-MongoDB-14-320.jpg)
![User Data in MongoDB
{ "_id": 4558992,
"events" : {
"all_ids": [ 116706, 179487, 16389, 827496 ],
"curr_ids": [ 827496 ],
},
"nns" : [
[ 2816442, 0.2 ],
[ 1615962, 0.047619047619047616 ],
],
"fb" : {
"_id" : 4808871, Facebook data
"name" : "Brian Zambrano",
"location" : "San Francisco, California",
"friends" : [ 568876525, 569507467, 569559792 ],
},
}
15
Friday, December 3, 2010](https://image.slidesharecdn.com/eventbrite-mongosv-social-graph-120822011050-phpapp02/85/Building-a-Social-Network-with-MongoDB-15-320.jpg)
![MongoDB Indexes
{ "_id": 4558992,
"events" : {
"all_ids": [ 116706, 179487, 16389, 827496 ],
"curr_ids": [ 827496 ],
},
"nns" : [
[ 2816442, 0.2 ],
[ 1615962, 0.047619047619047616 ],
],
"fb" : {
"_id" : 4808871,
"name" : "Brian Zambrano",
"location" : "San Francisco, California",
"friends" : [ 568876525, 569507467, 569559792],
},
}
16
Friday, December 3, 2010](https://image.slidesharecdn.com/eventbrite-mongosv-social-graph-120822011050-phpapp02/85/Building-a-Social-Network-with-MongoDB-16-320.jpg)
![Events Collection
> db.events.findOne({_id: 799177})
{
"_id" : 799177,
"uid" : 2989008,
"title" : "MongoSV",
"venue" : {
"loc" : [
37.413042,
-122.071106
],
"state" : "CA",
"id" : 508093,
"city" : "Mountain View"
},
"logo" : "758915938.png",
"shortname" : "mongosv",
"start_date" : "Fri Dec 03 2010 01:00:00 GMT-0800 (PST)"
}
17
Friday, December 3, 2010](https://image.slidesharecdn.com/eventbrite-mongosv-social-graph-120822011050-phpapp02/85/Building-a-Social-Network-with-MongoDB-17-320.jpg)
Building a Social Network with MongoDB
- 1. Building a Social Network with MongoDB Brian Zambrano MongoSV December 3, 2010 1 Friday, December 3, 2010
- 2. Eventbrite Brand Tenets 2 Friday, December 3, 2010
- 3. Eventbrite Brand Tenets 3 Friday, December 3, 2010
- 4. Social Recommendations 4 Friday, December 3, 2010
- 5. EventbriteŹ¼s Social Graph 5 Friday, December 3, 2010
- 6. EventbriteŹ¼s Social Graph 6 Friday, December 3, 2010
- 7. Neighbors 7 Friday, December 3, 2010
- 8. Challenges ā¢ Dynamic ā¢ Neighbors change often ā¢ NeighborsŹ¼ events change often ā¢ Flexibility ā¢ Want to incorporate other social graphs ā¢ Product may evolve quickly ā¢ Performance ā¢ We need really fast reads ā¢ Frequent writes 8 Friday, December 3, 2010
- 9. Why MongoDB? ā¢ Performance ā¢ Flexible schema design ā¢ Easy to work with ā¢ We felt comfortable MongoDB would mature as our needs became more demanding 9 Friday, December 3, 2010
- 10. Providing Recommendations 1. User visits http://eventbrite.com/mytickets/ 2. Fetch neighbors 3. Fetch neighborsŹ¼ events 4. Score each possible event 5. Return recommendations 10 Friday, December 3, 2010
- 11. MongoDB setup ā¢ One non-sharded replica set ā¢ Two DBs on Large EC2 instances ā¢ One arbiter ā¢ Three collections ā¢ Users ā¢ Events ā¢ Orders 11 Friday, December 3, 2010
- 12. User Data in MongoDB { "_id": 4558992, Unique User Id } 12 Friday, December 3, 2010
- 13. User Data in MongoDB { "_id": 4558992, "events" : { "all_ids": [ 116706, 179487, 16389, 827496 ], "curr_ids": [ 827496 ], } }, Past and current attendance 13 Friday, December 3, 2010
- 14. User Data in MongoDB { "_id": 4558992, "events" : { "all_ids": [ 116706, 179487, 16389, 827496 ], "curr_ids": [ 827496 ], }, "nns" : [ [ 2816442, 0.2 ], [ 1615962, 0.047619047619047616 ], ], } Nearest neighbors (user_id, score) 14 Friday, December 3, 2010
- 15. User Data in MongoDB { "_id": 4558992, "events" : { "all_ids": [ 116706, 179487, 16389, 827496 ], "curr_ids": [ 827496 ], }, "nns" : [ [ 2816442, 0.2 ], [ 1615962, 0.047619047619047616 ], ], "fb" : { "_id" : 4808871, Facebook data "name" : "Brian Zambrano", "location" : "San Francisco, California", "friends" : [ 568876525, 569507467, 569559792 ], }, } 15 Friday, December 3, 2010
- 16. MongoDB Indexes { "_id": 4558992, "events" : { "all_ids": [ 116706, 179487, 16389, 827496 ], "curr_ids": [ 827496 ], }, "nns" : [ [ 2816442, 0.2 ], [ 1615962, 0.047619047619047616 ], ], "fb" : { "_id" : 4808871, "name" : "Brian Zambrano", "location" : "San Francisco, California", "friends" : [ 568876525, 569507467, 569559792], }, } 16 Friday, December 3, 2010
- 17. Events Collection > db.events.findOne({_id: 799177}) { "_id" : 799177, "uid" : 2989008, "title" : "MongoSV", "venue" : { "loc" : [ 37.413042, -122.071106 ], "state" : "CA", "id" : 508093, "city" : "Mountain View" }, "logo" : "758915938.png", "shortname" : "mongosv", "start_date" : "Fri Dec 03 2010 01:00:00 GMT-0800 (PST)" } 17 Friday, December 3, 2010
- 18. Orders Collection > db.orders.find({_eid: 799177}) { "_id" : 17464215, "_uid" : 1111195, "_eid" : 799177 } { "_id" : 17575729, "_uid" : 6970539, "_eid" : 799177 } { "_id" : 17582343, "_uid" : 3092687, "_eid" : 799177 } { "_id" : 17588693, "_uid" : 2255017, "_eid" : 799177 } { "_id" : 17589589, "_uid" : 6976917, "_eid" : 799177 } { "_id" : 17601979, "_uid" : 885441, "_eid" : 799177 } { "_id" : 17603085, "_uid" : 2500199, "_eid" : 799177 } { "_id" : 17608289, "_uid" : 6984367, "_eid" : 799177 } { "_id" : 17681965, "_uid" : 628459, "_eid" : 799177 } { "_id" : 17684489, "_uid" : 7017999, "_eid" : 799177 } { "_id" : 17689673, "_uid" : 7020133, "_eid" : 799177 } { "_id" : 17728267, "_uid" : 7036607, "_eid" : 799177 } has more 18 Friday, December 3, 2010
- 19. Recommended Events Query Two + n queries 1. Get neighbors nns = db.users.find({_id : {$in : user.nn_ids}}) 2. Get possible event recommendations: db.events.find({_id : {$in : nns.events.all}}) n.For each event, get total attendee count db.orders.find({_eid : event_id}) 19 Friday, December 3, 2010
- 20. Recommended Events Query Two + n queries 1. Get neighbors nns = db.users.find({_id : {$in : user.nn_ids}}) 2. Get possible event recommendations: db.events.find({_id : {$in : nns.events.all}}) n.For each event, get total attendee count db.orders.find({_eid : event_id}) Optimization opportunity: Embed orders in Event records 20 Friday, December 3, 2010
- 21. Updating Neighbors Two queries, one update 1. Get all orders for a userŹ¼s past events: uids = db.orders.find({_id : {$in : user.events.all}}) 2. Get all neighbors: nns = db.users.find({_id : {$in : uids}}) ā”Score neighbors 3. Update nn_ids db.users.update({_id : uid}, {$set : {nn_ids: nn}}) 21 Friday, December 3, 2010
- 22. Facebook FriendŹ¼s Events Two queries 1. Get FB friends db.users.find({fb._id : {$in : fb.friends}}) 2. Get events FB friends are attending db.events.find({_id : {$in : fb_friends_events}}) 22 Friday, December 3, 2010
- 23. The Future ā¢ Incorporate other social networks ā¢ Iterate scoring algorithm ā¢ Count recommendation impressions 23 Friday, December 3, 2010
- 24. WeŹ¼re hiring! http://www.eventbrite.com/jobs/ 24 Friday, December 3, 2010
- 25. Thanks! Brian Zambrano <[email protected]> EventbriteŹ¼s new Facebook recommendations power social event discovery: http://bit.ly/gRVS7I Social Commerce: A First Look at the Numbers: http://bit.ly/gXeg9Q 25 Friday, December 3, 2010