Schema design short

Schema Design Basics Roger Bodamer roger @ 10gen.com @rogerb

A brief history of Data Modeling ISAM COBOL Network Hiearchical Relational 1970 E.F.Codd introduces 1 st Normal Form (1NF) 1971 E.F.Codd introduces 2 nd and 3 rd Normal Form (2NF, 3NF 1974 Codd & Boyce define Boyce/Codd Normal Form (BCNF) 2002 Date, Darween, Lorentzos define 6 th Normal Form (6NF) Object

Modeling goals Goals: Avoid anomalies when inserting, updating or deleting Minimize redesign when extending the schema Make the model informative to users Avoid bias towards a particular style of query * source : wikipedia

Relational made normalized data look like this

Document databases make normalized data look like this

Some terms before we proceed RDBMS Document DBs Table Collection View / Row(s) JSON Document Index Index Join Embedding & Linking across documents Partition Shard Partition Key Shard Key

Recap Design documents that simply map to your application post = { author : “roger”, date : new Date(), text : “I love J.Biebs...”, tags : [“rockstar”,“puppy-love”]}

Query operators Conditional operators: $ne, $in, $nin, $mod, $all, $size, $exists, $type, .. $lt, $lte, $gt, $gte, $ne, // find posts with any tags >db.posts.find({ tags : {$exists: true}})

Query operators Conditional operators: $ne, $in, $nin, $mod, $all, $size, $exists, $type, .. $lt, $lte, $gt, $gte, $ne, // find posts with any tags >db.posts.find({ tags : {$exists: true}}) Regular expressions: // posts where author starts with k >db.posts.find({ author : /^r*/i })

Query operators Conditional operators: $ne, $in, $nin, $mod, $all, $size, $exists, $type, .. $lt, $lte, $gt, $gte, $ne, // find posts with any tags >db.posts.find({ tags : {$exists: true}}) Regular expressions: // posts where author starts with k >db.posts.find({ author : /^r*/i }) Counting: // posts written by mike >db.posts.find({ author : “roger”}).count()

Extending the Schema new_comment = { author : “Gretchen”, date : new Date(), text : “Biebs is Toll!!!!”} new_info = { ‘$push’: { comments : new_comment}, ‘ $inc’: { comments_count : 1}} >db.posts.update({ _id : “...” }, new_info)

{ _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), author : ”roger", date : "Sat Jul 24 2010 19:47:11 GMT-0700 (PDT)", text : " I love J.Biebs... ", tags : [ ”rockstar", ”puppy-love" ], comments_count : 1, comments : [ { author : ”Gretchen", date : "Sat Jul 24 2010 20:51:03 GMT-0700 (PDT)", text : ” Biebs is Toll!!!! " } ]} Extending the Schema

// create index on nested documents: >db.posts.ensureIndex({"comments.author": 1}) >db.posts.find({comments.author:”Gretchen”}) // find last 5 posts: >db.posts.find().sort({ date :-1}).limit(5) // most commented post: >db.posts.find().sort({ comments_count :-1}).limit(1) When sorting, check if you need an index Extending the Schema

Single Table Inheritance >db.shapes.find() { _id : ObjectId("..."), type : "circle", area : 3.14, radius : 1} { _id : ObjectId("..."), type : "square", area : 4, d : 2} { _id : ObjectId("..."), type : "rect", area : 10, length : 5, width : 2} // find shapes where radius > 0 >db.shapes.find({ radius : { $gt : 0}}) // create index >db.shapes.ensureIndex({ radius : 1})

One to Many - Embedded Array / Using Array Keys - slice operator to return subset of array - hard to find latest comments across all documents

One to Many - Embedded Array / Array Keys - slice operator to return subset of array - hard to find latest comments across all documents - Embedded tree - Single document - Natural

One to Many - Embedded Array / Array Keys - slice operator to return subset of array - hard to find latest comments across all documents - Embedded tree - Single document - Natural - Normalized (2 collections) - most flexible - more queries

Many - Many Example: - Product can be in many categories - Category can have many products Products - product_id Category - category_id Prod_Categories id product_id category_id

products: { _id : ObjectId("4c4ca23933fb5941681b912e"), name : "Sumatra Dark Roast", category_ids : [ ObjectId("4c4ca25433fb5941681b912f"), ObjectId("4c4ca25433fb5941681b92af”]} Many – Many

products: { _id : ObjectId("4c4ca23933fb5941681b912e"), name : "Sumatra Dark Roast", category_ids : [ ObjectId("4c4ca25433fb5941681b912f"), ObjectId("4c4ca25433fb5941681b92af”]} categories: { _id : ObjectId("4c4ca25433fb5941681b912f"), name : "Indonesia", product_ids : [ ObjectId("4c4ca23933fb5941681b912e"), ObjectId("4c4ca30433fb5941681b9130"), ObjectId("4c4ca30433fb5941681b913a"]} Many – Many

products: { _id : ObjectId("4c4ca23933fb5941681b912e"), name : "Sumatra Dark Roast", category_ids : [ ObjectId("4c4ca25433fb5941681b912f"), ObjectId("4c4ca25433fb5941681b92af”]} categories: { _id : ObjectId("4c4ca25433fb5941681b912f"), name : "Indonesia", product_ids : [ ObjectId("4c4ca23933fb5941681b912e"), ObjectId("4c4ca30433fb5941681b9130"), ObjectId("4c4ca30433fb5941681b913a"]} //All categories for a given product >db.categories.find({ product_ids : ObjectId("4c4ca23933fb5941681b912e")}) Many - Many

products: { _id : ObjectId("4c4ca23933fb5941681b912e"), name : "Sumatra Dark Roast", category_ids : [ ObjectId("4c4ca25433fb5941681b912f"), ObjectId("4c4ca25433fb5941681b92af”]} categories: { _id : ObjectId("4c4ca25433fb5941681b912f"), name : "Indonesia", product_ids : [ ObjectId("4c4ca23933fb5941681b912e"), ObjectId("4c4ca30433fb5941681b9130"), ObjectId("4c4ca30433fb5941681b913a"]} //All categories for a given product >db.categories.find({ product_ids : ObjectId("4c4ca23933fb5941681b912e")}) //All products for a given category >db.products.find({ category_ids : ObjectId("4c4ca25433fb5941681b912f")}) Many - Many

products: { _id : ObjectId("4c4ca23933fb5941681b912e"), name : "Sumatra Dark Roast", category_ids : [ ObjectId("4c4ca25433fb5941681b912f"), ObjectId("4c4ca25433fb5941681b92af”]} categories: { _id : ObjectId("4c4ca25433fb5941681b912f"), name : "Indonesia"} Alternative

products: { _id : ObjectId("4c4ca23933fb5941681b912e"), name : "Sumatra Dark Roast", category_ids : [ ObjectId("4c4ca25433fb5941681b912f"), ObjectId("4c4ca25433fb5941681b92af”]} categories: { _id : ObjectId("4c4ca25433fb5941681b912f"), name : "Indonesia"} // All products for a given category >db.products.find({ category_ids : ObjectId("4c4ca25433fb5941681b912f")}) Alternative

products: { _id : ObjectId("4c4ca23933fb5941681b912e"), name : "Sumatra Dark Roast", category_ids : [ ObjectId("4c4ca25433fb5941681b912f"), ObjectId("4c4ca25433fb5941681b92af”]} categories: { _id : ObjectId("4c4ca25433fb5941681b912f"), name : "Indonesia"} // All products for a given category >db.products.find({ category_ids : ObjectId("4c4ca25433fb5941681b912f")}) // All categories for a given product product = db.products.find( _id : some_id) >db.categories.find({ _id : {$in : product.category_ids}}) Alternative

Trees Full Tree in Document { comments : [ { author : “rpb”, text : “...”, replies : [ { author : “Fred”, text : “...”, replies : []} ]} ]} Pros: Single Document, Performance, Intuitive Cons: Hard to search, 4MB limit

Trees - continued Parent Links - Each node is stored as a document - Contains the id of the parent Child Links - Each node contains the id’s of the children - Can support graphs (multiple parents / child)

Array of Ancestors - Store Ancestors of a node { _id : "a" } { _id : "b", ancestors : [ "a" ], parent : "a" } { _id : "c", ancestors : [ "a", "b" ], parent : "b" } { _id : "d", ancestors : [ "a", "b" ], parent : "b" } { _id : "e", ancestors : [ "a" ], parent : "a" } { _id : "f", ancestors : [ "a", "e" ], parent : "e" } { _id : "g", ancestors : [ "a", "b", "d" ], parent : "d" }

Array of Ancestors - Store Ancestors of a node { _id : "a" } { _id : "b", ancestors : [ "a" ], parent : "a" } { _id : "c", ancestors : [ "a", "b" ], parent : "b" } { _id : "d", ancestors : [ "a", "b" ], parent : "b" } { _id : "e", ancestors : [ "a" ], parent : "a" } { _id : "f", ancestors : [ "a", "e" ], parent : "e" } { _id : "g", ancestors : [ "a", "b", "d" ], parent : "d" } //find all descendants of b: >db.tree2.find({ ancestors : ‘b’})

Array of Ancestors - Store Ancestors of a node { _id : "a" } { _id : "b", ancestors : [ "a" ], parent : "a" } { _id : "c", ancestors : [ "a", "b" ], parent : "b" } { _id : "d", ancestors : [ "a", "b" ], parent : "b" } { _id : "e", ancestors : [ "a" ], parent : "a" } { _id : "f", ancestors : [ "a", "e" ], parent : "e" } { _id : "g", ancestors : [ "a", "b", "d" ], parent : "d" } //find all descendants of b: >db.tree2.find({ ancestors : ‘b’}) //find all ancestors of f: >ancestors = db.tree2.findOne({ _id :’f’}).ancestors >db.tree2.find({ _id : { $in : ancestors})

Variable Keys How to index ? { "_id" : "uuid1", "field1" : { "ctx1" : { "ctx3" : 5, … }, "ctx8" : { "ctx3" : 5, … } }} db.MyCollection.find({ "field1.ctx1.ctx3" : { $exists : true} }) Rewrite: { "_id" : "uuid1", "field1" : { key: "ctx1”, value : { k:"ctx3”, v : 5, … }, key: "ctx8”, value : { k: "ctx3”, v : 5, … } }} db.x.ensureIndex({“field1.key.k”, 1})

findAndModify Queue example //Example: find highest priority job and mark job = db.jobs.findAndModify({ query : {inprogress: false}, sort : {priority: -1), update : {$set: {inprogress: true, started: new Date()}}, new : true})

Learn More Kyle’s presentation + video: http://www.slideshare.net/kbanker/mongodb-schema-design http://www.blip.tv/file/3704083 Dwight’s presentation http://www.slideshare.net/mongosf/schema-design-with-mongodb-dwight-merriman Documentation Trees: http://www.mongodb.org/display/DOCS/Trees+in+MongoDB Queues: http://www.mongodb.org/display/DOCS/findandmodify+Command Aggregration: http://www.mongodb.org/display/DOCS/Aggregation Capped Col. : http://www.mongodb.org/display/DOCS/Capped+Collections Geo: http://www.mongodb.org/display/DOCS/Geospatial+Indexing GridFS: http://www.mongodb.org/display/DOCS/GridFS+Specification

Download MongoDB http://www.mongodb.org and let us know what you think @mongodb

DBRef DBRef { $ref : collection, $id : id_value} - Think URL - YDSMV: your driver support may vary Sample Schema: nr = { note_refs : [{"$ref" : "notes", "$id" : 5}, ... ]} Dereferencing: nr.forEach(function(r) { printjson(db[r.$ref].findOne({ _id : r.$id})); }

BSON Mongodb stores data in BSON internally Lightweight, Traversable, Efficient encoding Typed boolean, integer, float, date, string, binary, array...

Schema design short

More Related Content

What's hot (20)

Similar to Schema design short (20)

More from MongoDB (20)

Recently uploaded (20)

Schema design short

Editor's Notes