MongoDB Sharding Fundamentals
2 likes1,304 views
The document provides an overview of MongoDB sharding, including: - Sharding allows horizontal scaling of data by partitioning a database across multiple servers or shards. - The MongoDB sharding architecture consists of shards to hold data, config servers to store metadata, and mongos processes to route requests. - Data is partitioned into chunks based on a shard key and chunks can move between shards as the data distribution changes.
Ad
More Related Content
What's hot (20)
Viewers also liked (6)
Ad
Similar to MongoDB Sharding Fundamentals (20)
Ad
More from Antonios Giannopoulos (15)
Recently uploaded (20)
![PRE-NATAL GRnnnmnnnnmmOWTH seminar[1].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/pre-natalgrowthseminar1-250427093235-de04befc-thumbnail.jpg?width=560&fit=bounds)
MongoDB Sharding Fundamentals
- 1. MongoDB Sharding fundamentals Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace
- 2. Background -‐ 14 years in databases and system engineering -‐ NoSQL DBA @ ObjectRocket by Rackspace -‐ MongoDB CerFfied DBA
- 3. What is sharding? -‐ A mechanism for horizontal scaling -‐ Distributes the dataset over mulFple servers (shards) -‐ Each shard is an independent database -‐ All shards consists a single logical database
- 4. Why Sharding? -‐ Increases cluster throughput – Read/Write Scaling -‐ Reduces costs -‐ Many small servers VS one big box -‐ Eliminates HW and SW hard limits
- 5. MongoDB Sharding -‐ Consists of three elements: Shards, Config Servers and Mongos -‐ Shards: Hold the cluster data, databases, collecFons, documents (Data nodes) -‐ Config Servers: Hold the cluster metadata, map the cluster architecture. -‐ Mongos: Serve all drivers requests. Route each request to a shard or shards (Router nodes)
- 6. ApplicaFon / Driver Layer Mongos01 Mongos02 MongosN ConfigSrv01 ConfigSrv02 ConfigSrv03 Shard02 ShardN MongoDB Sharded Cluster Shard01 … …
- 7. How Sharding works? -‐ Range parFFoning per collecFon (chunks) -‐ Shard key to define chunks (field(s)) -‐ Chunks are “metadata” on the config servers -‐ Chunks can move, split and merge
- 8. How Sharding works? -‐ Example { "name" : "Angelina", "surname" : "Jolie", "posiFon" : "Windows Eng.", "phone" : "555-‐5555" } { "name" : "Emma", "surname" : "Stone", "posiFon" : "Windows Eng.", "phone" : "555-‐5555" } { "name" : "Charlize", "surname" : "Theron", "posiFon" : "Linux Eng.", "phone" : "555-‐5555" } { "name" : "Olivia", "surname" : "Wilde", "posiFon" : "Linux Eng.", "phone" : "555-‐5555" } { "name" : "Jessica", "surname" : "Alba", "posiFon" : "Sr Linux Eng.", "phone" : "555-‐5555" } { "name" : "Scarlef", "surname" : "Johansson", "posiFon" : "Sr Windows Eng.", "phone" : "555-‐5555" } { "name" : "Megan", "surname" : "Fox", "posiFon" : "Networks Eng.", "phone" : "555-‐5555" } { "name" : "Mila", "surname" : "Kunis", "posiFon" : "Sr Networks Eng.", "phone" : "555-‐5555" } { "name" : "Natalie", "surname" : "Portman", "posiFon" : "Database Eng", "phone" : "555-‐5555" } { "name" : "Anne", "surname" : "Hathaway", "posiFon" : "Sr Database Eng", "phone" : "555-‐5555" } -‐ CollecFon employees for an IT company -‐ Shard key “posi-on”
- 9. How Sharding works? -‐ Example { "min" : { "posiFon" : { "$minKey" : 1 } }, "max" : { "posiFon" : "Database Eng" }, "shard" : ”Shard01" } { "min" : { "posiFon" : "Database Eng" }, "max" : { "posiFon" : "Sr Database Eng" }, "shard" : "Shard01" } { "min" : { "posiFon" : "Sr Database Eng" }, "max" : { "posiFon" : "Windows Eng." }, "shard" : "Shard02" } { "min" : { "posiFon" : "Windows Eng." }, "max" : { "posiFon" : { "$maxKey" : 1 } }, "shard" : "Shard02" } -‐ Lower/upper bound and shard (server)
- 10. Choose a shard key -‐ High Cardinality -‐ Not Null values -‐ Immutable field(s) -‐ Not Monotonically increased fields
- 11. Choose a shard key -‐ Even read/write distribuFon -‐ Even data distribuFon -‐ Read targeFng -‐ Read locality
- 12. Choose a shard key -‐ Hashed shard keys for randomness -‐ Compound shard keys for cardinality -‐ Unique indexes are good -‐ {_id:”hashed”} scales writes
- 13. LimitaFons of Sharding -‐ Unique indexes – Just one… -‐ IniFal collecFon size – Avoid collecFons > 256G, hard limit is a funcFon of key and chunk size , for 64MB chunk/512B key is more than 1TB -‐ Number of documents per chunk (250K)
- 14. LimitaFons of Sharding -‐ Shard key size < 512 bytes -‐ MulFkey,text, geo indexes are prohibited -‐ Some operaFons won’t run (for example group, db.eval(), $isolated, $snapshot, geoSearch)
- 15. “Sharding” – Other players -‐ ApplicaFon level sharding -‐ Mysql (MaxScale, Fabric,…) -‐ Postgres (pg_shard) -‐ ElasFcSearch (Document ID or rouFng) -‐ Cassandra (Hash-‐based -‐ Ring topology)
- 16. Contact www.objectrocket.com www.rackspace.co.uk/objectrocket/mongodb [email protected] We are hiring! (DevOps, DBAs and more) hfp://objectrocket.com/careers
- 17. QuesFons? Thank you!!! MongoDB Meetup What's new in MongoDB 3.0 Tuesday, November 10 , 7:00 pm @ Harokopio University