Mongo performance tuning: tips and tricks
8 likes6,457 views
This document discusses various aspects of MongoDB performance. It covers monitoring MongoDB performance through the HTTP console, SSH console, and custom monitoring. It also discusses tuning MongoDB performance through optimizing indexes, avoiding growing documents, preallocating documents, and using explain queries. The conclusion provides some tips on dealing with disk fragmentation, concurrency, and why it's good to upgrade MongoDB versions.
Ad
More Related Content
What's hot (20)
Similar to Mongo performance tuning: tips and tricks (20)
Ad
Recently uploaded (20)
Ad
Mongo performance tuning: tips and tricks
- 1. Mongo performance Stuff that matters Monitoring Tuning Conclusion [email protected]
- 2. Mongo performance Stuff that matters
- 3. Mongo Pain. Write lock Multiple readers / Single-writer lock ( http://en.wikipedia.org/wiki/Readers-writer_lock) ver < 2.0 - has a process-wide write lock 2.1.1+ per-database locking ( https://jira.mongodb.org/browse/SERVER-4328 ) per-collection locking ( https://jira.mongodb.org/browse/SERVER-1240 )
- 4. Mongo Pain. mmap http://docs.mongodb.org/manual/faq/storage/ Page faults: will occur if you’re attempting to access part of a memory-mapped file that isn’t in memory. ( http://docs.mongodb.org/manual/reference/glossary/#term-page-fault ) Performance: page fault can take around 40,000 times longer than a nonfaulting memory operation ver 2.0+, this is addressed by detecting the likelihood of a page fault and releasing the lock before faulting - YIELD operation; SSD is a doctor's stuff in this case;
- 5. Mongo Atomicity Write operations are atomic on the level of a single document: no single write operation can atomically affect more than one document or more than one collection. (http://docs.mongodb.org/manual/tutorial/isolate-sequence-of-operations/) hint: isolates a write operation that affects multiple documents from other write operations (http://docs.mongodb.org/manual/reference/operator/isolated/) http://docs.mongodb.org/manual/reference/command/findAndModify/
- 7. Monitoring: HTTP Console http://docs.mongodb.org/ecosystem/tools/http-interf aces/#http-console http://dbhost.net:28017/ Replica Set Admin UI, Oplog status clients DBTOP write lock % time in write lock, by 4 sec periods (form ~12% to ~6%) Log - By default the slow operation threshold is 100 millis
- 8. Monitoring: SSH Console netstat -n | wc -l ps aux | grep mongodb cat /proc/{pid}/limits | grep "Max open files" ulimit -n 65000 (http://www.andrewrollins.com/2010/10/20/mongodb-open-file-limit/)
- 9. Monitoring: Custom http://sj.malyk/admin/MongoProfileLog/ milestone marks: do fake update before and after app execution http://docs.mongodb.org/manual/tutorial/manage-the-database-profi ler/ http://docs.mongodb.org/manual/reference/database-profiler/ https://gist.github.com/vladimir-malyk/5047107 https://gist.github.com/kgorman/995a3aa5b35e92e5ab57
- 11. Tuning: Heavy tricks degradation - don't write to an unimportant collections; partitioning - move an unimportant collections to a separate replicaset.
- 12. Tuning: Indexes * Create Indexes to Support Your Queries * Use Compound Indexes to Support Several Different Queries * -10% write performance for each additional index (http://www.slideshare.net/mongodb/mongodb-performance-tuning) * Create Indexes that Support Covered Queries * Use Indexes to Sort Query Results * http://blog.mongolab.com/2012/06/cardinal-ins/ * First, fields on which you will query for exact values. * Second, fields on which you will sort. * Finally, fields on which you will query for a range of val. * Ensure Indexes Fit RAM * Creating index on a big collection takes a lot of time with write lock
- 13. Tuning: Common tricks http://blog.serverdensity.com/mongodb-schema-design-pitfalls/ * Avoid growing documents (“moved” in system.profile) * Use field modifiers: Instead of sending a whole new document to update an existing one, you can set or remove specific fields * Preallocate documents: preallocate the document with placeholder values, then use the $set field modifier to change the actual value later * Field names take up space: http://blog.serverdensity.com/on-shortened-field-names-in-mongod b/ * Consider using _id for your own purposes
- 14. Tuning: Explain http://docs.mongodb.org/manual/reference/explain/ * http://docs.mongodb.org/manual/reference/explain/#explain.cursor * http://docs.mongodb.org/manual/reference/explain/#explain.net * http://docs.mongodb.org/manual/reference/explain/#explain.nscanned * http://docs.mongodb.org/manual/reference/explain/#explain.n * http://docs.mongodb.org/manual/reference/explain/#explain.scanAndOrder * http://docs.mongodb.org/manual/reference/explain/#explain.indexOnly * http://docs.mongodb.org/manual/reference/explain/#explain.nYields * http://docs.mongodb.org/manual/reference/explain/#explain.allPlans
- 16. Mongo: Tips and tricks * deal with db disk fragmentation: * manual defragmentation (via replica resync); * http://docs.mongodb.org/manual/reference/command/compact/ * Can't take a write lock while out of disk space: * you need some free space - db defrag will help you;
- 17. Mongo: Concurrency http://docs.mongodb.org/manual/faq/concurrency/ http://docs.mongodb.org/manual/faq/concurrency/#how-do-i-see-the-status- of-locks-on-my-mongod-instances http://docs.mongodb.org/manual/faq/concurrency/#which-operations-lock-th e-database http://docs.mongodb.org/manual/faq/concurrency/#which-administrative-co mmands-lock-the-database * db.auth() - lock the database but only hold the lock for a very short time http://blog.serverdensity.com/goodbye-global-lock-mongodb-2-0-vs-2-2/ why it's a good idea to upgrade your mongodb