MongoDB Database Replication
3 likes4,925 views
Database replication involves keeping identical copies of data on different servers to provide redundancy and minimize downtime. Replication is recommended for databases in production from the start. A MongoDB replica set consists of a primary server that handles client requests and secondary servers that copy the primary's data. Replica sets can include up to 50 members with 7 voting members and use an oplog to replicate operations from the primary to secondaries. For elections and writes to succeed, a majority of voting members must be reachable.
![Pro Tip :]
Auto Format Query Response
Add
DBQuery.prototype._prettyShell = true;
to your ~/.mongorc.js](https://image.slidesharecdn.com/replicatingdatawithmongodb-150526104720-lva1-app6891/85/MongoDB-Database-Replication-9-320.jpg)
![Setup Your Replica Set
Part 2: Example Configuration
$: rs.config();
{
"_id" : "myReplica",
"version": 3,
"members" : [
{
"_id" : 0,
"host" : "my-host:31000"
},
{
"_id" : 1,
"host" : "my-host:31001"
},
{
"_id" : 2,
"host" : "my-host:31002"
}
]
}](https://image.slidesharecdn.com/replicatingdatawithmongodb-150526104720-lva1-app6891/85/MongoDB-Database-Replication-14-320.jpg)
![Replica Set Reconfiguration
Update existing member
$: mongo --port 31000 # current primary
$: config = rs.config()
$: config.members[2].priority = 0;
$: rs.reconfig(config);](https://image.slidesharecdn.com/replicatingdatawithmongodb-150526104720-lva1-app6891/85/MongoDB-Database-Replication-19-320.jpg)
![Custom Write Concerns
Step 1: Tags
$: config = rs.config()
$: config.members[0].tags = {dc: ‘dc-1’};
$: config.members[3].tags = {dc: ‘dc-2’};
$: rs.reconfig(config);](https://image.slidesharecdn.com/replicatingdatawithmongodb-150526104720-lva1-app6891/85/MongoDB-Database-Replication-40-320.jpg)
![Custom Write Concerns
Step 2: Custom Last Error Mode
$: config = rs.config()
$: config.settings.lastErrorModes = [{
“each-dc”: {dc: 2}
}];
$: rs.reconfig(config);](https://image.slidesharecdn.com/replicatingdatawithmongodb-150526104720-lva1-app6891/85/MongoDB-Database-Replication-41-320.jpg)
Ad
More Related Content
What's hot (20)
Similar to MongoDB Database Replication (20)
Ad
Recently uploaded (20)
Ad
MongoDB Database Replication
- 2. Replication Process of keeping IDENTICAL COPY of your data on different servers. Replication offers Minimum Downtime in case of hardware failure, software failure or maintenance
- 3. Replication is highly recommended for all databases in production from the very first day.
- 4. Replica Set MongoDB’s Terminology for Replication A replica set is a group of servers with one primary, the server taking client requests, and multiple secondaries, servers that keep copies of the primary’s data.
- 5. Give A Go To MongoDB’s Replica Set In 60 Seconds! #ToBeTriedOnlyAtHome
- 6. Step 1: Setup Replica Set $: mongo --nodb $: myReplica = new ReplSetTest({nodes: 3}); $: myReplica.startSet(); $: myReplica.initiate();
- 7. Congratulations! 3 member of replica set running on ports 31000, 31001 and 31002
- 8. Step 2: Connect To Primary $: mongo --port 31000 testReplSet:PRIMARY> use surryhills_db testReplSet:PRIMARY> db.offices.insert({name: ‘MongoDB’}); testReplSet:PRIMARY> show dbs testReplSet:PRIMARY> show collections testReplSet:PRIMARY> db.offices.find().pretty()
- 9. Pro Tip :] Auto Format Query Response Add DBQuery.prototype._prettyShell = true; to your ~/.mongorc.js
- 10. Step 3: Connect To A Secondary $: mongo --port 31001 testReplSet:SECONDARY> db.slaveOk() testReplSet:SECONDARY> show dbs testReplSet:SECONDARY> use surryhills_db testReplSet:SECONDARY> show collections testReplSet:SECONDARY> db.offices.find()
- 11. Step 4: Shutdown Replica Set $: myReplica.stopSet()
- 12. Setup Your Replica Set Part 1: Run replica members $: mongod --port 31000 --fork --logpath ./path/to/log --dbpath /path/to/srv1 --replSet myReplica $: mongod --port 31001 --fork --logpath ./path/to/log --dbpath /path/to/srv2 --replSet myReplica $: mongod --port 31002 --fork --logpath ./path/to/log --dbpath /path/to/srv3 --replSet myReplica
- 13. Setup Your Replica Set Part 2: Configure the set $: mongo --port 31000 $: rs.initiate(); $: rs.add(‘my-host:31001’); $: rs.add(‘my-host:31002’);
- 14. Setup Your Replica Set Part 2: Example Configuration $: rs.config(); { "_id" : "myReplica", "version": 3, "members" : [ { "_id" : 0, "host" : "my-host:31000" }, { "_id" : 1, "host" : "my-host:31001" }, { "_id" : 2, "host" : "my-host:31002" } ] }
- 15. Auto Connect To Current Primary $: mongo --host myReplica/my-host:31000,my-host:31001,my-host:31002
- 16. Replica Set Good To Knows 1. Always start your instance with `replSet` 2. You can’t initiate a replica set with more than one member with existing data on it. 3. If one of set members has data, you must run rs.initiate() against that member 4. `system.replset` collection on `local` database of all members contains set configuration. 5. Replica sets can have up to 50 members, 7 voting 6. By default, MongoDB reads from primary
- 17. Replica Set Reconfiguration Add new member $: mongo --port 31000 # current primary $: rs.add(‘my-host:31003’) # accepts object as well.
- 18. Replica Set Reconfiguration Remove existing member $: mongo --port 31000 # current primary $: rs.remove(‘my-host:31003’)
- 19. Replica Set Reconfiguration Update existing member $: mongo --port 31000 # current primary $: config = rs.config() $: config.members[2].priority = 0; $: rs.reconfig(config);
- 20. How Replication Works MongoDB keeps a log of every write operations that primary does in database Logs are saved in “oplog.rs” collection of “local” db Secondaries apply changes by reading from oplog.rs and applieing to their data Secondaries maintain their own “oplog.rs” as well.
- 21. “oplog.rs” Facts “oplog.rs” is a capped collection which means it can save limited amount of operation logs. Bulk operations fill “oplog” quickly. A bulk removal of 1K documents adds 1K documents to “oplog” It’s important to have a “oplog.rs” to keep logs for atleast last 24 hrs.
- 22. “Stale” Secondary :( Secondary misses a copy of last operation it applied in syncing source. Secondary queries other members to find another source to continue syncing. Secondary missed some operations permanently. Admin must resyncs from scratch.
- 23. Heartbeat Members PING each other every 2 seconds. Members should reply within 10 seconds otherwise they’re counted as unreachable.
- 24. Change in replica set configuration Current primary steps down Replica Set Election When It Happens? Current primary is unreachable
- 25. Majority The Single Most Important Fact About Replica Set More than half of All Voting Members in the set must be reachable.
- 26. You need a majority of voting members to elect a primary. A primary can only stay primary so long as it can reach a majority of voting members. Majority
- 27. Replica Set with two members. Three Common Replica Set Design Mistakes :( Even number of members in two data center. A set with more than one arbiter
- 28. Keep a majority of members in preferred data center and rest in the other one. Two Common Replica Set Design Patterns :) Keep even number of members in two data center and a tie breaker in third one.
- 29. Meet Arbiter Budget Friendly Member Of Replica Set Does not hold data It’s a lightweight process Can vote on elections
- 30. Start An Arbiter Process $: mongod --port 31005 --replSet myReplica -- nojournal --smallfiles Arbiter does not save data, no need for journaling Don’t waste disk space and use --smallfiles
- 31. Add An Arbiter To Replica Set $: rs.addArb(‘arbiter-host-name:port’);
- 32. Arbiter Good To Know You can’t change an arbiter to non-arbiter member If possible, use normal members instead of arbiters At most you need one arbiter in your set Do not add an arbiter to a set with odd members
- 33. Replica Set Useful Commands $: rs.config() $: rs.reconfig() $: rs.isMaster() $: rs.status() $: rs.stepDown()
- 34. Rollback :( Primary does a write and goes down before at least one secondary has a chance to replicate the write.
- 35. Rollback Main Reasons Network Partitions Different write rates between primary and secondary
- 36. Rollback Process 1. Former primary reverts write operations 2. Saves affected documents into /rollback directory 3. Admin browses documents using mongorestore 4. Admin applies changes to the current primary
- 37. Durability Write Concerns And Replica Set 1. Unacknowledged 2. Acknowledged 3. Journaled 4. Replica Acknowledged 5. Custom Writer Concern
- 38. Improve Durability db.runCommand({getLastError:1}) options Returns the error status of the preceding write operation on the current connection. j: boolean w: number | majority | custom role wtimeout: number (milliseconds) fsync: boolean
- 39. “wtimeout” GOOD TO KNOW MongoDB does not “rollback” or ” undo modifications” made before the “wtimeout” interval expired.
- 40. Custom Write Concerns Step 1: Tags $: config = rs.config() $: config.members[0].tags = {dc: ‘dc-1’}; $: config.members[3].tags = {dc: ‘dc-2’}; $: rs.reconfig(config);
- 41. Custom Write Concerns Step 2: Custom Last Error Mode $: config = rs.config() $: config.settings.lastErrorModes = [{ “each-dc”: {dc: 2} }]; $: rs.reconfig(config);
- 42. Custom Write Concerns Step 3: Use it! $: db.runCommand({getLastError:1, w: ‘each-dc’})
- 43. Replica Set & Mixed Storage Engines You can have both WiredTiger & MMAPv1 ● Due to migration ● Due to different use cases
- 44. Keen To Go Further? Register for M102: MONGODB FOR DBAS course! Starts tomorrow! It’s awesome! https://university.mongodb.com/courses/M102/about
- 45. Keen To Go Further? docs.mongodb.org/manual/replication MongoDB The Definite Guide
- 46. Stay In Touch ● Follow me on Twitter @mehdivk ● Add me to your Linkedin network https://au. linkedin.com/in/valikhani ● Subscribe to my blog blog.mehdivk.net ● Send me an email [email protected]
- 47. Thank You!