MongoDB.local Atlanta: Modern Data Backup and Recovery from On-Premises to the Public Cloud
Download as PPTX, PDF1 like270 views
This document summarizes MongoDB's modern data backup and recovery strategies, both for on-premises and cloud environments. It outlines MongoDB's current on-premises backup architecture and its limitations. The document then introduces MongoDB's new backup strategy of taking WiredTiger checkpoints directly to long-term storage to reduce overhead. It also discusses MongoDB Atlas, which provides automatic multi-region backups and point-in-time restore capabilities. The document demos these backup features and previews future improvements like direct agent backups to optimize the process.
Ad
More Related Content
What's hot (20)
Similar to MongoDB.local Atlanta: Modern Data Backup and Recovery from On-Premises to the Public Cloud (20)
Ad
More from MongoDB (20)
![MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]](https://cdn.slidesharecdn.com/ss_thumbnails/315pminfosysfinalsfoversionvocalpart1-200120221508-thumbnail.jpg?width=560&fit=bounds)
Ad
Recently uploaded (20)
MongoDB.local Atlanta: Modern Data Backup and Recovery from On-Premises to the Public Cloud
- 2. Modern Data Backup and Recovery from On-Premises to the Public Cloud Benjamin Cefalo Senior Product Manager, Cloud
- 3. Benjamin Cefalo Senior Product Manager, Cloud MongoDB [email protected] @bencefalo https://github.com/bencefalo
- 4. This presentation contains “forward-looking statements” within the meaning of Section 27A of the Securities Act of 1933, as amended, and Section 21E of the Securities Exchange Act of 1934, as amended. Such forward- looking statements are subject to a number of risks, uncertainties, assumptions and other factors that could cause actual results and the timing of certain events to differ materially from future results expressed or implied by the forward-looking statements. Factors that could cause or contribute to such differences include, but are not limited to, those identified our filings with the Securities and Exchange Commission. You should not rely upon forward-looking statements as predictions of future events. Furthermore, such forward-looking statements speak only as of the date of this presentation. In particular, the development, release, and timing of any features or functionality described for MongoDB products remains at MongoDB’s sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality. Except as required by law, we undertake no obligation to update any forward-looking statements to reflect events or circumstances after the date of such statements. Safe Harbor Statement
- 5. Quick level set — We’re not going to talk about disaster recovery
- 7. Application Driver Primary Secondary Secondary Replication But what happens when you have data corruption in your primary or someone accidentally does something they shouldn’t have... ?
- 10. Data Recovery - That’s what we’re going to focus on today
- 11. Private DBaaS: On-Prem Public DBaaS: Fully Managed Built on the Same Code Base, Same API, Same Management UI MongoDB Offerings Hybrid DBaaS
- 12. The Future of Backup for Self Managing or On- Premises
- 13. On-Premises Backup Architecture Today Ops Manager OM01 OM02 OM03 OM04 OM05 OM06
- 14. On-Premises Backup Architecture Today Ops Manager OM01 OM02 OM03 OM04 OM05 OM06 Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent OM Group
- 15. On-Premises Backup Architecture Today Ops Manager OM01 OM02 OM03 OM04 OM05 OM06 Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent OM Group BackupLoadBalancer App Server for Backup App Server for Backup App Server for Backup
- 16. On-Premises Backup Architecture Today Ops Manager OM01 OM02 OM03 OM04 OM05 OM06 Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent OM Group BackupLoadBalancer App Server for Backup App Server for Backup App Server for Backup Backup Daemon HeadDB Backup Daemon HeadDB Backup Daemon HeadDB
- 17. ... ... Blockstore06 Blockstore04 Blockstore02 On-Premises Backup Architecture Today Ops Manager OM01 OM02 OM03 OM04 OM05 OM06 Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent OM Group BackupLoadBalancer App Server for Backup App Server for Backup App Server for Backup Backup Daemon HeadDB Backup Daemon HeadDB Backup Daemon HeadDB Blockstore01 Blockstore03 Blockstore05
- 18. ... ... ... ... Oplog04 Oplog02 Blockstore06 Blockstore04 Blockstore02 Oplog06 On-Premises Backup Architecture Today Ops Manager OM01 OM02 OM03 OM04 OM05 OM06 Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent OM Group BackupLoadBalancer App Server for Backup App Server for Backup App Server for Backup Oplog01 Oplog03 Oplog05 Backup Daemon HeadDB Backup Daemon HeadDB Backup Daemon HeadDB Blockstore01 Blockstore03 Blockstore05
- 19. Backup Strategy ● Reduce the footprint and architectural complexities needed for backup ● Reduce cost ● Improve performance
- 21. What is it? ● Enhanced WiredTiger to take checkpoints of itself ● Checkpoints are then moved to long term storage ● HeadDB’s are completely eliminated — no more initial syncs! ○ Reduced storage requirements ○ Reduces architectural complexities ○ Reduces infrastructure required ● Consolidated agents, from three agents to one
- 22. ... ... ... ... Oplog04 Oplog02 Blockstore06 Blockstore04 Blockstore02 Oplog06 On-Premises Backup Architecture Today Ops Manager OM01 OM02 OM03 OM04 OM05 OM06 Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent Backup Agent Monitoring Agent OM Group BackupLoadBalancer App Server for Backup App Server for Backup App Server for Backup Oplog01 Oplog03 Oplog05 Backup Daemon HeadDB Backup Daemon HeadDB Backup Daemon HeadDB Blockstore01 Blockstore03 Blockstore05
- 23. Oplog02 Blockstore New Backup Architecture — Phase 1 Ops Manager OM01 OM02 OM03 OM04 OM05 OM06 MongoDB Agent MongoDB Agent MongoDB Agent MongoDB Agent MongoDB Agent MongoDB Agent OM Group BackupLoadBalancer App Server/Backup Daemon App Server/Backup Daemon Oplog01 S3 Snapshot Store
- 24. S3 Oplog Blockstore New Backup Architecture — Phase 2 Ops Manager OM01 OM02 OM03 OM04 OM05 OM06 MongoDB Agent MongoDB Agent MongoDB Agent MongoDB Agent MongoDB Agent MongoDB Agent OM Group BackupLoadBalancer App Server for Backup App Server for Backup Oplog01 S3 Snapshot Store
- 25. Future Improvements ● Agents read/write directly to OpLog and snapshots store
- 26. S3 Oplog New Backup Architecture — Future Ops Manager OM01 OM02 OM03 OM04 OM05 OM06 MongoDB Agent MongoDB Agent MongoDB Agent MongoDB Agent MongoDB Agent MongoDB Agent OM Group S3 Snapshot Store
- 27. Future Improvements ● Agents read/write directly to OpLog and snapshots store ● Leave a checkpoint behind
- 28. What this means for you... ● Less people, time, and energy you have to spend to deliver a best-in- class data recovery solution for your on-premises and self-managed MongoDB deployments
- 29. Atlas
- 30. Private DBaaS: On-Prem Public DBaaS: Fully Managed Built on the Same Code Base, Same API, Same Management UI Atlas: Same Features as Ops Manager & Cloud Manager Hybrid DBaaS
- 31. The Latest MongoDB Features • MongoDB Atlas comes out-of-the-box with MongoDB 3.4, 3.6, 4.0 • Transactions (4.0) • Change Streams (3.6) • JSON Schema (3.6) • Expressive nested array updates (3.6) • Expressive joins: $lookup (3.6) • Graph queries (3.4) • Facets & expressive aggregations (3.4) • Minor updates and major upgrades without downtime ● Data Explorer ○ Inspect schema & index utilization ● Real-Time Performance Panel ○ Live telemetry: in-flight operations & resource consumption ● Performance Advisor ○ Always-on index recommendations
- 32. Multi-AZ Disaster Recovery by Default AZ IIAZ I AZ III
- 33. ...with Optional Multi-Region Fault Tolerance AZ IIAZ I AZ III Preferred Region AZ I AZ II AZ I AZ II Failover Region Failover Region
- 34. Atlas Backups: Data Recovery Roll back the clock when you run into issues triggered by user or application errors that are replicated from the primary to the rest of your cluster.
- 35. Point-in-Time Data Recovery • Lets you select a restore time based on your PIT window • Restores the closest snapshot and rolls ahead • Reduces the possibility of data loss
- 36. What About Small Disasters? • The application is working fine • But there is data missing or has been altered • No time to do a full restore
- 37. Queryable Backups • Ability to query your snapshots and restore data at the document level in minutes. • Reduces the operational overhead associated with: • Identifying whether data of interest has been altered • Pinpointing the best point in time to restore a database
- 38. Sample Queryable Script db = source.locations db2 = destination.locations zips = db.zipcodes zips2 = db2.zipcodes def restore(): print "Finding Missing Data" query = {'state': 'CO'} try: cursor = zips.find(query) except Exception as e: print "Unexpected error:", type(e), e for doc in cursor: zips2.insert(doc)
- 39. Demo
- 41. Cloud Provider Snapshots (CPS) • Utilizes each providers native snapshot capabilities • Granular backup region selection • Satisfy data sovereignty requirements • Supports replica sets and sharded clusters • Pricing is based on snapshot size, not datasize • Less expensive, starting at $0.14 per GB of snapshot size • Available on AWS and Azure • AWS snapshots are incremental 10 GB 15 GB (5 new) 20 GB (5 new) Incremental snapshots for Atlas customers deploying on AWS
- 42. Cloud Provider Snapshots (CPS) Updates — Continued • Bring your own keys • AWS Key Management Service • Azure Key Vault • Configure snapshot time and retention • Choose when you want to take a snapshot • Choose how many snapshots you want to retain • On-demand snapshots (quicksave) • Takes a snapshot immediately if there’s not already one in progress • API for pipeline integrations
- 45. Demo
- 46. Whats Next?
- 47. Cloud Provider Snapshots Future ● New Modernized Scheduler ● Launch on GCP ○ GCP KMS (Bring your own Keys) ● Point in Time Restores with customizable window
- 48. Pick the fully managed backup solution that’s right for you Within Atlas is a modern data recovery platform that enables you to choose how you want to backup your data, depending on your requirements. Continuous ● Point-in-time restore ○ All but eliminate the possibility of data loss ● Queryable snapshots ○ Rapidly restore granular data Cloud Provider Snapshots ● Localized backup ○ Satisfy data sovereignty requirements ● Bring your own keys ● Take snapshots on-demand ● The cost effective option
- 49. Thank You!