Ad
More Related Content
What's hot (20)
Viewers also liked (9)
Ad
Similar to DevOpsDays SLC - Getting Along With Your DBOps Team (20)
Ad
Recently uploaded (20)
DevOpsDays SLC - Getting Along With Your DBOps Team
- 1. Endurance International Group Getting Along with Your DBOps Team SLC DevOpsDays @nickdemaster
- 2. Why do we care?
- 3. Why do we care? Your data is super important! @nickdemaster #slcdevopsdays
- 4. What does the DB Ops team care about? (and how does that apply to dev)
- 6. ACID • Atomicity • Atomicity requires that each transaction be "all or nothing": if one part of the transaction fails, then the entire transaction fails, and the database state is left unchanged. An atomic system must guarantee atomicity in each and every situation, including power failures, errors, and crashes. To the outside world, a committed transaction appears (by its effects on the database) to be indivisible ("atomic"), and an aborted transaction does not happen. @nickdemaster #slcdevopsdays
- 7. ACID DevOps Translation: Each code deployment involving a database must be successful and be progressive, not regressive. In the event of a failure in the code deployment, the rollback must be transparent to the user. @nickdemaster #slcdevopsdays • Atomicity • Atomicity requires that each transaction be "all or nothing": if one part of the transaction fails, then the entire transaction fails, and the database state is left unchanged. An atomic system must guarantee atomicity in each and every situation, including power failures, errors, and crashes. To the outside world, a committed transaction appears (by its effects on the database) to be indivisible ("atomic"), and an aborted transaction does not happen.
- 8. ACID • Atomicity • Consistency @nickdemaster #slcdevopsdays
- 9. • Consistency • The consistency property ensures that any transaction will bring the database from one valid state to another. Any data written to the database must be valid according to all defined rules, including constraints, cascades, triggers, and any combination thereof. This does not guarantee correctness of the transaction in all ways the application programmer might have wanted (that is the responsibility of application-level code) but merely that any programming errors cannot result in the violation of any defined rules. @nickdemaster #slcdevopsdays ACID
- 10. • Consistency • The consistency property ensures that any transaction will bring the database from one valid state to another. Any data written to the database must be valid according to all defined rules, including constraints, cascades, triggers, and any combination thereof. This does not guarantee correctness of the transaction in all ways the application programmer might have wanted (that is the responsibility of application-level code) but merely that any programming errors cannot result in the violation of any defined rules. DevOps Translation: Understand the underlying data you are changing with each code deploy. Make sure that the data structures you use or change are consistent with the ones currently in place. Keep data types for similar data the same. i.e. If you use database datestamps, don’t switch over to unix timestamps, unless part of a larger initiative. @nickdemaster #slcdevopsdays ACID
- 11. • Atomicity • Consistency • Isolation ACID @nickdemaster #slcdevopsdays
- 12. ACID • Isolation • The isolation property ensures that the concurrent execution of transactions results in a system state that would be obtained if transactions were executed serially, i.e., one after the other. Providing isolation is the main goal of concurrency control. Depending on the concurrency control method (i.e., if it uses strict - as opposed to relaxed - serializability), the effects of an incomplete transaction might not even be visible to another transaction. @nickdemaster #slcdevopsdays
- 13. DevOps Translation: When doing code deploys, always be on the same page with the way the data is going to be changed with each deploy. This is most pertinent if you have multiple projects or initiatives going on that use the same database model. i.e. If you need a column for X and someone else needs the same column for Y, be sure that you don’t crush someone else’s (or your own) changes with an out of order alter/CRUD action. @nickdemaster #slcdevopsdays ACID • Isolation • The isolation property ensures that the concurrent execution of transactions results in a system state that would be obtained if transactions were executed serially, i.e., one after the other. Providing isolation is the main goal of concurrency control. Depending on the concurrency control method (i.e., if it uses strict - as opposed to relaxed - serializability), the effects of an incomplete transaction might not even be visible to another transaction.
- 14. • Atomicity • Consistency • Isolation • Durability @nickdemaster #slcdevopsdays ACID
- 15. • Durability • The durability property ensures that once a transaction has been committed, it will remain so, even in the event of power loss, crashes, or errors. In a relational database, for instance, once a group of SQL statements execute, the results need to be stored permanently (even if the database crashes immediately thereafter). To defend against power loss, transactions (or their effects) must be recorded in a non-volatile memory. @nickdemaster #slcdevopsdays ACID
- 16. DevOps Translation: Understand the mechanics of the durability property of the database. If you are not using a truly ACID database, you may have to program in checks to assure the data is stored and retrievable correctly – even in the event of a connection or system failure. @nickdemaster #slcdevopsdays ACID • Durability • The durability property ensures that once a transaction has been committed, it will remain so, even in the event of power loss, crashes, or errors. In a relational database, for instance, once a group of SQL statements execute, the results need to be stored permanently (even if the database crashes immediately thereafter). To defend against power loss, transactions (or their effects) must be recorded in a non-volatile memory.
- 17. In Practice – Schema
- 18. @nickdemaster #slcdevopsdays In Practice – Database Schema • Know your data! (love your data)
- 19. @nickdemaster #slcdevopsdays In Practice – Database Schema • Know your data! (love your data) • If your database uses them, Explicit Primary Keys
- 20. @nickdemaster #slcdevopsdays In Practice – Database Schema • Know your data! (love your data) • If your database uses them, Explicit Primary Keys • Correctly sized data fields
- 21. @nickdemaster #slcdevopsdays In Practice – Database Schema • Know your data! (love your data) • If your database uses them, Explicit Primary Keys • Correctly sized data fields • Be able to forecast growth patterns
- 22. In Practice – Database Schema @nickdemaster #slcdevopsdays • Know your data! (love your data) • If your database uses them, Explicit Primary Keys • Correctly sized data fields • Be able to reasonably forecast growth • ConSistentNaming Conventions
- 23. @nickdemaster #slcdevopsdays In Practice – Database Schema • Know your data! (love your data) • If your database uses them, Explicit Primary Keys • Correctly sized data fields • Be able to reasonably forecast growth • ConSistentNaming Conventions • Understand storage conventions of the database engine (i.e. ACID vs eventual consistency)
- 24. • Know your data! (love your data) • If your database uses them, Explicit Primary Keys • Correctly sized data fields • Be able to reasonably forecast growth • ConSistentNaming Conventions • Understand storage conventions of the database engine (i.e. ACID vs eventual consistency) • Bonus: example queries with explain paths @nickdemaster #slcdevopsdays In Practice – Database Schema
- 25. In Practice – DBA
- 26. @nickdemaster #slcdevopsdays In Practice – DBA • Continuous deployment of code, have a system/agreement in place
- 27. @nickdemaster #slcdevopsdays In Practice – DBA • Continuous deployment of code, have a system/agreement in place • Audit/Change Management (and if possible roll-forward, roll back)
- 28. @nickdemaster #slcdevopsdays In Practice – DBA • Continuous deployment of code, have a system/agreement in place • Audit/Change Management (and if possible roll-forward, roll back) • Data Migrations plans – cause and effect analysis (application/system)
- 29. @nickdemaster #slcdevopsdays In Practice – DBA • Continuous deployment of code, have a system/agreement in place • Audit/Change Management (and if possible roll-forward, roll back) • Data Migrations plans – cause and effect analysis (application/system) • Rapid development infrastructure (development > staging > production)
- 30. • Continuous deployment of code, have a system/agreement in place • Audit/Change Management (and if possible roll-forward, roll back) • Data Migrations plans – cause and effect analysis (application/system) • Rapid development infrastructure (development > staging > production) • Backups and restores – TEST OFTEN @nickdemaster #slcdevopsdays In Practice – DBA
- 31. Getting Along with your DB Ops Team
- 32. @nickdemaster #slcdevopsdays Getting Along with your DB Ops Team • Buy into the same goals
- 33. @nickdemaster #slcdevopsdays Getting Along with your DB Ops Team • Buy into the same goals • Communicate early and often
- 34. @nickdemaster #slcdevopsdays Getting Along with your DB Ops Team • Buy into the same goals • Communicate early and often • Be ready to compromise
- 35. @nickdemaster #slcdevopsdays Getting Along with your DB Ops Team • Buy into the same goals • Communicate early and often • Be ready to compromise • Be able to define stakeholders for each data release
- 36. @nickdemaster #slcdevopsdays Getting Along with your DB Ops Team • Buy into the same goals • Communicate early and often • Be ready to compromise • Be able to define stakeholders for each data release • Be accountable (bonus: be on pager duty)
- 37. • Buy into the same goals • Communicate early and often • Be ready to compromise • Be able to define stakeholders for each data release • Be accountable (bonus: be on pager duty) • TEAM @nickdemaster #slcdevopsdays Getting Along with your DB Ops Team
- 38. I am the DB Ops team (and the dev team.. and the ops team… and the printer person...)
- 39. • Explain plans @nickdemaster #slcdevopsdays Troubleshooting tips
- 40. • Explain plans • Mongo: db.collection.find().explain() @nickdemaster #slcdevopsdays Troubleshooting tips
- 41. • Explain plans • Mongo: db.collection.find().explain() • MySQL/Postgres: EXPLAIN select ... @nickdemaster #slcdevopsdays Troubleshooting tips
- 42. Troubleshooting tips @nickdemaster #slcdevopsdays • Explain plans • Monitor Monitor Monitor (alert alert alert)
- 43. @nickdemaster #slcdevopsdays Troubleshooting tips • Explain plans • Monitor Monitor Monitor (alert alert alert) • Practice your deploys on a staging environment
- 44. • Explain plans • Monitor Monitor Monitor (alert alert alert) • Practice your deploys on a staging environment • Automation is key! @nickdemaster #slcdevopsdays Troubleshooting tips
- 45. Endurance International Group Getting Along with Your DBOps Team SLC DevOpsDays @nickdemaster
Editor's Notes
- #2: Two MySQL DBAs walk to a NoSQL bar, but they had to leave because they couldn't find any tables!
- #27: What does that mean for database changes Alter tables, schema changes?
- #28: What does that mean for database changes Alter tables, schema changes?
- #29: What does that mean for database changes Alter tables, schema changes?
- #30: What does that mean for database changes Alter tables, schema changes?
- #31: What does that mean for database changes Alter tables, schema changes?
- #46: Two MySQL DBAs walk to a NoSQL bar, but they had to leave because they couldn't find any tables!