Data Agility for Devops - OSI 2018
Download as PPTX, PDF2 likes245 views
About the Talk: Cloud native ecosystem is bringing a huge change in the way of DevOps in every cloud native organisation. Developers and operators in cloud native organisations are using tools and platforms like Kubernetes to achieve the agility promised by DevOps and microservices. The tools and best practices for stateless applications have been well established and the results can be seen in the agility of teams using these stateless applications. However, stateful applications pose new challenges to DevOps teams in achieving the agility as the best practices around persistent storage management are still emerging. In this talk, first we discuss the challenges faced by DevOps while dealing with persistent storage handling in stateful applications. Then we discuss the open source tools and best practices for DevOps teams to achieve data agility of cloud native applications.
Ad
More Related Content
What's hot (20)
Similar to Data Agility for Devops - OSI 2018 (20)
![[OpenInfra Days Vietnam 2019] Innovation with open sources and app modernizat...](https://cdn.slidesharecdn.com/ss_thumbnails/innovationwithopensourcesandappmodernizationfordevelopers-dist-190824044544-thumbnail.jpg?width=560&fit=bounds)
Ad
More from OpenEBS (20)
Ad
Recently uploaded (20)
Data Agility for Devops - OSI 2018
- 1. Cloud Native Storage & Data Agility for DevOps @uma_mukkara 12 October 2018
- 2. ● MayaData delivers Data Agility ● Sponsor of OpenEBS and Litmus projects ● Large Bangalore based OSS contributor ● https://mayaonline.io is SaaS software used to manage stateful workloads on Kubernetes Uma Mukkara Co-founder & COO
- 3. ● Popular open source storage project for stateful applications on Kubernetes ● Team of 50+ hackers with storage and DevOps experience ● Leading project in CAS category ● 250+ contributors ● 5K+ github stars ● 1M+ docker pulls slack.openebs.io
- 4. DevOps Micro- services Agility Cloud Native technologies and practices Docker Kubernetes Istio OpenEBS HelmPrometheus Adopt Enable Achieve
- 6. Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil. Definition from What is “Cloud Native” ?
- 7. Data Agility Challenges in Cloud Native Space
- 8. 1. HA within a cluster • Microservices architecture separates every component. A legacy application now becomes • A stateless application and/or • A stateful application with a database in it • Stateless applications are mobile across nodes in a cluster but Stateful applications need persistent data for easy movement Synchronous replication of cloud native data
- 9. 2. HA across clusters • multi-cloud is now a reality • Multi Cloud example • Code is in Github • Dev uses On-Prem • CI/CD in Google • Pre-Prod in Azure • Production in AWS Move data across clouds
- 10. 3. CI Pipelines • State of the data moves to CI pipelines • Developer needs access to the exact state of the data when a pipeline fails • Pipelines have to snapshot the data • Pipelines have to clone the data and automate the access to the developer Data snapshots in CI pipelines
- 11. 4. E2E Tests • For gaining credibility to push the stateful application into production, building an E2E test suite is a must • The stateful application that is containerized need to be tested for all state transitions and changes in the infrastructure • E2E matrix for stateful applications is large and complex App specific data resilience and availability
- 12. 5. Data Backup and Restore • The traditional Backup and Restore systems do NOT work in Cloud Native eco-systems • Irrespective of the reliable infrastructure designs that Cloud Native systems provide, DevOps still need to have Backup and Restore strategy • Converting the application into microservices architecture is the one challenge but finding and integrating/automating the Backup/Restore process is another Backup and DR for Cloud Native Data
- 13. Choose Cloud Native technologies to resolve Data Agility challenges
- 14. Cloud Native Tech for Data Agility * These are applicable when you are using Kubernetes as Orchestration platform Synchronous replication of cloud native data Move data across clusters, clouds Data snapshots in CI pipelines App specific data resilience and availability Backup and DR for Cloud Native Data Data Agility Challenge Choose any CAS like OpenEBS Choose any CAS like OpenEBS OpenEBS & Litmus Litmus Heptio ARK with OpenEBS Plugin Cloud Native Technology
- 15. Cloud Native Tech for Data Agility
- 16. What if storage for container native applications was itself container native? https://www.cncf.io/blog/2018/04/19/container-attached-storage-a-primer/ Blog White Paper https://openebs.io/assets/docs/WP-OpenEBS-0_7.pdf Container Attached Storage or CAS - OpenEBS
- 17. Container Attached Storage or CAS - OpenEBS Architecture
- 18. Container Attached Storage or CAS - OpenEBS
- 19. Chaos Engineering Framework - Litmus Litmus- Chaos Engineering Framework for Stateful applications on Kubernetes It is a K8S JOB K8S deployers Provider specific deployers Test cases (ansible playbooks)Framework contains Standard & customized Workload containers Tools to help execute tests (Chaos, logging etc)
- 20. Litmus - For CI Pipelines / E2E Chaos Engineering Framework - Litmus
- 21. Backup and Restore framework - Heptio ARK+OpenEBS Heptio ARK OpenEBS Plugin + Cloud Native incremental data backup
- 22. Backup and Restore framework - Heptio ARK+OpenEBS
- 24. DevOps Application agility Data agility Cloud Native tools and best practices (sufficient expertise is already in the public domain) Use CAS for data storage and data resiliency Build Snapshot/aware CI pipelines Build data aware E2E suites Choose Cloud Native Backup and DR tools
- 25. Data Agility Example DevOps Developer DevOps Architect Dev Cluster CI/CD Cluster Production Cluster 1.Develop Application YAML Volume: Name : Volume 1 Mount : /mnt/volume PVC : OpenEBS-PVC Persistent volumes are provisioned automatically for developer. Code is merged2. Code is merged 3. CI/CD Pipeline Build Stage Setup Stage Litmus CD 4. Deploy
- 26. Data Agility Example DevOps Developer DevOps Architect Dev Cluster CI/CD Cluster Production Cluster 1.Develop 2. Code is merged 3. CI/CD Pipeline Build Stage Setup Stage Litmus One test fails Snapshot Clone Access clone and debug
- 27. Data Agility Example DevOps Developer DevOps Architect Dev Cluster CI/CD Cluster Production Cluster 1.Develop 2. Code is merged 3. CI/CD Pipeline Build Stage Setup Stage Litmus CD 4. Deploy 5. Chaos Engineering using Litmus
- 28. Data Agility Example DevOps Developer DevOps Architect Dev Cluster CI/CD Cluster Production Cluster 1.Develop 2. Code is merged 3. CI/CD Pipeline Build Stage Setup Stage Litmus CD 4. Deploy 6. Backup Heptio Ark + OpenEBS
- 29. Data Agility . Delivered
- 30. QUESTIONS?
- 31. We are Hiring DevOps Engineers [email protected]