Taming Multi-Cloud, Hybrid Cloud, Docker, and Kubernetes
1 like237 views
This document discusses best practices for optimizing log management in modern, complex IT environments. It notes the rise of technologies like microservices, containers, and serverless computing have greatly increased infrastructure complexity. It recommends taking a unified approach to log management, monitoring both logs and metrics across bare metal, virtual machines, Kubernetes, and serverless platforms. This will help increase visibility, simplify instrumentation for developers, and provide a basis for continuous compliance.
Taming Multi-Cloud, Hybrid Cloud, Docker, and Kubernetes
- 1. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Taming Multi-Cloud, Hybrid Cloud, Docker, and Kubernetes: Five Best Practices to Optimize Log Management for DevOps, SREs, and IT Operations EMA Torsten Volk (Research Director) Twitter: @torstenvolk SolarWinds Bradley Sisson (Technical Account Manager) Email: [email protected]
- 2. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Agenda Slide 2 © 2019 Enterprise Management Associates, Inc. Taming Multi-Cloud, Hybrid Cloud, Docker, and Kubernetes: Five Best Practices to Optimize Log Management for DevOps, SREs, and IT Operations Agenda • Five critical questions to ask log analytics vendors • Balance log analytics requirements between traditional and modern apps • Impact of log analytics on security and compliance • Increase DevOps efficiency and lower operational risk
- 3. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Unified Monitoring of Data Center and Public Cloud Rapid increase in application complexity between 2016 and today Slide 3 © 2019 Enterprise Management Associates, Inc. 335 714 685 883 1,908 5,214 (+173%) 5,018 (-4%) 6,359 (+27%) October 2019October 2018October 2017October 2016 Each dot represents a category of developer questions on stackoverflow.com (on Nov 15, 2019)
- 4. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Unified Monitoring of Data Center and Public Cloud 62% increase in AWS complexity Slide 4 © 2019 Enterprise Management Associates, Inc. 2018 2019 YTD 12,957 issue categories 20,950 issue categories (+62%) Data source: stackoverflow.com
- 5. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Unified Monitoring of Data Center and Public Cloud Slide 5 © 2019 Enterprise Management Associates, Inc. Top 10 fastest growing public cloud services between January 2018 and today The rise of DevOps and continuous delivery brings further complexity Data source: devopshelp.com
- 6. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Unified Monitoring of Data Center and Public Cloud Slide 6 © 2019 Enterprise Management Associates, Inc. 335 714 (+113%) 685 (-4%) 883 (+29%) October 2019October 2018October 2017October 2016 Application complexity directly translates into infrastructure complexity Data source: serverfault.com
- 7. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Monitor Bare Metal, VMs, Kubernetes, and Serverless Slide 7 © 2019 Enterprise Management Associates, Inc. 36 different databases 15 networking options 16 storage platforms 22 automation platforms 12 container runtimes 5 container registries Data source: CNCF dataset
- 8. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Monitor Bare Metal, VMs, Kubernetes, and Serverless Slide 8 © 2019 Enterprise Management Associates, Inc. And 35 application management technologies… … and • 12 API gateways • 25 service mesh platforms • 21 Kubernetes distributions • And counting Data source: CNCF dataset
- 9. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Monitor Bare Metal, VMs, Kubernetes, and Serverless Slide 9 © 2019 Enterprise Management Associates, Inc. Serverless technologies grew 1,200% since 2016 Data source: stackoverflow.com
- 10. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Monitor Bare Metal, VMs, Kubernetes, and Serverless Slide 10 © 2019 Enterprise Management Associates, Inc. Serverless added over 1,000 more problem categories Data source: StackOverflow.com
- 11. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Jointly Monitor Logs and Metrics Slide 11 © 2019 Enterprise Management Associates, Inc. Issues related to monitoring logs and metrics doubled since Q1, 2018. Logs and metrics are often monitored separately Data source: stackoverflow.com
- 12. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Simplify Instrumentation for Developers Consistency is critical for code instrumentation Recent progress: OpenTracing and OpenCensus merged into OpenTelemetry Slide 12 © 2019 Enterprise Management Associates, Inc. Source: stackoverflow.com
- 13. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Jointly Monitor Metrics and Logs Slide 13 © 2019 Enterprise Management Associates, Inc. Metrics Logging Data source: github.com
- 14. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Guidance for Operators Rapid growth of microservices (orange) and serverless applications (red) Result: Logs and metrics becoming a bigger challenge Slide 14 © 2019 Enterprise Management Associates, Inc. Source: stackoverflow.com
- 15. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Guidance for Operators Slide 15 © 2019 Enterprise Management Associates, Inc. Source: stackoverflow.com Microservices have added tremendous operational complexity
- 16. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Summary Slide 16 © 2019 Enterprise Management Associates, Inc. • Simplicity is key • Eliminate all silos • Applications and infrastructure are becoming one • Unified logs, metrics, and traces are the basis for continuous compliance • No excuses