Improve monitoring and observability for kubernetes with oss tools
0 likes87 views
Slide deck from the Azure Community Conference (https://azconf.dev/) presented on 29th October 2021. The session covered following topics - Need for centralized logging - Using ElasticSearch, Fluentd and Kibana (EFK) with Kubernetes - Need for monitoring - Using Prometheus & Grafana for infrastructure, application and third party services - Integration of application with Sentry for Exception aggregation
Ad
More Related Content
What's hot (20)
Similar to Improve monitoring and observability for kubernetes with oss tools (20)
Ad
More from Nilesh Gule (20)
Ad
Recently uploaded (20)
Improve monitoring and observability for kubernetes with oss tools
- 1. Azure Community Conference 2021 Indiaâs largest Azure Conference #AzConfDev
- 2. #AzConfDev Title Sponsor Gold Sponsor Community Partner
- 7. #AzConfDev Pre-requisites Self contained application with all its dependencies Docker â Orchestrates containers â Self healing â Service discovery â Scaling Kubernetes â Scalable apps in dynamic environments (public / private / hybrid clouds) â Exemplified by Containers, service meshes, microservices, immutable infrastructure & declarative APIs â Loosely coupled systems, resilient, observable & manageable â Robust automation Cloud Native Applications
- 8. #AzConfDev
- 11. #AzConfDev â Application specific â Long term log retention for compliance reasons â Workloads scheduled on different nodes during application restarts / updates â Autoscaling workloads â Kubernetes upgrades â Auto healing can reschedule workloads â Underlying nodes added / deleted during cluster scaling â Underlying nodes replaced during cluster upgrades Container based workloads Why centralized logging â Not much control over underlying infra â Relies on cloud prover specific logging and monitoring solution PaaS / Serverless services
- 12. #AzConfDev Tech Talks EFK integration Log collector Log storage Log search, visualise, dashboards rabbitmq-producer-service rabbitmq-consumer-deployment
- 13. #AzConfDev Demo 1 â Log Aggregation with EFK
- 15. #AzConfDev âĒ Application specific âĒ Monitor resource usage âĒ Monitor scaling needs âĒ Monitor anomalies / outliers âĒ Kubernetes platform level âĒ Monitor cluster resources (CPU / RAM) âĒ API health âĒ Autoscaling Container based workloads Why Monitoring & Alerting âĒ Monitor resource usage âĒ Scaling âĒ Bottlenecks PaaS / Serverless services
- 17. #AzConfDev Demo 2 â Metrics using Prometheus & Grafana
- 18. #AzConfDev Spring Boot Conference App integration https://github.com/NileshGule/spring-boot-conference-app/tree/mssql-server conference-demo-service-monitor conference-demo-service
- 20. #AzConfDev Sentry Architecture https://develop.sentry.dev/architecture/
- 21. #AzConfDev Spring Boot Sentry integration conference-demo-service Managed Kubernetes cluster
- 22. #AzConfDev Demo 3 â Exception aggregation using Sentry
- 24. #AzConfDev End to End Observability
- 25. #AzConfDev Summary Modern day cloud native applications need new ways to address observability & monitoring Log Aggregation â EFK stack helps in centralized logging â Kibana is used to visualize logs and build dashboards Monitoring & Alerting â Prometheus provides easy to use metrics for platforms, applications â Grafana provides visualization capabilities to build intuitive dashboards Exception Aggregation â Sentry provides Exception Aggregation capabilities â Excellent telemetry data captured by Sentry to help diagnose problems
- 26. #AzConfDev References Log Aggregation â Elastic stack â Kibana â Fluentd Monitoring & Alerting â Prometheus â Grafana â Kube Prometheus stack â Houssem Dellai â Prometheus & Grafana for monitoring Kubernetes Sentry â Sentry docs
- 27. #AzConfDev Source Code & slide deck Tech Talks https://github.com/NileshGule/pd-tech-fest-2019 Conference app https://github.com/NileshGule/spring-boot-conference-app/tree/mssql-server https://speakerdeck.com/nileshgule/ https://www.slideshare.net/nileshgule/
- 28. Nilesh Gule ARCHITECT | MICROSOFT MVP âCode with Passion and Strive for Excellenceâ nileshgule @nileshgule Nilesh Gule NileshGule www.handsonarchitect.com https://bit.ly/youtube-nileshgule
- 29. #AzConfDev Q & A