SlideShare a Scribd company logo
Kubernetes Logging
Implement and test self-managed logs
collection and visualization solution for
your k8s cluster
Real world requirements
●
Quick and easy deployment and testing
(do it like a DevOps)
●
Reliable enough
●
Maximum performance by default
●
Simple to understand and debug (KISS)
●
Reuse existing solutions as much as possible
Stack decisions and tools
●
Playground: Minikube/Microk8s/Kubeadm/Kops
●
FluentD/Fluent Bit/Logstash/Telegraf
●
EL(F)K/TICK (InfluxDB) stacks, Graphana?
●
Using k8s operator, reliability concerns
●
Helm/GitKube/Kapitan
●
Ingress controller (Nginx/ALB)
Kubernetes on localhost
Minikube
●
Needs a VM
●
Needs special
treatment
●
Good for developing
services, not
operations testing
Microk8s
●
Super simple
●
Real cluster
●
Docker (in Docker)
●
Too much magic
●
We need Cloud for UI
Load balancing and
ingress
Kubernetes in the Cloud
Kubeadm
●
Not really for cloud
●
Needs manual job
●
No mature enough
automations available
Kops
●
Easy, scalable
●
AWS native
●
Cluster-as-a-Code
●
Addons
●
Generates Terraform!
●
Actively developed
Logs collector and processor
●
Logstash: heavy, do we need all its features?
●
Telegraf (InfluxDB): looks cool, where is k8s
support?
●
FluentD: good stuff, recommended upstream,
Kops plugin
●
Fluent Bit: lightweight and fast, native support for
Kubernetes, minimal configuration
EFK stack for logs visualisation
●
ElasticSearch: deploy real cluster, rotate
indexes, resize, backup, kubernetes operator is
the only option
●
Fluent Bit: read all container logs, enrich with
kubernetes metadata and system metrics, does
buffering and retries, deployed via DaemonSet
●
Kibana: discover the index in ES, configure
timestamp field, watch and query.
TODO: create some pretty dashboards
Turn-key deployment
●
GitKube looks nice…
●
Kapitan is more Pythonic
●
Helm appears to be widely adopted, many
examples available
●
Install all stack components from single chart
●
Helm is great about vendorizing dependencies
●
GitHub repo could become Helm repo!
●
TODO: enable ingress rule
Wanna try?
https://github.com/vutny/kube-efk
Concerns and future improvements
●
Authentication, sensitive data, ES hardening,
Cerebro, capasity, proper TLS setup
●
“IFG” stack: InfluxDB, FluentD, Grafana
Thank you.
Questions?
●
https://github.com/vutny
●
https://www.linkedin.com/in/denyshavrysh/
Ad

More Related Content

What's hot (20)

Serverless stream processing of Debezium data change events with Knative | De...
Serverless stream processing of Debezium data change events with Knative | De...Serverless stream processing of Debezium data change events with Knative | De...
Serverless stream processing of Debezium data change events with Knative | De...
Red Hat Developers
 
Multi-cloud Kubernetes BCDR with Velero
Multi-cloud Kubernetes BCDR with VeleroMulti-cloud Kubernetes BCDR with Velero
Multi-cloud Kubernetes BCDR with Velero
Kublr
 
K8s Pod Scheduling - Deep Dive. By Tsahi Duek.
K8s Pod Scheduling - Deep Dive. By Tsahi Duek.K8s Pod Scheduling - Deep Dive. By Tsahi Duek.
K8s Pod Scheduling - Deep Dive. By Tsahi Duek.
Cloud Native Day Tel Aviv
 
Kubernetes Deployments: A "Hands-off" Approach
Kubernetes Deployments: A "Hands-off" ApproachKubernetes Deployments: A "Hands-off" Approach
Kubernetes Deployments: A "Hands-off" Approach
Rodrigo Reis
 
"Using Automation Tools To Deploy And Operate Applications In Real World Scen...
"Using Automation Tools To Deploy And Operate Applications In Real World Scen..."Using Automation Tools To Deploy And Operate Applications In Real World Scen...
"Using Automation Tools To Deploy And Operate Applications In Real World Scen...
ConSol Consulting & Solutions Software GmbH
 
Monitoring Cockpit for OpenShift Clusters
Monitoring Cockpit for OpenShift ClustersMonitoring Cockpit for OpenShift Clusters
Monitoring Cockpit for OpenShift Clusters
ConSol Consulting & Solutions Software GmbH
 
Kubernetes-native or not? When should you ditch your traditional CI/CD server...
Kubernetes-native or not? When should you ditch your traditional CI/CD server...Kubernetes-native or not? When should you ditch your traditional CI/CD server...
Kubernetes-native or not? When should you ditch your traditional CI/CD server...
Red Hat Developers
 
GPU enablement for data science on OpenShift | DevNation Tech Talk
GPU enablement for data science on OpenShift | DevNation Tech TalkGPU enablement for data science on OpenShift | DevNation Tech Talk
GPU enablement for data science on OpenShift | DevNation Tech Talk
Red Hat Developers
 
OpenShift Application Development | DO288 | Red Hat OpenShift
OpenShift Application Development | DO288 | Red Hat OpenShiftOpenShift Application Development | DO288 | Red Hat OpenShift
OpenShift Application Development | DO288 | Red Hat OpenShift
Global Knowledge Technologies
 
Building CI/CD Pipelines with Jenkins and Kubernetes
Building CI/CD Pipelines with Jenkins and KubernetesBuilding CI/CD Pipelines with Jenkins and Kubernetes
Building CI/CD Pipelines with Jenkins and Kubernetes
Janakiram MSV
 
OSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoringOSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoring
Gianluca Arbezzano
 
Using kubernetes to lose your fear of using containers
Using kubernetes to lose your fear of using containersUsing kubernetes to lose your fear of using containers
Using kubernetes to lose your fear of using containers
josfuecas
 
Kubernetes Helm: Why It Matters
Kubernetes Helm: Why It MattersKubernetes Helm: Why It Matters
Kubernetes Helm: Why It Matters
Platform9
 
Kubernetes: The evolution of distributed systems | DevNation Tech Talk
Kubernetes: The evolution of distributed systems | DevNation Tech TalkKubernetes: The evolution of distributed systems | DevNation Tech Talk
Kubernetes: The evolution of distributed systems | DevNation Tech Talk
Red Hat Developers
 
OpenStack on Kubernetes (BOS Summit / May 2017 update)
OpenStack on Kubernetes (BOS Summit / May 2017 update)OpenStack on Kubernetes (BOS Summit / May 2017 update)
OpenStack on Kubernetes (BOS Summit / May 2017 update)
rhirschfeld
 
Centralizing Kubernetes and Container Operations
Centralizing Kubernetes and Container OperationsCentralizing Kubernetes and Container Operations
Centralizing Kubernetes and Container Operations
Kublr
 
Remote debugging of Application in Kubernetes
Remote debugging of Application in KubernetesRemote debugging of Application in Kubernetes
Remote debugging of Application in Kubernetes
ConSol Consulting & Solutions Software GmbH
 
Democratizing machine learning on kubernetes
Democratizing machine learning on kubernetesDemocratizing machine learning on kubernetes
Democratizing machine learning on kubernetes
Docker, Inc.
 
Kubernetes fundamentals
Kubernetes fundamentalsKubernetes fundamentals
Kubernetes fundamentals
Victor Morales
 
KubeCon EU 2016: Heroku to Kubernetes
KubeCon EU 2016: Heroku to KubernetesKubeCon EU 2016: Heroku to Kubernetes
KubeCon EU 2016: Heroku to Kubernetes
KubeAcademy
 
Serverless stream processing of Debezium data change events with Knative | De...
Serverless stream processing of Debezium data change events with Knative | De...Serverless stream processing of Debezium data change events with Knative | De...
Serverless stream processing of Debezium data change events with Knative | De...
Red Hat Developers
 
Multi-cloud Kubernetes BCDR with Velero
Multi-cloud Kubernetes BCDR with VeleroMulti-cloud Kubernetes BCDR with Velero
Multi-cloud Kubernetes BCDR with Velero
Kublr
 
K8s Pod Scheduling - Deep Dive. By Tsahi Duek.
K8s Pod Scheduling - Deep Dive. By Tsahi Duek.K8s Pod Scheduling - Deep Dive. By Tsahi Duek.
K8s Pod Scheduling - Deep Dive. By Tsahi Duek.
Cloud Native Day Tel Aviv
 
Kubernetes Deployments: A "Hands-off" Approach
Kubernetes Deployments: A "Hands-off" ApproachKubernetes Deployments: A "Hands-off" Approach
Kubernetes Deployments: A "Hands-off" Approach
Rodrigo Reis
 
"Using Automation Tools To Deploy And Operate Applications In Real World Scen...
"Using Automation Tools To Deploy And Operate Applications In Real World Scen..."Using Automation Tools To Deploy And Operate Applications In Real World Scen...
"Using Automation Tools To Deploy And Operate Applications In Real World Scen...
ConSol Consulting & Solutions Software GmbH
 
Kubernetes-native or not? When should you ditch your traditional CI/CD server...
Kubernetes-native or not? When should you ditch your traditional CI/CD server...Kubernetes-native or not? When should you ditch your traditional CI/CD server...
Kubernetes-native or not? When should you ditch your traditional CI/CD server...
Red Hat Developers
 
GPU enablement for data science on OpenShift | DevNation Tech Talk
GPU enablement for data science on OpenShift | DevNation Tech TalkGPU enablement for data science on OpenShift | DevNation Tech Talk
GPU enablement for data science on OpenShift | DevNation Tech Talk
Red Hat Developers
 
OpenShift Application Development | DO288 | Red Hat OpenShift
OpenShift Application Development | DO288 | Red Hat OpenShiftOpenShift Application Development | DO288 | Red Hat OpenShift
OpenShift Application Development | DO288 | Red Hat OpenShift
Global Knowledge Technologies
 
Building CI/CD Pipelines with Jenkins and Kubernetes
Building CI/CD Pipelines with Jenkins and KubernetesBuilding CI/CD Pipelines with Jenkins and Kubernetes
Building CI/CD Pipelines with Jenkins and Kubernetes
Janakiram MSV
 
OSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoringOSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoring
Gianluca Arbezzano
 
Using kubernetes to lose your fear of using containers
Using kubernetes to lose your fear of using containersUsing kubernetes to lose your fear of using containers
Using kubernetes to lose your fear of using containers
josfuecas
 
Kubernetes Helm: Why It Matters
Kubernetes Helm: Why It MattersKubernetes Helm: Why It Matters
Kubernetes Helm: Why It Matters
Platform9
 
Kubernetes: The evolution of distributed systems | DevNation Tech Talk
Kubernetes: The evolution of distributed systems | DevNation Tech TalkKubernetes: The evolution of distributed systems | DevNation Tech Talk
Kubernetes: The evolution of distributed systems | DevNation Tech Talk
Red Hat Developers
 
OpenStack on Kubernetes (BOS Summit / May 2017 update)
OpenStack on Kubernetes (BOS Summit / May 2017 update)OpenStack on Kubernetes (BOS Summit / May 2017 update)
OpenStack on Kubernetes (BOS Summit / May 2017 update)
rhirschfeld
 
Centralizing Kubernetes and Container Operations
Centralizing Kubernetes and Container OperationsCentralizing Kubernetes and Container Operations
Centralizing Kubernetes and Container Operations
Kublr
 
Democratizing machine learning on kubernetes
Democratizing machine learning on kubernetesDemocratizing machine learning on kubernetes
Democratizing machine learning on kubernetes
Docker, Inc.
 
Kubernetes fundamentals
Kubernetes fundamentalsKubernetes fundamentals
Kubernetes fundamentals
Victor Morales
 
KubeCon EU 2016: Heroku to Kubernetes
KubeCon EU 2016: Heroku to KubernetesKubeCon EU 2016: Heroku to Kubernetes
KubeCon EU 2016: Heroku to Kubernetes
KubeAcademy
 

Similar to Kubernetes Logging (20)

To Russia with Love: Deploying Kubernetes in Exotic Locations On Prem
To Russia with Love: Deploying Kubernetes in Exotic Locations On PremTo Russia with Love: Deploying Kubernetes in Exotic Locations On Prem
To Russia with Love: Deploying Kubernetes in Exotic Locations On Prem
CloudOps2005
 
[WSO2Con USA 2018] Deploying Applications in K8S and Docker
[WSO2Con USA 2018] Deploying Applications in K8S and Docker[WSO2Con USA 2018] Deploying Applications in K8S and Docker
[WSO2Con USA 2018] Deploying Applications in K8S and Docker
WSO2
 
[WSO2Con EU 2018] Deploying Applications in K8S and Docker
[WSO2Con EU 2018] Deploying Applications in K8S and Docker[WSO2Con EU 2018] Deploying Applications in K8S and Docker
[WSO2Con EU 2018] Deploying Applications in K8S and Docker
WSO2
 
Kubernetes 101
Kubernetes 101Kubernetes 101
Kubernetes 101
Stanislav Pogrebnyak
 
Virtual Flink Forward 2020: Integrate Flink with Kubernetes natively - Yang Wang
Virtual Flink Forward 2020: Integrate Flink with Kubernetes natively - Yang WangVirtual Flink Forward 2020: Integrate Flink with Kubernetes natively - Yang Wang
Virtual Flink Forward 2020: Integrate Flink with Kubernetes natively - Yang Wang
Flink Forward
 
Scalable Spark deployment using Kubernetes
Scalable Spark deployment using KubernetesScalable Spark deployment using Kubernetes
Scalable Spark deployment using Kubernetes
datamantra
 
[WSO2Con Asia 2018] Deploying Applications in K8S and Docker
[WSO2Con Asia 2018] Deploying Applications in K8S and Docker[WSO2Con Asia 2018] Deploying Applications in K8S and Docker
[WSO2Con Asia 2018] Deploying Applications in K8S and Docker
WSO2
 
Kubernetes Manchester - 6th December 2018
Kubernetes Manchester - 6th December 2018Kubernetes Manchester - 6th December 2018
Kubernetes Manchester - 6th December 2018
David Stockton
 
Kubernates : An Small introduction for Beginners by Rajiv Vishwkarma
Kubernates : An Small introduction for Beginners by Rajiv VishwkarmaKubernates : An Small introduction for Beginners by Rajiv Vishwkarma
Kubernates : An Small introduction for Beginners by Rajiv Vishwkarma
Rajiv Vishwkarma
 
DevEx | there’s no place like k3s
DevEx | there’s no place like k3sDevEx | there’s no place like k3s
DevEx | there’s no place like k3s
Haggai Philip Zagury
 
Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...
Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...
Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...
javier ramirez
 
reBuy on Kubernetes
reBuy on KubernetesreBuy on Kubernetes
reBuy on Kubernetes
Stephan Lindauer
 
Kubernetes: The Next Research Platform
Kubernetes: The Next Research PlatformKubernetes: The Next Research Platform
Kubernetes: The Next Research Platform
Bob Killen
 
Kubernetes: Managed or Not Managed?
Kubernetes: Managed or Not Managed?Kubernetes: Managed or Not Managed?
Kubernetes: Managed or Not Managed?
Mathieu Herbert
 
Session 4 GCCP.pptx
Session 4 GCCP.pptxSession 4 GCCP.pptx
Session 4 GCCP.pptx
DSCIITPatna
 
OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...
NETWAYS
 
Kubernetes & Google Container Engine @ mabl
Kubernetes & Google Container Engine @ mablKubernetes & Google Container Engine @ mabl
Kubernetes & Google Container Engine @ mabl
Joseph Lust
 
Containers, Serverless and Functions in a nutshell
Containers, Serverless and Functions in a nutshellContainers, Serverless and Functions in a nutshell
Containers, Serverless and Functions in a nutshell
Eugene Fedorenko
 
Free GitOps Workshop + Intro to Kubernetes & GitOps
Free GitOps Workshop + Intro to Kubernetes & GitOpsFree GitOps Workshop + Intro to Kubernetes & GitOps
Free GitOps Workshop + Intro to Kubernetes & GitOps
Weaveworks
 
Javantura v4 - Self-service app deployment with Kubernetes and OpenShift - Ma...
Javantura v4 - Self-service app deployment with Kubernetes and OpenShift - Ma...Javantura v4 - Self-service app deployment with Kubernetes and OpenShift - Ma...
Javantura v4 - Self-service app deployment with Kubernetes and OpenShift - Ma...
HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association
 
To Russia with Love: Deploying Kubernetes in Exotic Locations On Prem
To Russia with Love: Deploying Kubernetes in Exotic Locations On PremTo Russia with Love: Deploying Kubernetes in Exotic Locations On Prem
To Russia with Love: Deploying Kubernetes in Exotic Locations On Prem
CloudOps2005
 
[WSO2Con USA 2018] Deploying Applications in K8S and Docker
[WSO2Con USA 2018] Deploying Applications in K8S and Docker[WSO2Con USA 2018] Deploying Applications in K8S and Docker
[WSO2Con USA 2018] Deploying Applications in K8S and Docker
WSO2
 
[WSO2Con EU 2018] Deploying Applications in K8S and Docker
[WSO2Con EU 2018] Deploying Applications in K8S and Docker[WSO2Con EU 2018] Deploying Applications in K8S and Docker
[WSO2Con EU 2018] Deploying Applications in K8S and Docker
WSO2
 
Virtual Flink Forward 2020: Integrate Flink with Kubernetes natively - Yang Wang
Virtual Flink Forward 2020: Integrate Flink with Kubernetes natively - Yang WangVirtual Flink Forward 2020: Integrate Flink with Kubernetes natively - Yang Wang
Virtual Flink Forward 2020: Integrate Flink with Kubernetes natively - Yang Wang
Flink Forward
 
Scalable Spark deployment using Kubernetes
Scalable Spark deployment using KubernetesScalable Spark deployment using Kubernetes
Scalable Spark deployment using Kubernetes
datamantra
 
[WSO2Con Asia 2018] Deploying Applications in K8S and Docker
[WSO2Con Asia 2018] Deploying Applications in K8S and Docker[WSO2Con Asia 2018] Deploying Applications in K8S and Docker
[WSO2Con Asia 2018] Deploying Applications in K8S and Docker
WSO2
 
Kubernetes Manchester - 6th December 2018
Kubernetes Manchester - 6th December 2018Kubernetes Manchester - 6th December 2018
Kubernetes Manchester - 6th December 2018
David Stockton
 
Kubernates : An Small introduction for Beginners by Rajiv Vishwkarma
Kubernates : An Small introduction for Beginners by Rajiv VishwkarmaKubernates : An Small introduction for Beginners by Rajiv Vishwkarma
Kubernates : An Small introduction for Beginners by Rajiv Vishwkarma
Rajiv Vishwkarma
 
DevEx | there’s no place like k3s
DevEx | there’s no place like k3sDevEx | there’s no place like k3s
DevEx | there’s no place like k3s
Haggai Philip Zagury
 
Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...
Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...
Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...
javier ramirez
 
Kubernetes: The Next Research Platform
Kubernetes: The Next Research PlatformKubernetes: The Next Research Platform
Kubernetes: The Next Research Platform
Bob Killen
 
Kubernetes: Managed or Not Managed?
Kubernetes: Managed or Not Managed?Kubernetes: Managed or Not Managed?
Kubernetes: Managed or Not Managed?
Mathieu Herbert
 
Session 4 GCCP.pptx
Session 4 GCCP.pptxSession 4 GCCP.pptx
Session 4 GCCP.pptx
DSCIITPatna
 
OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...
NETWAYS
 
Kubernetes & Google Container Engine @ mabl
Kubernetes & Google Container Engine @ mablKubernetes & Google Container Engine @ mabl
Kubernetes & Google Container Engine @ mabl
Joseph Lust
 
Containers, Serverless and Functions in a nutshell
Containers, Serverless and Functions in a nutshellContainers, Serverless and Functions in a nutshell
Containers, Serverless and Functions in a nutshell
Eugene Fedorenko
 
Free GitOps Workshop + Intro to Kubernetes & GitOps
Free GitOps Workshop + Intro to Kubernetes & GitOpsFree GitOps Workshop + Intro to Kubernetes & GitOps
Free GitOps Workshop + Intro to Kubernetes & GitOps
Weaveworks
 
Ad

Recently uploaded (20)

Explaining GitHub Actions Failures with Large Language Models Challenges, In...
Explaining GitHub Actions Failures with Large Language Models Challenges, In...Explaining GitHub Actions Failures with Large Language Models Challenges, In...
Explaining GitHub Actions Failures with Large Language Models Challenges, In...
ssuserb14185
 
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Eric D. Schabell
 
The Significance of Hardware in Information Systems.pdf
The Significance of Hardware in Information Systems.pdfThe Significance of Hardware in Information Systems.pdf
The Significance of Hardware in Information Systems.pdf
drewplanas10
 
F-Secure Freedome VPN 2025 Crack Plus Activation New Version
F-Secure Freedome VPN 2025 Crack Plus Activation  New VersionF-Secure Freedome VPN 2025 Crack Plus Activation  New Version
F-Secure Freedome VPN 2025 Crack Plus Activation New Version
saimabibi60507
 
WinRAR Crack for Windows (100% Working 2025)
WinRAR Crack for Windows (100% Working 2025)WinRAR Crack for Windows (100% Working 2025)
WinRAR Crack for Windows (100% Working 2025)
sh607827
 
Secure Test Infrastructure: The Backbone of Trustworthy Software Development
Secure Test Infrastructure: The Backbone of Trustworthy Software DevelopmentSecure Test Infrastructure: The Backbone of Trustworthy Software Development
Secure Test Infrastructure: The Backbone of Trustworthy Software Development
Shubham Joshi
 
Adobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage Dashboards
Adobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage DashboardsAdobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage Dashboards
Adobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage Dashboards
BradBedford3
 
Pixologic ZBrush Crack Plus Activation Key [Latest 2025] New Version
Pixologic ZBrush Crack Plus Activation Key [Latest 2025] New VersionPixologic ZBrush Crack Plus Activation Key [Latest 2025] New Version
Pixologic ZBrush Crack Plus Activation Key [Latest 2025] New Version
saimabibi60507
 
Meet the Agents: How AI Is Learning to Think, Plan, and Collaborate
Meet the Agents: How AI Is Learning to Think, Plan, and CollaborateMeet the Agents: How AI Is Learning to Think, Plan, and Collaborate
Meet the Agents: How AI Is Learning to Think, Plan, and Collaborate
Maxim Salnikov
 
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Lionel Briand
 
Adobe After Effects Crack FREE FRESH version 2025
Adobe After Effects Crack FREE FRESH version 2025Adobe After Effects Crack FREE FRESH version 2025
Adobe After Effects Crack FREE FRESH version 2025
kashifyounis067
 
Exploring Wayland: A Modern Display Server for the Future
Exploring Wayland: A Modern Display Server for the FutureExploring Wayland: A Modern Display Server for the Future
Exploring Wayland: A Modern Display Server for the Future
ICS
 
Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)
Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)
Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)
Andre Hora
 
Avast Premium Security Crack FREE Latest Version 2025
Avast Premium Security Crack FREE Latest Version 2025Avast Premium Security Crack FREE Latest Version 2025
Avast Premium Security Crack FREE Latest Version 2025
mu394968
 
How to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud PerformanceHow to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud Performance
ThousandEyes
 
TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...
TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...
TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...
Andre Hora
 
Automation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath CertificateAutomation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath Certificate
VICTOR MAESTRE RAMIREZ
 
Top 10 Client Portal Software Solutions for 2025.docx
Top 10 Client Portal Software Solutions for 2025.docxTop 10 Client Portal Software Solutions for 2025.docx
Top 10 Client Portal Software Solutions for 2025.docx
Portli
 
How can one start with crypto wallet development.pptx
How can one start with crypto wallet development.pptxHow can one start with crypto wallet development.pptx
How can one start with crypto wallet development.pptx
laravinson24
 
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
steaveroggers
 
Explaining GitHub Actions Failures with Large Language Models Challenges, In...
Explaining GitHub Actions Failures with Large Language Models Challenges, In...Explaining GitHub Actions Failures with Large Language Models Challenges, In...
Explaining GitHub Actions Failures with Large Language Models Challenges, In...
ssuserb14185
 
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Eric D. Schabell
 
The Significance of Hardware in Information Systems.pdf
The Significance of Hardware in Information Systems.pdfThe Significance of Hardware in Information Systems.pdf
The Significance of Hardware in Information Systems.pdf
drewplanas10
 
F-Secure Freedome VPN 2025 Crack Plus Activation New Version
F-Secure Freedome VPN 2025 Crack Plus Activation  New VersionF-Secure Freedome VPN 2025 Crack Plus Activation  New Version
F-Secure Freedome VPN 2025 Crack Plus Activation New Version
saimabibi60507
 
WinRAR Crack for Windows (100% Working 2025)
WinRAR Crack for Windows (100% Working 2025)WinRAR Crack for Windows (100% Working 2025)
WinRAR Crack for Windows (100% Working 2025)
sh607827
 
Secure Test Infrastructure: The Backbone of Trustworthy Software Development
Secure Test Infrastructure: The Backbone of Trustworthy Software DevelopmentSecure Test Infrastructure: The Backbone of Trustworthy Software Development
Secure Test Infrastructure: The Backbone of Trustworthy Software Development
Shubham Joshi
 
Adobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage Dashboards
Adobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage DashboardsAdobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage Dashboards
Adobe Marketo Engage Champion Deep Dive - SFDC CRM Synch V2 & Usage Dashboards
BradBedford3
 
Pixologic ZBrush Crack Plus Activation Key [Latest 2025] New Version
Pixologic ZBrush Crack Plus Activation Key [Latest 2025] New VersionPixologic ZBrush Crack Plus Activation Key [Latest 2025] New Version
Pixologic ZBrush Crack Plus Activation Key [Latest 2025] New Version
saimabibi60507
 
Meet the Agents: How AI Is Learning to Think, Plan, and Collaborate
Meet the Agents: How AI Is Learning to Think, Plan, and CollaborateMeet the Agents: How AI Is Learning to Think, Plan, and Collaborate
Meet the Agents: How AI Is Learning to Think, Plan, and Collaborate
Maxim Salnikov
 
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Lionel Briand
 
Adobe After Effects Crack FREE FRESH version 2025
Adobe After Effects Crack FREE FRESH version 2025Adobe After Effects Crack FREE FRESH version 2025
Adobe After Effects Crack FREE FRESH version 2025
kashifyounis067
 
Exploring Wayland: A Modern Display Server for the Future
Exploring Wayland: A Modern Display Server for the FutureExploring Wayland: A Modern Display Server for the Future
Exploring Wayland: A Modern Display Server for the Future
ICS
 
Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)
Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)
Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)
Andre Hora
 
Avast Premium Security Crack FREE Latest Version 2025
Avast Premium Security Crack FREE Latest Version 2025Avast Premium Security Crack FREE Latest Version 2025
Avast Premium Security Crack FREE Latest Version 2025
mu394968
 
How to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud PerformanceHow to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud Performance
ThousandEyes
 
TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...
TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...
TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...
Andre Hora
 
Automation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath CertificateAutomation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath Certificate
VICTOR MAESTRE RAMIREZ
 
Top 10 Client Portal Software Solutions for 2025.docx
Top 10 Client Portal Software Solutions for 2025.docxTop 10 Client Portal Software Solutions for 2025.docx
Top 10 Client Portal Software Solutions for 2025.docx
Portli
 
How can one start with crypto wallet development.pptx
How can one start with crypto wallet development.pptxHow can one start with crypto wallet development.pptx
How can one start with crypto wallet development.pptx
laravinson24
 
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
steaveroggers
 
Ad

Kubernetes Logging

  • 1. Kubernetes Logging Implement and test self-managed logs collection and visualization solution for your k8s cluster
  • 2. Real world requirements ● Quick and easy deployment and testing (do it like a DevOps) ● Reliable enough ● Maximum performance by default ● Simple to understand and debug (KISS) ● Reuse existing solutions as much as possible
  • 3. Stack decisions and tools ● Playground: Minikube/Microk8s/Kubeadm/Kops ● FluentD/Fluent Bit/Logstash/Telegraf ● EL(F)K/TICK (InfluxDB) stacks, Graphana? ● Using k8s operator, reliability concerns ● Helm/GitKube/Kapitan ● Ingress controller (Nginx/ALB)
  • 4. Kubernetes on localhost Minikube ● Needs a VM ● Needs special treatment ● Good for developing services, not operations testing Microk8s ● Super simple ● Real cluster ● Docker (in Docker) ● Too much magic ● We need Cloud for UI Load balancing and ingress
  • 5. Kubernetes in the Cloud Kubeadm ● Not really for cloud ● Needs manual job ● No mature enough automations available Kops ● Easy, scalable ● AWS native ● Cluster-as-a-Code ● Addons ● Generates Terraform! ● Actively developed
  • 6. Logs collector and processor ● Logstash: heavy, do we need all its features? ● Telegraf (InfluxDB): looks cool, where is k8s support? ● FluentD: good stuff, recommended upstream, Kops plugin ● Fluent Bit: lightweight and fast, native support for Kubernetes, minimal configuration
  • 7. EFK stack for logs visualisation ● ElasticSearch: deploy real cluster, rotate indexes, resize, backup, kubernetes operator is the only option ● Fluent Bit: read all container logs, enrich with kubernetes metadata and system metrics, does buffering and retries, deployed via DaemonSet ● Kibana: discover the index in ES, configure timestamp field, watch and query. TODO: create some pretty dashboards
  • 8. Turn-key deployment ● GitKube looks nice… ● Kapitan is more Pythonic ● Helm appears to be widely adopted, many examples available ● Install all stack components from single chart ● Helm is great about vendorizing dependencies ● GitHub repo could become Helm repo! ● TODO: enable ingress rule
  • 10. Concerns and future improvements ● Authentication, sensitive data, ES hardening, Cerebro, capasity, proper TLS setup ● “IFG” stack: InfluxDB, FluentD, Grafana