eBPF - Observability In Deep
2 likes813 views
In the Cloud Native community, eBPF is gaining popularity, which can often be the best solution for solving different challenges with deep observability of system. Currently, eBPF is being embraced by major players. Mydbops co-Founder, Kabilesh P.R (MySQL and Mongo Consultant) illustrates on debugging linux issues with eBPF. A brief about BPF & eBPF, BPF internals and the tools in actions for faster resolution.
1 of 25
Downloaded 26 times
Ad
Recommended
Understanding eBPF in a Hurry!
Understanding eBPF in a Hurry!Ray Jenkins eBPF is an exciting new technology that is poised to transform Linux performance engineering. eBPF enables users to dynamically and programatically trace any kernel or user space code path, safely and efficiently. However, understanding eBPF is not so simple. The goal of this talk is to give audiences a fundamental understanding of eBPF, how it interconnects existing Linux tracing technologies, and provides a powerful aplatform to solve any Linux performance problem.
BPF: Tracing and more
BPF: Tracing and moreBrendan Gregg Video: https://www.youtube.com/watch?v=JRFNIKUROPE . Talk for linux.conf.au 2017 (LCA2017) by Brendan Gregg, about Linux enhanced BPF (eBPF). Abstract:
A world of new capabilities is emerging for the Linux 4.x series, thanks to enhancements that have been included in Linux for to Berkeley Packet Filter (BPF): an in-kernel virtual machine that can execute user space-defined programs. It is finding uses for security auditing and enforcement, enhancing networking (including eXpress Data Path), and performance observability and troubleshooting. Many new open source tools that have been written in the past 12 months for performance analysis that use BPF. Tracing superpowers have finally arrived for Linux!
For its use with tracing, BPF provides the programmable capabilities to the existing tracing frameworks: kprobes, uprobes, and tracepoints. In particular, BPF allows timestamps to be recorded and compared from custom events, allowing latency to be studied in many new places: kernel and application internals. It also allows data to be efficiently summarized in-kernel, including as histograms. This has allowed dozens of new observability tools to be developed so far, including measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more.
This talk will summarize BPF capabilities and use cases so far, and then focus on its use to enhance Linux tracing, especially with the open source bcc collection. bcc includes BPF versions of old classics, and many new tools, including execsnoop, opensnoop, funcccount, ext4slower, and more (many of which I developed). Perhaps you'd like to develop new tools, or use the existing tools to find performance wins large and small, especially when instrumenting areas that previously had zero visibility. I'll also summarize how we intend to use these new capabilities to enhance systems analysis at Netflix.
Accelerating Envoy and Istio with Cilium and the Linux Kernel
Accelerating Envoy and Istio with Cilium and the Linux KernelThomas Graf The document discusses how Cilium can accelerate Envoy and Istio by using eBPF/XDP to provide transparent acceleration of network traffic between Kubernetes pods and sidecars without any changes required to applications or Envoy. Cilium also provides features like service mesh datapath, network security policies, load balancing, and visibility/tracing capabilities. BPF/XDP in Cilium allows for transparent TCP/IP acceleration during the data phase of communications between pods and sidecars.
eBPF Workshop
eBPF WorkshopMichael Kehoe This document provides instructions for setting up and attending an eBPF workshop. It includes links for setting up the workshop platform, background slides, and code repository. It also lists an agenda with topics that will be covered, including setting up the eBPF lab, an introduction, eBPF 101, writing eBPF programs, BCC, and a tutorial. Attendees are asked to let the presenter know if they have any problems setting up.
Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)Brendan Gregg This document provides an overview of using eBPF (extended Berkeley Packet Filter) to quickly get performance wins as a sysadmin. It recommends installing BCC and bpftrace tools to easily find issues like periodic processes, misconfigurations, unexpected TCP sessions, or slow file system I/O. A case study examines using biosnoop to identify which processes were causing disk latency issues. The document suggests thinking like a sysadmin first by running tools, then like a programmer if a problem requires new tools. It also outlines recommended frontends depending on use cases and provides references to learn more about BPF.
eBPF Basics
eBPF BasicsMichael Kehoe This document provides an overview of cBPF and eBPF. It discusses the history and implementation of cBPF, including how it was originally used for packet filtering. It then covers eBPF in more depth, explaining what it is, its history, implementation including different program types and maps. It also discusses several uses of eBPF including networking, firewalls, DDoS mitigation, profiling, security, and chaos engineering. Finally, it introduces XDP and DPDK, comparing XDP's benefits over DPDK.
Using eBPF for High-Performance Networking in Cilium
Using eBPF for High-Performance Networking in CiliumScyllaDB The Cilium project is a popular networking solution for Kubernetes, based on eBPF. This talk uses eBPF code and demos to explore the basics of how Cilium makes network connections, and manipulates packets so that they can avoid traversing the kernel's built-in networking stack. You'll see how eBPF enables high-performance networking as well as deep network observability and security.
Introduction to eBPF and XDP
Introduction to eBPF and XDPlcplcp1 This document provides an introduction to eBPF and XDP. It discusses the history of BPF and how it evolved into eBPF. Key aspects of eBPF covered include the instruction set, JIT compilation, verifier, helper functions, and maps. XDP is introduced as a way to program the data plane using eBPF programs attached early in the receive path. Example use cases and performance benchmarks for XDP are also mentioned.
LinuxCon 2015 Linux Kernel Networking Walkthrough
LinuxCon 2015 Linux Kernel Networking WalkthroughThomas Graf This presentation features a walk through the Linux kernel networking stack for users and developers. It will cover insights into both, existing essential networking features and recent developments and will show how to use them properly. Our starting point is the network card driver as it feeds a packet into the stack. We will follow the packet as it traverses through various subsystems such as packet filtering, routing, protocol stacks, and the socket layer. We will pause here and there to look into concepts such as networking namespaces, segmentation offloading, TCP small queues, and low latency polling and will discuss how to configure them.
eBPF Trace from Kernel to Userspace
eBPF Trace from Kernel to UserspaceSUSE Labs Taipei This document discusses eBPF (extended Berkeley Packet Filter), which allows tracing from the Linux kernel to userspace using BPF programs. It provides an overview of eBPF including extended registers, verification, maps, and probes. Examples are given of using eBPF for tracing functions like kfree_skb() and the C library function malloc. The Berkeley Compiler Collection (BCC) makes it easy to write eBPF programs in C and Python.
Kubernetes Networking
Kubernetes NetworkingCJ Cullen The document discusses Kubernetes networking. It describes how Kubernetes networking allows pods to have routable IPs and communicate without NAT, unlike Docker networking which uses NAT. It covers how services provide stable virtual IPs to access pods, and how kube-proxy implements services by configuring iptables on nodes. It also discusses the DNS integration using SkyDNS and Ingress for layer 7 routing of HTTP traffic. Finally, it briefly mentions network plugins and how Kubernetes is designed to be open and customizable.
Linux Performance Analysis: New Tools and Old Secrets
Linux Performance Analysis: New Tools and Old SecretsBrendan Gregg Talk for USENIX/LISA2014 by Brendan Gregg, Netflix. At Netflix performance is crucial, and we use many high to low level tools to analyze our stack in different ways. In this talk, I will introduce new system observability tools we are using at Netflix, which I've ported from my DTraceToolkit, and are intended for our Linux 3.2 cloud instances. These show that Linux can do more than you may think, by using creative hacks and workarounds with existing kernel features (ftrace, perf_events). While these are solving issues on current versions of Linux, I'll also briefly summarize the future in this space: eBPF, ktap, SystemTap, sysdig, etc.
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDPThomas Graf This talk will start with a deep dive and hands on examples of BPF, possibly the most promising low level technology to address challenges in application and network security, tracing, and visibility. We will discuss how BPF evolved from a simple bytecode language to filter raw sockets for tcpdump to the a JITable virtual machine capable of universally extending and instrumenting both the Linux kernel and user space applications. The introduction is followed by a concrete example of how the Cilium open source project applies BPF to solve networking, security, and load balancing for highly distributed applications. We will discuss and demonstrate how Cilium with the help of BPF can be combined with distributed system orchestration such as Docker to simplify security, operations, and troubleshooting of distributed applications.
[KubeCon EU 2021] Introduction and Deep Dive Into Containerd![[KubeCon EU 2021] Introduction and Deep Dive Into Containerd](https://cdn.slidesharecdn.com/ss_thumbnails/containerd-kubeconeu2021-210505123042-thumbnail.jpg?width=560&fit=bounds)
[KubeCon EU 2021] Introduction and Deep Dive Into ContainerdAkihiro Suda Join containerd maintainers and reviewers in a combined introduction and deep dive session. They will discuss the overview and the recent updates of containerd as well as how it is being used by Kubernetes, Docker and other container-based systems. The brief introduction about its architecture and service design will be included. The talk will also deep dive into how to leverage contained by extending and customizing it for your use case with low-level plugins like remote snapshotters, as well as by implementing your own containerd client. Upcoming features and recent discussion in containerd community will also be covered.
- - -
https://kccnceu2021.sched.com/event/iE6v/introduction-and-deep-dive-into-containerd-kohei-tokunaga-akihiro-suda-ntt-corporation?iframe=no
Introduction to eBPF
Introduction to eBPFRogerColl2 This document provides an introduction to eBPF (Extended Berkeley Packet Filter), which allows running user-space code in the Linux kernel without needing to compile a kernel module. It describes how eBPF avoids unnecessary copying of packets between kernel and user-space for improved performance. Examples are given of using eBPF for networking tasks like SDN configuration, DDoS mitigation, intrusion detection, and load balancing. The document concludes by noting eBPF provides alternatives to iptables that are better suited for microservices architectures.
BPF / XDP 8월 세미나 KossLab
BPF / XDP 8월 세미나 KossLabTaeung Song BPF (Berkeley Packet Filter) allows for safe dynamic program injection into the Linux kernel. It provides an in-kernel virtual machine and instruction set for running custom programs. The BPF infrastructure includes a verifier that checks programs for safety, helper functions to access kernel APIs, and maps for inter-process communication. BPF has become a core kernel subsystem and is used for applications like XDP, tracing, networking, and more.
Tutorial: Using GoBGP as an IXP connecting router
Tutorial: Using GoBGP as an IXP connecting routerShu Sugimoto - Show you how GoBGP can be used as a software router in conjunction with quagga
- (Tutorial) Walk through the setup of IXP connecting router using GoBGP
CI/CD with Github Actions
CI/CD with Github ActionsMd. Minhazul Haque This document discusses setting up a CI/CD pipeline using GitHub Actions. It begins with an introduction to CI/CD pipelines and their importance. It then provides an overview of GitHub Actions and how they can be used to automate builds, tests, releases and deployments. The document demonstrates a sample GitHub Actions workflow file and explains its key components like jobs, steps and actions. It also covers topics like workflow events, jobs and steps/actions that can be used in GitHub Actions.
BPF Internals (eBPF)
BPF Internals (eBPF)Brendan Gregg USENIX LISA2021 talk by Brendan Gregg (https://www.youtube.com/watch?v=_5Z2AU7QTH4). This talk is a deep dive that describes how BPF (eBPF) works internally on Linux, and dissects some modern performance observability tools. Details covered include the kernel BPF implementation: the verifier, JIT compilation, and the BPF execution environment; the BPF instruction set; different event sources; and how BPF is used by user space, using bpftrace programs as an example. This includes showing how bpftrace is compiled to LLVM IR and then BPF bytecode, and how per-event data and aggregated map data are fetched from the kernel.
Cilium - Bringing the BPF Revolution to Kubernetes Networking and Security
Cilium - Bringing the BPF Revolution to Kubernetes Networking and SecurityThomas Graf BPF is one of the fastest emerging technologies of the Linux kernel. The talk provides an introduction to Cilium which brings the powers of BPF to Kubernetes and other orchestration systems to provide highly scalable and efficient networking, security and load balancing for containers and microservices. The talk will provide an introduction to the capabilities of Cilium today but also deep dives into the emerging roadmap involving networking at the socket layer and service mesh datapath capabilities to provide highly efficient connectivity between cloud native apps and sidecar proxies.
Monitoring, Logging and Tracing on Kubernetes
Monitoring, Logging and Tracing on KubernetesMartin Etmajer The document discusses monitoring, logging and tracing tools for Kubernetes including Heapster, Grafana, Fluentd, Elastic Stack, Jolokia and OpenTracing. It provides examples of deploying Heapster with InfluxDB and Grafana for metrics collection, Fluentd to ingest container logs into Elasticsearch, and using Jolokia and OpenTracing for remote access to JMX metrics and distributed tracing functionality.
Linux Network Stack
Linux Network StackAdrien Mahieux - The document discusses Linux network stack monitoring and configuration. It begins with definitions of key concepts like RSS, RPS, RFS, LRO, GRO, DCA, XDP and BPF.
- It then provides an overview of how the network stack works from the hardware interrupts and driver level up through routing, TCP/IP and to the socket level.
- Monitoring tools like ethtool, ftrace and /proc/interrupts are described for viewing hardware statistics, software stack traces and interrupt information.
Linux BPF Superpowers
Linux BPF SuperpowersBrendan Gregg Video: https://www.facebook.com/atscaleevents/videos/1693888610884236/ . Talk by Brendan Gregg from Facebook's Performance @Scale: "Linux performance analysis has been the domain of ancient tools and metrics, but that's now changing in the Linux 4.x series. A new tracer is available in the mainline kernel, built from dynamic tracing (kprobes, uprobes) and enhanced BPF (Berkeley Packet Filter), aka, eBPF. It allows us to measure latency distributions for file system I/O and run queue latency, print details of storage device I/O and TCP retransmits, investigate blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. This talk will summarize this new technology and some long-standing issues that it can solve, and how we intend to use it at Netflix."
Xdp and ebpf_maps
Xdp and ebpf_mapslcplcp1 This document discusses XDP (eXpress Data Path), a high-performance network data path that allows programs to run on the receive path of a network interface card. XDP enables packet processing using eBPF programs before packets reach the Linux networking stack. The document provides an overview of XDP and its performance advantages over other packet processing methods. It also discusses XDP's current status and support in the Linux kernel as well as example use cases and benchmarks.
왜 쿠버네티스는 systemd로 cgroup을 관리하려고 할까요
왜 쿠버네티스는 systemd로 cgroup을 관리하려고 할까요Jo Hoon 왜 쿠버네티스는 cgroup을 관리하는데 systemd를 쓰려고 하는지 고찰한 문서입니다.
특히 v1.22로 올라가면서 systemd를 사용하도록 설정된 부분을 포함하고 있습니다.
eBPF maps 101
eBPF maps 101SUSE Labs Taipei The document provides an overview of eBPF maps and how they can be used to share data between eBPF programs running in the kernel and userspace applications. It describes how maps are created via the BPF syscall using the BPF_MAP_CREATE command. It also explains how keys and values can be looked up, updated, and deleted from maps using commands like BPF_MAP_LOOKUP_ELEM, BPF_MAP_UPDATE_ELEM, and BPF_MAP_DELETE_ELEM. Finally, it lists the different types of eBPF maps available.
Replacing iptables with eBPF in Kubernetes with Cilium
Replacing iptables with eBPF in Kubernetes with CiliumMichal Rostecki Cilium is an open source project which provides networking, security and load balancing for application services that are deployed using Linux container technologies by using the native eBPF technology in the Linux kernel. In this presentation we talked about:
- The evolution of the BPF filters and explained the advantages of eBPF Filters and its use cases today in Linux especially on how Cilium networking utilizes the eBPF Filters to secure the Kubernetes workload with increased performance when compared to legacy iptables.
- How Cilium uses SOCKMAP for layer 7 policy enforcement - How Cilium integrates with Istio and handles L7 Network Policies with Envoy Proxies.
- The new features since the last release such as running Kubernetes cluster without kube-proxy, providing clusterwide NetworkPolicies, providing fully distributed networking and security observability platform for cloud native workloads etc.
CNCF Singapore - Introduction to Envoy
CNCF Singapore - Introduction to EnvoyHarish Envoy is an open source edge and service proxy developed by Lyft to be used as a communication bus and network proxy. It can be used to add capabilities like traffic management, observability and security to services without code modification. Some key features include lightweight and low latency design, dynamic configuration via xDS APIs, rich observability, and built-in load balancing, failure recovery and traffic control capabilities. It uses concepts like clusters, listeners and routes. The Envoy admin API provides endpoints to interact with it and retrieve stats, configs and logs. Dynamic configuration allows adding and updating resources like clusters and listeners at runtime. Circuit breaking, outlier detection and health checking can be configured and demoed.
The Open Source Ecosystem for eBPF in Kubernetes
The Open Source Ecosystem for eBPF in KubernetesAll Things Open Presented at All Things Open 2024
Presented by Andre Fredette, Ph.D. & Billy McFall - Red Hat
Title: The Open Source Ecosystem for eBPF in Kubernetes
Abstract: Kubernetes has become the de facto open source solution for orchestrating containerized applications. However, as the complexity and scale of deployments grow, traditional tools often fall short of providing the granularity and efficiency required by advanced applications. To meet the demand, applications are increasingly leveraging eBPF (extended Berkeley Packet Filters) due to the revolutionary kernel capabilities it enables.
This talk will explore the integration of the eBPF ecosystem with Kubernetes, highlighting its potential to transform how operators and developers observe, secure, and troubleshoot their deployments. We will start by introducing eBPF and its core concepts, including its architecture, programming model, and key benefits such as minimal overhead, improved visibility, and dynamic tracing capabilities. We will then review real-world examples of open source tools which leverage eBPF for networking, security, and observability in Kubernetes environments. We will also explore an open source project called bpfman (https://bpfman.io), an eBPF Manager focusing on simplifying the deployment, administration and visibility of eBPF programs in Kubernetes clusters.
This presentation is designed for Kubernetes operators, security professionals, and developers seeking to deepen their understanding of eBPF and its applications in cloud-native environments. No prior knowledge of eBPF is required, but familiarity with Kubernetes concepts and architecture will be beneficial.
Find more info about All Things Open:
On the web: https://www.allthingsopen.org/
Twitter: https://twitter.com/AllThingsOpen
LinkedIn: https://www.linkedin.com/company/all-things-open/
Instagram: https://www.instagram.com/allthingsopen/
Facebook: https://www.facebook.com/AllThingsOpen
Mastodon: https://mastodon.social/@allthingsopen
Threads: https://www.threads.net/@allthingsopen
2024 conference: https://2024.allthingsopen.org/
Introduction of eBPF - 時下最夯的Linux Technology 
Introduction of eBPF - 時下最夯的Linux Technology Jace Liang @ 2020/04/20 SDN x Cloud Native Meetup #27
隨著CNCF將Falco納入incubator project，eBPF這藏於Linux核心內的技術也開始受到矚目。
eBPF，一個從1992年就出現的技術，一路走來經過了甚麼樣的變化?
對於你我目前，或未來的工作又會有甚麼影響呢?
本次分享將會介紹eBPF的前世今生，帶各位了解何謂eBPF。並透過實際範例演示eBPF工具的特殊用法。
Ad
More Related Content
What's hot (20)
LinuxCon 2015 Linux Kernel Networking Walkthrough
LinuxCon 2015 Linux Kernel Networking WalkthroughThomas Graf This presentation features a walk through the Linux kernel networking stack for users and developers. It will cover insights into both, existing essential networking features and recent developments and will show how to use them properly. Our starting point is the network card driver as it feeds a packet into the stack. We will follow the packet as it traverses through various subsystems such as packet filtering, routing, protocol stacks, and the socket layer. We will pause here and there to look into concepts such as networking namespaces, segmentation offloading, TCP small queues, and low latency polling and will discuss how to configure them.
eBPF Trace from Kernel to Userspace
eBPF Trace from Kernel to UserspaceSUSE Labs Taipei This document discusses eBPF (extended Berkeley Packet Filter), which allows tracing from the Linux kernel to userspace using BPF programs. It provides an overview of eBPF including extended registers, verification, maps, and probes. Examples are given of using eBPF for tracing functions like kfree_skb() and the C library function malloc. The Berkeley Compiler Collection (BCC) makes it easy to write eBPF programs in C and Python.
Kubernetes Networking
Kubernetes NetworkingCJ Cullen The document discusses Kubernetes networking. It describes how Kubernetes networking allows pods to have routable IPs and communicate without NAT, unlike Docker networking which uses NAT. It covers how services provide stable virtual IPs to access pods, and how kube-proxy implements services by configuring iptables on nodes. It also discusses the DNS integration using SkyDNS and Ingress for layer 7 routing of HTTP traffic. Finally, it briefly mentions network plugins and how Kubernetes is designed to be open and customizable.
Linux Performance Analysis: New Tools and Old Secrets
Linux Performance Analysis: New Tools and Old SecretsBrendan Gregg Talk for USENIX/LISA2014 by Brendan Gregg, Netflix. At Netflix performance is crucial, and we use many high to low level tools to analyze our stack in different ways. In this talk, I will introduce new system observability tools we are using at Netflix, which I've ported from my DTraceToolkit, and are intended for our Linux 3.2 cloud instances. These show that Linux can do more than you may think, by using creative hacks and workarounds with existing kernel features (ftrace, perf_events). While these are solving issues on current versions of Linux, I'll also briefly summarize the future in this space: eBPF, ktap, SystemTap, sysdig, etc.
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDPThomas Graf This talk will start with a deep dive and hands on examples of BPF, possibly the most promising low level technology to address challenges in application and network security, tracing, and visibility. We will discuss how BPF evolved from a simple bytecode language to filter raw sockets for tcpdump to the a JITable virtual machine capable of universally extending and instrumenting both the Linux kernel and user space applications. The introduction is followed by a concrete example of how the Cilium open source project applies BPF to solve networking, security, and load balancing for highly distributed applications. We will discuss and demonstrate how Cilium with the help of BPF can be combined with distributed system orchestration such as Docker to simplify security, operations, and troubleshooting of distributed applications.
[KubeCon EU 2021] Introduction and Deep Dive Into Containerd
[KubeCon EU 2021] Introduction and Deep Dive Into ContainerdAkihiro Suda Join containerd maintainers and reviewers in a combined introduction and deep dive session. They will discuss the overview and the recent updates of containerd as well as how it is being used by Kubernetes, Docker and other container-based systems. The brief introduction about its architecture and service design will be included. The talk will also deep dive into how to leverage contained by extending and customizing it for your use case with low-level plugins like remote snapshotters, as well as by implementing your own containerd client. Upcoming features and recent discussion in containerd community will also be covered.
- - -
https://kccnceu2021.sched.com/event/iE6v/introduction-and-deep-dive-into-containerd-kohei-tokunaga-akihiro-suda-ntt-corporation?iframe=no
Introduction to eBPF
Introduction to eBPFRogerColl2 This document provides an introduction to eBPF (Extended Berkeley Packet Filter), which allows running user-space code in the Linux kernel without needing to compile a kernel module. It describes how eBPF avoids unnecessary copying of packets between kernel and user-space for improved performance. Examples are given of using eBPF for networking tasks like SDN configuration, DDoS mitigation, intrusion detection, and load balancing. The document concludes by noting eBPF provides alternatives to iptables that are better suited for microservices architectures.
BPF / XDP 8월 세미나 KossLab
BPF / XDP 8월 세미나 KossLabTaeung Song BPF (Berkeley Packet Filter) allows for safe dynamic program injection into the Linux kernel. It provides an in-kernel virtual machine and instruction set for running custom programs. The BPF infrastructure includes a verifier that checks programs for safety, helper functions to access kernel APIs, and maps for inter-process communication. BPF has become a core kernel subsystem and is used for applications like XDP, tracing, networking, and more.
Tutorial: Using GoBGP as an IXP connecting router
Tutorial: Using GoBGP as an IXP connecting routerShu Sugimoto - Show you how GoBGP can be used as a software router in conjunction with quagga
- (Tutorial) Walk through the setup of IXP connecting router using GoBGP
CI/CD with Github Actions
CI/CD with Github ActionsMd. Minhazul Haque This document discusses setting up a CI/CD pipeline using GitHub Actions. It begins with an introduction to CI/CD pipelines and their importance. It then provides an overview of GitHub Actions and how they can be used to automate builds, tests, releases and deployments. The document demonstrates a sample GitHub Actions workflow file and explains its key components like jobs, steps and actions. It also covers topics like workflow events, jobs and steps/actions that can be used in GitHub Actions.
BPF Internals (eBPF)
BPF Internals (eBPF)Brendan Gregg USENIX LISA2021 talk by Brendan Gregg (https://www.youtube.com/watch?v=_5Z2AU7QTH4). This talk is a deep dive that describes how BPF (eBPF) works internally on Linux, and dissects some modern performance observability tools. Details covered include the kernel BPF implementation: the verifier, JIT compilation, and the BPF execution environment; the BPF instruction set; different event sources; and how BPF is used by user space, using bpftrace programs as an example. This includes showing how bpftrace is compiled to LLVM IR and then BPF bytecode, and how per-event data and aggregated map data are fetched from the kernel.
Cilium - Bringing the BPF Revolution to Kubernetes Networking and Security
Cilium - Bringing the BPF Revolution to Kubernetes Networking and SecurityThomas Graf BPF is one of the fastest emerging technologies of the Linux kernel. The talk provides an introduction to Cilium which brings the powers of BPF to Kubernetes and other orchestration systems to provide highly scalable and efficient networking, security and load balancing for containers and microservices. The talk will provide an introduction to the capabilities of Cilium today but also deep dives into the emerging roadmap involving networking at the socket layer and service mesh datapath capabilities to provide highly efficient connectivity between cloud native apps and sidecar proxies.
Monitoring, Logging and Tracing on Kubernetes
Monitoring, Logging and Tracing on KubernetesMartin Etmajer The document discusses monitoring, logging and tracing tools for Kubernetes including Heapster, Grafana, Fluentd, Elastic Stack, Jolokia and OpenTracing. It provides examples of deploying Heapster with InfluxDB and Grafana for metrics collection, Fluentd to ingest container logs into Elasticsearch, and using Jolokia and OpenTracing for remote access to JMX metrics and distributed tracing functionality.
Linux Network Stack
Linux Network StackAdrien Mahieux - The document discusses Linux network stack monitoring and configuration. It begins with definitions of key concepts like RSS, RPS, RFS, LRO, GRO, DCA, XDP and BPF.
- It then provides an overview of how the network stack works from the hardware interrupts and driver level up through routing, TCP/IP and to the socket level.
- Monitoring tools like ethtool, ftrace and /proc/interrupts are described for viewing hardware statistics, software stack traces and interrupt information.
Linux BPF Superpowers
Linux BPF SuperpowersBrendan Gregg Video: https://www.facebook.com/atscaleevents/videos/1693888610884236/ . Talk by Brendan Gregg from Facebook's Performance @Scale: "Linux performance analysis has been the domain of ancient tools and metrics, but that's now changing in the Linux 4.x series. A new tracer is available in the mainline kernel, built from dynamic tracing (kprobes, uprobes) and enhanced BPF (Berkeley Packet Filter), aka, eBPF. It allows us to measure latency distributions for file system I/O and run queue latency, print details of storage device I/O and TCP retransmits, investigate blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. This talk will summarize this new technology and some long-standing issues that it can solve, and how we intend to use it at Netflix."
Xdp and ebpf_maps
Xdp and ebpf_mapslcplcp1 This document discusses XDP (eXpress Data Path), a high-performance network data path that allows programs to run on the receive path of a network interface card. XDP enables packet processing using eBPF programs before packets reach the Linux networking stack. The document provides an overview of XDP and its performance advantages over other packet processing methods. It also discusses XDP's current status and support in the Linux kernel as well as example use cases and benchmarks.
왜 쿠버네티스는 systemd로 cgroup을 관리하려고 할까요
왜 쿠버네티스는 systemd로 cgroup을 관리하려고 할까요Jo Hoon 왜 쿠버네티스는 cgroup을 관리하는데 systemd를 쓰려고 하는지 고찰한 문서입니다.
특히 v1.22로 올라가면서 systemd를 사용하도록 설정된 부분을 포함하고 있습니다.
eBPF maps 101
eBPF maps 101SUSE Labs Taipei The document provides an overview of eBPF maps and how they can be used to share data between eBPF programs running in the kernel and userspace applications. It describes how maps are created via the BPF syscall using the BPF_MAP_CREATE command. It also explains how keys and values can be looked up, updated, and deleted from maps using commands like BPF_MAP_LOOKUP_ELEM, BPF_MAP_UPDATE_ELEM, and BPF_MAP_DELETE_ELEM. Finally, it lists the different types of eBPF maps available.
Replacing iptables with eBPF in Kubernetes with Cilium
Replacing iptables with eBPF in Kubernetes with CiliumMichal Rostecki Cilium is an open source project which provides networking, security and load balancing for application services that are deployed using Linux container technologies by using the native eBPF technology in the Linux kernel. In this presentation we talked about:
- The evolution of the BPF filters and explained the advantages of eBPF Filters and its use cases today in Linux especially on how Cilium networking utilizes the eBPF Filters to secure the Kubernetes workload with increased performance when compared to legacy iptables.
- How Cilium uses SOCKMAP for layer 7 policy enforcement - How Cilium integrates with Istio and handles L7 Network Policies with Envoy Proxies.
- The new features since the last release such as running Kubernetes cluster without kube-proxy, providing clusterwide NetworkPolicies, providing fully distributed networking and security observability platform for cloud native workloads etc.
CNCF Singapore - Introduction to Envoy
CNCF Singapore - Introduction to EnvoyHarish Envoy is an open source edge and service proxy developed by Lyft to be used as a communication bus and network proxy. It can be used to add capabilities like traffic management, observability and security to services without code modification. Some key features include lightweight and low latency design, dynamic configuration via xDS APIs, rich observability, and built-in load balancing, failure recovery and traffic control capabilities. It uses concepts like clusters, listeners and routes. The Envoy admin API provides endpoints to interact with it and retrieve stats, configs and logs. Dynamic configuration allows adding and updating resources like clusters and listeners at runtime. Circuit breaking, outlier detection and health checking can be configured and demoed.
Similar to eBPF - Observability In Deep (20)
The Open Source Ecosystem for eBPF in Kubernetes
The Open Source Ecosystem for eBPF in KubernetesAll Things Open Presented at All Things Open 2024
Presented by Andre Fredette, Ph.D. & Billy McFall - Red Hat
Title: The Open Source Ecosystem for eBPF in Kubernetes
Abstract: Kubernetes has become the de facto open source solution for orchestrating containerized applications. However, as the complexity and scale of deployments grow, traditional tools often fall short of providing the granularity and efficiency required by advanced applications. To meet the demand, applications are increasingly leveraging eBPF (extended Berkeley Packet Filters) due to the revolutionary kernel capabilities it enables.
This talk will explore the integration of the eBPF ecosystem with Kubernetes, highlighting its potential to transform how operators and developers observe, secure, and troubleshoot their deployments. We will start by introducing eBPF and its core concepts, including its architecture, programming model, and key benefits such as minimal overhead, improved visibility, and dynamic tracing capabilities. We will then review real-world examples of open source tools which leverage eBPF for networking, security, and observability in Kubernetes environments. We will also explore an open source project called bpfman (https://bpfman.io), an eBPF Manager focusing on simplifying the deployment, administration and visibility of eBPF programs in Kubernetes clusters.
This presentation is designed for Kubernetes operators, security professionals, and developers seeking to deepen their understanding of eBPF and its applications in cloud-native environments. No prior knowledge of eBPF is required, but familiarity with Kubernetes concepts and architecture will be beneficial.
Find more info about All Things Open:
On the web: https://www.allthingsopen.org/
Twitter: https://twitter.com/AllThingsOpen
LinkedIn: https://www.linkedin.com/company/all-things-open/
Instagram: https://www.instagram.com/allthingsopen/
Facebook: https://www.facebook.com/AllThingsOpen
Mastodon: https://mastodon.social/@allthingsopen
Threads: https://www.threads.net/@allthingsopen
2024 conference: https://2024.allthingsopen.org/
Introduction of eBPF - 時下最夯的Linux Technology
Introduction of eBPF - 時下最夯的Linux Technology Jace Liang @ 2020/04/20 SDN x Cloud Native Meetup #27
隨著CNCF將Falco納入incubator project，eBPF這藏於Linux核心內的技術也開始受到矚目。
eBPF，一個從1992年就出現的技術，一路走來經過了甚麼樣的變化?
對於你我目前，或未來的工作又會有甚麼影響呢?
本次分享將會介紹eBPF的前世今生，帶各位了解何謂eBPF。並透過實際範例演示eBPF工具的特殊用法。
eBPF — Divulging The Hidden Super Power.pdf
eBPF — Divulging The Hidden Super Power.pdfSGBSeo If you are a performance engineer/network engineer or even security engineer, the chance of you encountering eBPF technology in the future is very high.
Efficient System Monitoring in Cloud Native Environments
Efficient System Monitoring in Cloud Native EnvironmentsGergely Szabó This document discusses efficient system monitoring in cloud native environments using eBPF. It provides an overview of eBPF and how it can be used for monitoring applications like Prometheus. Specific topics covered include BPF, Linux kernel tracing using kprobes and tracepoints, eBPF maps and programs, and an example Prometheus exporter that leverages eBPF to export metrics.
eBPF — Divulging The Hidden Super Power.pdf
eBPF — Divulging The Hidden Super Power.pdfseo18 If you are a performance engineer/network engineer or even security engineer, the chance of you encountering eBPF technology in the future is very high. eBPF now has a huge community of users, including big players like Meta, Google, Cloudflare, and Netflix all using this tech in their daily operations.
eBPF Debugging Infrastructure - Current Techniques
eBPF Debugging Infrastructure - Current TechniquesNetronome eBPF (extended Berkeley Packet Filter), in particular with its driver-level hook XDP (eXpress Data Path), has increased in importance over the past few years. As a result, the ability to rapidly debug and diagnose problems is becoming more relevant. This talk will cover common issues faced and techniques to diagnose them, including the use of bpftool for map and program introspection, the use of disassembly to inspect generated assembly code and other methods such as using debug prints and how to apply these techniques when eBPF programs are offloaded to the hardware.
The talk will also explore where the current gaps in debugging infrastructure are and suggest some of the next steps to improve this, for example, integrations with tools such as strace, valgrind or even the LLDB debugger.
story_of_bpf-1.pdf
story_of_bpf-1.pdfhegikip775 BPF began as an instruction set for filtering network packets over 30 years ago. In 2011, a startup sought to virtualize networking and build software-defined networks using a technology called iovisor. This led to challenges around crashing kernels and security issues. To address these, the startup developed a new instruction set called extended BPF and a verifier. Getting this technology accepted upstream required building reputation in the kernel and LLVM communities over several years by addressing concerns, making ideas familiar, and demonstrating performance benefits before the first patches were accepted in 2014-2015.
Unifying Network Filtering Rules for the Linux Kernel with eBPF
Unifying Network Filtering Rules for the Linux Kernel with eBPFNetronome At the core of fast network packet processing lies the ability to filter packets, or in other words, to apply a set of rules on packets, usually consisting of a pattern to match (L2 to L4 source and destination addresses and ports, protocols, etc.) and corresponding actions (redirect to a given queue, or drop the packet, etc.). Over the years, several filtering frameworks have been added to Linux. While at the lower level, ethtool can be used to configure N-tuple rules on the receive side for the hardware, the upper layers of the stack got equipped with rules for firewalling (Netfilter), traffic shaping (TC), or packet switching (Open vSwitch for example).
In this presentation, Quentin Monnet reviewed the needs for those filtering frameworks and the particularities of each one. Then focuses on the changes brought by eBPF and XDP in this landscape: as BPF programs allow for very flexible processing and can be attached very low in the stack—at the driver level, or even run on the NIC itself—they offer filtering capabilities with no precedent in terms of performance and versatility in the kernel. Lastly, the third part explores potential leads in order to create bridges between the different rule formats and to make it easier for users to build their filtering eBPF programs.
Building Linux IPv6 DNS Server (Complete Soft Copy)
Building Linux IPv6 DNS Server (Complete Soft Copy)Hari The document discusses building a Linux IPv6 DNS server. It provides an overview of the project which aims to configure a DNS server in Linux with IPv6 name resolution. It discusses the hardware and software requirements including using Red Hat Linux, kernel version 2.4 or higher, and BIND version 9. It also summarizes the key steps in building the DNS server such as creating a new kernel, making the DNS server support IPv6, and providing a backup of the existing kernel.
SRE NL MeetUp - eBPF.pdf
SRE NL MeetUp - eBPF.pdfSiteReliabilityEngin The presentation from Joseph Muli and Rajesh Dutta from Xebia on "How eBPF boost up Kubernetes service networking performance" - as presented on the 13th of April, 2023 at the Site Reliability Engineering NL MeetUp.
ebpf and IO Visor: The What, how, and what next!
ebpf and IO Visor: The What, how, and what next!Affan Syed Extended BPF (eBPF) provides a mechanism for running custom programs inside the Linux kernel that can be used for filtering network packets, monitoring system activity, and more. eBPF programs are written in a restricted subset of C and compiled to bytecode that is verified by the kernel for safety before being run. The BCC toolkit makes it easier to write and load eBPF programs. The IO Visor project aims to further develop eBPF and provide tools and use cases for networking, security, and system tracing applications.
DCSF 19 eBPF Superpowers
DCSF 19 eBPF SuperpowersDocker, Inc. It's been three years since Netflix's Brendan Gregg described the Berkeley Packet Filter as "Superpowers for Linux". Since then there has been an explosion of capabilities and tools based on eBPF, so you've probably heard the term, but do you know what it is and how to use it? In this demo-rich talk we'll explore some of the powerful things we can do with this technology, especially in the context of containers.
Security Monitoring with eBPF
Security Monitoring with eBPFAlex Maestretti Using the new extended Berkley Packet Filter capabilities in Linux to the improve performance of auditing security relevant kernel events around network, file and process actions.
stackconf 2024 | Buzzing across the eBPF Landscape and into the Hive by Bill ...
stackconf 2024 | Buzzing across the eBPF Landscape and into the Hive by Bill ...NETWAYS The buzz around the Linux kernel technology eBPF is growing quickly and it can be hard to know where to start or how to keep up with this technology that is reshaping our infrastructure stack. In this talk, Bill will trace how he got into eBPF, explore some of the applications leveraging eBPF today, and teach others how to dive into the hive of activity around eBPF. People just beginning with eBPF will learn how eBPF makes it possible to have efficient networking, observability without instrumentation, effortless tracing, and real-time security (among other things) without needing your own kernel team. Those already familiar with eBPF will get an overview of the eBPF landscape and learn about many new and expanding eBPF applications that allow them to harness the power without needing to dive into the bytecode. The audience will walk away with an understanding of the buzz around eBPF and knowledge of new tools that may solve some of their problems in networking, observability, and security.
Kernel Recipes 2019 - BPF at Facebook
Kernel Recipes 2019 - BPF at FacebookAnne Nicolas This talk will provide several examples of how Facebook engineers use BPF to scale the networking, prevent denial of service, secure containers, analyze performance. It’s suitable for BPF newbies and experts.
Alexei Starovoitov, Facebook
Ceph Day Beijing - SPDK for Ceph
Ceph Day Beijing - SPDK for CephDanielle Womboldt The document discusses using the Storage Performance Development Kit (SPDK) to optimize Ceph performance. SPDK provides userspace libraries and drivers to unlock the full potential of Intel storage technologies. It summarizes current SPDK support in Ceph's BlueStore backend and proposes leveraging SPDK further to accelerate Ceph's block services through optimized SPDK targets and caching. Collaboration is needed between the SPDK and Ceph communities to fully realize these optimizations.
Ceph Day Beijing - SPDK in Ceph
Ceph Day Beijing - SPDK in CephCeph Community The document discusses using the Storage Performance Development Kit (SPDK) to optimize Ceph storage performance. SPDK provides userspace libraries and drivers to unlock the full potential of Intel storage technologies. It summarizes current SPDK support in Ceph's BlueStore backend and proposes leveraging SPDK further to accelerate Ceph's block services using optimized SPDK targets and caching. Collaboration is needed between the SPDK and Ceph communities to fully realize these performance benefits.
Using eBPF to Measure the k8s Cluster Health
Using eBPF to Measure the k8s Cluster HealthScyllaDB As a k8s cluster-admin your app teams have a certain expectation of your cluster to be available to deploy services at any time without problems. While there is no shortage on metrics in k8s its important to have the right metrics to alert on issues and giving you enough data to react to potential availability issues. Prometheus has become a standard and sheds light on the inner behaviour of Kubernetes clusters and workloads. Lots of KPIs (CPU, IO, network. Etc) in our On-Premise environment are less precise when we start to work in a Cloud environment. Ebpf is the perfect technology that fulfills that requirement as it gives us information down to the kernel level. In 2018 Cloudflare shared an opensource project to expose custom ebpf metrics in Prometheus. Join this session and learn about: • What is ebpf? • What type of metrics we can collect? • How to expose those metrics in a K8s environment. This session will try to deliver a step-by-step guide on how to take advantage of the ebpf exporter.
OpenEmbedded
OpenEmbeddedScott Garman An overview of OpenEmbedded for embedded Linux systems. Presentation given for the Portland Linux User Group Advanced Topics meeting on February 17, 2010.
Building a Linux IPv6 DNS Server Project review PPT v3.0 First review
Building a Linux IPv6 DNS Server Project review PPT v3.0 First reviewHari Building a Linux IPv6 DNS Server Project review PPT v3.0 First review ;For more updates visit : http://techiehari.pbworks.com/
Ad
More from Mydbops (20)
Scaling TiDB for Large-Scale Application
Scaling TiDB for Large-Scale ApplicationMydbops Mydbops MyWebinar 42: Scaling TiDB for Large-Scale Applications
Presenter: Kabilesh P.R., Founding Partner, Mydbops
Is your database slowing down as your business grows?
Scaling databases is a challenge, especially when dealing with high traffic and large workloads. TiDB is designed for scalability, but without the right approach, you may face slow queries, downtime, and migration hurdles.
Join Kabilesh P.R. as he shares real-world use cases, proven strategies, and common pitfalls in scaling TiDB for large applications. This session will help you understand how to optimize performance, improve reliability, and scale seamlessly.
What You'll Learn:
* How to scale TiDB efficiently for large applications
* Common mistakes in scaling and how to avoid them
* Real-world case studies of successful migrations
* Best practices for maintaining performance and reliability
https://www.mydbops.com/
[email protected]
AWS MySQL Showdown - RDS vs RDS Multi AZ vs Aurora vs Serverless - Mydbops...
AWS MySQL Showdown - RDS vs RDS Multi AZ vs Aurora vs Serverless - Mydbops...Mydbops AWS MySQL Showdown - RDS vs RDS Multi AZ vs Aurora vs Serverless - Mydbops Webinar 41
Key takeaways:
* Performance & Scalability – How each service handles workloads
* High Availability & Failover – Ensuring uptime and reliability
* Cost & Efficiency – Which solution gives the best value
* Architecture Deep Dive – Comparing Multi-AZ RDS and Aurora’s distributed model
Who Should Attend?
* Database Architects & Engineers
* DevOps & Cloud Professionals
* CTOs & Tech Decision-Makers
Don't miss out!
#aws #mysql #rds #aurora #serverless #cloud #database #scalability #highavailability #performance #cloudcomputing #devops #tech #engineering #webinar #automation #costoptimization #failover #replication #opensource #datamanagement #cloudarchitecture #cloudservices #datastorage #techcommunity #itprofessionals #dba #event #community #databasemanagement
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: [email protected]
Visit: https://www.mydbops.com/
Mastering Vector Search with MongoDB Atlas - Manosh Malai - Mydbops MyWebinar 39
Mastering Vector Search with MongoDB Atlas - Manosh Malai - Mydbops MyWebinar 39Mydbops Mastering Vector Search with MongoDB Atlas - Manosh Malai - Mydbops MyWebinar 39
In this session, explore how to harness MongoDB's native vector search capabilities to enhance your database and search functionality. From the basics to advanced techniques, gain insights into building intelligent solutions that drive innovation.
What You’ll Learn:
* The fundamentals of vector search in MongoDB Atlas.
* How to store vector embeddings and create efficient indexes.
* Performing similarity queries for applications like semantic search and personalized recommendations.
* Best practices for optimizing performance and scaling vector-based systems effectively.
Whether you’re a developer, data scientist, or database administrator, this webinar will equip you with practical skills to elevate your projects with MongoDB’s advanced features.
Download presentation here: https://www.mydbops.com/webinars/mastering-vector-search-with-mongodb-atlas
This webinar is ideal for database administrators, data engineers, system architects, and anyone involved in MongoDB database management.
#Webinar #mongodb #mongodbatlas #MyWebinar #Mydbops #DatabaseManagement #DevOps #TechWebinar #database #dbms #dba #vectorsearch
Migration Journey To TiDB - Kabilesh PR - Mydbops MyWebinar 38
Migration Journey To TiDB - Kabilesh PR - Mydbops MyWebinar 38Mydbops Migration Journey To TiDB - Kabilesh PR - Mydbops MyWebinar 38
Youtube video link: https://youtu.be/_WgXm1Ykj8c
What You Will Learn
* Data Migration Strategies – Understand the best approaches for transferring data to TiDB with minimal disruption.
* Seamless Replication – Learn how to maintain data consistency and minimize downtime during the migration process.
* Schema Design Adjustments – Explore the key schema design adjustments necessary for optimal TiDB performance.
* Challenges & Solutions – Gain practical insights into tackling common migration challenges to ensure a smooth transition.
This webinar is ideal for database administrators, data engineers, system architects, and anyone involved in database management and migrations. Whether you are considering TiDB as a new solution or already exploring it, this session will equip you with valuable knowledge to streamline your migration journey.
#Webinar #TiDB #MyWebinar #Mydbops #DatabaseManagement #migration #DevOps #TechWebinar #database #dbms #dba #distributedsql #sql #HTAP
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, PostgreSQL and TiDB.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: [email protected]
Visit: https://www.mydbops.com/
AWS Blue Green Deployment for Databases - Mydbops
AWS Blue Green Deployment for Databases - MydbopsMydbops Mastering AWS Blue/Green Deployment for Databases - Mydbops MyWebinar 37
What You Will Learn
* Key Principles of Blue/Green Deployment: Understand the fundamental concepts that drive this deployment strategy.
* Step-by-Step Implementation: A detailed walkthrough of the processes involved in setting up Blue/Green deployments using AWS services.
* Best Practices: Discover industry best practices to minimize risks and avoid common pitfalls during deployments.
* Database Management with AWS: Learn how to effectively use AWS services like RDS and Aurora for safe database upgrades, including rollback options in the event of deployment issues.
This webinar is ideal for database administrators, DevOps engineers, cloud architects, and anyone interested in mastering AWS deployment strategies. Whether you are new to AWS or looking to enhance your skills, this session will provide valuable insights and practical knowledge.
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: [email protected]
Visit: https://www.mydbops.com/
What's New In MySQL 8.4 LTS Mydbops MyWebinar Edition 36
What's New In MySQL 8.4 LTS Mydbops MyWebinar Edition 36Mydbops What's New in MySQL 8.4? Mydbops MyWebinar Edition 36 - Vinoth Kanna, Founding Partner, Mydbops
Join us as we explore the latest advancements in MySQL 8.4 and discover how these updates can enhance your database management.
Key highlights:
* GTID Tags for improved replication
* Automatic histogram updates for query optimization
* Clone Plugin for faster replication
* Backward-compatible backups with mysqldump
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: [email protected]
What's New in PostgreSQL 17? - Mydbops MyWebinar Edition 35
What's New in PostgreSQL 17? - Mydbops MyWebinar Edition 35Mydbops What's New in PostgreSQL 17? - Mydbops MyWebinar Edition 35
Key Features of PostgreSQL 17:
• Discover how PostgreSQL 17 has optimized performance, making your queries run faster and more efficiently.
• Learn about the new indexing techniques that provide quicker access to data and reduce the load on your system.
• Explore the expanded support for various data types, allowing for more flexibility in how you store and manipulate data.
• PostgreSQL 17 introduces new functions that simplify data manipulation and enhance your ability to handle complex queries.
• Understand the improvements in logical replication that make data synchronization more robust and easier to manage.
• Get insights into the latest security enhancements designed to protect your data more effectively than ever before.
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: [email protected]
Visit: https://www.mydbops.com/
What's New in MongoDB 8.0 - Mydbops MyWebinar Edition 34
What's New in MongoDB 8.0 - Mydbops MyWebinar Edition 34Mydbops What's New in MongoDB 8.0 - Mydbops MyWebinar Edition 34
* Performance Enhancements: Discover the impressive speed boosts in write and read performance, with benchmarks showing up to a 54% improvement in write-heavy workloads and a 27% improvement in read-heavy workloads.
* Time Series Enhancements: Learn about the new block processing feature and the transition to columnar storage, which promises faster queries and smarter use of storage space.
* Command Path Optimization: Understand the major overhaul of the command path for faster response times and more efficient database operations.
* Express Path Efficiency: Explore the new Express Path designed to optimize specific queries for speed and reduced overhead.
* Resource Efficiency: Learn about the reduced memory fragmentation and enhanced peak load behavior for better overall system performance.
* Advanced Sharding Capabilities: Discover the new capabilities for moving and converting collections between shards.
* Queryable Encryption Enhancements: Gain insights into the support for range queries within encrypted fields, enhancing security and functionality.
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: [email protected]
Visit: https://www.mydbops.com/
Follow us on LinkedIn: https://in.linkedin.com/company/mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsMydbops This presentation, delivered at the Postgres Bangalore (PGBLR) Meetup-2 on June 29th, 2024, dives deep into connection pooling for PostgreSQL databases. Aakash M, a PostgreSQL Tech Lead at Mydbops, explores the challenges of managing numerous connections and explains how connection pooling optimizes performance and resource utilization.
Key Takeaways:
* Understand why connection pooling is essential for high-traffic applications
* Explore various connection poolers available for PostgreSQL, including pgbouncer
* Learn the configuration options and functionalities of pgbouncer
* Discover best practices for monitoring and troubleshooting connection pooling setups
* Gain insights into real-world use cases and considerations for production environments
This presentation is ideal for:
* Database administrators (DBAs)
* Developers working with PostgreSQL
* DevOps engineers
* Anyone interested in optimizing PostgreSQL performance
Contact [email protected] for PostgreSQL Managed, Consulting and Remote DBA Services
Read/Write Splitting using MySQL Router - Mydbops Meetup16
Read/Write Splitting using MySQL Router - Mydbops Meetup16Mydbops Read/Write Splitting using MySQL Router - Mydbops Meetup16
Topic: Scale Your Database Traffic with Read/Write Splitting Using MySQL Router
Date & Time: 8th June | 10 AM - 1 PM IST
Abstract:
This session dives deep into the power of Read/Write splitting, a technique that significantly improves database performance and application scalability.
* Challenges of managing read/write workloads on a single server.
* How MySQL Router enables transparent read/write splitting.
* Step-by-step guidance for implementation in your MySQL environment.
* Real-world use cases and benefits.
* No code changes required!
TiDB - From Data to Discovery: Exploring the Intersection of Distributed Dat...
TiDB - From Data to Discovery: Exploring the Intersection of Distributed Dat...Mydbops Speaker: Sreedharma Vijayan, India Director at PingCAP, at Mydbops Open Source Meetup 16.
Topic: From Data to Discovery: Exploring the Intersection of Distributed Databases and AI
Date & Time: 8th June | 10 AM - 1 PM IST
In this session, Sreedharma Vijayan delves into the exciting intersection of distributed databases and AI.
You'll discover how TiDB empowers digital industries to:
* Handle explosive data growth that challenges traditional databases.
* Optimize data distribution and access logic within your applications.
* Unlock valuable insights to fuel AI-powered workflows.
MySQL InnoDB Storage Engine: Deep Dive - Mydbops
MySQL InnoDB Storage Engine: Deep Dive - MydbopsMydbops This presentation, titled "MySQL - InnoDB" and delivered by Mayank Prasad at the Mydbops Open Source Database Meetup 16 on June 8th, 2024, covers dynamic configuration of REDO logs and instant ADD/DROP columns in InnoDB.
This presentation dives deep into the world of InnoDB, exploring two ground-breaking features introduced in MySQL 8.0:
• Dynamic Configuration of REDO Logs: Enhance your database's performance and flexibility with on-the-fly adjustments to REDO log capacity. Unleash the power of the snake metaphor to visualize how InnoDB manages REDO log files.
• Instant ADD/DROP Columns: Say goodbye to costly table rebuilds! This presentation unveils how InnoDB now enables seamless addition and removal of columns without compromising data integrity or incurring downtime.
Key Learnings:
• Grasp the concept of REDO logs and their significance in InnoDB's transaction management.
• Discover the advantages of dynamic REDO log configuration and how to leverage it for optimal performance.
• Understand the inner workings of instant ADD/DROP columns and their impact on database operations.
• Gain valuable insights into the row versioning mechanism that empowers instant column modifications.
Demystifying Real time Analytics with TiDB
Demystifying Real time Analytics with TiDBMydbops Are you struggling to gain real-time insights from your data?
Mydbops MyWebinar Edition 33 can help you.
Discover how TiDB can revolutionize your analytics game!
Topic: Demystifying Real-Time Analytics with TiDB
Presenter: Kabilesh PR, Founding Partner, Mydbops
In today's data-driven world, real-time analytics is essential for businesses to make quick decisions based on immediate insights. This webinar will explore how TiDB empowers organizations to unlock the full potential of their data. We'll delve into TiDB's powerful capabilities, including:
• Hybrid Transactional/Analytical Processing (HTAP): Run high-speed transactions and complex queries simultaneously without sacrificing performance.
• Real-time Analytics: Gain immediate insights from your data to make informed decisions faster.
• Scalability & Flexibility: Effortlessly scale your database to accommodate growing data volumes.
Download our previous webinar presentations here for free: https://www.mydbops.com/webinars
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top open-source databases: MySQL, MongoDB, MariaDB, PostgreSQL, TiDB and Cassandra.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: [email protected]
Visit: https://www.mydbops.com/
Follow us on LinkedIn: / mydbops
Blogs: https://www.mydbops.com/blog/
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMydbops Mydbops Opensource Database Meetup 16
Topic: Must-Know PostgreSQL Extensions for Developers and DBAs During Migration
Speaker: Deepak Mahto, Founder of DataCloudGaze Consulting
Date & Time: 8th June | 10 AM - 1 PM IST
Venue: Bangalore International Centre, Bangalore
Abstract: Discover how PostgreSQL extensions can be your secret weapon! This talk explores how key extensions enhance database capabilities and streamline the migration process for users moving from other relational databases like Oracle.
Key Takeaways:
* Learn about crucial extensions like oracle_fdw, pgtt, and pg_audit that ease migration complexities.
* Gain valuable strategies for implementing these extensions in PostgreSQL to achieve license freedom.
* Discover how these key extensions can empower both developers and DBAs during the migration process.
* Don't miss this chance to gain practical knowledge from an industry expert and stay updated on the latest open-source database trends.
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: [email protected]
Visit: https://www.mydbops.com/
Follow us on LinkedIn: https://in.linkedin.com/company/mydbops
For more details and updates, please follow up the below links.
Meetup Page : https://www.meetup.com/mydbops-databa...
Twitter: https://twitter.com/mydbopsofficial
Blogs: https://www.mydbops.com/blog/
Facebook(Meta): https://www.facebook.com/mydbops/
Efficient MySQL Indexing and what's new in MySQL Explain
Efficient MySQL Indexing and what's new in MySQL ExplainMydbops Efficient MySQL Indexing & What's New in MySQL Explain - Mydbops MyWebinar Edition 32
This session will delve into:
• Strategic indexing techniques: Learn how to optimize your MySQL database by implementing effective indexing strategies, including when to avoid fulltext indexes to prevent wasted resources.
• Demystifying the new MySQL Explain: We'll explore the latest enhancements to the MySQL Explain plan's JSON output format. Discover how to store the output in a variable for further analysis – a valuable addition introduced in MySQL 8.3. You'll also learn about the explain_json_format_version variable, which empowers you to choose between different JSON output versions for greater flexibility.
• Live Chat Engagement: We encourage you to actively participate throughout the webinar! Use the chat functionality to ask questions and share your experiences with indexing and Explain.
This webinar is perfect for:
• Database administrators (DBAs)
• Developers
• Anyone seeking to optimize MySQL performance and streamline database queries
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: [email protected]
Visit: https://www.mydbops.com/
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops Scale your database traffic with Read & Write split using MySQL Router
This webinar recording dives into the world of MySQL Router and its capabilities for effectively managing high database traffic loads.
You'll learn:
• The challenges of scaling database traffic
• How MySQL Router facilitates read/write splitting
• The benefits of implementing read/write splitting
• Step-by-step demonstrations for configuring MySQL Router for:
1. Static read/write routing for standalone servers
2. Dynamic read/write split for InnoDB Cluster & Replica Set
• A comparison of popular load balancers (MySQL Router, ProxySQL, Maxscale)
Mydbops is a trusted database management and consultancy provider, helping businesses achieve optimal database performance and scalability.
Connect with Mydbops!
Website: https://www.mydbops.com/
Email: [email protected]
PostgreSQL Schema Changes with pg-osc - Mydbops @ PGConf India 2024
PostgreSQL Schema Changes with pg-osc - Mydbops @ PGConf India 2024Mydbops Title: PostgreSQL Schema Changes with Minimal Downtime using pg_osc
Speaker: Aakash M, Mydbops
Event: PGConf India, 2024
Description:
This presentation explores pg_osc, a tool that enables efficient schema changes in PostgreSQL tables with minimal downtime and locking. It addresses the challenges of traditional ALTER statements and provides a smoother alternative.
Key points covered:
• Introduction to pg_osc and its benefits.
• Limitations of ALTER statements and how pg_osc overcomes them.
• Step-by-step explanation of the pg_osc process.
• Prominent features and considerations for using pg_osc.
• References and resources for further exploration.
Target Audience:
• Database administrators
• Developers working with PostgreSQL
• Anyone interested in optimizing schema changes
This presentation provides valuable insights for anyone seeking to streamline schema modifications in PostgreSQL while minimizing disruptions.
Choosing the Right Database: Exploring MySQL Alternatives for Modern Applicat...
Choosing the Right Database: Exploring MySQL Alternatives for Modern Applicat...Mydbops Choosing the Right Database: Exploring MySQL Alternatives for Modern Applications by Bhanu Jamwal, Head of Solution Engineering, PingCAP at the Mydbops Opensource Database Meetup 14.
This presentation discusses the challenges in choosing the right database for modern applications, focusing on MySQL alternatives. It highlights the growth of new applications, the need to improve infrastructure, and the rise of cloud-native architecture.
The presentation explores alternatives to MySQL, such as MySQL forks, database clustering, and distributed SQL. It introduces TiDB as a distributed SQL database for modern applications, highlighting its features and top use cases.
Case studies of companies benefiting from TiDB are included. The presentation also outlines TiDB's product roadmap, detailing upcoming features and enhancements.
Mastering Aurora PostgreSQL Clusters for Disaster Recovery
Mastering Aurora PostgreSQL Clusters for Disaster RecoveryMydbops The presentation "Mastering Aurora PostgreSQL Clusters for Disaster Recovery" by Bhuvanesh, Co-Founder & CTO of ShellKode, at the Mydbops OpenSource Database Meetup 14 covers advanced topics in managing Aurora PostgreSQL clusters for disaster recovery purposes.
Bhuvanesh discusses key features of Aurora, such as its decoupled storage and compute layers, auto scaling capabilities, and native replication, highlighting its benefits over traditional RDS instances. He also explores Aurora Global Databases, explaining how they enable replication of data across regions for geo-span applications with low latency.
The presentation includes architecture details, such as physical and log replication, and managed failover options for ensuring high availability. Bhuvanesh shares real-world experiences and best practices for managing Aurora clusters, including handling replication lag and TLS certificate management.
Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open So...
Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open So...Mydbops Navigating Transactions: ACID Complexity in Modern Databases- Mydbops Open Source Database Meetup 15
Shivji explores the evolution of transactions, implementation challenges, and insights into distributed database environments. Whether you're a database enthusiast or a tech enthusiast, this presentation offers valuable insights into the world of database management.
Contents:
• Historical perspective of transactions
• Implementing transactions
• Challenges and trade-offs in ACID properties
• Distributed transactions in modern databases like Amazon Aurora, DynamoDB, and Google Spanner
Key Takeaways:
• Understanding the evolution of transactions in databases
• Insights into the challenges of implementing ACID properties
• Exploration of distributed transaction models in leading database systems
Ad
Recently uploaded (20)
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroUiPathCommunity This session is designed to equip developers with the skills needed to build mission-critical, end-to-end processes that seamlessly orchestrate agents, people, and robots.
📕 Here's what you can expect:
- Modeling: Build end-to-end processes using BPMN.
- Implementing: Integrate agentic tasks, RPA, APIs, and advanced decisioning into processes.
- Operating: Control process instances with rewind, replay, pause, and stop functions.
- Monitoring: Use dashboards and embedded analytics for real-time insights into process instances.
This webinar is a must-attend for developers looking to enhance their agentic automation skills and orchestrate robust, mission-critical processes.
👨🏫 Speaker:
Andrei Vintila, Principal Product Manager @UiPath
This session streamed live on April 29, 2025, 16:00 CET.
Check out all our upcoming Dev Dives sessions at https://community.uipath.com/dev-dives-automation-developer-2025/.
Semantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AIartmondano By 2026, AI agents will consume 10x more enterprise data than humans, but with none of the contextual understanding that prevents catastrophic misinterpretations.
What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...Vishnu Singh Chundawat The MCP (Model Context Protocol) is a framework designed to manage context and interaction within complex systems. This SlideShare presentation will provide a detailed overview of the MCP Model, its applications, and how it plays a crucial role in improving communication and decision-making in distributed systems. We will explore the key concepts behind the protocol, including the importance of context, data management, and how this model enhances system adaptability and responsiveness. Ideal for software developers, system architects, and IT professionals, this presentation will offer valuable insights into how the MCP Model can streamline workflows, improve efficiency, and create more intuitive systems for a wide range of use cases.
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix Talk at the final event of Data Fusion Dynamics: A Collaborative UK-Saudi Initiative in Cybersecurity and Artificial Intelligence funded by the British Council UK-Saudi Challenge Fund 2024, Cardiff Metropolitan University, 29th April 2025
https://alandix.com/academic/talks/CMet2025-AI-Changes-Everything/
Is AI just another technology, or does it fundamentally change the way we live and think?
Every technology has a direct impact with micro-ethical consequences, some good, some bad. However more profound are the ways in which some technologies reshape the very fabric of society with macro-ethical impacts. The invention of the stirrup revolutionised mounted combat, but as a side effect gave rise to the feudal system, which still shapes politics today. The internal combustion engine offers personal freedom and creates pollution, but has also transformed the nature of urban planning and international trade. When we look at AI the micro-ethical issues, such as bias, are most obvious, but the macro-ethical challenges may be greater.
At a micro-ethical level AI has the potential to deepen social, ethnic and gender bias, issues I have warned about since the early 1990s! It is also being used increasingly on the battlefield. However, it also offers amazing opportunities in health and educations, as the recent Nobel prizes for the developers of AlphaFold illustrate. More radically, the need to encode ethics acts as a mirror to surface essential ethical problems and conflicts.
At the macro-ethical level, by the early 2000s digital technology had already begun to undermine sovereignty (e.g. gambling), market economics (through network effects and emergent monopolies), and the very meaning of money. Modern AI is the child of big data, big computation and ultimately big business, intensifying the inherent tendency of digital technology to concentrate power. AI is already unravelling the fundamentals of the social, political and economic world around us, but this is a world that needs radical reimagining to overcome the global environmental and human challenges that confront us. Our challenge is whether to let the threads fall as they may, or to use them to weave a better future.
Leading AI Innovation As A Product Manager - Michael Jidael
Leading AI Innovation As A Product Manager - Michael JidaelMichael Jidael Unlike traditional product management, AI product leadership requires new mental models, collaborative approaches, and new measurement frameworks. This presentation breaks down how Product Managers can successfully lead AI Innovation in today's rapidly evolving technology landscape. Drawing from practical experience and industry best practices, I shared frameworks, approaches, and mindset shifts essential for product leaders navigating the unique challenges of AI product development.
In this deck, you'll discover:
- What AI leadership means for product managers
- The fundamental paradigm shift required for AI product development.
- A framework for identifying high-value AI opportunities for your products.
- How to transition from user stories to AI learning loops and hypothesis-driven development.
- The essential AI product management framework for defining, developing, and deploying intelligence.
- Technical and business metrics that matter in AI product development.
- Strategies for effective collaboration with data science and engineering teams.
- Framework for handling AI's probabilistic nature and setting stakeholder expectations.
- A real-world case study demonstrating these principles in action.
- Practical next steps to begin your AI product leadership journey.
This presentation is essential for Product Managers, aspiring PMs, product leaders, innovators, and anyone interested in understanding how to successfully build and manage AI-powered products from idea to impact. The key takeaway is that leading AI products is about creating capabilities (intelligence) that continuously improve and deliver increasing value over time.
tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdffjgm517 descaripcion detallada del avance de las tecnologias en mesopotamia, egipto, roma y grecia.
Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp This is the keynote of the Into the Box conference, highlighting the release of the BoxLang JVM language, its key enhancements, and its vision for the future.
Procurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptxJon Hansen Procurement Insights integrated Historic Procurement Industry Archives, serves as a powerful complement — not a competitor — to other procurement industry firms. It fills critical gaps in depth, agility, and contextual insight that most traditional analyst and association models overlook.
Learn more about this value- driven proprietary service offering here.
Automation Dreamin': Capture User Feedback From Anywhere
Automation Dreamin': Capture User Feedback From AnywhereLynda Kane Slide Deck from Automation Dreamin' 2022 presentation Capture User Feedback from Anywhere
Linux Professional Institute LPIC-1 Exam.pdf
Linux Professional Institute LPIC-1 Exam.pdfRHCSA Guru Introduction to LPIC-1 Exam - overview, exam details, price and job opportunities
"PHP and MySQL CRUD Operations for Student Management System"
"PHP and MySQL CRUD Operations for Student Management System"Jainul Musani Php with MySQL database Connectivity - CRUD Operations
Network Security. Different aspects of Network Security.
Network Security. Different aspects of Network Security.gregtap1 Network Security. Different aspects of Network Security.
Buckeye Dreamin 2024: Assessing and Resolving Technical Debt
Buckeye Dreamin 2024: Assessing and Resolving Technical DebtLynda Kane Slide Deck from Buckeye Dreamin' 2024 presentation Assessing and Resolving Technical Debt. Focused on identifying technical debt in Salesforce and working towards resolving it.
Learn the Basics of Agile Development: Your Step-by-Step Guide
Learn the Basics of Agile Development: Your Step-by-Step GuideMarcel David New to Agile? This step-by-step guide is your perfect starting point. "Learn the Basics of Agile Development" simplifies complex concepts, providing you with a clear understanding of how Agile can improve software development and project management. Discover the benefits of iterative work, team collaboration, and flexible planning.
Rusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond Sparkcarlyakerly1 Spark is a powerhouse for large datasets, but when it comes to smaller data workloads, its overhead can sometimes slow things down. What if you could achieve high performance and efficiency without the need for Spark?
At S&P Global Commodity Insights, having a complete view of global energy and commodities markets enables customers to make data-driven decisions with confidence and create long-term, sustainable value. 🌍
Explore delta-rs + CDC and how these open-source innovations power lightweight, high-performance data applications beyond Spark! 🚀
Salesforce AI Associate 2 of 2 Certification.docx
Salesforce AI Associate 2 of 2 Certification.docxJosé Enrique López Rivera Salesforce AI Associate 2 of 2
AI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global TrendsInData Labs In this infographic, we explore how businesses can implement effective governance frameworks to address AI data privacy. Understanding it is crucial for developing effective strategies that ensure compliance, safeguard customer trust, and leverage AI responsibly. Equip yourself with insights that can drive informed decision-making and position your organization for success in the future of data privacy.
This infographic contains:
-AI and data privacy: Key findings
-Statistics on AI data privacy in the today’s world
-Tips on how to overcome data privacy challenges
-Benefits of AI data security investments.
Keep up-to-date on how AI is reshaping privacy standards and what this entails for both individuals and organizations.
What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...Vishnu Singh Chundawat
eBPF - Observability In Deep
- 1. Kabilesh P R Co-Founder, Mydbops October 3rd, 2020 Mydbops Database Meetup -7 EBPF Observability In Deep
- 2. Interested in Open Source technologies Building Solutions on MySQL, Cloud & MongoDB Active Tech Speaker/Blogger Co-Founder, Mydbops Kabilesh PR About Me
- 3. Consulting Services Managed Services Focuses on Top Opensource database MySQL, MongoDB and PostgreSQL Mydbops Services
- 4. 400 + Clients In 4 Yrs. of Operations Our Clients
- 5. Agenda BPF & EBPF BPF Internals Tools in action
- 6. BPF Vs EBPF BPF(Berkely Packet Filter) an "in-kernal VM" for packet filter developed in 1992 by Steven McCanne and Van Jacobson Introduced with the Linux Kernel 2.1.75 in 1997 EBPF(Extended BPF), has been there from 2014 with LinuX kernel from version 3.15... It's no more a acronym, Since it not only does packet filtering
- 7. Pre BPF Analysis 1. uptime 2. dmesg -T | tail 3. vmstat 1 4. mpstat -P ALL 1 5. pidstat 1 6. iostat -xdz 1 7. free -m 8. sar -n DEV 1 9. sar -n TCP,ETCP 1 10. top Mostly based on the procfs (/proc) -- post-process manner
- 8. What is BPF eBPF can prepare user information in kernel context and transfer only needed information to user space. So far, support of kprobes, tracepoints, uprobes and perf_events filtering using eBPF have been implemented in the upstream kernel A user can interact with the eBPF kernel using a bpf() system call whose prototype is: int bpf(int cmd, union bpf_attr *attr, unsigned int size); eBPF is a revolutionary technology that can run sandboxed programs in the Linux kernel without changing kernel source code or loading kernel modules. By making the Linux kernel programmable, infrastructure software can leverage existing layers, making them more intelligent and feature-rich without continuing to add additional layers of complexity to the system (Dynamic tracing)
- 9. update to modify the data). Microkernel model
- 10. Development is being actively done later is better Execution Plan Analysis Linux Kernel support for eBPF
- 11. Improvements with Kernel Versions
- 12. BPF Internals
- 13. BPF Internals
- 14. BPF instruction are verified against end-loop and sanity (Verifier) Interpretor/Compiler uses GCC/LVM compiler Just-in-time (JIT) compiler translates eBPF bytecode into a host system's assembly code Registers to store instructions eBPF uses map as generic key/value data structure for data transfer between Kernel and user space BPF Internals
- 15. BPF Internals
- 16. Current State of BPF
- 18. Closer view of Sub-Systems CPU (Scheduling) Memory Disks File Systems Networking Applications Kernel Hypervisors Containers
- 19. Newer Kernal is better, Great to have > 4.4 https://github.com/iovisor/bcc/blob/master/INSTALL.md (choose according to the version) Installation Ubuntu Package Ubuntu Packages Source packages and the binary packages produced from them can be found at packages.ubuntu.com. #sudo apt-get install bpfcc-tools linux-headers-$(uname -r) The tools are installed in /sbin (/usr/sbin in Ubuntu 18.04) with a -bpfcc extension. Try running sudo opensnoop-bpfcc.
- 20. Iovisor: BPF Compiler Collection (BCC). BCC is a toolkit for creating efficient kernel tracing and manipulation programs, and includes several useful tools Installation Installation Ubuntu #sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4052245BD4284CDD #echo "deb https://repo.iovisor.org/apt/$(lsb_release -cs) $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/iovisor.list #sudo apt-get update #sudo apt-get install bcc-tools libbcc-examples linux-headers-$(uname - r)
- 21. Installation Installation Centos8 With Centos8, the package comes with the default repo #yum install bcc-tools Location: /usr/share/bcc/tools
- 22. Tools In Action
- 23. GUI tools Cloud-Flare has developed a open-source exporter around EPBF tool to ingest data to prometheus and Grafana for visualisation PMM also supports EBPF with external exporter support Netflix has their own GUI tool called Vector built on top of EBPF Many more enhancements are in progress
- 25. Reach Us : [email protected] Thank You