Bpf performance tools chapter 4 bcc
1 like395 views
Study report about BPF Performance Tools (book) The video is in https://www.youtube.com/watch?v=GlKtraMn4Ic
1 of 19
Download to read offline
![task_switch.py
[/path/to/bcc/examples/tracing]% sudo python task_switch.py
task_switch[ 0-> 2520]=10
task_switch[14980-> 0]=1
task_switch[ 1096-> 0]=1
task_switch[19347->14055]=1
task_switch[14423-> 0]=1
task_switch[ 0-> 53]=1
task_switch[ 0->28879]=60](https://image.slidesharecdn.com/bpfperformancetoolschapter4-bcc-200223205012/85/Bpf-performance-tools-chapter-4-bcc-8-320.jpg)
Ad
Recommended
Spying on the Linux kernel for fun and profit
Spying on the Linux kernel for fun and profitAndrea Righi Do you ever wonder what the kernel is doing while your code is running? This talk will explore some methodologies and techniques (eBPF, ftrace, etc.) to look under the hood of the Linux kernel and understand what it’s actually doing behind the scenes.
Linux kernel tracing superpowers in the cloud
Linux kernel tracing superpowers in the cloudAndrea Righi The Linux 4.x series introduced a new powerful engine of programmable tracing (BPF) that allows to actually look inside the kernel at runtime. This talk will show you how to exploit this engine in order to debug problems or identify performance bottlenecks in a complex environment like a cloud. This talk will cover the latest Linux superpowers that allow to see what is happening “under the hood” of the Linux kernel at runtime. I will explain how to exploit these “superpowers” to measure and trace complex events at runtime in a cloud environment. For example, we will see how we can measure latency distribution of filesystem I/O, details of storage device operations, like individual block I/O request timeouts, or TCP buffer allocations, investigating stack traces of certain events, identify memory leaks, performance bottlenecks and a whole lot more.
eBPF Trace from Kernel to Userspace
eBPF Trace from Kernel to UserspaceSUSE Labs Taipei This document discusses eBPF (extended Berkeley Packet Filter), which allows tracing from the Linux kernel to userspace using BPF programs. It provides an overview of eBPF including extended registers, verification, maps, and probes. Examples are given of using eBPF for tracing functions like kfree_skb() and the C library function malloc. The Berkeley Compiler Collection (BCC) makes it easy to write eBPF programs in C and Python.
BPF Internals (eBPF)
BPF Internals (eBPF)Brendan Gregg USENIX LISA2021 talk by Brendan Gregg (https://www.youtube.com/watch?v=_5Z2AU7QTH4). This talk is a deep dive that describes how BPF (eBPF) works internally on Linux, and dissects some modern performance observability tools. Details covered include the kernel BPF implementation: the verifier, JIT compilation, and the BPF execution environment; the BPF instruction set; different event sources; and how BPF is used by user space, using bpftrace programs as an example. This includes showing how bpftrace is compiled to LLVM IR and then BPF bytecode, and how per-event data and aggregated map data are fetched from the kernel.
eBPF Perf Tools 2019
eBPF Perf Tools 2019Brendan Gregg The document describes a biolatency tool that traces block device I/O latency using eBPF. It discusses how the tool was originally written in the bcc framework using C/BPF, but has since been rewritten in the bpftrace framework using a simpler one-liner script. It provides examples of the bcc and bpftrace implementations of biolatency.
Understanding eBPF in a Hurry!
Understanding eBPF in a Hurry!Ray Jenkins eBPF is an exciting new technology that is poised to transform Linux performance engineering. eBPF enables users to dynamically and programatically trace any kernel or user space code path, safely and efficiently. However, understanding eBPF is not so simple. The goal of this talk is to give audiences a fundamental understanding of eBPF, how it interconnects existing Linux tracing technologies, and provides a powerful aplatform to solve any Linux performance problem.
eBPF Basics
eBPF BasicsMichael Kehoe This document provides an overview of cBPF and eBPF. It discusses the history and implementation of cBPF, including how it was originally used for packet filtering. It then covers eBPF in more depth, explaining what it is, its history, implementation including different program types and maps. It also discusses several uses of eBPF including networking, firewalls, DDoS mitigation, profiling, security, and chaos engineering. Finally, it introduces XDP and DPDK, comparing XDP's benefits over DPDK.
Linux 4.x Tracing: Performance Analysis with bcc/BPF
Linux 4.x Tracing: Performance Analysis with bcc/BPFBrendan Gregg Talk about bcc/eBPF for SCALE15x (2017) by Brendan Gregg. "BPF (Berkeley Packet Filter) has been enhanced in the Linux 4.x series and now powers a large collection of performance analysis and observability tools ready for you to use, included in the bcc (BPF Complier Collection) open source project. BPF nowadays can do system tracing, software defined networks, and kernel fast path: much more than just filtering packets! This talk will focus on the bcc/BPF tools for performance analysis, which make use of other built in Linux capabilities: dynamic tracing (kprobes and uprobes) and static tracing (tracepoints and USDT). There are now bcc tools for measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. Tracing superpowers have finally arrived, built in to Linux."
Performance Analysis Tools for Linux Kernel
Performance Analysis Tools for Linux Kernellcplcp1 Perf is a collection of Linux kernel tools for performance monitoring and profiling. It provides sampling and profiling of the system to analyze performance bottlenecks. Perf supports hardware events from the CPU performance counters, software events from the kernel, and tracepoint events from the kernel and loaded modules. It offers tools like perf record to sample events and store them, perf report to analyze stored samples, and perf trace to trace system events in real-time.
bcc/BPF tools - Strategy, current tools, future challenges
bcc/BPF tools - Strategy, current tools, future challengesIO Visor Project Brendan Gregg discusses the current state and future potential of BPF and BCC tools for observability in Linux. He outlines 18 areas where BPF support has progressed and 16 areas still needing work. Gregg also discusses challenges like dynamic tracing stability, overhead, ease of coding, and developing visualizations. He proposes finishing ports of his old DTrace tools and links to resources on BPF, BCC, and flame graphs.
Linux 4.x Tracing Tools: Using BPF Superpowers
Linux 4.x Tracing Tools: Using BPF SuperpowersBrendan Gregg Talk for USENIX LISA 2016 by Brendan Gregg.
"Linux 4.x Tracing Tools: Using BPF Superpowers
The Linux 4.x series heralds a new era of Linux performance analysis, with the long-awaited integration of a programmable tracer: Enhanced BPF (eBPF). Formally the Berkeley Packet Filter, BPF has been enhanced in Linux to provide system tracing capabilities, and integrates with dynamic tracing (kprobes and uprobes) and static tracing (tracepoints and USDT). This has allowed dozens of new observability tools to be developed so far: for example, measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. Tracing superpowers have finally arrived.
In this talk I'll show you how to use BPF in the Linux 4.x series, and I'll summarize the different tools and front ends available, with a focus on iovisor bcc. bcc is an open source project to provide a Python front end for BPF, and comes with dozens of new observability tools (many of which I developed). These tools include new BPF versions of old classics, and many new tools, including: execsnoop, opensnoop, funccount, trace, biosnoop, bitesize, ext4slower, ext4dist, tcpconnect, tcpretrans, runqlat, offcputime, offwaketime, and many more. I'll also summarize use cases and some long-standing issues that can now be solved, and how we are using these capabilities at Netflix."
LPC2019 BPF Tracing Tools
LPC2019 BPF Tracing ToolsBrendan Gregg This document discusses Brendan Gregg's opinions on various tracing tools including sysdig, perf, ftrace, eBPF, bpftrace, and BPF perf tools. It provides a table comparing the scope, capability, and ease of use of these tools. It then gives an example of using BPF perf tools to analyze readahead performance. Finally, it outlines desired additions to tracing capabilities and BPF helpers as well as challenges in areas like function tracing without frame pointers.
UM2019 Extended BPF: A New Type of Software
UM2019 Extended BPF: A New Type of SoftwareBrendan Gregg BPF (Berkeley Packet Filter) has evolved from a limited virtual machine for efficient packet filtering to a new type of software called extended BPF. Extended BPF allows for custom, efficient, and production-safe performance analysis tools and observability programs to be run in the Linux kernel through BPF. It enables new event-based applications running as BPF programs attached to various kernel events like kprobes, uprobes, tracepoints, sockets, and more. Major companies like Facebook, Google, and Netflix are using BPF programs for tasks like intrusion detection, container security, firewalling, and observability with over 150,000 AWS instances running BPF programs. BPF provides a new program model and security features compared
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021Valeriy Kravchuk Bpftrace is a relatively new eBPF-based open source tracer for modern Linux versions (kernels 5.x.y) that is useful for analyzing production performance problems and troubleshooting software. Basic usage of the tool, as well as bpftrace one liners and advanced scripts useful for MariaDB DBAs are presented. Problems of MariaDB Server dynamic tracing with bpftrace and some possible solutions and alternative tracing tools are discussed.
Security Monitoring with eBPF
Security Monitoring with eBPFAlex Maestretti Using the new extended Berkley Packet Filter capabilities in Linux to the improve performance of auditing security relevant kernel events around network, file and process actions.
Low Overhead System Tracing with eBPF
Low Overhead System Tracing with eBPFAkshay Kapoor In the Container world, there is a need to build observability around apps and backing services running in containers. The observability should allow to capture on demand low level metrics at a low overhead. The proposal is to use ebpf as the tracing technology to capture details at kernel & user level, and generate on demand flamegraphs, heat maps for applications & backing services. The Linux kernel has a built-in BPF JIT compiler, and an in-kernel verifier which is used to validate eBPF programs. This allows user defined instrumentation on a live kernel image that can never crash, hang or interfere with the kernel negatively. eBPF provides in-kernel implementation of storage maps such as histograms and hash-maps, which helps in efficient copy of summarized monitoring data from kernel to user space with low overhead.
These features make eBPF programs safe to run in production and allow admins and engineers to collect crucial data from systems for performance analysis and monitoring.
Building Network Functions with eBPF & BCC
Building Network Functions with eBPF & BCCKernel TLV eBPF (Extended Berkeley Packet Filter) is an in-kernel virtual machine that allows running user-supplied sandboxed programs inside of the kernel. It is especially well-suited to network programs and it's possible to write programs that filter traffic, classify traffic and perform high-performance custom packet processing.
BCC (BPF Compiler Collection) is a toolkit for creating efficient kernel tracing and manipulation programs. It makes use of eBPF.
BCC provides an end-to-end workflow for developing eBPF programs and supplies Python bindings, making eBPF programs much easier to write.
Together, eBPF and BCC allow you to develop and deploy network functions safely and easily, focusing on your application logic (instead of kernel datapath integration).
In this session, we will introduce eBPF and BCC, explain how to implement a network function using BCC, discuss some real-life use-cases and show a live demonstration of the technology.
About the speaker
Shmulik Ladkani, Chief Technology Officer at Meta Networks,
Long time network veteran and kernel geek.
Shmulik started his career at Jungo (acquired by NDS/Cisco) implementing residential gateway software, focusing on embedded Linux, Linux kernel, networking and hardware/software integration.
Some billions of forwarded packets later, Shmulik left his position as Jungo's lead architect and joined Ravello Systems (acquired by Oracle) as tech lead, developing a virtual data center as a cloud-based service, focusing around virtualization systems, network virtualization and SDN.
Recently he co-founded Meta Networks where he's been busy architecting secure, multi-tenant, large-scale network infrastructure as a cloud-based service.
eBPF Workshop
eBPF WorkshopMichael Kehoe This document provides instructions for setting up and attending an eBPF workshop. It includes links for setting up the workshop platform, background slides, and code repository. It also lists an agenda with topics that will be covered, including setting up the eBPF lab, an introduction, eBPF 101, writing eBPF programs, BCC, and a tutorial. Attendees are asked to let the presenter know if they have any problems setting up.
Performance Tuning EC2 Instances
Performance Tuning EC2 InstancesBrendan Gregg Talk for AWS re:Invent 2014. Video: https://www.youtube.com/watch?v=7Cyd22kOqWc . Netflix tunes Amazon EC2 instances for maximum performance. In this session, you learn how Netflix configures the fastest possible EC2 instances, while reducing latency outliers. This session explores the various Xen modes (e.g., HVM, PV, etc.) and how they are optimized for different workloads. Hear how Netflix chooses Linux kernel versions based on desired performance characteristics and receive a firsthand look at how they set kernel tunables, including hugepages. You also hear about Netflix’s use of SR-IOV to enable enhanced networking and their approach to observability, which can exonerate EC2 issues and direct attention back to application performance.
netfilter and iptables
netfilter and iptablesKernel TLV netfilter is a framework provided by the Linux kernel that allows various networking-related operations to be implemented in the form of customized handlers.
iptables is a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall (implemented as different netfilter modules) and the chains and rules it stores.
Many systems use iptables/netfilter, Linux's native packet filtering/mangling framework since Linux 2.4, be it home routers or sophisticated cloud network stacks.
In this session, we will talk about the netfilter framework and its facilities, explain how basic filtering and mangling use-cases are implemented using iptables, and introduce some less common but powerful extensions of iptables.
Shmulik Ladkani, Chief Architect at Nsof Networks.
Long time network veteran and kernel geek.
Shmulik started his career at Jungo (acquired by NDS/Cisco) implementing residential gateway software, focusing on embedded Linux, Linux kernel, networking and hardware/software integration.
Some billions of forwarded packets later, Shmulik left his position as Jungo's lead architect and joined Ravello Systems (acquired by Oracle) as tech lead, developing a virtual data center as a cloud-based service, focusing around virtualization systems, network virtualization and SDN.
Recently he co-founded Nsof Networks, where he's been busy architecting network infrastructure as a cloud-based service, gazing at internet routes in astonishment, and playing the chkuku.
BPF / XDP 8월 세미나 KossLab
BPF / XDP 8월 세미나 KossLabTaeung Song BPF (Berkeley Packet Filter) allows for safe dynamic program injection into the Linux kernel. It provides an in-kernel virtual machine and instruction set for running custom programs. The BPF infrastructure includes a verifier that checks programs for safety, helper functions to access kernel APIs, and maps for inter-process communication. BPF has become a core kernel subsystem and is used for applications like XDP, tracing, networking, and more.
eBPF maps 101
eBPF maps 101SUSE Labs Taipei The document provides an overview of eBPF maps and how they can be used to share data between eBPF programs running in the kernel and userspace applications. It describes how maps are created via the BPF syscall using the BPF_MAP_CREATE command. It also explains how keys and values can be looked up, updated, and deleted from maps using commands like BPF_MAP_LOOKUP_ELEM, BPF_MAP_UPDATE_ELEM, and BPF_MAP_DELETE_ELEM. Finally, it lists the different types of eBPF maps available.
Performance Wins with BPF: Getting Started
Performance Wins with BPF: Getting StartedBrendan Gregg Keynote by Brendan Gregg for the eBPF summit, 2020. How to get started finding performance wins using the BPF (eBPF) technology. This short talk covers the quickest and easiest way to find performance wins using BPF observability tools on Linux.
Staring into the eBPF Abyss
Staring into the eBPF AbyssSasha Goldshtein eBPF (extended Berkeley Packet Filters) is a modern kernel technology that can be used to introduce dynamic tracing into a system that wasn't prepared or instrumented in any way. The tracing programs run in the kernel, are guaranteed to never crash or hang your system, and can probe every module and function -- from the kernel to user-space frameworks such as Node and Ruby.
In this workshop, you will experiment with Linux dynamic tracing first-hand. First, you will explore BCC, the BPF Compiler Collection, which is a set of tools and libraries for dynamic tracing. Many of your tracing needs will be answered by BCC, and you will experiment with memory leak analysis, generic function tracing, kernel tracepoints, static tracepoints in user-space programs, and the "baked" tools for file I/O, network, and CPU analysis. You'll be able to choose between working on a set of hands-on labs prepared by the instructors, or trying the tools out on your own test system.
Next, you will hack on some of the bleeding edge tools in the BCC toolkit, and build a couple of simple tools of your own. You'll be able to pick from a curated list of GitHub issues for the BCC project, a set of hands-on labs with known "school solutions", and an open-ended list of problems that need tools for effective analysis. At the end of this workshop, you will be equipped with a toolbox for diagnosing issues in the field, as well as a framework for building your own tools when the generic ones do not suffice.
Kernel development
Kernel developmentNuno Martins Mechanisms and tools of development and monitoring in Linux Kernel
A brief presentation about the tools and mechanisms about development for the Linux Kernel to help understanding of what is required.
Tools that create an execution profile or provide instrumentation through static or dynamic methods, the Linux Kernel code, will be presented.
Also discussed will be the GDB debugger and how through a remote virtual serial connection to a virtual machine it can be used to debug a live Kernel and Linux Kernel modules. Also demonstrated will be how a deeper understanding of the code can be attained by attaching the memory locations used by the Kernel module to the GDB session.
Lastly, some of the Kernel execution contexts, such as, interrupts, deferrable work, context, etc. are presented.
New Ways to Find Latency in Linux Using Tracing
New Ways to Find Latency in Linux Using TracingScyllaDB Ftrace is the official tracer of the Linux kernel. It originated from the real-time patch (now known as PREEMPT_RT), as developing an operating system for real-time use requires deep insight and transparency of the happenings of the kernel. Not only was tracing useful for debugging, but it was critical for finding areas in the kernel that was causing unbounded latency. It's no wonder why the ftrace infrastructure has a lot of tooling for seeking out latency. Ftrace was introduced into mainline Linux in 2008, and several talks have been done on how to utilize its tracing features. But a lot has happened in the past few years that makes the tooling for finding latency much simpler. Other talks at P99 will discuss the new ftrace tracers "osnoise" and "timerlat", but this talk will focus more on the new flexible and dynamic aspects of ftrace that facilitates finding latency issues which are more specific to your needs. Some of this work may still be in a proof of concept stage, but this talk will give you the advantage of knowing what tools will be available to you in the coming year.
Yet another introduction to Linux RCU
Yet another introduction to Linux RCUViller Hsiao RCU is a kind of read/write synchronization mechanism. In the slide, we talk about the concept and implementation of Linux RCU.
Berkeley Packet Filters
Berkeley Packet FiltersKernel TLV This talk is all about the Berkeley Packet Filters (BPF) and their uses in Linux.
Agenda:
* What is a BPF and why do we need it?
* Writing custom BPFs
* Notes on BPF implementation in the kernel
* Usage examples: SOCKET_FILTER & seccomp
Speaker:
Kfir Gollan, senior embedded software developer, Linux kernel hacker and software team leader.
Systems@Scale 2021 BPF Performance Getting Started
Systems@Scale 2021 BPF Performance Getting StartedBrendan Gregg Talk for Facebook Systems@Scale 2021 by Brendan Gregg: "BPF (eBPF) tracing is the superpower that can analyze everything, helping you find performance wins, troubleshoot software, and more. But with many different front-ends and languages, and years of evolution, finding the right starting point can be hard. This talk will make it easy, showing how to install and run selected BPF tools in the bcc and bpftrace open source projects for some quick wins. Think like a sysadmin, not like a programmer."
Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)Brendan Gregg This document provides an overview of using eBPF (extended Berkeley Packet Filter) to quickly get performance wins as a sysadmin. It recommends installing BCC and bpftrace tools to easily find issues like periodic processes, misconfigurations, unexpected TCP sessions, or slow file system I/O. A case study examines using biosnoop to identify which processes were causing disk latency issues. The document suggests thinking like a sysadmin first by running tools, then like a programmer if a problem requires new tools. It also outlines recommended frontends depending on use cases and provides references to learn more about BPF.
Ad
More Related Content
What's hot (20)
Performance Analysis Tools for Linux Kernel
Performance Analysis Tools for Linux Kernellcplcp1 Perf is a collection of Linux kernel tools for performance monitoring and profiling. It provides sampling and profiling of the system to analyze performance bottlenecks. Perf supports hardware events from the CPU performance counters, software events from the kernel, and tracepoint events from the kernel and loaded modules. It offers tools like perf record to sample events and store them, perf report to analyze stored samples, and perf trace to trace system events in real-time.
bcc/BPF tools - Strategy, current tools, future challenges
bcc/BPF tools - Strategy, current tools, future challengesIO Visor Project Brendan Gregg discusses the current state and future potential of BPF and BCC tools for observability in Linux. He outlines 18 areas where BPF support has progressed and 16 areas still needing work. Gregg also discusses challenges like dynamic tracing stability, overhead, ease of coding, and developing visualizations. He proposes finishing ports of his old DTrace tools and links to resources on BPF, BCC, and flame graphs.
Linux 4.x Tracing Tools: Using BPF Superpowers
Linux 4.x Tracing Tools: Using BPF SuperpowersBrendan Gregg Talk for USENIX LISA 2016 by Brendan Gregg.
"Linux 4.x Tracing Tools: Using BPF Superpowers
The Linux 4.x series heralds a new era of Linux performance analysis, with the long-awaited integration of a programmable tracer: Enhanced BPF (eBPF). Formally the Berkeley Packet Filter, BPF has been enhanced in Linux to provide system tracing capabilities, and integrates with dynamic tracing (kprobes and uprobes) and static tracing (tracepoints and USDT). This has allowed dozens of new observability tools to be developed so far: for example, measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. Tracing superpowers have finally arrived.
In this talk I'll show you how to use BPF in the Linux 4.x series, and I'll summarize the different tools and front ends available, with a focus on iovisor bcc. bcc is an open source project to provide a Python front end for BPF, and comes with dozens of new observability tools (many of which I developed). These tools include new BPF versions of old classics, and many new tools, including: execsnoop, opensnoop, funccount, trace, biosnoop, bitesize, ext4slower, ext4dist, tcpconnect, tcpretrans, runqlat, offcputime, offwaketime, and many more. I'll also summarize use cases and some long-standing issues that can now be solved, and how we are using these capabilities at Netflix."
LPC2019 BPF Tracing Tools
LPC2019 BPF Tracing ToolsBrendan Gregg This document discusses Brendan Gregg's opinions on various tracing tools including sysdig, perf, ftrace, eBPF, bpftrace, and BPF perf tools. It provides a table comparing the scope, capability, and ease of use of these tools. It then gives an example of using BPF perf tools to analyze readahead performance. Finally, it outlines desired additions to tracing capabilities and BPF helpers as well as challenges in areas like function tracing without frame pointers.
UM2019 Extended BPF: A New Type of Software
UM2019 Extended BPF: A New Type of SoftwareBrendan Gregg BPF (Berkeley Packet Filter) has evolved from a limited virtual machine for efficient packet filtering to a new type of software called extended BPF. Extended BPF allows for custom, efficient, and production-safe performance analysis tools and observability programs to be run in the Linux kernel through BPF. It enables new event-based applications running as BPF programs attached to various kernel events like kprobes, uprobes, tracepoints, sockets, and more. Major companies like Facebook, Google, and Netflix are using BPF programs for tasks like intrusion detection, container security, firewalling, and observability with over 150,000 AWS instances running BPF programs. BPF provides a new program model and security features compared
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021Valeriy Kravchuk Bpftrace is a relatively new eBPF-based open source tracer for modern Linux versions (kernels 5.x.y) that is useful for analyzing production performance problems and troubleshooting software. Basic usage of the tool, as well as bpftrace one liners and advanced scripts useful for MariaDB DBAs are presented. Problems of MariaDB Server dynamic tracing with bpftrace and some possible solutions and alternative tracing tools are discussed.
Security Monitoring with eBPF
Security Monitoring with eBPFAlex Maestretti Using the new extended Berkley Packet Filter capabilities in Linux to the improve performance of auditing security relevant kernel events around network, file and process actions.
Low Overhead System Tracing with eBPF
Low Overhead System Tracing with eBPFAkshay Kapoor In the Container world, there is a need to build observability around apps and backing services running in containers. The observability should allow to capture on demand low level metrics at a low overhead. The proposal is to use ebpf as the tracing technology to capture details at kernel & user level, and generate on demand flamegraphs, heat maps for applications & backing services. The Linux kernel has a built-in BPF JIT compiler, and an in-kernel verifier which is used to validate eBPF programs. This allows user defined instrumentation on a live kernel image that can never crash, hang or interfere with the kernel negatively. eBPF provides in-kernel implementation of storage maps such as histograms and hash-maps, which helps in efficient copy of summarized monitoring data from kernel to user space with low overhead.
These features make eBPF programs safe to run in production and allow admins and engineers to collect crucial data from systems for performance analysis and monitoring.
Building Network Functions with eBPF & BCC
Building Network Functions with eBPF & BCCKernel TLV eBPF (Extended Berkeley Packet Filter) is an in-kernel virtual machine that allows running user-supplied sandboxed programs inside of the kernel. It is especially well-suited to network programs and it's possible to write programs that filter traffic, classify traffic and perform high-performance custom packet processing.
BCC (BPF Compiler Collection) is a toolkit for creating efficient kernel tracing and manipulation programs. It makes use of eBPF.
BCC provides an end-to-end workflow for developing eBPF programs and supplies Python bindings, making eBPF programs much easier to write.
Together, eBPF and BCC allow you to develop and deploy network functions safely and easily, focusing on your application logic (instead of kernel datapath integration).
In this session, we will introduce eBPF and BCC, explain how to implement a network function using BCC, discuss some real-life use-cases and show a live demonstration of the technology.
About the speaker
Shmulik Ladkani, Chief Technology Officer at Meta Networks,
Long time network veteran and kernel geek.
Shmulik started his career at Jungo (acquired by NDS/Cisco) implementing residential gateway software, focusing on embedded Linux, Linux kernel, networking and hardware/software integration.
Some billions of forwarded packets later, Shmulik left his position as Jungo's lead architect and joined Ravello Systems (acquired by Oracle) as tech lead, developing a virtual data center as a cloud-based service, focusing around virtualization systems, network virtualization and SDN.
Recently he co-founded Meta Networks where he's been busy architecting secure, multi-tenant, large-scale network infrastructure as a cloud-based service.
eBPF Workshop
eBPF WorkshopMichael Kehoe This document provides instructions for setting up and attending an eBPF workshop. It includes links for setting up the workshop platform, background slides, and code repository. It also lists an agenda with topics that will be covered, including setting up the eBPF lab, an introduction, eBPF 101, writing eBPF programs, BCC, and a tutorial. Attendees are asked to let the presenter know if they have any problems setting up.
Performance Tuning EC2 Instances
Performance Tuning EC2 InstancesBrendan Gregg Talk for AWS re:Invent 2014. Video: https://www.youtube.com/watch?v=7Cyd22kOqWc . Netflix tunes Amazon EC2 instances for maximum performance. In this session, you learn how Netflix configures the fastest possible EC2 instances, while reducing latency outliers. This session explores the various Xen modes (e.g., HVM, PV, etc.) and how they are optimized for different workloads. Hear how Netflix chooses Linux kernel versions based on desired performance characteristics and receive a firsthand look at how they set kernel tunables, including hugepages. You also hear about Netflix’s use of SR-IOV to enable enhanced networking and their approach to observability, which can exonerate EC2 issues and direct attention back to application performance.
netfilter and iptables
netfilter and iptablesKernel TLV netfilter is a framework provided by the Linux kernel that allows various networking-related operations to be implemented in the form of customized handlers.
iptables is a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall (implemented as different netfilter modules) and the chains and rules it stores.
Many systems use iptables/netfilter, Linux's native packet filtering/mangling framework since Linux 2.4, be it home routers or sophisticated cloud network stacks.
In this session, we will talk about the netfilter framework and its facilities, explain how basic filtering and mangling use-cases are implemented using iptables, and introduce some less common but powerful extensions of iptables.
Shmulik Ladkani, Chief Architect at Nsof Networks.
Long time network veteran and kernel geek.
Shmulik started his career at Jungo (acquired by NDS/Cisco) implementing residential gateway software, focusing on embedded Linux, Linux kernel, networking and hardware/software integration.
Some billions of forwarded packets later, Shmulik left his position as Jungo's lead architect and joined Ravello Systems (acquired by Oracle) as tech lead, developing a virtual data center as a cloud-based service, focusing around virtualization systems, network virtualization and SDN.
Recently he co-founded Nsof Networks, where he's been busy architecting network infrastructure as a cloud-based service, gazing at internet routes in astonishment, and playing the chkuku.
BPF / XDP 8월 세미나 KossLab
BPF / XDP 8월 세미나 KossLabTaeung Song BPF (Berkeley Packet Filter) allows for safe dynamic program injection into the Linux kernel. It provides an in-kernel virtual machine and instruction set for running custom programs. The BPF infrastructure includes a verifier that checks programs for safety, helper functions to access kernel APIs, and maps for inter-process communication. BPF has become a core kernel subsystem and is used for applications like XDP, tracing, networking, and more.
eBPF maps 101
eBPF maps 101SUSE Labs Taipei The document provides an overview of eBPF maps and how they can be used to share data between eBPF programs running in the kernel and userspace applications. It describes how maps are created via the BPF syscall using the BPF_MAP_CREATE command. It also explains how keys and values can be looked up, updated, and deleted from maps using commands like BPF_MAP_LOOKUP_ELEM, BPF_MAP_UPDATE_ELEM, and BPF_MAP_DELETE_ELEM. Finally, it lists the different types of eBPF maps available.
Performance Wins with BPF: Getting Started
Performance Wins with BPF: Getting StartedBrendan Gregg Keynote by Brendan Gregg for the eBPF summit, 2020. How to get started finding performance wins using the BPF (eBPF) technology. This short talk covers the quickest and easiest way to find performance wins using BPF observability tools on Linux.
Staring into the eBPF Abyss
Staring into the eBPF AbyssSasha Goldshtein eBPF (extended Berkeley Packet Filters) is a modern kernel technology that can be used to introduce dynamic tracing into a system that wasn't prepared or instrumented in any way. The tracing programs run in the kernel, are guaranteed to never crash or hang your system, and can probe every module and function -- from the kernel to user-space frameworks such as Node and Ruby.
In this workshop, you will experiment with Linux dynamic tracing first-hand. First, you will explore BCC, the BPF Compiler Collection, which is a set of tools and libraries for dynamic tracing. Many of your tracing needs will be answered by BCC, and you will experiment with memory leak analysis, generic function tracing, kernel tracepoints, static tracepoints in user-space programs, and the "baked" tools for file I/O, network, and CPU analysis. You'll be able to choose between working on a set of hands-on labs prepared by the instructors, or trying the tools out on your own test system.
Next, you will hack on some of the bleeding edge tools in the BCC toolkit, and build a couple of simple tools of your own. You'll be able to pick from a curated list of GitHub issues for the BCC project, a set of hands-on labs with known "school solutions", and an open-ended list of problems that need tools for effective analysis. At the end of this workshop, you will be equipped with a toolbox for diagnosing issues in the field, as well as a framework for building your own tools when the generic ones do not suffice.
Kernel development
Kernel developmentNuno Martins Mechanisms and tools of development and monitoring in Linux Kernel
A brief presentation about the tools and mechanisms about development for the Linux Kernel to help understanding of what is required.
Tools that create an execution profile or provide instrumentation through static or dynamic methods, the Linux Kernel code, will be presented.
Also discussed will be the GDB debugger and how through a remote virtual serial connection to a virtual machine it can be used to debug a live Kernel and Linux Kernel modules. Also demonstrated will be how a deeper understanding of the code can be attained by attaching the memory locations used by the Kernel module to the GDB session.
Lastly, some of the Kernel execution contexts, such as, interrupts, deferrable work, context, etc. are presented.
New Ways to Find Latency in Linux Using Tracing
New Ways to Find Latency in Linux Using TracingScyllaDB Ftrace is the official tracer of the Linux kernel. It originated from the real-time patch (now known as PREEMPT_RT), as developing an operating system for real-time use requires deep insight and transparency of the happenings of the kernel. Not only was tracing useful for debugging, but it was critical for finding areas in the kernel that was causing unbounded latency. It's no wonder why the ftrace infrastructure has a lot of tooling for seeking out latency. Ftrace was introduced into mainline Linux in 2008, and several talks have been done on how to utilize its tracing features. But a lot has happened in the past few years that makes the tooling for finding latency much simpler. Other talks at P99 will discuss the new ftrace tracers "osnoise" and "timerlat", but this talk will focus more on the new flexible and dynamic aspects of ftrace that facilitates finding latency issues which are more specific to your needs. Some of this work may still be in a proof of concept stage, but this talk will give you the advantage of knowing what tools will be available to you in the coming year.
Yet another introduction to Linux RCU
Yet another introduction to Linux RCUViller Hsiao RCU is a kind of read/write synchronization mechanism. In the slide, we talk about the concept and implementation of Linux RCU.
Berkeley Packet Filters
Berkeley Packet FiltersKernel TLV This talk is all about the Berkeley Packet Filters (BPF) and their uses in Linux.
Agenda:
* What is a BPF and why do we need it?
* Writing custom BPFs
* Notes on BPF implementation in the kernel
* Usage examples: SOCKET_FILTER & seccomp
Speaker:
Kfir Gollan, senior embedded software developer, Linux kernel hacker and software team leader.
Similar to Bpf performance tools chapter 4 bcc (20)
Systems@Scale 2021 BPF Performance Getting Started
Systems@Scale 2021 BPF Performance Getting StartedBrendan Gregg Talk for Facebook Systems@Scale 2021 by Brendan Gregg: "BPF (eBPF) tracing is the superpower that can analyze everything, helping you find performance wins, troubleshoot software, and more. But with many different front-ends and languages, and years of evolution, finding the right starting point can be hard. This talk will make it easy, showing how to install and run selected BPF tools in the bcc and bpftrace open source projects for some quick wins. Think like a sysadmin, not like a programmer."
Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)Brendan Gregg This document provides an overview of using eBPF (extended Berkeley Packet Filter) to quickly get performance wins as a sysadmin. It recommends installing BCC and bpftrace tools to easily find issues like periodic processes, misconfigurations, unexpected TCP sessions, or slow file system I/O. A case study examines using biosnoop to identify which processes were causing disk latency issues. The document suggests thinking like a sysadmin first by running tools, then like a programmer if a problem requires new tools. It also outlines recommended frontends depending on use cases and provides references to learn more about BPF.
Efficient System Monitoring in Cloud Native Environments
Efficient System Monitoring in Cloud Native EnvironmentsGergely Szabó This document discusses efficient system monitoring in cloud native environments using eBPF. It provides an overview of eBPF and how it can be used for monitoring applications like Prometheus. Specific topics covered include BPF, Linux kernel tracing using kprobes and tracepoints, eBPF maps and programs, and an example Prometheus exporter that leverages eBPF to export metrics.
Kernel bug hunting
Kernel bug huntingAndrea Righi Debugging is an essential part of Linux kernel development. In
user-space we have the support of the kernel and many debugging tools, tracking down a kernel bug, instead, can be very difficult if you don't know the proper methodologies. This talk will cover some techniques to understand how the kernel works, hunt down and fix kernel bugs in order to become a better kernel developer.
Turbo C
Turbo Cnat236 This document provides an overview and introduction to the Turbo C integrated development environment. It describes what is included in the Turbo C package, the system requirements, and new features of version 2.0. It also summarizes the contents and purpose of the two manuals included: Volume I covers using Turbo C for beginners, while Volume II is a reference guide for experienced C programmers.
ebpf and IO Visor: The What, how, and what next!
ebpf and IO Visor: The What, how, and what next!Affan Syed Extended BPF (eBPF) provides a mechanism for running custom programs inside the Linux kernel that can be used for filtering network packets, monitoring system activity, and more. eBPF programs are written in a restricted subset of C and compiled to bytecode that is verified by the kernel for safety before being run. The BCC toolkit makes it easier to write and load eBPF programs. The IO Visor project aims to further develop eBPF and provide tools and use cases for networking, security, and system tracing applications.
Using eBPF to Measure the k8s Cluster Health
Using eBPF to Measure the k8s Cluster HealthScyllaDB As a k8s cluster-admin your app teams have a certain expectation of your cluster to be available to deploy services at any time without problems. While there is no shortage on metrics in k8s its important to have the right metrics to alert on issues and giving you enough data to react to potential availability issues. Prometheus has become a standard and sheds light on the inner behaviour of Kubernetes clusters and workloads. Lots of KPIs (CPU, IO, network. Etc) in our On-Premise environment are less precise when we start to work in a Cloud environment. Ebpf is the perfect technology that fulfills that requirement as it gives us information down to the kernel level. In 2018 Cloudflare shared an opensource project to expose custom ebpf metrics in Prometheus. Join this session and learn about: • What is ebpf? • What type of metrics we can collect? • How to expose those metrics in a K8s environment. This session will try to deliver a step-by-step guide on how to take advantage of the ebpf exporter.
Dynamic tracing of MariaDB on Linux - problems and solutions (MariaDB Server ...
Dynamic tracing of MariaDB on Linux - problems and solutions (MariaDB Server ...Valeriy Kravchuk Linux with kernels 2.6+. provides different ways to add user probes to almost every other line of code dynamically, and collect the resulting trace and profiling data in a safe and efficient way. This session discusses basic use of ftrace, perf, bcc tools and bpftrace utility, highlights typical problems MariaDB DBAs and developers may hit while trying to apply them, as well as solutions to some of them.
eBPF Debugging Infrastructure - Current Techniques
eBPF Debugging Infrastructure - Current TechniquesNetronome eBPF (extended Berkeley Packet Filter), in particular with its driver-level hook XDP (eXpress Data Path), has increased in importance over the past few years. As a result, the ability to rapidly debug and diagnose problems is becoming more relevant. This talk will cover common issues faced and techniques to diagnose them, including the use of bpftool for map and program introspection, the use of disassembly to inspect generated assembly code and other methods such as using debug prints and how to apply these techniques when eBPF programs are offloaded to the hardware.
The talk will also explore where the current gaps in debugging infrastructure are and suggest some of the next steps to improve this, for example, integrations with tools such as strace, valgrind or even the LLDB debugger.
[KubeCon EU 2020] containerd Deep Dive![[KubeCon EU 2020] containerd Deep Dive](https://cdn.slidesharecdn.com/ss_thumbnails/containerd-deep-dive-200819115038-thumbnail.jpg?width=560&fit=bounds)
[KubeCon EU 2020] containerd Deep DiveAkihiro Suda - containerd overview
- Upcoming features in v1.4
- External plugins
https://kccnceu20.sched.com/event/ZexS/containerd-deep-dive-akihiro-suda-ntt-wei-fu-alibaba
Shifter singularity - june 7, 2018 - bw symposium
Shifter singularity - june 7, 2018 - bw symposiuminside-BigData.com In this video from the Blue Waters 2018 Symposium, Maxim Belkin presents a tutorial on Containers: Shifter and Singularity on Blue Waters.
Container solutions are a great way to seamlessly execute code on a variety of platforms. Not only they are used to abstract away from the software stack of the underlying operating system, they also enable reproducible computational research. In this mini-tutorial, I will review the process of working with Shifter and Singularity on Blue Waters.
Watch the video: https://wp.me/p3RLHQ-iXO
Learn more: https://bluewaters.ncsa.illinois.edu/blue-waters-symposium-2018
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Not breaking userspace: the evolving Linux ABI
Not breaking userspace: the evolving Linux ABIAlison Chaiken This document discusses not breaking userspace through maintaining the Linux application binary interface (ABI). It begins by defining what an ABI is and how it guarantees compatibility between userspace applications and the Linux kernel. It then discusses several examples where ABI breaks have occurred or may need to occur, such as for the 2038 time issue, priority inheritance in threading, and changes to tools like BPF programs. The document provides methods for avoiding ABI breaks, such as unused function parameters and exporting information to sysfs. It concludes that maintaining the ABI is important but also sometimes unavoidable when new features are needed.
Differences of Deep Learning Frameworks
Differences of Deep Learning FrameworksSeiya Tokui This is the slide for the second session of the "Deep Learning Implementations and Frameworks" tutorial at PAKDD 2016.
Using R on High Performance Computers
Using R on High Performance ComputersDave Hiltbrand This document provides an overview of using R and high performance computers (HPC). It discusses why HPC is useful when data becomes too large for a local machine, and strategies like moving to more powerful hardware, using parallel packages, or rewriting code. It also covers topics like accessing HPC resources through batch jobs, setting up the R environment, profiling code, and using packages like purrr and foreach to parallelize workflows. The overall message is that HPC can scale up R analyses, but developers must adapt their code for parallel and distributed processing.
Measurement .Net Performance with BenchmarkDotNet
Measurement .Net Performance with BenchmarkDotNetVasyl Senko This document discusses using BenchmarkDotNet to measure .NET performance. It provides an overview of profiling vs benchmarking vs microbenchmarking and best practices for benchmarking. It then demonstrates how to install BenchmarkDotNet, design a benchmark, analyze results, and configure columns, jobs, and diagnosers. Key aspects of how BenchmarkDotNet works and useful statistics and tools it provides are also summarized.
Talk Python To Me: Stream Processing in your favourite Language with Beam on ...
Talk Python To Me: Stream Processing in your favourite Language with Beam on ...Aljoscha Krettek Flink is a great stream processor, Python is a great programming language, Apache Beam is a great programming model and portability layer. Using all three together is a great idea! We will demo and discuss writing Beam Python pipelines and running them on Flink. We will cover Beam's portability vision that led here, what you need to know about how Beam Python pipelines are executed on Flink, and where Beam's portability framework is headed next (hint: Python pipelines reading from non-Python connectors)
Fast federated SQL with Apache Calcite
Fast federated SQL with Apache CalciteChris Baynes This document discusses Apache Calcite, an open source framework for federated SQL queries. It provides an introduction to Calcite and its components. It then evaluates Calcite's performance on single data sources through benchmarks. Lastly, it proposes a hybrid approach to enable efficient federated queries using Calcite and Spark.
workshop_8_c__.pdf
workshop_8_c__.pdfAtulAvhad2 This document provides instructions for students to implement the execution of map and reduce tasks on workers in a MapReduce system. It details expected outcomes such as deploying MapReduce applications to Azure Kubernetes and developing an HTTP interface. It also provides suggestions on implementing Python code wrappers to execute mapper and reducer functions, saving intermediate and final results, and delivering required code and demos.
Introduction to eBPF and XDP
Introduction to eBPF and XDPlcplcp1 This document provides an introduction to eBPF and XDP. It discusses the history of BPF and how it evolved into eBPF. Key aspects of eBPF covered include the instruction set, JIT compilation, verifier, helper functions, and maps. XDP is introduced as a way to program the data plane using eBPF programs attached early in the receive path. Example use cases and performance benchmarks for XDP are also mentioned.
FIFODC
FIFODCsumeet jain This document describes the gate-level synthesis of a FIFO design using Synopsys Design Compiler. It discusses the FIFO description, introduces Design Compiler and the libraries used. It then outlines the steps to set up Design Compiler and synthesize the design, including specifying libraries, reading the HDL file, setting constraints, and compiling. Timing and reference reports are generated and the synthesized netlist is written out.
Ad
More from Viller Hsiao (9)
Prerequisite knowledge for shared memory concurrency
Prerequisite knowledge for shared memory concurrencyViller Hsiao The document provides an overview of prerequisite knowledge for shared memory concurrency, including memory hierarchy, data consistency across memory levels, issues with simple spinlock implementations, and atomic instructions supported by CPUs like ARM and RISC-V. It discusses examples of implementing spinlocks using ARMv8 atomic instructions like ldrex/strex. It also covers memory ordering, memory barrier instructions, cache coherence protocols, and memory consistency models for architectures like ARMv8 and RISC-V.
twlkh-linux-vsyscall-and-vdso
twlkh-linux-vsyscall-and-vdsoViller Hsiao Viller Hsiao presents information on Linux vsyscall and vDSO. vDSO (virtual dynamic shared object) is mapped into userspace and contains implementations of common system calls to make them faster. It gets benefits from ASLR and allows additional system calls compared to the older vsyscall method. The kernel generates the vDSO shared object which is then loaded and accessed by the glibc dynamic linker to provide optimized system call implementations to applications.
Linux kernel tracing
Linux kernel tracingViller Hsiao This document discusses tracing in the Linux kernel. It describes various tracing mechanisms like ftrace, tracepoints, kprobes, perf, and eBPF. Ftrace allows tracing functions via compiler instrumentation or dynamically. Tracepoints define custom trace events that can be inserted at specific points. Kprobes and related probes like jprobes allow tracing kernel functions. Perf provides performance monitoring capabilities. eBPF enables custom tracing programs to be run efficiently in the kernel via just-in-time compilation. Tracing tools like perf, systemtap, and LTTng provide user interfaces.
Meet cute-between-ebpf-and-tracing
Meet cute-between-ebpf-and-tracingViller Hsiao This document discusses how eBPF (extended Berkeley Packet Filter) can be used for kernel tracing. It provides an overview of BPF and eBPF, how eBPF programs are compiled and run in the kernel, the use of BPF maps, and how eBPF enables new possibilities for dynamic kernel instrumentation through techniques like Kprobes and ftrace.
mbed-os 3.0 modules dependency graph
mbed-os 3.0 modules dependency graphViller Hsiao I use module.json, the one mbed-OS used to describe the module dependncy, to draw the dependency graph. It helps to under stand the relationship among modules.
Introduction to ARM mbed-OS 3.0 uvisor
Introduction to ARM mbed-OS 3.0 uvisorViller Hsiao Viller Hsiao presents on mbed-OS uvisor, a self-contained software hypervisor that creates independent secure domains on ARM Cortex-M3 and M4 microcontrollers. It protects resources by implementing access control lists and private box contexts to isolate domains. Secure function calls are enabled through a secure gateway that executes functions from the context of a secure box. Low-level APIs provide interrupt management across boxes. Memory is laid out to separate box contexts protected by an MPU. uvisor aims to provide security and sandboxing for applications on ARM microcontrollers.
My first-crawler-in-python
My first-crawler-in-pythonViller Hsiao This document discusses using Python to scrape financial report information from websites. It provides steps for scraping web pages, parsing content, and calculating stock values from tables of data. It also recommends Python modules like urllib, requests, and BeautifulSoup that can be used to parse DOM structures and extract information. Examples are given showing how to make HTTP requests, parse response bodies, and extract table data from HTML. The goal is to practice Python skills like web scraping, parsing, and analyzing financial data.
Trace kernel code tips
Trace kernel code tipsViller Hsiao This document discusses dynamically tracing C code using runtime execution logs. It provides tips for building the environment, using the menuconfig system, makefiles, toolchains and runtime tracing with Qemu logs and Linux ftrace. Ftrace in particular allows tracing kernel functions and call graphs to help understand code flow and task relationships when statically tracing C/C++ sources proves difficult due to macros and configurations.
f9-microkernel-ktimer
f9-microkernel-ktimerViller Hsiao The document discusses the ktimer subsystem in the F9 Microkernel. It provides an overview of operating system kernel time subsystems and concepts like system time, ticks, and tickless kernels. It then describes the features of the F9 Microkernel ktimer, including supporting time events, being tickless, and using a two-stage interrupt model. The ktimer aims for efficiency and flexibility while providing time services to the microkernel.
Ad
Recently uploaded (20)
π0.5: a Vision-Language-Action Model with Open-World Generalization
π0.5: a Vision-Language-Action Model with Open-World GeneralizationNABLAS株式会社 今回の資料「Transfusion / π0 / π0.5」は、画像・言語・アクションを統合するロボット基盤モデルについて紹介しています。
拡散×自己回帰を融合したTransformerをベースに、π0.5ではオープンワールドでの推論・計画も可能に。
This presentation introduces robot foundation models that integrate vision, language, and action.
Built on a Transformer combining diffusion and autoregression, π0.5 enables reasoning and planning in open-world settings.
railway wheels, descaling after reheating and before forging
railway wheels, descaling after reheating and before forgingJavad Kadkhodapour railway wheels, descaling after reheating and before forging
fluke dealers in bangalore..............
fluke dealers in bangalore..............Haresh Vaswani The Fluke 925 is a vane anemometer, a handheld device designed to measure wind speed, air flow (volume), and temperature. It features a separate sensor and display unit, allowing greater flexibility and ease of use in tight or hard-to-reach spaces. The Fluke 925 is particularly suitable for HVAC (heating, ventilation, and air conditioning) maintenance in both residential and commercial buildings, offering a durable and cost-effective solution for routine airflow diagnostics.
DATA-DRIVEN SHOULDER INVERSE KINEMATICS YoungBeom Kim1 , Byung-Ha Park1 , Kwa...
DATA-DRIVEN SHOULDER INVERSE KINEMATICS YoungBeom Kim1 , Byung-Ha Park1 , Kwa...charlesdick1345 This paper proposes a shoulder inverse kinematics (IK) technique. Shoulder complex is comprised of the sternum, clavicle, ribs, scapula, humerus, and four joints.
some basics electrical and electronics knowledge
some basics electrical and electronics knowledgenguyentrungdo88 This chapter discribe about common electrical divices such as passive component, the internaltional system unit and international system prefixes.
ELectronics Boards & Product Testing_Shiju.pdf
ELectronics Boards & Product Testing_Shiju.pdfShiju Jacob This presentation provides a high level insight about DFT analysis and test coverage calculation, finalizing test strategy, and types of tests at different levels of the product.
Lidar for Autonomous Driving, LiDAR Mapping for Driverless Cars.pptx
Lidar for Autonomous Driving, LiDAR Mapping for Driverless Cars.pptxRishavKumar530754 LiDAR-Based System for Autonomous Cars
Autonomous Driving with LiDAR Tech
LiDAR Integration in Self-Driving Cars
Self-Driving Vehicles Using LiDAR
LiDAR Mapping for Driverless Cars
The Gaussian Process Modeling Module in UQLab
The Gaussian Process Modeling Module in UQLabJournal of Soft Computing in Civil Engineering We introduce the Gaussian process (GP) modeling module developed within the UQLab software framework. The novel design of the GP-module aims at providing seamless integration of GP modeling into any uncertainty quantification workflow, as well as a standalone surrogate modeling tool. We first briefly present the key mathematical tools on the basis of GP modeling (a.k.a. Kriging), as well as the associated theoretical and computational framework. We then provide an extensive overview of the available features of the software and demonstrate its flexibility and user-friendliness. Finally, we showcase the usage and the performance of the software on several applications borrowed from different fields of engineering. These include a basic surrogate of a well-known analytical benchmark function; a hierarchical Kriging example applied to wind turbine aero-servo-elastic simulations and a more complex geotechnical example that requires a non-stationary, user-defined correlation function. The GP-module, like the rest of the scientific code that is shipped with UQLab, is open source (BSD license).
"Boiler Feed Pump (BFP): Working, Applications, Advantages, and Limitations E...
"Boiler Feed Pump (BFP): Working, Applications, Advantages, and Limitations E...Infopitaara A Boiler Feed Pump (BFP) is a critical component in thermal power plants. It supplies high-pressure water (feedwater) to the boiler, ensuring continuous steam generation.
⚙️ How a Boiler Feed Pump Works
Water Collection:
Feedwater is collected from the deaerator or feedwater tank.
Pressurization:
The pump increases water pressure using multiple impellers/stages in centrifugal types.
Discharge to Boiler:
Pressurized water is then supplied to the boiler drum or economizer section, depending on design.
🌀 Types of Boiler Feed Pumps
Centrifugal Pumps (most common):
Multistage for higher pressure.
Used in large thermal power stations.
Positive Displacement Pumps (less common):
For smaller or specific applications.
Precise flow control but less efficient for large volumes.
🛠️ Key Operations and Controls
Recirculation Line: Protects the pump from overheating at low flow.
Throttle Valve: Regulates flow based on boiler demand.
Control System: Often automated via DCS/PLC for variable load conditions.
Sealing & Cooling Systems: Prevent leakage and maintain pump health.
⚠️ Common BFP Issues
Cavitation due to low NPSH (Net Positive Suction Head).
Seal or bearing failure.
Overheating from improper flow or recirculation.
Introduction to FLUID MECHANICS & KINEMATICS
Introduction to FLUID MECHANICS & KINEMATICSnarayanaswamygdas Fluid mechanics is the branch of physics concerned with the mechanics of fluids (liquids, gases, and plasmas) and the forces on them. Originally applied to water (hydromechanics), it found applications in a wide range of disciplines, including mechanical, aerospace, civil, chemical, and biomedical engineering, as well as geophysics, oceanography, meteorology, astrophysics, and biology.
It can be divided into fluid statics, the study of various fluids at rest, and fluid dynamics.
Fluid statics, also known as hydrostatics, is the study of fluids at rest, specifically when there's no relative motion between fluid particles. It focuses on the conditions under which fluids are in stable equilibrium and doesn't involve fluid motion.
Fluid kinematics is the branch of fluid mechanics that focuses on describing and analyzing the motion of fluids, such as liquids and gases, without considering the forces that cause the motion. It deals with the geometrical and temporal aspects of fluid flow, including velocity and acceleration. Fluid dynamics, on the other hand, considers the forces acting on the fluid.
Fluid dynamics is the study of the effect of forces on fluid motion. It is a branch of continuum mechanics, a subject which models matter without using the information that it is made out of atoms; that is, it models matter from a macroscopic viewpoint rather than from microscopic.
Fluid mechanics, especially fluid dynamics, is an active field of research, typically mathematically complex. Many problems are partly or wholly unsolved and are best addressed by numerical methods, typically using computers. A modern discipline, called computational fluid dynamics (CFD), is devoted to this approach. Particle image velocimetry, an experimental method for visualizing and analyzing fluid flow, also takes advantage of the highly visual nature of fluid flow.
Fundamentally, every fluid mechanical system is assumed to obey the basic laws :
Conservation of mass
Conservation of energy
Conservation of momentum
The continuum assumption
For example, the assumption that mass is conserved means that for any fixed control volume (for example, a spherical volume)—enclosed by a control surface—the rate of change of the mass contained in that volume is equal to the rate at which mass is passing through the surface from outside to inside, minus the rate at which mass is passing from inside to outside. This can be expressed as an equation in integral form over the control volume.
The continuum assumption is an idealization of continuum mechanics under which fluids can be treated as continuous, even though, on a microscopic scale, they are composed of molecules. Under the continuum assumption, macroscopic (observed/measurable) properties such as density, pressure, temperature, and bulk velocity are taken to be well-defined at "infinitesimal" volume elements—small in comparison to the characteristic length scale of the system, but large in comparison to molecular length scale
Artificial Intelligence (AI) basics.pptx
Artificial Intelligence (AI) basics.pptxaditichinar its all about Artificial Intelligence(Ai) and Machine Learning and not on advanced level you can study before the exam or can check for some information on Ai for project
Compiler Design_Lexical Analysis phase.pptx
Compiler Design_Lexical Analysis phase.pptxRushaliDeshmukh2 The role of the lexical analyzer
Specification of tokens
Finite state machines
From a regular expressions to an NFA
Convert NFA to DFA
Transforming grammars and regular expressions
Transforming automata to grammars
Language for specifying lexical analyzers
Data Structures_Searching and Sorting.pptx
Data Structures_Searching and Sorting.pptxRushaliDeshmukh2 Sorting Order and Stability in Sorting.
Concept of Internal and External Sorting.
Bubble Sort,
Insertion Sort,
Selection Sort,
Quick Sort and
Merge Sort,
Radix Sort, and
Shell Sort,
External Sorting, Time complexity analysis of Sorting Algorithms.
Bpf performance tools chapter 4 bcc
- 1. BPF Performance Tools chapter 4 - bcc Viller Hsiao Feb. 17, 2020
- 2. What’s in Chapter 4
- 3. BPF Compiler Collection (BCC) BCC is a toolkit for creating efficient kernel tracing and manipulation programs, and includes several useful tools and examples. It makes use of extended BPF (Berkeley Packet Filters), formally known as eBPF, a new feature that was first added to Linux 3.15. Much of what BCC uses requires Linux 4.1 and above. (4.9 and up is better)
- 4. Overview (1/3) ● Use bcc ○ How to install bcc ■ with man page ■ example files ○ Tools overview ● Develop bcc program ○ bcc internals ○ debugging bcc
- 5. Overview (2/3) ● Use bcc ○ How to install bcc ○ Tools overview ● Develop bcc program ○ bcc internals ○ debugging bcc
- 6. Overview (3/3) ● Use bcc ○ How to install bcc ○ Tools overview ● Develop bcc program ○ bcc internals ○ debugging bcc
- 8. task_switch.py [/path/to/bcc/examples/tracing]% sudo python task_switch.py task_switch[ 0-> 2520]=10 task_switch[14980-> 0]=1 task_switch[ 1096-> 0]=1 task_switch[19347->14055]=1 task_switch[14423-> 0]=1 task_switch[ 0-> 53]=1 task_switch[ 0->28879]=60
- 10. Main Parts 1. BPF program to run in kernel 2. Kernel tracing tool to use 3. Retrieve data and do some post processing
- 11. What’s the Program Do? Translation When kernel run to <target>, use <tracing mechanism> to execute <handler>. The handler record some <statistics> or do <something>. (Like a function decorator in Python)
- 12. Actions When kernel run to finish_task_switch, use kprobe to execute cond_sched. The handler store counts of task switch as hash map, which is indexed by (prev_pid, curr_pid) pairs. static struct rq * finish_task_switch(struct task_struct *prev) BPF helper
- 13. Recorded Data Manipulation When kernel run to finish_task_switch, use kprobe to execute cond_sched. The handler store counts of task switch as hash map, which is indexed by (prev_pid, curr_pid) pairs.
- 14. 翻訳コンニャク When kernel run to finish_task_switch, use kprobe to execute cond_sched. The handler store counts of task switch as hash map, which is indexed by (prev_pid, curr_pid) pairs.
- 15. Resources for bcc Programming ● bcc Reference Guide ○ BPF C ○ bcc Python ● bpftrace -l
- 16. Other resources ● If it’s hard to use bpftrace on your machine ... ● /sys/kernel/tracing ○ available_events ■ tracepoint ○ tracepoint format ■ /sys/kernel/tracing/<type>/<event>/format ■ e.g. events/irq/softirq_entry/format ○ available_filter_functions ■ kprobe on function entry ● “perf list” ○ pmu events
- 18. Alternative Profiling Tools ● bpftrace ● ftrace / perf ○ perf-tools, by Brendan Gregg, before bcc ○ iosnoop vs biosnoop-bpfcc ● Other frontends ○ systemtap, LTTng and so on
- 19. Q & A