DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
24 likes11,104 views
This talk will start with a deep dive and hands on examples of BPF, possibly the most promising low level technology to address challenges in application and network security, tracing, and visibility. We will discuss how BPF evolved from a simple bytecode language to filter raw sockets for tcpdump to the a JITable virtual machine capable of universally extending and instrumenting both the Linux kernel and user space applications. The introduction is followed by a concrete example of how the Cilium open source project applies BPF to solve networking, security, and load balancing for highly distributed applications. We will discuss and demonstrate how Cilium with the help of BPF can be combined with distributed system orchestration such as Docker to simplify security, operations, and troubleshooting of distributed applications.
![BPF: Toolchain – from user to kernel
USER SPACE
SOURCE CODE [C]
</>](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-36-320.jpg)
![BPF: LLVM compiles program code to bytecode
USER SPACE
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-37-320.jpg)
![BPF: Bytecode is loaded and verified into kernel
USER SPACE
KERNELVERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-38-320.jpg)
![BPF: Bytecode runs inside safe kernel sandbox
USER SPACE
KERNELVERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>
SANDBOX
BPF](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-39-320.jpg)
![BPF: Program is attached to event (packet-in)
USER SPACE
KERNELVERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>
SANDBOX
BPF](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-40-320.jpg)
![BPF: Program can redirect to netns & sockets
USER SPACE
KERNELVERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>
SANDBOX
BPF](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-41-320.jpg)
![BPF: Leverage user space tool chain
USER SPACE
KERNEL
connect()
VERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-52-320.jpg)
![BPF: Attach program to connect() syscall (LSM)
USER SPACE
KERNEL
connect()
VERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>
BPF
LSM Hook](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-53-320.jpg)
![BPF: Return EACCESS – No packets created at all
USER SPACE
KERNEL
connect()
EACCESS
VERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>
BPF
LSM Hook](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-54-320.jpg)
![XDP [Express Data Path] mode
BPF
Driver
Run BPF Program inside network
driver with access to DMA buffer
Software Stack](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-59-320.jpg)
![XDP [Express Data Path] mode
BPF
Driver
Can drop millions of packets per
Second while under DDoS
Software Stack
drop](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-60-320.jpg)
![XDP [Express Data Path] mode
BPF
Driver
Can pass packets to network stack
Software Stack
drop
Stack](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-61-320.jpg)
![XDP [Express Data Path] mode
BPF
Driver
Can perform loadbalancing and
transmit out the wire again
Software Stack
drop
LB & TX
Stack](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-62-320.jpg)
![75
140
205
240
325
365 370 365
410 412 425
445 450 460 460
490 495 505 515 525
545
565
0
100
200
300
400
500
600
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
BPF redirect() performance
[GBit per core]
Intel Xeon 3.5Ghz Sandy Bridge, 24 Cores,
(1 TCP GSO flow per core, netperf -t TCP_SENDFILE, 10K policies)](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-81-320.jpg)
![[MeetUp][1st] 오리뎅이의_쿠버네티스_네트워킹](https://cdn.slidesharecdn.com/ss_thumbnails/3-191024235922-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (20)
Similar to DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP (20)
![[Cisco Connect 2018 - Vietnam] Anh duc le reap the benefits of sdn with cisco...](https://cdn.slidesharecdn.com/ss_thumbnails/anhducle-reapthebenefitsofsdnwithciscoacivirtualedge-180504035355-thumbnail.jpg?width=560&fit=bounds)
Ad
More from Thomas Graf (11)
Ad
Recently uploaded (20)
![Minitab 22 Full Crack Plus Product Key Free Download [Latest] 2025](https://cdn.slidesharecdn.com/ss_thumbnails/avtpastoralvisits20252026-250407114051-0c8f0fbf-250407123507-bf8f5b04-250426175635-00bc6197-thumbnail.jpg?width=560&fit=bounds)
![Download Wondershare Filmora Crack [2025] With Latest](https://cdn.slidesharecdn.com/ss_thumbnails/neo4j-howkgsareshapingthefutureofgenerativeaiatawssummitlondonapril2024-240426125209-2d9db05d-250419-250428115407-a04afffa-thumbnail.jpg?width=560&fit=bounds)
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
- 1. CILIUM: NETWORK AND APPLICATION SECURITY WITH BPF AND XDP Thomas Graf Co-founder & CTO Covalent
- 2. Who is this guy?
- 3. Helped build the biggest monolith ever … Who is this guy?
- 4. Helped build the biggest monolith ever … Who is this guy?
- 5. Time to rethink the kernel
- 6. syscalls syscalls Net IOBlock IO Time to rethink the kernel
- 7. Time to rethink the kernel From monolith to “microkernel” with BPF syscalls syscalls BPF BPF BPF BPF Net IOBlock IO
- 8. Time to rethink the kernel From monolith to “microkernel” with BPF syscalls syscalls BPF BPF BPF BPF BPF BPF Security Networking Net IOBlock IO
- 9. BPF is revolutionizing… • Tracing / Profiling
- 10. BPF is revolutionizing… • Tracing / Profiling Container Performance Analysis Brendan Gregg Wed 1:30pm “Black Belt”
- 11. BPF is revolutionizing… • Tracing / Profiling • Networking Container Performance Analysis Brendan Gregg Wed 1:30pm “Black Belt”
- 12. BPF is revolutionizing… • Tracing / Profiling • Networking • Security Container Performance Analysis Brendan Gregg Wed 1:30pm “Black Belt”
- 13. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low Delivery Frequency
- 14. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low 3-Tier App Monthly Moderate Delivery Frequency
- 15. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low Distributed Microservices 10-100 x’s / day Extreme 3-Tier App Monthly Moderate Delivery Frequency
- 16. Network Security has not evolved $ iptables -A INPUT -p tcp -s 15.15.15.3 --dport 80 -m conntrack --ctstate NEW -j ACCEPT The world still runs on iptables matching IPs and ports:
- 17. Your HTTP ports be like …
- 18. Network Security for Microservices Example Gordon is looking for a job…
- 19. Gordon Job Postings Example: Security for Microservices
- 20. GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API Gordon Job Postings Example: Security for Microservices
- 21. GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API GET /jobs/331 Gordon Job Postings Example: Security for Microservices
- 22. L3/L4 GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API iptables -s 10.1.1.1 -p tcp --dport 80 -j ACCEPT GET /jobs/331 Gordon Job Postings Example: Security for Microservices
- 23. L3/L4 GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API exposed exposed exposed GET /jobs/331 Gordon Job Postings Example: Security for Microservices iptables -s 10.1.1.1 -p tcp --dport 80 -j ACCEPT
- 24. Not exactly least privilege Security team is not amused
- 25. GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API GET /jobs/331 Gordon Job Postings Example: Security for Microservices
- 26. L3/L4 GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API FROM Gordon ALLOW GET /jobs/.* GET /jobs/331 Gordon Job Postings Example: Security for Microservices
- 27. We demand a demo
- 28. BPF - The Superpowers inside Linux
- 29. SANDBOX BPF GET /foo BPF: Transparent redirection into proxy
- 30. SANDBOX BPF Proxy GET /foo redirect rules sk BPF: Transparent redirection into proxy
- 31. SANDBOX BPF Proxy GET /foo redirect rules sk Shared State • Orig Dest IP • Identity BPF: Transparent redirection into proxy
- 32. SANDBOX BPF Proxy GET /foo redirect reinject rules sk sk Shared State • Orig Dest IP • Identity BPF: Transparent redirection into proxy
- 33. SANDBOX BPF Proxy GET /foo rules BPF: Transparent redirection into proxy sk sk 403 Access Denied
- 34. So what is BPF exactly?
- 35. .insns = { BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8), BPF_ST_MEM(BPF_DW, BPF_REG_2, 0, 0), BPF_LD_MAP_FD(BPF_REG_1, 0), BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem), BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -152), BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0), BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 2), BPF_LDX_MEM(BPF_DW, BPF_REG_3, BPF_REG_1, 0), BPF_ST_MEM(BPF_DW, BPF_REG_3, 0, 42), BPF_EXIT_INSN(), } What is BPF? Learn more about BPF: docs.cilium.io
- 36. BPF: Toolchain – from user to kernel USER SPACE SOURCE CODE [C] </>
- 37. BPF: LLVM compiles program code to bytecode USER SPACE SOURCE CODE [C] </> BYTE CODE [BPF] </>
- 38. BPF: Bytecode is loaded and verified into kernel USER SPACE KERNELVERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </>
- 39. BPF: Bytecode runs inside safe kernel sandbox USER SPACE KERNELVERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> SANDBOX BPF
- 40. BPF: Program is attached to event (packet-in) USER SPACE KERNELVERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> SANDBOX BPF
- 41. BPF: Program can redirect to netns & sockets USER SPACE KERNELVERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> SANDBOX BPF
- 42. BPF – An opportunity to rethink security policy enforcement
- 43. Status Quo: Policy Enforcement connect()
- 44. Status Quo: Policy Enforcement connect() TCP
- 45. Status Quo: Policy Enforcement connect() TCP Network packets
- 46. Status Quo: Policy Enforcement connect() TCP Network packets veth veth namespace boundary
- 47. Status Quo: Policy Enforcement connect() TCP Network packets iptables veth veth namespace boundary
- 48. Status Quo: Policy Enforcement connect() drop TCP Network packets iptables veth veth namespace boundary
- 49. Status Quo: Policy Enforcement connect() drop TCP Network packets ETIMEDOUT iptables veth veth namespace boundary
- 50. Status Quo: Policy Enforcement connect() drop TCP Network packets ETIMEDOUT/ ECONNREFUSED iptables RST veth veth namespace boundary
- 51. Can we do better? connect()
- 52. BPF: Leverage user space tool chain USER SPACE KERNEL connect() VERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </>
- 53. BPF: Attach program to connect() syscall (LSM) USER SPACE KERNEL connect() VERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> BPF LSM Hook
- 54. BPF: Return EACCESS – No packets created at all USER SPACE KERNEL connect() EACCESS VERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> BPF LSM Hook
- 55. XDP/BPF – The software loadbalancer of the future
- 56. WHAT IF I TOLD YOU XDP allows for 10x IPVS performance
- 57. Source: https://www.netdevconf.org/2.1/slides/apr6/zhou-netdev-xdp-2017.pdf FB moves from IPVS to BPF/XDP for L3/L4 LB XDP throughput IPVS throughput Source:
- 58. Regular BPF mode BPF Driver Software Stack
- 59. XDP [Express Data Path] mode BPF Driver Run BPF Program inside network driver with access to DMA buffer Software Stack
- 60. XDP [Express Data Path] mode BPF Driver Can drop millions of packets per Second while under DDoS Software Stack drop
- 61. XDP [Express Data Path] mode BPF Driver Can pass packets to network stack Software Stack drop Stack
- 62. XDP [Express Data Path] mode BPF Driver Can perform loadbalancing and transmit out the wire again Software Stack drop LB & TX Stack
- 63. How can I use BPF with Docker?
- 72. Cilium Architecture BPF BPF BPF Cilium Agent CLI Monitor Policy Plugins
- 73. Project Status • Initial release two weeks ago • Docker & Kubernetes integration • Looking for feedback and contributions
- 74. Getting Started • Play with our vagrant box: $ git clone https://github.com/cilium/cilium $ cd cilium/examples/getting-started $ vagrant up
- 75. Summary
- 78. Summary • BPF/XDP will drive the future of software based networking on Linux. • Never underestimate the Jedi • Traditional L3/L4 network policies are insufficient for microservices. Least privilege requires HTTP / API / Function awareness.
- 79. Summary • Never underestimate the Jedi • Traditional L3/L4 network policies are insufficient for microservices. Least privilege requires HTTP / API / Function awareness. • BPF/XDP will drive the future of software based networking on Linux. • Cilium brings BPF/XDP and L7 policies to containers and microservices.
- 80. Thank You! github.com/cilium/cilium http://cilium.io/ @ciliumproject Want to chat? DM me! @tgraf__ Don’t forget to vote and grab a shirt on the way out!
- 81. 75 140 205 240 325 365 370 365 410 412 425 445 450 460 460 490 495 505 515 525 545 565 0 100 200 300 400 500 600 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 BPF redirect() performance [GBit per core] Intel Xeon 3.5Ghz Sandy Bridge, 24 Cores, (1 TCP GSO flow per core, netperf -t TCP_SENDFILE, 10K policies)