Accelerating Envoy and Istio with Cilium and the Linux Kernel
23 likes7,783 views
The document discusses how Cilium can accelerate Envoy and Istio by using eBPF/XDP to provide transparent acceleration of network traffic between Kubernetes pods and sidecars without any changes required to applications or Envoy. Cilium also provides features like service mesh datapath, network security policies, load balancing, and visibility/tracing capabilities. BPF/XDP in Cilium allows for transparent TCP/IP acceleration during the data phase of communications between pods and sidecars.
![The Rise of BPF [and XDP]](https://image.slidesharecdn.com/2018kubeconeucilium-acceleratingenvoy-180505125951/85/Accelerating-Envoy-and-Istio-with-Cilium-and-the-Linux-Kernel-3-320.jpg)
![BPF/XDP: DDoS mitigation
Metric iptables / ipset BPF / XDP
DDoS rate [packets/s] 11.6M 11.6M
Drop rate [packets/s] 7.1M 11.6M
Latency under load [ms] 2.3ms 0.1ms
Requests/s under DDoS [Requests/s] 280 82’800
Sender: Send 64B packets as fast as possible
Receiver: Drop as fast as possible
Source: http://schd.ws/hosted_files/ossna2017/da/BPFandXDP.pdf](https://image.slidesharecdn.com/2018kubeconeucilium-acceleratingenvoy-180505125951/85/Accelerating-Envoy-and-Istio-with-Cilium-and-the-Linux-Kernel-5-320.jpg)
![So many more examples….
• Cloudflare DDoS mitigation [https://www.netdevconf.org/2.1/slides/apr6/bertin_Netdev-XDP.pdf]
• BCC [https://github.com/iovisor/bcc]
• Systemtap [https://sourceware.org/git/gitweb.cgi?p=systemtap.git;a=summary]
• Weave Scope [https://github.com/weaveworks/scope]
• Suricata [http://suricata.readthedocs.io/en/latest/capture-hardware/ebpf-xdp.html]
• systemd [http://0pointer.net/blog/ip-accounting-and-access-lists-with-systemd.html]
• gobpf [https://github.com/iovisor/gobpf]
• ply [https://github.com/wkz/ply]
• bpfps [https://github.com/genuinetools/bpfps]
• Perf [https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/perf]
• bpftrace [https://github.com/ajor/bpftrace]
• Open vSwitch [http://www.openvswitch.org//support/ovscon2016/7/1120-tu.pdf]
• PCP [https://github.com/performancecopilot/pcp]
• …](https://image.slidesharecdn.com/2018kubeconeucilium-acceleratingenvoy-180505125951/85/Accelerating-Envoy-and-Istio-with-Cilium-and-the-Linux-Kernel-12-320.jpg)
![Cilium in a Nutshell
• Current Release: 1.0.1
• Highly efficient BPF datapath
• Fully Distributed
• Service Mesh datapath
• CNI and CMM plugin
• Network Security on both Packet and API level
• Identity Based
• IP/CIDR as fallback
• API Aware (HTTP, gRPC, Kafka, [more coming soon])
• Distributed and Scalable Load Balancing
• Simplified Networking Model
• Overlay
• Direct Routing
• Delegation to other plugin (1.1)
• Visibility / Tracing](https://image.slidesharecdn.com/2018kubeconeucilium-acceleratingenvoy-180505125951/85/Accelerating-Envoy-and-Istio-with-Cilium-and-the-Linux-Kernel-28-320.jpg)
Ad
More Related Content
What's hot (20)
Similar to Accelerating Envoy and Istio with Cilium and the Linux Kernel (20)
Ad
More from Thomas Graf (13)
Ad
Recently uploaded (20)
Accelerating Envoy and Istio with Cilium and the Linux Kernel
- 1. Cilium Accelerating Envoy and Istio with Cilium and the Linux Kernel Thomas Graf, Covalent
- 2. BPF - The Superpowers inside Linux
- 3. The Rise of BPF [and XDP]
- 4. Source: https://www.netdevconf.org/2.1/slides/apr6/zhou-netdev-xdp-2017.pdf Facebook published BPF numbers for L3/L4 LB at NetDev 2.1 BPF/XDP performance IPVS performance
- 5. BPF/XDP: DDoS mitigation Metric iptables / ipset BPF / XDP DDoS rate [packets/s] 11.6M 11.6M Drop rate [packets/s] 7.1M 11.6M Latency under load [ms] 2.3ms 0.1ms Requests/s under DDoS [Requests/s] 280 82’800 Sender: Send 64B packets as fast as possible Receiver: Drop as fast as possible Source: http://schd.ws/hosted_files/ossna2017/da/BPFandXDP.pdf
- 6. BPF: “dtrace for Linux” Source: http://www.brendangregg.com/blog/
- 8. What is your favorite iptables memory?
- 9. What is your favorite iptables memory?
- 10. Kernel developers are saying goodbye to iptables https://lwn.net/Articles/747504/
- 11. Early performance benchmark Source: https://bit.ly/2Ks9gJH
- 12. So many more examples…. • Cloudflare DDoS mitigation [https://www.netdevconf.org/2.1/slides/apr6/bertin_Netdev-XDP.pdf] • BCC [https://github.com/iovisor/bcc] • Systemtap [https://sourceware.org/git/gitweb.cgi?p=systemtap.git;a=summary] • Weave Scope [https://github.com/weaveworks/scope] • Suricata [http://suricata.readthedocs.io/en/latest/capture-hardware/ebpf-xdp.html] • systemd [http://0pointer.net/blog/ip-accounting-and-access-lists-with-systemd.html] • gobpf [https://github.com/iovisor/gobpf] • ply [https://github.com/wkz/ply] • bpfps [https://github.com/genuinetools/bpfps] • Perf [https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/perf] • bpftrace [https://github.com/ajor/bpftrace] • Open vSwitch [http://www.openvswitch.org//support/ovscon2016/7/1120-tu.pdf] • PCP [https://github.com/performancecopilot/pcp] • …
- 13. BPF toolchain
- 14. How does BPF apply to Envoy and Service Mesh?
- 15. Kubernetes Pod Service Kubernetes Pod Service Mesh / Sidecar Architecture Network Service SidecarSidecar
- 16. Kubernetes Pod Service Kubernetes Pod Operating System Socket Socket Sidecar Injection Network Service Socket Socket Socket Socket
- 17. Kubernetes Pod Service Kubernetes Pod Operating System Socket TCP/IP Socket TCP/IP Sidecar Injection Network Service Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP
- 18. Kubernetes Pod Service Kubernetes Pod Sidecar Injection Network Service Operating System Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Ethernet Ethernet Ethernet Ethernet EthernetEthernet
- 19. Kubernetes Pod Service Kubernetes Pod Operating System Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Sidecar Injection (Non-transparent) Network Service Ethernet Ethernet Loopback Ethernet eth0 Ethernet Ethernet Loopback Ethernet eth0
- 20. Kubernetes Pod Service Kubernetes Pod Operating System Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Socket TCP/IP Sidecar Injection (Transparent) Network Service Ethernet Ethernet Loopback Ethernet eth0 Ethernet Ethernet Loopback Ethernet eth0 iptables iptables iptablesiptables
- 21. Why use TCP and Ethernet in a single-node, lossless environment?
- 22. Kubernetes Pod Service Kubernetes Pod Operating System Socket Socket Socket TCP/IP Socket TCP/IP Socket Socket Transparent Sidecar Injection with Cilium Network Service Ethernet eth0 Ethernet eth0 CiliumCilium
- 24. Socket TCP/IP Socket TCP/IP Service A Service B Ethernet Ethernet Loopback TCP Handshake Phase Socket Socket Service A Process B Data Phase Cilium Transparent acceleration during data phase • Transparent acceleration, behavior stays the same • No changes to application or Envoy needed 1 2
- 25. Socket TCP/IP Socket TCP/IP Process A Process B Ethernet Ethernet Cilium But wait… Works with any local socket communication TCP Handshake Phase Socket Socket Process A Process B Data Phase Cilium vethveth
- 26. How do I get this?
- 27. http://github.com/cilium/cilium Cilium Sidecar Accel: - Kernel: 4.16 - Cilium: 1.1/1.2 Cilium: - Kernel: 4.9
- 28. Cilium in a Nutshell • Current Release: 1.0.1 • Highly efficient BPF datapath • Fully Distributed • Service Mesh datapath • CNI and CMM plugin • Network Security on both Packet and API level • Identity Based • IP/CIDR as fallback • API Aware (HTTP, gRPC, Kafka, [more coming soon]) • Distributed and Scalable Load Balancing • Simplified Networking Model • Overlay • Direct Routing • Delegation to other plugin (1.1) • Visibility / Tracing
- 29. It gets better…
- 30. It gets better… … and a little bit scary
- 31. Service A Service B Mutual TLS TLS in Service Mesh Clear Text Encrypted Sidecar Sidecar Key
- 32. Service A Accessing external services via sidecar using TLS GitHub API Clear Text Encrypted Sidecar Encrypt Decrypt
- 33. Service A Option #1: Inject Root CA GitHub API Clear Text Encrypted Sidecar Inject new root CA Real root CA Encrypt Decrypt Encrypt Decrypt
- 34. kTLS Kubernetes Pod App Kubernetes Pod Operating System Socket Socket Network App Socket Socket Socket Socket SSL SSL … … … … … … Ethernet Ethernet Loopback Ethernet eth0 Ethernet Ethernet Loopback Ethernet eth0 Symmetric encryption offloaded to kernel ~4% CPU gain
- 35. Kubernetes Pod App Kubernetes Pod Operating System Socket Socket Socket TCP/IP Socket TCP/IP Socket Socket kTLS + Cilium + Envoy Network App Ethernet eth0 Ethernet eth0 iptablesiptables CiliumCilium SSL SSL
- 36. Service A Option #2: kTLS GitHub API Clear Text Encrypted Sidecar Deferred kTLS encryption Decrypt
- 37. Sidecar for TLS encrypted connections without CA injection & decryption
- 38. What date is today?