Tectonic Summit 2016: Networking for Kubernetes
Download as PPTX, PDF2 likes1,099 views
The document discusses the complexities of networking in Kubernetes, particularly focusing on integration with OpenStack and the challenges of multitenancy and legacy networks. It highlights the implementation of network 2.0, which abstracts network boundaries, allocates IP addresses via an IPAM controller, and dynamically configures pods with their respective IPs. Additionally, it covers load balancing strategies and future work on global network policy enforcement and federated ingress solutions.
Tectonic Summit 2016: Networking for Kubernetes
- 1. Networking for Kubernetes A Tale from the Trenches Cloud Engineering, eBay Sreekanth Pothanis
- 2. Networking is inherently hard! Complexities of running on openstack Scale Multitenancy Interoperability with Legacy
- 3. Private Network model with Openstack SDN Dedicated kube router provisioned in neutron Private Networks Subnet per node
- 4. L3 Routed Model NIPAP as IPAM Subnet per node Fully routable pods
- 5. Network 2.0 Abstract out network boundaries from nodes to arbitrary network scopes IP blocks are allocated to these network scopes Scopes can represent a host or a higher level aggregation Supports legacy and other complex network zoning
- 6. Network 2.0 node pod pod Network Scope IPAM node pod pod Allocation Pools Network Scope Allocation Pool 1 uuid1 2 uuid2
- 7. IPAM controller Cluster admin creates network scopes + allocation pools Kubernetes Nodes are associated with Scopes IPAM Controller assigns IP based on scope of the node selected by Kube scheduler Pods are annotated with IPs Tessnet plugin configures the pods with annotated IP Kube Scheduler IPAM controller Tess NetPlugin Pod: myPod Host: A Pod:myPod Node A notMyPod myPod 10.10.11.4 Tessnet Pluginkubelet Network Scope1 Allocation Pools 10.10.12.0/22 10.11.1.0/24 Node: A Node: B Node: C Network Scope2 API Server Host: A IP: 10.10.1.4 Pod: myPod Host: A IP: 10.10.1.4 Pod: myPod “network_scope”: “netscope1”
- 8. Networking 2.0 -- host OVS ARP Proxy
- 9. Service to POD Kube’s default implementation creates LBs on Nodes Load balance on pods directly Neutron LBaaS Pool Neutron LBaaS VIP POD POD POD POD
- 10. eBay Ingress Application Topology POOL Application VIP VIP GTM Load Balanced Pool POOL VIP POOL VIP Region 1 Region 2 Region 3 Global Name (omg.g.ebay.com) MONITOR MONITOR MONITOR Application VIPApplication VIP
- 11. Ingress controller Ingress: myIngress Status: VIP-1 IP GTM name Ingress controller API Server LBMS DNS GTM Ingress: myIngress Ingress: myIngress Status: VIP-1 IP Ingress: myIngress Status: VIP-1 IP GTM name
- 13. Future work Network Policy Enforcement Globally federated Ingress -- SLB based
Editor's Notes
- #4: ebays architecture, multiple generations of cloud, scale, use of lot of physical hardware and LBs have sdn and routed l3 netowrks Dont want to make it more harder why not private ips
- #5: IPTables work because one end of veth was still in host We had a few options, we went with what we knew best that worked on an overlay and routed L3 network Issues: Floating IPs need to created for every pod that needs to be routable Complex router/network setup. Works only overlay
- #6: Static allocation of per minion subnet Bootstrapping issues
- #7: Why central IPAM
- #8: Statically route to tor
- #9: Finalize with IPAM and Network plugin
- #10: Bonus: Hairpin ovs flow routes
- #15: Fully software based implementation of ingress collab with community