PKS: The What and How of Enterprise-Grade Kubernetes
5 likes2,640 views
SpringOne Platform 2017 Cornelia Davis, Pivotal; Fred Melo, Pivotal Because of its well thought out and powerful abstractions, robust and cloud-native architecture, and the vibrant community around it, the use of Kubernetes for containerized workloads has surged. And while Kubernetes is theoretically ready to run applications in production, the actual viability is highly dependent on how Kubernetes itself is managed. In this session Cornelia and Fred will cover role of the container orchestration system in your IT landscape, and they’ll dive under the covers to show how it provides the enterprise-class Kubernetes services you need to trust your most critical workloads to it. Yes, technical details revealed!
Ad
More Related Content
What's hot (20)
Similar to PKS: The What and How of Enterprise-Grade Kubernetes (20)
Ad
More from VMware Tanzu (20)
Ad
Recently uploaded (20)
PKS: The What and How of Enterprise-Grade Kubernetes
- 1. PKS: The What and How of Enterprise-Grade Kubernetes (or: Because the Cool Kids All Spell Kontainer with a K ;-)) Cornelia Davis, Sr. Director of Technology, Pivotal, @cdavisafc Fred Melo, Director of Technology, Pivotal, @fredmelo_br 1
- 2. Disclaimer
- 3. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Safe Harbor Statement The following is intended to outline the general direction of Pivotal's offerings. It is intended for information purposes only and may not be incorporated into any contract. Any information regarding pre-release of Pivotal offerings, future updates or other planned modifications is subject to ongoing evaluation by Pivotal and is subject to change. This information is provided without warranty or any kind, express or implied, and is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions regarding Pivotal's offerings. These purchasing decisions should only be based on features currently available. The development, release, and timing of any features or functionality described for Pivotal's offerings in this presentation remain at the sole discretion of Pivotal. Pivotal has no obligation to update forward looking information in this presentation. 3
- 4. The Why
- 5. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Companies have Many Types of Workloads 5 CONTAINERS EVENT-DRIVEN FUNCTIONS DATA SERVICES MICROSERVICES Batches MONOLITHIC APPLICATIONS
- 6. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ …And Different SDLCs 6 “Stable” maintenance Active Development RUN BUILD + RUN
- 7. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 7 Monolithic PAAS Cloud Native RUN BUILD + RUN
- 8. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 8 Traditional IaaS Cloud-native App Platform Monolithic PAAS Cloud Native RUN BUILD + RUN
- 9. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 9 Infrastructure Orchestrated CaaS Cloud-native App Platform Monolithic PAAS Cloud Native RUN BUILD + RUN Traditional IaaS
- 10. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ PCF2.0 CONFIDENTIAL 10 Infrastructure Orchestrated CaaS Cloud-native App Platform Monolithic PAAS Cloud Native RUN BUILD + RUN Traditional IaaS
- 11. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 11 Infrastructure Traditional IaaS Orchestrated CaaS Cloud-native App Platform Monolithic PAAS Cloud Native PCF2.0 RUN BUILD + RUN
- 12. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 12 Hardware IaaS Container Orchestrator Application Platform Serverless Functions Strategic goal: Push as many workloads as technically feasible to the top of the platform hierarchy Higher flexibility and less enforcement of standards Lower development complexity and higher operational efficiency
- 13. The What
- 14. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 14
- 15. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Operational Challenges with any platform 15 Patches Patching platform components with thousands of apps running should feel normal. Scaling Seamlessly scale platform components to accommodate changing demand. Upgrades. How do you roll out new versions of the platform with the lights on? Operating Effort Operating a platform should require very few resources and minimum manual intervention. Otherwise, is it really providing operational benefits? Multi-cloud Provide a reliable and smooth experience for any cloud. Open APIs Allow platform operations from different toolsets and the creation of CD pipelines. Consistency Provide a consistent setup experience, across different cloud environment configurations. Setup time How long does it take to setup a real world working environment? Think hours, not weeks. Day 1 - Build Day 2 - Operate
- 16. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Kubernetes - especially hard to operationalize 16 High Availability. No out-of-the-box fault- tolerance for the cluster components themselves (masters and etcd nodes). Scaling. Kubernetes clusters handle scaling the pod/service within the Nodes, but doesn’t provide a mechanism to scale Masters & etcd VMs. Health checks and healing. The Kubernetes cluster does routine health checks for the health of Nodes only. Upgrades. Rolling upgrades on a large fleet of clusters is hard. Who manages the system it runs on?
- 17. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ PIVOTAL CLOUD FOUNDRY OPS Powered by BOSH 17 BOSH is an open source tool for release engineering, deployment, lifecycle management, and monitoring of distributed systems. Packaging w/ embedded OS Server provisioning on any IaaS Software deployment across availability zones Health monitoring (server AND processes) Self-healing w/ Resurrector Storage management Rolling upgrades via canaries Easy scaling of clusters
- 18. Workeretcd etcd Kubernetes etcd Master Master Worker Worker Packaging w/ embedded OS Server provisioning on any IaaS Software deployment across availability zones Health monitoring (server AND processes) Self-healing w/ Resurrector Storage management Rolling upgrades via canaries Easy scaling of clusters
- 19. Project Kubo Uniform way to instantiate, deploy, and manage highly available Kubernetes clusters. On any cloud. Launched by Pivotal & Google Feb 2017, Donated to Cloud Foundry Foundation June 2017 Committers: Pivotal, Google, VMware “Day 1” Build " Deploy Kubernetes cluster via BOSH “Day 2” Operate " Self-healing VMs and monitoring via BOSH " Elastic scaling for clusters " Rolling upgrades to latest Kubernetes release " High-availability and multi-AZ support
- 20. Workeretcd etcd Kubernetes etcd Master Master Worker Worker Kubo Provides Specification of K8S Components 20 Platform team is then responsible for assembly into desired clustersRelease templates Manifest Kubo Release bosh deploy
- 21. Pivotal Container Service (PKS) Provides the control plane for provisioning and managing Kubo releases Joint development effort between Pivotal and VMWare Kubernetes Dial Tone: • Health management • Framework for Metrics and Logging • Autoscaling • Persistence interface • Networking interface Control Plane: • Provisioning Engine • Self-service Clusters • Software Update Automation • Load balancing • Networking • Persistence • Multi-tenancy
- 22. PKS: Provisioning Engine 22 PKS Service Broker Release templates Manifest Kubo Release
- 23. 23 PKS Service Broker Release templates Manifest Kubo Release create cluster (with upgrade policy) PKS: Self-service Clusters
- 24. 24 PKS: Networking (different options available) BOSH network Worker Worker 10.0.30.12 ContainerContainerContainer 10.200.2.6C2C overlay 10.200.1.510.200.1.4 10.0.30.11 kube-proxy iptables kube-proxy iptables Service network Service
- 25. BOSH GCP Service Broker Harbor NSX-T Kubernetes K8s Cluster K8s Cluster K8s Cluster Built with open-source Kubernetes — Constant compatibility with the current stable release of Kubernetes, operated by BOSH. No proprietary extensions. Production-ready — Highly available from apps to infrastructure, no single points of failure. Built-in health checks, scaling, auto-healing and rolling upgrades. Multicloud — BOSH provides a reliable and consistent operational experience. For any cloud. Network management and security out-of-the-box with VMware NSX-T. Multi-cloud, multi-hypervisor. GCP APIs access — The GCP Service Broker allows apps to transparently access Google Cloud APIs, from anywhere. Easily move workloads to/from Google Container Engine (GKE). Fully automated Ops — Fully automated deploy, scale, patch, upgrade. No downtime. Use CD pipelines to deploy your platform, too. VMware GCP Azure Openstack AWS PKSController
- 26. Apps are constantly compatible between PKS and Google Container Engine (GKE) Google Container Engine (GKE) Pivotal Container Service (PKS) Private and Public Cloud Public Cloud (on GCP)
- 27. VMware PKS Analytics Automation SecurityOperations MonitoringLogging Physical Infrastructure Container Registry vSphere vSAN Kubernetes on BOSH (Kubo) NSX GCP Service Broker masteretcd workermasteretcd worker
- 28. Leveraging more than one abstraction Other Broker Services Platform Services Logging Metrics Monitoring Pivotal Application Service Application Application VMware GCP Azure Openstack AWS PKSController GCP Service Broker Harbor NSX-T Kubernetes K8s Cluster K8s Cluster K8s Cluster
- 29. Unified Platform Experience Shared Logging and Metrics Shared Networking Shared Security
- 30. The How
- 31. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 31 BOSH GCP Service Broker Harbor NSX-T Kubernetes K8s Cluster K8s Cluster K8s Cluster Built with open-source Kubernetes — Constant compatibility with the current stable release of Kubernetes, operated by BOSH. No proprietary extensions. Production-ready — Highly available from apps to infrastructure, no single points of failure. Built-in health checks, scaling, auto-healing and rolling upgrades. Multicloud — BOSH provides a reliable and consistent operational experience. For any cloud. Network management and security out-of-the-box with VMware NSX-T. Multi-cloud, multi-hypervisor. GCP APIs access — The GCP Service Broker allows apps to transparently access Google Cloud APIs, from anywhere. Easily move workloads to/from Google Container Engine (GKE). Fully automated Ops — Fully automated deploy, scale, patch, upgrade. No downtime. Use CD pipelines to deploy your platform, too. VMware GCP Azure Openstack AWS PKSController How does this all work? • Availability and Resiliency • Operational Efficiency • Multi-tenancy isolation • Security • Extensibility Guiding Principles
- 32. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 32 BOSH PKSControlPlane Pivotal Container Service
- 33. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ BOSH Pivotal Container Service PKSControlPlane Create cluster
- 34. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 34 Availability Zone B Availability Zone A HA and Health Management - Kubelet watches and restart containers - Bosh director watches and restarts nodes - Bosh agent watches and restarts processes - Bosh distributes deployments across AZ’s Kubelet Kube-proxy Pod Pod K8s Node Pod API Server Kube Scheduler K8s Master Controller Manager Bosh agent Bosh agentBosh director Watches and restarts VMs Availability Zone A Availability Zone B Create cluster
- 35. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 35 BOSH Pivotal Container ServicePKSControlPlane Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube-proxy Pod Pod Pod K8s Worker API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Etcd GCP Service BrokerHarbor NSX-T Proxy NCP T1 Routers NSX-T T1 Routers Other components Included component, but optional usage CFCR / Kubo
- 36. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 36 BOSH Pivotal Container ServicePKSControlPlane Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube-proxy Pod Pod Pod K8s Worker API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Etcd GCP Service BrokerHarbor NSX-T Proxy NCP T1 Routers NSX-T T1 Routers Other components Included component, but optional usage CFCR / Kubo Kubernetes
- 37. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Kubernetes components API Server Kube Scheduler K8s Master Controller Manager Etcd K-V Store Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod CNI Persistent Volume Virtual switch Network edge router Volume mount Aws, Azure, GCP, vSphere SCSI, FC, NFS, Photon, Scale IO, … Provisioner NSX-T, Flannel, Calico, GCE, Nuange, OVN, Kube-router… CNI Plugin Protobuf Network edge routerNetwork edge router
- 38. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 38 API Server Kube Scheduler K8s Master Controller Manager Etcd K-V Store Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod CNI Persistent Volume Virtual switch Network edge router Volume mount Aws, Azure, GCP, vSphere SCSI, FC, NFS, Photon, Scale IO, … Provisioner NSX-T, Flannel, Calico, GCE, Nuage, OVN, Kube-router… CNI Plugin Protobuf Network edge routerNetwork edge router Pluggable Networking (CNI)
- 39. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 39 BOSH Pivotal Container Service PKSController GCP Service Broker BOSH network 10.0.1.5 Harbor 10.0.1.2 Etcd 10.0.1.6 Etcd 10.0.1.6 API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token 10.0.1.4 Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node 10.0.1.3 Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node 10.0.1.3 Pod Pod Pod K8s Worker Kubelet Kube-proxy 10.0.1.3 Networking
- 40. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 40 BOSH PKSController GCP Service Broker Harbor Etcd API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Base Network 10.0.0.11 10.0.0.16 10.0.0.17 10.0.0.13 BOSH Network NSX-T Mgr NSX-T NCP .18 .19 10.0.0.21 10.0.0.22 Kubelet NSX Kube-Proxy OVS K8s Worker K8s Worker Kubelet
- 41. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 41 BOSH PKSController GCP Service Broker Harbor Etcd API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Base Network 10.0.0.11 10.0.0.16 10.0.0.17 10.0.0.13 BOSH Network NSX-T Mgr NSX-T NCP .18 .19 10.0.0.21 10.0.0.22 Kubelet Pod Pod NSX Kube-Proxy OVS K8s Worker Pod Pod NSX Kube-Proxy K8s Worker K8K8s NameSpace A = NSX Logical Switch K8K8s NameSpace B = NSX Logical Switch T1 10.172.10.2 10.172.100.0/24 192.168.100.0/24 NO_NAT NAT T1 10.172.10.3 T0 10.172.10.1 Kubelet OVS NSX CNI PLUGIN Overlay Networks
- 42. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 42 -T Pivotal Container Service Network Policy Management Multi-Tenancy Isolation Network Security Groups Dynamic Load Balancing & Ingress Comprehensive VM + Container security model
- 43. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 43 kubectl apply -f <file.yml> Network Policies Management BOSH Pivotal Container Service PKSController NSX-T Proxy NCP T1 RoutersT1 Routers Other components NSX-T Kubelet Kube- proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube- proxy Fluentd Pod Pod Pod K8s NodeNode NSX Router Network policy management API Server Kube Scheduler K8s Master Controller Manager Etcd new policy Apply Kubelet K. Proxy Pod Pod Pod
- 44. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 44 Deployment Topologies & Multi-Tenancy Multi-cluster Single cluster K8s Cluster A K8s Cluster BOSH PKSController Namespace A Namespace B Namespace C BOSH PKSController K8s Cluster B K8s Cluster C
- 45. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 45 admin@k8s-master:~$ kubectl create namespace foo namespace ”foo" created admin@k8s-master:~$ kubectl create namespace bar namespace ”bar" created admin@k8s-master:~$ kubectl run nginx-foo --image=nginx -n foo deployment "nginx-foo" created admin@k8s-master:~$ kubectl run nginx-bar --image=nginx -n bar deployment "nginx-bar" created Namespace: foo Namespace: bar NSX / K8s topology 10.24.0.0/24 10.24.1.0/24 10.24.2.0/24 NAT boundary NAT boundary K8s nodesK8s Masters NameSpace as a Tenancy Construct NSX-T Automatically Provides Each Namespace a secure network & routing
- 46. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 46 admin@k8s-master:~$ kubectl label pods nginx-foo-3492604561-nltrf secgroup=web -n foo Pod "nginx-nsx-3492604561-nltrf" labeled admin@k8s-master:~$ kubectl label pods nginx-bar-2789337611-z09x2 secgroup=db -n bar pod "nginx-k8s-2789337611-z09x2" labeled admin@k8s-master:~$ kubectl get pods --all-namespaces -Lsecgroup NAMESPACE NAME READY STATUS RESTARTS AGE SECGROUP k8s nginx-foo-2789337611-z09x2 1/1 Running 0 58m web nsx nginx-bar-3492604561-nltrf 1/1 Running 0 1h db Namespace: foo Namespace: bar NSX / K8s topology 10.24.0.0/24 10.24.1.0/24 114.4.10.0/26 NAT boundary NAT boundary Web • Security Groups are defined in NSX with ingress and egress policy • Each Security Group could be micro-segmented to protect Pods from each other DB
- 47. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 47 App / Container Deployment, Services & Routing cf push App myapp.mydomain.net Pivotal Cloud Foundry Application Runtime Pivotal Container Service kubectl run ???? ????
- 48. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 48 App / Container Deployment, Services & Routing cf push App myapp.mydomain.net Pivotal Cloud Foundry Application Runtime Pivotal Container Service kubectl run ???? ????
- 49. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 49 BOSH Pivotal Container Service PKSController Kubelet Kube-proxy Pod K8s Worker K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token kubectl run <image> Harbor Etcd Pod API Server Kube Scheduler Pod Pod
- 50. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 50 ??? kubectl run <docker img> K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 API Server Kube Scheduler K8s Master Controller Manager API Server Kube Scheduler K8s Master Controller Manager K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 K8s Worker Kubelet Kube-proxy Pod Pod 192.168.0.4 192.168.0.3 Pod 192.168.0.2 K8s Cluster Base Network Overlay Network Cluster network (conceptual)
- 51. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 51 kubectl run <docker img> K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 Cluster network (conceptual) API Server Kube Scheduler K8s Master Controller Manager API Server Kube Scheduler K8s Master Controller Manager K8s Cluster K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 Base Network Overlay Network Service K8s Worker Kubelet Kube-proxy Pod Pod 192.168.0.4 192.168.0.3 Pod 192.168.0.2
- 52. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 52 NodePort K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 Cluster network (conceptual) API Server Kube Scheduler K8s Master Controller Manager API Server Kube Scheduler K8s Master Controller Manager K8s Cluster K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 Base Network Overlay Network kubectl expose deployment NodePort NodePort K8s Worker Kubelet Kube-proxy Pod Pod 192.168.0.4 192.168.0.3 Pod 192.168.0.2 host:port NodePort Load Balancer host
- 53. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 53 NodePort K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 API Server Kube Scheduler K8s Master Controller Manager API Server Kube Scheduler K8s Master Controller Manager K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 kubectl expose deployment Load Balancer NodePort K8s Worker Kubelet Kube-proxy Pod Pod 192.168.0.4 192.168.0.3 Pod 192.168.0.2 NodePort Cloud Load Balancer host Dynamically provisioned Kubernetes Cluster Cloud 35.190.151.218 Public IP -T
- 54. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 54 Services: Ingress An API object that manages external access to the services in a cluster, typically HTTP. Ingress can provide load balancing, SSL termination and name-based virtual hosting. K8s Worker Kubelet Kube-proxy PodPodPod K8s Worker Kubelet Kube-proxy PodPodPodS2 S1 K8s Worker Kubelet Kube-proxy PodPod Pod Ingress /foo /bar foo.bar.com Deploy
- 55. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 5 BOSH Pivotal Container Service PKSController Kubelet Kube-proxy Fluentd Pod Pod K8s Node API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Pod 192.168.0.1 192.168.0.4 192.168.0.3 192.168.0.2 Pivotal Application Service Route-sync GO Router myapp.apps.myorg.net kubectl expose deployment (NodePort w/ route sync) NodePort Integration with CFAR Routing
- 56. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 56 API Server Kube Scheduler K8s Master Controller Manager Etcd K-V Store Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod CNI Persistent Volume Virtual switch Network edge router Volume mount Aws, Azure, GCP, vSphere SCSI, FC, NFS, Photon, Scale IO, … Provisioner Protobuf Network edge routerNetwork edge router Pluggable Persistent Volumes
- 57. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 57 Pod Container Container Host mount Fiber Channel, iSCSI, NFS GCP, Azure, AWS, vSphere Volumes and Persistent Volumes Legacy Systems Stateful Systems Database
- 58. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 58 kubectl apply -f <file.yml> Pivotal Container Service PKSController K8s Master Etcd Worker Kubelet K. Proxy Pod Pod Worker Kubelet K. Proxy Pod Pod Worker Kubelet K. Proxy Pod Pod BOSH
- 60. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 60 Stateful Sets Stable, unique network identifiers Stable, persistent storage Ordered, graceful deployment and scaling Ordered, automated rolling updates * stable = survives Pod rescheduling Pod Stateful set Name = web Replicas = 3 Pod web-0 web-1 web-2 Pod PV-claim-web-0 Each Pod gets an unique ID and Persistent Volumes PV-claim-web-1 PV-claim-web-2
- 61. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 61 Stateful Sets Apps / clusters using specific IPs / names to communicate Apps / clusters relying on persistent and durable storage on specific mount points Apps / clusters with specific starting order for components Stable, unique network identifiers Stable, persistent storage Ordered, graceful deployment and scaling Ordered, automated rolling updates * stable = survives Pod rescheduling Clustered, stateful workloads or legacy apps
- 62. Learn More. Stay Connected. 62 #springone@s1p