Dayta AI Seminar - Kubernetes, Docker and AI on Cloud
Download as PPTX, PDF1 like391 views
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery. Kubernetes services expose these units to enable dynamic load balancing while maintaining session affinity. It also provides self-healing capabilities by restarting containers that fail, replacing them, and killing containers that don't respond to their health check.
![[20200720]cloud native develoment - Nelson Lin](https://cdn.slidesharecdn.com/ss_thumbnails/20200720cloudnativedeveloment-200824031228-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (20)
Similar to Dayta AI Seminar - Kubernetes, Docker and AI on Cloud (20)
Ad
Recently uploaded (20)
![Pixologic ZBrush Crack Plus Activation Key [Latest 2025] New Version](https://cdn.slidesharecdn.com/ss_thumbnails/fashionevolution2-250322112409-f76abaa7-250428124909-b51264ff-250504160528-fc2bb1c5-thumbnail.jpg?width=560&fit=bounds)
![Download Wondershare Filmora Crack [2025] With Latest](https://cdn.slidesharecdn.com/ss_thumbnails/neo4j-howkgsareshapingthefutureofgenerativeaiatawssummitlondonapril2024-240426125209-2d9db05d-250419-250428115407-a04afffa-thumbnail.jpg?width=560&fit=bounds)
Ad
Dayta AI Seminar - Kubernetes, Docker and AI on Cloud
- 1. Dayta Engineering Seminar Kubernetes,Dockerand DaytaAI 12July, 2019
- 2. Dockerin a nutshell • Virtualization software for cross-OS deployment Program code Entrypoint (main.py) TODO In Remote Linux Instance: - apt-get all packages - git pull - virtualenv or conda - install requirements - if program is server: - Port mapping (http:80, https/tcp: 443) - Execute entrypoint program No Docker + CUDA & GPU Device Mapping Dockerfile FROM nvidia/cuda:9.0-runtime RUN apt-get update && apt- get install -y … RUN pip install -r requirements.txt COPY . . CMD python application.py With Docker TODO In Remote Linux Instance: - git pull - docker build - docker run Not using Git? You are fired
- 3. Dockerin a nutshell • Virtualization software for cross-OS deployment Program code Entrypoint (main.py) Dockerfile Docker Container docker build Docker Engine Host OS docker run App A App B New App
- 4. Dockerin Production /server codebase Entrypoint (worker.py) Dockerfile Worker Container docker build Docker Engine Host OS docker-compose run Worker /client codebase Dockerfile Client Container Nginx Container In Docker Hub Nginx client & server Worker Inference Codes in here
- 5. Dockerin Production:Problems TODO In Remote Linux Instance: - git pull - docker build - docker run • Requires SSH &Deployment Process (Infeasible) • No Automation (Continuous Deployment) (Unreliable) Docker Engine Host OS Worker Nginx client & server Worker CPU Util: 12% Mem Util: 20% • Difficult todetermine optimum resource utilization (Infeasible) • 1 Instance per service? (Unscalable) • Manual reaction tousage spikes (Unreliable) Profit margin space for cloud service providers
- 6. Dockerin Production:Problems Cloud Instance (Failed) Public IP Cloud Instance Public IP Cloud Instance Public IP Big & Expensive Instance (e.g. p3.2xlarge) Big & Expensive Instance (e.g. p3.2xlarge) Big & Expensive Instance (e.g. p3.2xlarge) 1. Marginally Additional Usage Cloud Instance Public IP Big & Expensive Instance (e.g. p3.2xlarge) 2. Instance OR Program Crash 3. Potential Network Security Vulnerability (SSH, DDoS, etc) 4. Not Enough Cross-Instance Load Balancing 5. Difficulty in Patch & Updates
- 7. Kubernetesin a nutshell • Virtualization software for docker container orchestration, Infrastructure as Code(IaC) Docker Engine Host OS Kubelet Program code Entrypoint (main.py) Dockerfile Docker Container docker build Container Registry DockerHub AWS ECR docker push kubectl apply (docker pull) K8s Pod Docker Container
- 8. Kubernetes:Components • Virtualization software for docker container orchestration K8s Pod Docker Container Pod - SmallestComponentin Kubernetes - Equivalentto singledocker-composedeployment Deployment - Entityof group of Pods - Defines& Maintainsthe Podconfig and numberof Pods K8s Deployment K8s Pod K8s Pod Service -Mapping,routingandnetworkaccess controllerfor deploymentsorpods -Defines&MaintainsthePodport& DNS mappinginside andoutsidek8scluster K8s Deployment K8s Pod K8s Pod K8s ServiceExternal Requests Internal Requests
- 9. Kubernetes:Components • Virtualization software for docker container orchestration ReplicaSet - Defines& MaintainsthePodconfig and numberof Pods - Deploymentmanagesthe ReplicaSetdefinedfor Deployment DaemonSet - ReplicaSetfor processesrequiredtorun when Instanceboots StatefulSet - ReplicaSetfor Podsthat requirestates(Pod-specific information) Job - ReplicaSetfor processesrequiredtorun once until completionortermination CronJob - JobConfigurationfor Jobsrequiredto run periodically
- 10. Kubernetes:Components • Virtualization software for docker container orchestration Node - Groupof Instanceswithidenticalconfiguration (InstanceType,NodeAffinity, AutoscalingTrigger) - Identicalto AWS AutoscalingGroup k8s::node::gpunode p3.2xlarge p3.2xlarge Auto-Scaler (AWS EC2 Autoscaling Group) Node Config: - Type: p3.2xlarge - maxSize: 5 - minSize: 1 - nodeLabel: - type:gpu k8s::node::gpunode apiVersion: kops/v1alpha2 kind: InstanceGroup metadata: labels: kops.k8s.io/cluster: somecluster.com name: gpunodes spec: hooks: - execContainer: image: dcwangmit01/nvidia-device-plugin:0.1.0 image: kope.io/k8s-1.10-debian-stretch-amd64-hvm-ebs- 2018-05-27 machineType: p3.2xlarge maxSize: 5 minSize: 1 nodeLabels: kops.k8s.io/instancegroup: gpunodes type: gpu role: Node subnets: - us-east-1a - us-east-1b - us-east-1c
- 11. Kubernetes:Components • Virtualization software for docker container orchestration Cluster - Groupof Nodesforming onesystemforKubernetes - LargestEntity in K8s k8s::cluster::somecluster.com Virtual Private Cloud k8s::ig::master-us-east-1a k8s::ig::master-us-east-1b k8s::ig::master-us-east-1c k8s::ig::nodes k8s::ig::gpunodes k8s::ig::redisnodes us-east-1a us-east-1b us-east-1c r5.xlarge p3.2xlarge r5.xlarge r5.xlarge m4.xlarge m4.xlarge m4.xlarge m4.xlarge m4.xlarge
- 12. Kubernetesin Production k8s.io/somecluster.com::deployment::web apiVersion: extensions/v1beta1 kind: Deployment metadata: name: web-deployment labels: app: web spec: replicas: 9 selector: matchLabels: app: web template: metadata: labels: app: web spec: containers: - name: web image: somerepo.com/web ports: - containerPort: 8080 - name: nginx image: nginx imagePullPolicy: IfNotPresent ports: - containerPort: 80 volumeMounts: - mountPath: /etc/nginx/conf.d readOnly: true name: web-nginx-conf volumes: - name: web-nginx-conf configMap: name: web-nginx-conf items: - key: nginx.conf path: nginx.conf k8s.io/somecluster.com::service::web apiVersion: v1 kind: Service metadata: name: web spec: type: LoadBalancer selector: app: web ports: - name: http port: 80 targetPort: 80 - name: https port: 443 targetPort: 80
- 13. Kubernetesin Production Deployment: web K8s Pod x 9 Service: webweb- deployment.yaml web- service.yaml kubectl kubectl apply kubectl translate cyclops cluster
- 14. Kubernetesin Production Deployment::web::Pod nginx web 808080 8080 Service::web podId:80 http(80) https(443)
- 15. Kubernetesin Production Deployment: web K8s Pod Service: web k8s::cluster::somecluster.com Virtual Private Cloud k8s::ig::master-us-east-1a k8s::ig::master-us-east-1b k8s::ig::master-us-east-1c k8s::ig::nodes k8s::ig::gpunodes k8s::ig::redisnodes us-east-1a us-east-1b us-east-1c r5.xlarge p3.2xlarge r5.xlarge r5.xlarge m4.xlarge m4.xlarge m4.xlarge m4.xlarge m4.xlarge K8s Pod K8s Pod K8s Pod K8s Pod K8s Pod K8s Pod K8s Pod K8s Pod
- 16. Kubernetesin Production Deployment: web Pod: 9/9 (Fulfilled) Service: web k8s::cluster::somecluster.com Virtual Private Cloud k8s::ig::master-us-east-1a k8s::ig::master-us-east-1b k8s::ig::master-us-east-1c k8s::ig::nodes k8s::ig::gpunodes (node affinity limit. Only GPU Processes) k8s::ig::redisnodes (node affinity limit. Only Redis Processes) us-east-1a us-east-1b us-east-1c m4.xlarge m4.xlarge m4.xlarge m4.xlarge m4.xlarge
- 17. Kubernetesin Production Deployment: web Pod: 9/9 (Fulfilled) Service: web k8s::cluster::somecluster.com Virtual Private Cloud k8s::ig::master-us-east-1a k8s::ig::master-us-east-1b k8s::ig::master-us-east-1c k8s::ig::nodes us-east-1a us-east-1b us-east-1c m4.xlarge m4.xlarge m4.xlarge m4.xlarge m4.xlarge t4.large kube-dns internal IP of pods (instanceIP/containerIP) Register serviceIP mapped to podIPs
- 18. Kubernetesin Production Deployment: web Service: web k8s::cluster::somecluster.com Virtual Private Cloud k8s::ig::master-us-east-1a k8s::ig::master-us-east-1b k8s::ig::master-us-east-1c k8s::ig::nodes us-east-1a us-east-1b us-east-1c m4.xlarge m4.xlarge m4.xlarge m4.xlarge m4.xlarge t4.large kube-dns VPC External IP (Public DNS) Internet Request
- 19. Kubernetesin Production Deployment: web Pod: 7/9 (unfulfilled) Service: web k8s::cluster::somecluster.com Virtual Private Cloud k8s::ig::master-us-east-1a k8s::ig::master-us-east-1b k8s::ig::master-us-east-1c k8s::ig::nodes (instances 4/5 (unfulfilled)) k8s::ig::gpunodes (node affinity limit. Only GPU Processes) k8s::ig::redisnodes (node affinity limit. Only Redis Processes) us-east-1a us-east-1b us-east-1c m4.xlarge m4.xlarge m4.xlarge (errored) m4.xlarge m4.xlarge
- 20. Kubernetesin Production Deployment: web Pod: 9/9 (fulfilled) Service: web k8s::cluster::somecluster.com Virtual Private Cloud k8s::ig::master-us-east-1a k8s::ig::master-us-east-1b k8s::ig::master-us-east-1c k8s::ig::nodes (instances 4/5 (unfulfilled)) k8s::ig::gpunodes (node affinity limit. Only GPU Processes) k8s::ig::redisnodes (node affinity limit. Only Redis Processes) us-east-1a us-east-1b us-east-1c m4.xlarge m4.xlarge m4.xlarge (errored) m4.xlarge m4.xlargem4.xlarge (booting) internal IP of pods (instanceIP/containerIP)
- 21. Kubernetesin Production Deployment: web Pod: 9/9 (fulfilled) Service: web k8s::cluster::somecluster.com Virtual Private Cloud k8s::ig::master-us-east-1a k8s::ig::master-us-east-1b k8s::ig::master-us-east-1c k8s::ig::nodes k8s::ig::gpunodes (node affinity limit. Only GPU Processes) k8s::ig::redisnodes (node affinity limit. Only Redis Processes) us-east-1a us-east-1b us-east-1c m4.xlarge m4.xlarge m4.xlarge (terminate d) m4.xlarge m4.xlargem4.xlarge internal IP of pods (instanceIP/containerIP)
- 22. Kubernetesin Production Deployment: web Service: some-system k8s::cluster::somecluster.com Virtual Private Cloud k8s::ig::master-us-east-1a k8s::ig::master-us-east-1b k8s::ig::master-us-east-1c k8s::ig::nodes us-east-1a us-east-1b us-east-1c m4.xlarge m4.xlarge m4.xlarge m4.xlarge m4.xlarge web t4.large kube-dns POST: /getAccount HTTP/1.1 Host: some-system some system
- 23. Cloud Instance (Failed) Private IP Cloud Instance Private IP Cloud Instance Private IP Big & Expensive Instance (e.g. p3.2xlarge) Big & Expensive Instance (e.g. p3.2xlarge) Big & Expensive Instance (e.g. p3.2xlarge) 1. Marginally Additional Usage Cloud Instance Private IP Big & Expensive Instance (e.g. p3.2xlarge) 2. Instance OR Program Crash 3. Potential Network Security Vulnerability (SSH, DDoS, etc) 4. Not Enough Cross-Instance Load Balancing 5. Difficulty in Patch & Updates - Direct Access to Instances blocked by Bastion (SSH Proxy) - Direct Access to Containers blocked by kube-dns - Marginally Additional Usage leads to auto-scaling of pods, not instances - K8s automatically maintains the deployment state of processes and instances - Whole purpose of k8s is cross-instance load balancing and easy orchestration of docker containers Dockerin Production:Problems Recap