0% found this document useful (0 votes)
25 views

Exploring Kubernetes (Online Learning Version)

The document provides an overview of Kubernetes, an open-source platform for orchestrating containerized applications, detailing its architecture, deployment strategies, and key components. It covers the installation of k3d for local development, the role of pods in managing containers, and the use of kubectl for interacting with the Kubernetes API. Additionally, it highlights the advantages of using Kubernetes over traditional deployment methods, emphasizing scalability, reliability, and ease of management.

Uploaded by

Andrew Koh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
25 views

Exploring Kubernetes (Online Learning Version)

The document provides an overview of Kubernetes, an open-source platform for orchestrating containerized applications, detailing its architecture, deployment strategies, and key components. It covers the installation of k3d for local development, the role of pods in managing containers, and the use of kubectl for interacting with the Kubernetes API. Additionally, it highlights the advantages of using Kubernetes over traditional deployment methods, emphasizing scalability, reliability, and ease of management.

Uploaded by

Andrew Koh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 453

Exploring Kubernetes

Copyright © Dell Inc. All Rights Reserved.


Topics
1. Introduction to Kubernetes
2. Understanding the Kubernetes Architecture
3. Deploying Applications in Kubernetes
4. Imperative and Declarative Workflows
5. Services and Networking
6. Applying Deployment Strategies
7. Externalizing Configuration with ConfigMaps and Secrets
8. Persisting Data with Volumes
9. Managing Namespaces and Resource Quotas
10. Best Practices and Troubleshooting

Copyright © Dell Inc. All Rights Reserved.


Introduction to Kubernetes

Copyright © Dell Inc. All Rights Reserved.


Learning Objectives

At the end of this topic, you will be able to:


• learn the fundamental concepts of container orchestration and why Kubernetes
is essential in modern software development and deployment
• compare and contrast Kubernetes with traditional deployment methods, and
appreciate their advantages and limitations
• install k3d, a lightweight Kubernetes distribution designed for local development
and testing as part of our hands-on learning with Kubernetes

Copyright © Dell Inc. All Rights Reserved.


What is Kubernetes?

• An open-source platform to orchestrate the deployment, scaling and


management of containerised applications
• Originally developed by Google
• Donated to the Cloud Native Computing Foundation (CNCF) in 2015
• The name Kubernetes originates from Greek, meaning helmsman or pilot
• Kubernetes is often abbreviated as k8s

k8s
kubernetes
Copyright © Dell Inc. All Rights Reserved.
Why Kubernetes?

Copyright © Dell Inc. All Rights Reserved.


A Typical Traditional Distributed Software
Deployment

App A App B Database

Host Host Host

Storage

Network

Copyright © Dell Inc. All Rights Reserved.


What could possibly go wrong?

App A App B Database

Host Host Host


Scalability challenges

Compute failures

Storage

Network failures Network

Copyright © Dell Inc. All Rights Reserved.


What could possibly go wrong?

App A App B Database

Operating system
failures
Host Host Host
Scalability challenges
Inconsistent
configurations

Compute failures

Storage

Network failures Network

Storage failures

Copyright © Dell Inc. All Rights Reserved.


What could possibly go wrong?
Service disruptions from
new app rollouts

App A App B Database


Service dependency issues

Operating system
failures
Host Host Host
Scalability challenges
Inconsistent
configurations

Compute failures

Storage

Network failures Network

Storage failures

Copyright © Dell Inc. All Rights Reserved.


Decouple the software from the infrastructure

App A App B App C App D

Kubernetes

Compute Compute Compute Compute Compute


✓ High Availability

✓ Reliability

✓ Resiliency Network
This is transparent to us
✓ Scalability

✓ Self-Healing
Storage Storage Storage

Copyright © Dell Inc. All Rights Reserved.


Running our apps on Kubernetes

• Running applications on Kubernetes requires them to be packaged


into container images
• Containers are well-suited for microservices architecture, where
applications are decomposed into smaller, independent services
• This enhances portability, consistency, scalability, and resource
efficiency

Copyright © Dell Inc. All Rights Reserved.


What is a Container?

Language Runtime

App
App Libraries and
Dependencies

Copyright © Dell Inc. All Rights Reserved.


Why do we need Containers?

Containers offer the promise of


“build once, run anywhere”

Copyright © Dell Inc. All Rights Reserved.


Immutable Images

• Applications and their dependencies are containerised into images


• When we need to make updates and changes to our applications,
new images are created to replace the existing images
• We do not apply incremental changes to existing images
• This is at the core of what we will deploy in Kubernetes

v1.0 v2.0 latest


myimage myimage v1.1 myimage myimage

Copyright © Dell Inc. All Rights Reserved.


Deploying containers to Kubernetes

• We tell Kubernetes what containers we want to run


• Kubernetes decides where to run the containers
• Kubernetes abstracts away the real physical processors, memory,
storage and networking from the containerised applications

Copyright © Dell Inc. All Rights Reserved.


Platform Portability

App A App B App C App D

Kubernetes

On-prem Cloud Hybrid Cloud Edge

Copyright © Dell Inc. All Rights Reserved.


A Kubernetes Cluster

Control Plane
We deploy our containerised
apps here

Node
Node
Node
Worker Data Plane
Node

Kubernetes Cluster

Copyright © Dell Inc. All Rights Reserved.


A Cluster is made up of a collection of Nodes

• Nodes are hosts that provide compute, memory, storage and


networking resources
• Control Plane
– controls and manages the Kubernetes cluster
– comprises of one or more Master Nodes

• Data Plane
– run the actual applications we deploy
– comprises of one or more Worker Nodes

Copyright © Dell Inc. All Rights Reserved.


We use kubectl to interact with the Kubernetes
Cluster

Control Plane

kubectl

Node
Node
Node
Worker Data Plane
Node

Copyright © Dell Inc. All Rights Reserved.


kubectl

• Kubernetes provides a declarative REST API to interact


with its set of abstractions, i.e. Kubernetes resources
• kubectl is a command line interface (CLI) for interacting
with the Kubernetes API
• Throughout this course, we will use it to create objects and
interact with the Kubernetes API kubectl

$ kubectl command

Copyright © Dell Inc. All Rights Reserved.


Widely Used Kubernetes Platforms and
Distributions
• Amazon Elastic Kubernetes Service • Red Hat OpenShift
(EKS)
• Rancher
• Azure Kubernetes Service (AKS)
• VMware Tanzu Kubernetes Grid
• Google Kubernetes Engine (GKE) (TKG)

Copyright © Dell Inc. All Rights Reserved.


Running Kubernetes locally with k3d
• For our learning purpose, we recommend running k3d
• k3d is a lightweight wrapper to run k3s in Docker
• k3s is Rancher Lab’s minimal Kubernetes distribution built for IOT and
Edge computing
• Provides an easy way to create single and multi-node clusters in
Docker

Requirements (as of this recording)


• Docker - minimal version v20.10.5 (runc >= v1.0.0-rc93)
• kubectl - to interact with the Kubernetes cluster

Copyright © Dell Inc. All Rights Reserved.


Install Docker
• As a prerequisite, we need to install Docker
• Docker Engine is available for the Windows, macOS and Linux
operating system platforms
• Refer to the official Docker documentation for the latest installation
instructions tailored to your operating system platform
• The installation process is relatively straightforward and simple to
follow

Copyright © Dell Inc. All Rights Reserved.


Docker Desktop

https://www.docker.com/products/docker-desktop

Copyright © Dell Inc. All Rights Reserved.


Install k3d
On macOS

brew install k3d

On Windows

choco install -y k3d

On Linux

wget -q -O - https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash

Refer to documentation guide at https://k3d.io

Copyright © Dell Inc. All Rights Reserved.


Checking k3d version

$ k3d version

k3d version v5.6.0


k3s version v1.27.4-k3s1 (default)

Copyright © Dell Inc. All Rights Reserved.


Install kubectl

Refer to documentation guide at https://kubernetes.io/docs/tasks/tools/#kubectl

Copyright © Dell Inc. All Rights Reserved.


Checking kubectl version

$ kubectl version --client

Client Version: v1.29.1


Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3

Copyright © Dell Inc. All Rights Reserved.


If you prefer other alternatives

https://minikube.sigs.k8s.io

minikube quickly sets up a local https://kind.sigs.k8s.io


Kubernetes cluster on macOS,
Linux, and Windows.
kind is a tool for running local
Kubernetes clusters using Docker
container “nodes”.
kind was primarily designed for
testing Kubernetes itself, but may be
used for local development or CI.

Copyright © Dell Inc. All Rights Reserved.


If you are too lazy to install on your own machine

The environment expires after an hour, requiring you to start a


new one. But this is good enough for our learning purposes.

https://killercoda.com/playgrounds/scenario/kubernetes

Copyright © Dell Inc. All Rights Reserved.


Create a single-node cluster with k3d

$ k3d cluster create

INFO[0000] Prep: Network


INFO[0000] Created network 'k3d-k3s-default'
INFO[0000] Created image volume k3d-k3s-default-images
INFO[0000] Starting new tools node...
INFO[0000] Starting Node 'k3d-k3s-default-tools'
INFO[0001] Creating node 'k3d-k3s-default-server-0'
INFO[0001] Creating LoadBalancer 'k3d-k3s-default-serverlb'
INFO[0001] Using the k3d-tools node to gather environment information
INFO[0001] HostIP: using network gateway 172.27.0.1 address
INFO[0001] Starting cluster 'k3s-default'
INFO[0001] Starting servers...
INFO[0001] Starting Node 'k3d-k3s-default-server-0'
INFO[0007] All agents already running.
INFO[0007] Starting helpers...
INFO[0007] Starting Node 'k3d-k3s-default-serverlb'
INFO[0014] Injecting records for hostAliases (incl. host.k3d.internal) and for 2 network members into CoreDNS configmap...
INFO[0016] Cluster 'k3s-default' created successfully!
WARN[0016] failed to open output file '/home/vagrant/.kube/config' or it's not a kubeconfig: open /home/vagrant/.kube/config: permission denied
INFO[0016] You can now use it like this:
kubectl cluster-info

Copyright © Dell Inc. All Rights Reserved.


Checking client and server version

$ kubectl version

Client Version: v1.29.1


Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.27.4+k3s1

Copyright © Dell Inc. All Rights Reserved.


Checking Cluster Status

$ kubectl cluster-info

Kubernetes control plane is running at https://0.0.0.0:34761


CoreDNS is running at
https://0.0.0.0:34761/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at
https://0.0.0.0:34761/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

This shows that our cluster is up and running!

Copyright © Dell Inc. All Rights Reserved.


Recap

• Kubernetes provides a standardised way to build and deploy reliable, scalable


distributed applications both in the cloud and on-premises
• Microservices can be packaged as containers and managed by Kubernetes at
scale
• A Kubernetes cluster is made up of a control plane and data plane with master
node(s) and worker node(s) respectively
• We will be using k3d as our Kubernetes environment for our learning purposes

Copyright © Dell Inc. All Rights Reserved.


Understanding the Kubernetes Architecture

Copyright © Dell Inc. All Rights Reserved.


Learning Objectives

At the end of this topic, you will be able to:


• understand the key components that make up the Kubernetes architecture for
the control plane and data plane in the cluster
• understand the role of a pod and its lifecycle, and how it is used to manage
containers in Kubernetes
• create a multi-node k3d cluster and deploy a pod to the running cluster

Copyright © Dell Inc. All Rights Reserved.


A Kubernetes Cluster

Control Plane

kubectl

Node
Node
Node
Worker Data Plane
Node

Copyright © Dell Inc. All Rights Reserved.


Worker Node

• Can be a physical machine (Bare Metal), or virtual machine (VM)


on-premise or in the cloud
• The worker node runs our containerised application workloads
• Worker nodes can be added or removed from the cluster to
accommodate changes in demand
• This can improve resource utilization, and enhance the overall
performance and availability of applications

VMVM Virtual Machines

Worker Node
Bare Metal

Copyright © Dell Inc. All Rights Reserved.


What’s inside a Worker Node?

Control Plane

kubectl

Node
Node
Container
Node Runtime Data Plane

Kubelet Kube-Proxy
Worker Node

Copyright © Dell Inc. All Rights Reserved.


Kubelet

• An agent running on every worker node and is responsible for


communicating with the control plane
• Manages the lifecycle of the containers, which run on worker
nodes via the container runtime

Container Runtime

Kubelet Kube-Proxy

Worker Node

Copyright © Dell Inc. All Rights Reserved.


Kube-Proxy

• A network proxy that runs on each worker node to maintain


network rules and enables network communication between
the pods
• It is responsible for implementing service discovery, load
balancing, and network policies

Container Runtime

Kubelet Kube-Proxy

Worker Node

Copyright © Dell Inc. All Rights Reserved.


Container Runtime

• Responsible for managing the containers which includes pulling


container images, creating and running containers, and managing
their lifecycle as part of the pod execution
• The kubelet communicates with the container runtime through a
well-defined interface known as the Container Runtime Interface
(CRI)
• Kubernetes can be configured to choose from a range of different
container runtime engines that complies with the CRI

Container Runtime

Kubelet Kube-Proxy

Worker Node

Copyright © Dell Inc. All Rights Reserved.


Let’s deploy an nginx web app on our cluster

$ kubectl run web --image=nginx

pod/web created

The name that we give to our app

The name of image to download from the


Docker Hub registry

Copyright © Dell Inc. All Rights Reserved.


Under the hood

Control Plane Docker Hub

$ kubectl run web --image=nginx Notifies Kubelet to


run nginx app

kubectl
Pulls the nginx image from Docker
Hub and runs the container web
on the worker node

web

Container Runtime
Instructs Container Runtime
to run the nginx image
Kubelet Kube-Proxy

Worker Node

Copyright © Dell Inc. All Rights Reserved.


How do we view our container?

$ kubectl get containers

error: the server doesn't have a resource type "containers"

Copyright © Dell Inc. All Rights Reserved.


Pod
• Kubernetes does not manage containers directly
• A Pod is the smallest deployable unit in Kubernetes
• A container resides inside of a Pod
• The Pod manages the containers
• A Pod has a unique IP address within the Kubernetes cluster

Fun Fact - Docker’s logo is a whale. A Pod is also a group of whales.


Copyright © Dell Inc. All Rights Reserved.
List all pods

$ kubectl get pods

NAME READY STATUS RESTARTS AGE


web 1/1 Running 0 36m

Our pod web is running successfully

Copyright © Dell Inc. All Rights Reserved.


Pod Lifecycle
• Upon creation, the pod goes through various life cycle phases
• The STATUS output of the pod reflects its current phase

ContainerCreating Unknown

Running

Succeeded Failed

Copyright © Dell Inc. All Rights Reserved.


List a specific pod by name

$ kubectl get pod web

NAME READY STATUS RESTARTS AGE


web 1/1 Running 0 36m

Name of the pod

Copyright © Dell Inc. All Rights Reserved.


Display additional info on the pod

$ kubectl get pod web -o wide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
web 1/1 Running 0 25h 10.42.0.10 k3d-mycluster-server-0 <none> <none>

The node on which our pod is


Display additional output
scheduled to run on
fields and details

Each pod in the cluster has


its own unique IP

Copyright © Dell Inc. All Rights Reserved.


List all nodes in our cluster

$ kubectl get nodes

NAME STATUS ROLES AGE VERSION


k3d-mycluster-server-0 Ready control-plane,master 11h v1.27.4+k3s1

We only have one node in our cluster. This node has the role of
control-plane and master. At the same time, it is also functioning as a
worker node where the pod web is deployed to

Copyright © Dell Inc. All Rights Reserved.


Both control plane and worker node can be on a single
node

Control Plane

This is fine for our learning purposes, but it is evidently not


appropriate for production environments
kubectl

Worker
Node
Single Node

Copyright © Dell Inc. All Rights Reserved.


Delete our single-node cluster in k3d

$ k3d cluster delete

INFO[0000] Deleting cluster 'k3s-default'


INFO[0001] Deleting cluster network 'k3d-k3s-default'
INFO[0001] Deleting 1 attached volumes...
INFO[0001] Removing cluster details from default kubeconfig...
INFO[0001] Removing standalone kubeconfig file (if there is one)...
INFO[0001] Successfully deleted cluster k3s-default!

Copyright © Dell Inc. All Rights Reserved.


Create a multi-node cluster with k3d

$ k3d cluster create mycluster --agents 3

INFO[0000] Prep: Network


INFO[0000] Created network 'k3d-mycluster-default'
INFO[0000] Created image volume k3d-mycluster-default-images We are creating a cluster with 3
INFO[0000] Starting new tools node...
INFO[0000] Starting Node 'k3d-mycluster-default-tools' worker nodes here
INFO[0001] Creating node 'k3d-mycluster-default-server-0'
INFO[0001] Creating node 'k3d-mycluster-default-agent-0'
INFO[0001] Creating node 'k3d-mycluster-default-agent-1'
INFO[0001] Creating node 'k3d-mycluster-default-agent-2'
INFO[0001] Creating LoadBalancer 'k3d-mycluster-default-serverlb'
INFO[0001] Using the k3d-tools node to gather environment information
INFO[0001] HostIP: using network gateway 172.29.0.1 address
INFO[0001] Starting cluster 'mycluster'
INFO[0001] Starting servers...
INFO[0001] Starting Node 'k3d-mycluster-default-server-0'
INFO[0006] Starting agents...
INFO[0006] Starting Node 'k3d-mycluster-default-agent-1'
INFO[0006] Starting Node 'k3d-mycluster-default-agent-0'
INFO[0006] Starting Node 'k3d-mycluster-default-agent-2'
INFO[0013] Starting helpers...
INFO[0013] Starting Node 'k3d-mycluster-serverlb'
INFO[0020] Injecting records for hostAliases (incl. host.k3d.internal) and for 5 network members into CoreDNS configmap...
INFO[0022] Cluster 'mycluster' created successfully!
INFO[0022] You can now use it like this:
kubectl cluster-info

Copyright © Dell Inc. All Rights Reserved.


Verify the cluster

$ kubectl cluster-info

Kubernetes control plane is running at https://0.0.0.0:37217


CoreDNS is running at https://0.0.0.0:37217/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at
https://0.0.0.0:37217/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Copyright © Dell Inc. All Rights Reserved.


List all nodes in our cluster

$ kubectl get nodes

NAME STATUS ROLES AGE VERSION


k3d-mycluster-agent-1 Ready <none> 7m25s v1.27.4+k3s1
k3d-mycluster-agent-0 Ready <none> 7m24s v1.27.4+k3s1
k3d-mycluster-agent-2 Ready <none> 7m23s v1.27.4+k3s1
k3d-mycluster-server-0 Ready control-plane,master 7m29s v1.27.4+k3s1

1 master node in the control plane

Our 3 worker nodes are ready!

Copyright © Dell Inc. All Rights Reserved.


Taint the master node to prevent pods from being scheduled

$ kubectl taint node k3d-mycluster-server-0 node-role.kubernetes.io/master=:NoSchedule

node/k3d-mycluster-server-0 tainted

This taint has the effect of marking the master node


with a "NoSchedule" effect, meaning that no Pods
without the corresponding toleration will be scheduled
on this node

Copyright © Dell Inc. All Rights Reserved.


Master Node

• Can be a physical machine or virtual machine (VM) on-prem or


in the cloud
• The master node manages the control plane components
• It orchestrates and coordinates the actions of the worker nodes,
ensuring that the desired state of the cluster is maintained and
responding to changes in workload or configuration

Control Plane

Copyright © Dell Inc. All Rights Reserved.


What’s inside the Control Plane?

Controller Manager

Scheduler
API-Server

etcd Assuming a single Master


Node in the Control Plane for
simplification purpose.
Control Plane
kubectl

Node
Node
Node Data Plane
Worker
Node

Copyright © Dell Inc. All Rights Reserved.


API Server

• Entrypoint to the Kubernetes cluster


• Provides a stateless front-end service that securely exposes the
Kubernetes API
• Enforces access controls, validates and persists cluster state
• Communicates with etcd to store and retrieve cluster data

Copyright © Dell Inc. All Rights Reserved.


Scheduler

• Watches for new pods with no assigned worker nodes and assigns
them to the target node for execution
• Its primary role is to ensure that pods are distributed across the
available nodes according to specified requirements and constraints

Copyright © Dell Inc. All Rights Reserved.


Controller Manager

• Tracks the state of the cluster and implements changes when


needed
• It is responsible for managing various resources, ensuring that the
declared configuration matches the actual state of the system

Copyright © Dell Inc. All Rights Reserved.


etcd

• Reliable, distributed key-value store used to store the


cluster configuration and state
• Acts as the storage for the API Server
• Responsible for all state storage
• Only the API Server can talk directly to etcd

Copyright © Dell Inc. All Rights Reserved.


Kubernetes Cluster Components

Controller Manager

Scheduler
API-Server

etcd

Control Plane
kubectl

Node
Node
Container
Node Runtime

Kubelet Kube-Proxy
Worker Node
Copyright © Dell Inc. All Rights Reserved.
Let’s deploy a pod on our new cluster again

$ kubectl run web --image=nginx

pod/web created

Copyright © Dell Inc. All Rights Reserved.


Display additional info on our pod

$ kubectl get pod web -o wide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
web 1/1 Running 0 83s 10.42.1.7 k3d-mycluster-agent-1 <none> <none>

Our pod has been assigned to the


k3d-mycluster-agent-1 worker node. Your
results might differ as it is up to the scheduler
to determine the worker node to assign to.

Copyright © Dell Inc. All Rights Reserved.


The complete workflow under the hood
Communicates with
scheduler to schedule pod
to run on the worker node Controller Manager
Sends request to
API-Server
Docker Hub
Scheduler
API-Server

etcd

Control Plane

kubectl Notifies Kubelet to run


nginx pod
Pulls the nginx image from Docker Hub
$ kubectl run web --image=nginx and runs the container on the worker
node

Container

Kubelet abstracts container


to run as a pod
pod

Instructs Container Runtime to


run the nginx image Container Runtime

Kubelet Kube-Proxy

Worker Node

Copyright © Dell Inc. All Rights Reserved.


Worker Node

• The Kubelet communicates with the


Container Container
container runtime to run the containers
Pod-2
Pod-2
associated with the pod
Pod-1 Pod-2
Pod-2
• The containers are managed by their
Container Runtime respective pods

Kubelet Kube-Proxy • We can have many pods deployed on a


worker node
Worker Node

Copyright © Dell Inc. All Rights Reserved.


Setting up an alias for kubectl

$ alias k=kubectl

We will be typing `kubectl` a lot. Setting up this short alias `k` can help us to save time on typing.

Throughout the slides, we will still be using the full name of kubectl for clarity and consistency.

Copyright © Dell Inc. All Rights Reserved.


kubectl command structure

$ kubectl run pod web --image=

Command Resource Type


Name Additional Flags
All kubectl commands
start with this

Copyright © Dell Inc. All Rights Reserved.


Recap

• A Kubernetes cluster consists of a control plane and worker nodes on which


your containers are run
• The master node and worker nodes has various components that coordinate
under the hood to run our containerised applications
• We interact with the cluster via the Kubernetes API using the kubectl command
line interface

Copyright © Dell Inc. All Rights Reserved.


Deploying Applications in Kubernetes

Copyright © Dell Inc. All Rights Reserved.


Learning Objectives

At the end of this topic, you will be able to:


• create deployments effectively on a Kubernetes cluster
• understand the differences between Deployment and ReplicaSets
• manually scale Deployments to ensure high availability

Copyright © Dell Inc. All Rights Reserved.


What happens if the pod is down?

web

Kubernetes

This is transparent to us

Copyright © Dell Inc. All Rights Reserved.


Let’s find out by deleting our pod

$ kubectl delete pod web

pod "web" deleted

Copyright © Dell Inc. All Rights Reserved.


List our pod

$ kubectl get pod web

Error from server (NotFound): pods "web" not found

Copyright © Dell Inc. All Rights Reserved.


What happens if the pod is down?

web

Kubernetes

✓ High Availability

✓ Reliability

✓ Resiliency What happened to these features that we mentioned earlier?


✓ Scalability

✓ Self-Healing

Copyright © Dell Inc. All Rights Reserved.


Let’s create a deployment

$ kubectl create deployment web-deploy --image=nginx

deployment.apps/web-deploy created

Copyright © Dell Inc. All Rights Reserved.


List all pods

$ kubectl get pods

NAME READY STATUS RESTARTS AGE


web-deploy-67cc48865-vfwbc 1/1 Running 0 15s

Notice there are 2 random suffixes separated by a ‘-’ after the


name of the deployment object as part of the pod name. The
suffix might differ from what you see here

Copyright © Dell Inc. All Rights Reserved.


Let’s try to delete this pod

$ kubectl delete pod web-deploy-67cc48865-vfwbc

pod "web-deploy-67cc48865-vfwbc" deleted

Copyright © Dell Inc. All Rights Reserved.


List all pods

$ kubectl get pods

NAME READY STATUS RESTARTS AGE


web-deploy-67cc48865-2xhfz 0/1 ContainerCreating 0 4s

A replacement pod is created when the previous


pod has been deleted

Copyright © Dell Inc. All Rights Reserved.


List all deployments

$ kubectl get deployments

NAME READY UP-TO-DATE AVAILABLE AGE


web-deploy 1/1 1 1 16m

1 pod is managed by the web-deploy deployment object.


The deployment ensures that there will always be 1 pod
running in the cluster

Copyright © Dell Inc. All Rights Reserved.


List a specific deployment by name

$ kubectl get deployment web-deploy

NAME READY UP-TO-DATE AVAILABLE AGE


web-deploy 1/1 1 1 21m

Name of the deployment

Copyright © Dell Inc. All Rights Reserved.


Self-healing

Controller Manager
Sends request to
API-Server
Scheduler
API-Server Monitors deployment
to have 1 pod running
etcd

Control Plane

kubectl Notifies Kubelet to


create replacement pod

$ kubectl delete pod XXXX

Container
Deletes pod

Notifies Kubelet to Pod


delete pod
Deployment

Container Runtime

Kubelet Kube-Proxy

Worker Node
Copyright © Dell Inc. All Rights Reserved.
What is a Deployment?

• A Deployment manages one or more identical pods


• In real world use cases, we almost never create pods directly
• The pods managed by the Deployment are monitored and automatically
restarted/recreated if they fail

Container Container Container Container

Pod Pod Pod Pod


Deployment

Copyright © Dell Inc. All Rights Reserved.


What happens if there is an increase in user traffic?

web-deploy

Kubernetes

This is transparent to us

Copyright © Dell Inc. All Rights Reserved.


Scale our deployment to increase more pods

$ kubectl scale deployment web-deploy --replicas=3

deployment.apps/web-deploy scaled

Increase up to 3 identical pods

Copyright © Dell Inc. All Rights Reserved.


List our deployment

$ kubectl get deployment web-deploy

NAME READY UP-TO-DATE AVAILABLE AGE


web-deploy 3/3 3 3 33m

The deployment is now


managing 3 pods

Copyright © Dell Inc. All Rights Reserved.


List all pods

$ kubectl get pods -o wide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
web-deploy-67cc48865-2xhfz 1/1 Running 0 73s 10.42.0.8 k3d-mycluster-agent-0 <none> <none>
web-deploy-67cc48865-m6zm7 1/1 Running 0 13s 10.42.0.9 k3d-mycluster-agent-1 <none> <none>
web-deploy-67cc48865-pv9p9 1/1 Running 0 13s 10.42.1.6 k3d-mycluster-agent-2 <none> <none>

2 more identical pods are created to


balance the load The 3 pods are assigned to various
worker nodes in the cluster

Copyright © Dell Inc. All Rights Reserved.


Manual Scaling
Sends request to Api-Server to
scale up to 3 replicas

Control Plane

Scheduler notifies the


kubelets in the 2 other
kubectl worker nodes to create
the pods

$ kubectl scale deployment … –replicas=3

Container Container Container

Pod Pod Pod


Deployment

The Deployment is designed to manage and ensureWorker Node-0 Worker Node-1 Worker Node-2
the desired number of replicas of our pods across
multiple worker nodes in the cluster

Copyright © Dell Inc. All Rights Reserved.


Let’s try to stop one of the worker nodes

$ sudo k3d node stop k3d-mycluster-agent-2

No expected output here

Copyright © Dell Inc. All Rights Reserved.


List all nodes in our cluster

$ kubectl get nodes

NAME STATUS ROLES AGE VERSION


k3d-mycluster-agent-1 Ready <none> 7m25s v1.27.4+k3s1
k3d-mycluster-agent-0 Ready <none> 7m24s v1.27.4+k3s1
k3d-mycluster-agent-2 NotReady <none> 7m23s v1.27.4+k3s1
k3d-mycluster-server-0 Ready control-plane,master 7m29s v1.27.4+k3s1

Worker node 2 is not ready

Copyright © Dell Inc. All Rights Reserved.


Delete the pod that is running on node 2

$ kubectl delete pod web-deploy-67cc48865-pv9p9 --force --grace-period=0

Warning: Immediate deletion does not wait for confirmation that the running resource has been
terminated. The resource may continue to run on the cluster indefinitely.
pod "web-deploy-67cc48865-pv9p9" force deleted

To save time, the pod will be immediately


terminated without waiting for any graceful
termination period

Copyright © Dell Inc. All Rights Reserved.


List all pods

$ kubectl get pods -o wide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
web-deploy-67cc48865-2xhfz 1/1 Running 0 73s 10.42.0.8 k3d-mycluster-agent-0 <none> <none>
web-deploy-67cc48865-m6zm7 1/1 Running 0 13s 10.42.0.9 k3d-mycluster-agent-1 <none> <none>
web-deploy-67cc48865-sp826 1/1 Running 0 13s 10.42.1.7 k3d-mycluster-agent-1 <none> <none>

A replacement pod has been created


The replacement pod has been assigned to a
running worker node

Copyright © Dell Inc. All Rights Reserved.


Automatic rescheduling of the pod to other
available nodes
Sends request to API-Server to delete
pod running on worker node-2

Control Plane

Controller monitoring the deployment


realises the mismatch between desired Scheduler notifies the
and actual state. Scheduler notifies the kubelet in worker node-2
kubelet in worker node-1 to create the to delete the pod
kubectl replacement pod

$ kubectl delete pod …

Container Container Container


Container

Pod pod Pod


Pod
Deployment

Worker Node-0 Worker Node-1 Worker Node-2

Worker node-2 is set to


unavailable

Copyright © Dell Inc. All Rights Reserved.


Actually …

• There is another Kubernetes resource called the ReplicaSet that


manages the identical pods of the application (replicas)
• It was the ReplicaSet that performs the self-healing, scaling and
automatic failover of a pod from a failed worker node to another
healthy worker node
• The Deployment abstracts the functionality of the ReplicaSet and
manages it internally
• We do not need to explicitly create and manage the ReplicaSet
ourselves

Copyright © Dell Inc. All Rights Reserved.


List our replicasets

$ kubectl get replicasets

NAME DESIRED CURRENT READY AGE


web-deploy-67cc48865 3 3 3 158m

Notice the random suffix is


appended to the name of the
The replicaset looks similar to the
deployment object to make up
deployment
the name of the replicaset

Copyright © Dell Inc. All Rights Reserved.


List all pods

$ kubectl get pods

NAME READY STATUS RESTARTS AGE


web-deploy-67cc48865-2xhfz 1/1 Running 0 162m
web-deploy-67cc48865-m6zm7 1/1 Running 0 162m
web-deploy-67cc48865-sp826 1/1 Running 0 160m

And then another random suffix is appended


to the name of the replicaset object to make
up the name of the pods

Copyright © Dell Inc. All Rights Reserved.


What is a ReplicaSet?

• A ReplicaSet ensures a specified number of replicas of a Pod


are running at all times
• It is part of the Kubernetes workload controllers and is used to
achieve high availability and fault tolerance for applications
• ReplicaSets are designed for stateless microservices

Container Container Container Container

Pod Pod Pod Pod


ReplicaSet

Copyright © Dell Inc. All Rights Reserved.


Reconciliation Loops

• The ReplicaSet uses a reconciliation loop to manage the replicas


• It ensures that the current state is always in sync with the desired
state, i.e. actual number of pods matches the desired number
• The reconciliation loop is constantly running and observing the actual
state
• It takes action when the current state is not in sync with the desired
state
observe

Desired State Actual State

adjust

Copyright © Dell Inc. All Rights Reserved.


The relationship between a Deployment,
ReplicaSet and its Pods

Deployment

Create and manage 3 replicas

ReplicaSet

Maintain set of 3 replicas

Pod Pod Pod

Copyright © Dell Inc. All Rights Reserved.


List all resources

$ kubectl get all

NAME READY STATUS RESTARTS AGE


pod/web-deploy-67cc48865-2xhfz 1/1 Running 0 176m
pod/web-deploy-67cc48865-m6zm7 1/1 Running 0 176m Our 3 identical pods
pod/web-deploy-67cc48865-sp826 1/1 Running 0 174m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE


service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 3h7m

NAME READY UP-TO-DATE AVAILABLE AGE


Our deployment
deployment.apps/web-deploy 3/3 3 3 176m

NAME DESIRED CURRENT READY AGE


replicaset.apps/web-deploy-67cc48865 3 3 3 176m Our replicaset

This is the Kubernetes service running in the cluster. We will


come back to services in the later topics

Copyright © Dell Inc. All Rights Reserved.


Delete our deployment

$ kubectl delete deployment web-deploy

deployment.apps "web-deploy" deleted

Copyright © Dell Inc. All Rights Reserved.


List our replicasets

$ kubectl get replicasets

No resources found in default namespace

The replicaset is deleted along


with the deployment

Copyright © Dell Inc. All Rights Reserved.


List our pods

$ kubectl get pods

No resources found in default namespace

The pods are deleted along with the deployment

Copyright © Dell Inc. All Rights Reserved.


Let’s get the worker node running again

$ sudo k3d node start k3d-mycluster-agent-2

INFO[0000] Starting Node 'k3d-mycluster-agent-2'

Copyright © Dell Inc. All Rights Reserved.


Recap

• Kubernetes provides a standardised way to run distributed applications both in


the cloud and on-premises
• The Deployment manages one or more identical pods
• ReplicaSets ensure that a specified number of pod replicas are running at all
times
• If a pod fails or is deleted, the ReplicaSet automatically creates a new pod to
replace it, maintaining the desired number of replicas

Copyright © Dell Inc. All Rights Reserved.


Imperative and Declarative Workflows

Copyright © Dell Inc. All Rights Reserved.


Learning Objectives

At the end of this topic, you will be able to:


• understand the fundamental differences between imperative and declarative
approaches in Kubernetes
• learn to use imperative commands to manage individual Kubernetes resources
• define Kubernetes resources using declarative manifests to specify the desired
state of the cluster

Copyright © Dell Inc. All Rights Reserved.


Imperative Workflow

• So far, we have been using the imperative approach to create and


manage our Kubernetes resources, such as pods and deployments
• Imperative is nothing more than running a series of commands in a
specific sequence
• Examples of the imperative commands we have run so far:

$ kubectl run web --image=nginx

$ kubectl create deployment web-deploy --image=nginx

$ kubectl scale deployment web-deploy --replicas=3

Copyright © Dell Inc. All Rights Reserved.


Declarative Configuration

• The Kubernetes API is based on a declarative model


• Everything in Kubernetes is a declarative configuration object that
represents the desired state of the system
• The desired state of these resources is defined using either YAML
or JSON manifests
• These manifests can be checked into version control for tracking
purposes for reproducibility
• It is the primary responsibility of Kubernetes to ensure the actual
state of the system matches the desired state defined by the
manifests

Copyright © Dell Inc. All Rights Reserved.


YAML

• Originally stood for Yet Another Markup


Language
• Subsequently renamed with the recursive
acronym YAML Ain’t Markup Language
• YAML is designed to be human-readable
and machine-consumable
• Widely used in the configuration of
software applications, and infrastructure
as code (IaC)

YAML

Copyright © Dell Inc. All Rights Reserved.


Format of a YAML manifest
# YAML example

• Key-value pairs - key and value are person:


separated by a colon (:) name: John Doe

• Indentation - uses spaces or tabs for age: 30


indentation to represent levels of address:
hierarchy and structure city: Anytown

• Lists - represented using hyphens (-) country: USA


interests:
• Comments - anything after the # on a line
is considered a comment and ignored - reading

during parsing - hiking


- coding
YAML

Copyright © Dell Inc. All Rights Reserved.


General structure of a Kubernetes YAML manifest

apiVersion
API version of the Kubernetes resource
kind
metadata
Type of the Kubernetes resource
E.g. pod, replicaset, deployment etc

spec
Describes the high level information about
the Kubernetes resource

Defines the desired state of the Kubernetes


resource

YAML

Copyright © Dell Inc. All Rights Reserved.


Basic YAML manifest of our Pod running nginx

apiVersion: v1
Using core Kubernetes API version 1
kind: Pod

metadata: This YAML describes a Pod


name: web

spec: Specifies the name of the Pod


containers:
- name: web
image: nginx
Specifies the name of the container

Specifies the container image to deploy and


run

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our pod

$ vim nginx-pod.yaml

apiVersion: v1
kind: Pod
metadata:
name: web
Press ‘i’ to type
spec:
containers:
- name: web
image: nginx

Enter Esc and :wq! to save configuration and exit edit


mode

Copyright © Dell Inc. All Rights Reserved.


Create the pod with the manifest

$ kubectl apply -f nginx-pod.yaml

pod/web created

The `kubectl apply -f` command is used in Kubernetes to create or


update resources based on the specifications provided in the YAML
manifest

Copyright © Dell Inc. All Rights Reserved.


List all pods

$ kubectl get pods

NAME READY STATUS RESTARTS AGE


web 1/1 Running 0 5s

Copyright © Dell Inc. All Rights Reserved.


Declarative way to deploy

Sends manifest to API-Server

Control Plane

YAML

Scheduler places the pod on the


Define desired state of the app in
kubectl YAML configuration
target worker node

$ kubectl apply -f X.yaml

Container

Pod

Worker Node

Copyright © Dell Inc. All Rights Reserved.


Delete the pod with the manifest

$ kubectl delete -f nginx-pod.yaml

pod “web” deleted

Copyright © Dell Inc. All Rights Reserved.


YAML manifest of a Deployment running nginx
apiVersion: apps/v1
The API version for Deployment is apps/v1
kind: Deployment

metadata:
name: web-deploy This YAML describes a Deployment
spec:
replicas: 3

selector:
3 parts in the .spec section
matchLabels:
app: web
● replicas - the desired number of pods
template:
metadata:
● label selector - pods with the specified label
labels: will be managed
app: web
spec: ● pod template - the spec for the pod
containers:
- name: web
image: nginx

Copyright © Dell Inc. All Rights Reserved.


YAML manifest of a Deployment running nginx
apiVersion: apps/v1

kind: Deployment

metadata:
name: web-deploy

spec:
replicas: 3

selector:
matchLabels:
app: web The pod labels must match the label selector of the
template: deployment, in our case, it will be app: web
metadata:
labels:
app: web
spec:
containers:
- name: web
image: nginx

Copyright © Dell Inc. All Rights Reserved.


YAML manifest of a Deployment running nginx
apiVersion: apps/v1

kind: Deployment

metadata:
name: web-deploy

spec:
replicas: 3
apiVersion: v1
selector:
kind: Pod
matchLabels:
app: web
metadata:
name: web
template:
metadata:
spec:
labels:
containers:
app: web
- name: web
spec:
image: nginx
containers:
- name: web
image: nginx

Copyright © Dell Inc. All Rights Reserved.


Resources and objects

• In Kubernetes, the terms "resource" and "object" are often


used interchangeably although there is a subtle difference
between the two
• A Kubernetes resource refers to the entity or concept that the
API represents and are defined by their corresponding API
types such as Nodes, Pods, Deployments etc
• A Kubernetes object is an instance of a particular resource type
defined in the Kubernetes API. Each object has a unique
identity within the cluster

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our deployment

$ vim nginx-deployment.yaml

apiVersion: apps/v1
template:
kind: Deployment metadata:
labels:
metadata: app: web
name: web-deploy spec:
containers:
spec: - name: web
replicas: 3 image: nginx

selector:
matchLabels:
app: web

Copyright © Dell Inc. All Rights Reserved.


Create the deployment with the manifest

$ kubectl apply -f nginx-deployment.yaml

deployment.apps/web-deploy created

Copyright © Dell Inc. All Rights Reserved.


List our deployment

$ kubectl get deployment web-deploy

NAME READY UP-TO-DATE AVAILABLE AGE


web-deploy 0/3 3 0 6s

The deployment is managing 3 pods since we


have specified 3 replicas in the manifest

Copyright © Dell Inc. All Rights Reserved.


List the pods and all their labels

$ kubectl get pods --show-labels

NAME READY STATUS RESTARTS AGE LABELS


web-deploy-7869bd98c5-8jw7f 1/1 Running 0 36s app=web,pod-template-hash=7869bd98c5
web-deploy-7869bd98c5-s6xwk 1/1 Running 0 36s app=web,pod-template-hash=7869bd98c5
web-deploy-7869bd98c5-f7t2k 1/1 Running 0 36s app=web,pod-template-hash=7869bd98c5

All 3 pods have the label


app=web

This flag shows the labels


assigned to the resource

Copyright © Dell Inc. All Rights Reserved.


List the pods and their specific label

$ kubectl get pods -l app

NAME READY STATUS RESTARTS AGE


web-deploy-7869bd98c5-8jw7f 1/1 Running 0 34m
web-deploy-7869bd98c5-s6xwk 1/1 Running 0 34m
web-deploy-7869bd98c5-f7t2k 1/1 Running 0 34m

Explicitly specify the label

Copyright © Dell Inc. All Rights Reserved.


Let’s change the label of one of the pods

$ kubectl label pod web-deploy-7869bd98c5-f7t2k app=db

error: 'app' already has a value (web), and --overwrite is false

We need to add the Change label to app=db


--overwrite flag

Copyright © Dell Inc. All Rights Reserved.


Let’s change the label of one of the pods

$ kubectl label pod web-deploy-7869bd98c5-f7t2k app=db --overwrite

pod/web-deploy-7869bd98c5-f7t2k labeled

Copyright © Dell Inc. All Rights Reserved.


List the pods and their specific label

$ kubectl get pods -L app

NAME READY STATUS RESTARTS AGE APP


web-deploy-7869bd98c5-8jw7f 1/1 Running 0 44m web
web-deploy-7869bd98c5-s6xwk 1/1 Running 0 44m web
web-deploy-7869bd98c5-f7t2k 1/1 Running 0 44m db
web-deploy-7869bd98c5-b97d6 1/1 Running 0 111s web

A new pod with the label app=web is The label of this pod is changed to app=db
created by the replicaset and is no longer managed by the
deployment

Copyright © Dell Inc. All Rights Reserved.


List our replicasets

$ kubectl get replicasets -L app

NAME DESIRED CURRENT READY AGE APP


web-deploy-7869bd98c5 3 3 3 50m web

The number of replicas is 3

Copyright © Dell Inc. All Rights Reserved.


Delete the unmanaged pod

$ kubectl delete pod web-deploy-7869bd98c5-f7t2k

pod "web-deploy-7869bd98c5-f7t2k" deleted

Copyright © Dell Inc. All Rights Reserved.


Labeling our objects

• Labels are key-value pairs that can be assigned to Kubernetes


objects
• They are used to identify and group objects in the cluster
• In the previous example, they can be used in selector queries to
group objects like Pods as part of a ReplicaSet
app=
web

ReplicaSet

Maintain set of 3 replicas with


label app=web

app= app= app= app=


web web web db

Pod Pod Pod Pod

Copyright © Dell Inc. All Rights Reserved.


An object can have more than one label

$ kubectl label pod web-deploy-7869bd98c5-8jw7f tier=frontend

pod/web-deploy-7869bd98c5-8jw7f labeled

Copyright © Dell Inc. All Rights Reserved.


List the pods and all their labels

$ kubectl get pods --show-labels

NAME READY STATUS RESTARTS AGE LABELS


web-deploy-7869bd98c5-8jw7f 1/1 Running 0 36s app=web,pod-template-hash=7869bd98c5,,tier=frontend
web-deploy-7869bd98c5-s6xwk 1/1 Running 0 36s app=web,pod-template-hash=7869bd98c5
web-deploy-7869bd98c5-f7t2k 1/1 Running 0 36s app=web,pod-template-hash=7869bd98c5

Additional label tier=frontend


Since the label app=web remains
unchanged, the 3 pods are still part
of the replicaset

Copyright © Dell Inc. All Rights Reserved.


Filter the pods by labels

$ kubectl get pods --selector tier=frontend --show-labels

NAME READY STATUS RESTARTS AGE LABELS


web-deploy-7869bd98c5-8jw7f 1/1 Running 0 16h app=web,pod-template-hash=7869bd98c5,tier=frontend

The --selector flag filters and selects


the objects by label

Copyright © Dell Inc. All Rights Reserved.


Filter the pods by labels

$ kubectl get pods -l tier=frontend --show-labels

NAME READY STATUS RESTARTS AGE LABELS


web-deploy-7869bd98c5-8jw7f 1/1 Running 0 16h app=web,pod-template-hash=7869bd98c5,tier=frontend

We can also use the -l flag as the


short-form notation of --selector

Copyright © Dell Inc. All Rights Reserved.


Filter the pods by labels

$ kubectl get pods -l tier=frontend,app=web --show-labels

NAME READY STATUS RESTARTS AGE LABELS


web-deploy-7869bd98c5-8jw7f 1/1 Running 0 16h app=web,pod-template-hash=7869bd98c5,tier=frontend

We can filter by multiple labels by separating each label with a comma. Take
note this only supports boolean AND, not boolean OR.

i.e. this filter returns pods that have both tier=frontend AND app=web labels

Copyright © Dell Inc. All Rights Reserved.


We can make changes to the manifest to scale more
replicas
$ vim nginx-deployment.yaml

apiVersion: apps/v1
template:
kind: Deployment metadata:
labels:
metadata: app: web
name: web-deploy spec:
containers:
spec: Change the replicas to 5 - name: web
replicas: 3 image: nginx

selector:
matchLabels:
app: web

Copyright © Dell Inc. All Rights Reserved.


Apply the changes to our deployment

$ kubectl apply -f nginx-deployment.yaml

deployment.apps/web-deploy configured

Copyright © Dell Inc. All Rights Reserved.


We can “watch” the status of the replicas created

$ kubectl get pods --watch

NAME READY STATUS RESTARTS AGE


web-deploy-7869bd98c5-s6xwk 1/1 Running 0 40h
web-deploy-7869bd98c5-b97d6 1/1 Running 0 39h
web-deploy-7869bd98c5-8jw7f 1/1 Running 0 40h
web-deploy-7869bd98c5-7nxw5 0/1 ContainerCreating 0 7s
web-deploy-7869bd98c5-657vg 0/1 ContainerCreating 0 7s

The --watch flag enables the continuous 2 additional replicas are in the
monitoring of changes to resources. Any process of being created
changes to the status will be streamed to the
console output

Copyright © Dell Inc. All Rights Reserved.


All 5 replicas are up and running now

$ kubectl get pods --watch

NAME READY STATUS RESTARTS AGE


web-deploy-7869bd98c5-s6xwk 1/1 Running 0 40h
web-deploy-7869bd98c5-b97d6 1/1 Running 0 39h
web-deploy-7869bd98c5-8jw7f 1/1 Running 0 40h
web-deploy-7869bd98c5-7nxw5 0/1 ContainerCreating 0 7s
web-deploy-7869bd98c5-657vg 0/1 ContainerCreating 0 7s
web-deploy-7869bd98c5-657vg 1/1 Running 0 13s
web-deploy-7869bd98c5-7nxw5 1/1 Running 0 14s

Enter Ctrl-C to escape from the command prompt

Copyright © Dell Inc. All Rights Reserved.


View the current YAML representation of the deployment

$ kubectl get deployment web-deploy -o yaml

apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
kubectl.kubernetes.io/last-applied-configuration: |

{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"name":"web-deploy","namespace":"default"},"spec":{"replica
s":5,"selector":{"matchLabels":{"app":"web"}},"template":{"metadata":{"labels":{"app":"web"}},"spec":{"containers":[{"image":"nginx"
,"name":"web"}]}}}}
creationTimestamp: "2024-02-14T08:03:21Z"
generation: 2
name: web-deploy
namespace: default Specifies the output format as YAML
resourceVersion: "24646"
uid: c888df99-8167-46be-84bd-1fbe1ae44fb3
spec:
progressDeadlineSeconds: 600
replicas: 5
revisionHistoryLimit: 10
selector:
matchLabels:
app: web

Copyright © Dell Inc. All Rights Reserved.


View the current YAML representation of the deployment

$ kubectl get deployment web-deploy -o yaml

strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: web
spec:
containers:
- image: nginx
imagePullPolicy: Always
name: web
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30

Copyright © Dell Inc. All Rights Reserved.


View the current YAML representation of the deployment

$ kubectl get deployment web-deploy -o yaml

status:
availableReplicas: 5
conditions:
- lastTransitionTime: "2024-02-14T08:03:21Z"
lastUpdateTime: "2024-02-14T08:03:38Z"
message: ReplicaSet "web-deploy-7869bd98c5" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: "2024-02-16T00:38:16Z"
lastUpdateTime: "2024-02-16T00:38:16Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 2
readyReplicas: 5
replicas: 5
updatedReplicas: 5

Copyright © Dell Inc. All Rights Reserved.


Viewing the YAML representation of an object

• The command is essentially querying the Kubernetes cluster for


information about the deployment object and then printing that
information in YAML format
• Notice that the output contains much more verbose details as
compared to the YAML manifest which we have used to create
the deployment
• This output represents the “live” version of the deployment
object

Copyright © Dell Inc. All Rights Reserved.


Viewing the YAML representation of an object

• The additional status section shows the actual state of the


deployment
• The Kubernetes controller is constantly monitoring the actual state
against the desired state of the object, and making changes to
ensure they are in sync

apiVersion

kind

metadata
Desired state of the object
spec

status Actual state of the object

YAML

Copyright © Dell Inc. All Rights Reserved.


View the YAML representation of the current deployment

$ kubectl get deployment web-deploy -o yaml

status:
availableReplicas: 5
conditions:
- lastTransitionTime: "2024-02-14T08:03:21Z"
lastUpdateTime: "2024-02-14T08:03:38Z"
message: ReplicaSet "web-deploy-7869bd98c5" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: "2024-02-16T00:38:16Z"
lastUpdateTime: "2024-02-16T00:38:16Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 2
readyReplicas: 5 We see 2 additional replicas have been added
replicas: 5
updatedReplicas: 5 to reach a total of 5 replicas, which is the
desired state we have specified

Copyright © Dell Inc. All Rights Reserved.


View the YAML representation of the current deployment

$ kubectl get deployment web-deploy -o yaml > mywebdeploy.yaml

Instead of printing the output directly to the screen, we can redirect it to a file
where we can use it for other purposes such as version control, make
additional changes or share the configuration etc

Copyright © Dell Inc. All Rights Reserved.


Reduce replicas using kubectl edit

$ kubectl edit deployment web-deploy

spec:
progressDeadlineSeconds: 600
replicas: 2
Change the replicas to 2
revisionHistoryLimit: 10
selector:
matchLabels:
app: web

Enter Esc and :wq! to save configuration and exit edit mode

Copyright © Dell Inc. All Rights Reserved.


3 replicas are being terminated

$ kubectl get pods --watch

NAME READY STATUS RESTARTS AGE


web-deploy-7869bd98c5-b97d6 1/1 Running 0 40h
web-deploy-7869bd98c5-8jw7f 1/1 Running 0 41h
web-deploy-7869bd98c5-5qcr5 0/1 Terminating 0 16s
web-deploy-7869bd98c5-sndrg 0/1 Terminating 0 16s
web-deploy-7869bd98c5-5qcr5 0/1 Terminating 0 17s
web-deploy-7869bd98c5-sndrg 1/1 Terminating 0 17s
web-deploy-7869bd98c5-5qcr5 0/1 Terminating 0 17s
web-deploy-7869bd98c5-sndrg 0/1 Terminating 0 17s
web-deploy-7869bd98c5-sndrg 0/1 Terminating 0 18s

Enter Ctl-C to escape from the command prompt

Copyright © Dell Inc. All Rights Reserved.


Only 2 replicas are running now

$ kubectl get pods

NAME READY STATUS RESTARTS AGE


web-deploy-7869bd98c5-b97d6 1/1 Running 0 40h
web-deploy-7869bd98c5-8jw7f 1/1 Running 0 41h

Keep in mind that using `kubectl edit` directly in a production environment may not be the best practice, as it can introduce
manual changes and potential human error. In production, it's often recommended to use declarative configuration files
(kubectl apply -f) or infrastructure-as-code practices to manage and version control your Kubernetes configurations

Copyright © Dell Inc. All Rights Reserved.


Generate YAML of the object without running it

$ kubectl create deployment test-deploy --image=nginx --dry-run=client -o yaml > mytestdeploy.yaml

Test a run without actually running anything

Copyright © Dell Inc. All Rights Reserved.


Generated YAML from --dry-run=client

$ cat mytestdeploy.yaml

The YAML manifest is automatically generated and


apiVersion: apps/v1 formatted nicely for you
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: test-deploy
name: test-deploy
spec:
replicas: 1
selector:
matchLabels:
app: test-deploy
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: test-deploy
spec:
containers:
- image: nginx
name: nginx
resources: {}
status: {}

Copyright © Dell Inc. All Rights Reserved.


Recap

• For imperative method, you specify how to achieve a desired state by running a
series of commands in sequence
• For declarative method, you define the desired end state using YAML manifest
and let Kubernetes figure out how to achieve it
• Labels are key-value pairs used to identify and group objects in the cluster
• Using declarative method rather than imperative is considered best practice as
you can manage and version control your Kubernetes configurations

Copyright © Dell Inc. All Rights Reserved.


Services and Networking

Copyright © Dell Inc. All Rights Reserved.


Learning Objectives

At the end of this topic, you will be able to:


• Understand the concepts of Kubernetes network design
• Understand the methods to expose applications and make it accessible from
outside the cluster
• Identify the different types of services and compare their characteristics and use
cases

Copyright © Dell Inc. All Rights Reserved.


Challenges with hard-coded static IP addresses

10.0.0.2

App A App B

Host Host

IP Address IP Address
10.0.0.1 10.0.0.2

Copyright © Dell Inc. All Rights Reserved.


Challenges with hard-coded static IP addresses
app=
web

10.42.0.9
Pod

IP Address
10.42.0.9
Deployment
Client
Pod

app=
web

Pod

? IP Address
10.42.1.9

app=
web

Pod

IP Address
10.42.2.9

Copyright © Dell Inc. All Rights Reserved.


Kubernetes Networking

• Kubernetes assigns an IP address to a pod after the pod has been


scheduled to a node
• Each pod gets its own IP address and can communicate with all
other pods in the cluster directly without needing NAT (Network
Address Translation)
• Containers within the pod shares the same IP address

Copyright © Dell Inc. All Rights Reserved.


Limitations with Pods

• Pods are ephemeral in nature


• They may come and go at any time due to situations like restarting
after a crash, scaling in and scaling out etc
• New pods are assigned new IP addresses dynamically
• When multiple identical pods are providing the same application
service, which pod shall the client call?

Copyright © Dell Inc. All Rights Reserved.


Services
• Implements an abstraction layer on top of the pods
• Has a fixed virtual IP address that fronts all the pods with the
matching label
• Keeps track of which pods are running and routes network traffic
to those available pods app=
web

Pod

IP Address
Deployment
10.42.0.9
app=
web
10.42.0.1
Client Service
Pod app=
web

Pod
Fixed IP Address
10.42.0.1
IP Address
10.42.1.9

app=
web

Pod
IP Address
10.42.2.9
Copyright © Dell Inc. All Rights Reserved.
Remember our Kube-Proxy?
• Keeps the mapping between the virtual IP address of the
services and the pods up to date as the pods come and go
• Kube-Proxy queries the API-Server to learn about new
services in cluster and updates the nodes’ iptables rules
accordingly

Container Container

Pod-2
Pod-2
Pod-1 Pod-2
Pod-2

Container Runtime

Kubelet Kube-Proxy

Worker Node

Copyright © Dell Inc. All Rights Reserved.


Types of Service
• There are 3 main service types for different use cases
▪ ClusterIP
▪ NodePort
▪ LoadBalancer

• Each type is responsible for exposing the service inside and/or outside of the
cluster

Copyright © Dell Inc. All Rights Reserved.


ClusterIP Service

• Default service type


• Exposes the service using a cluster-internal IP address
• Can only be accessed from pods running inside of the cluster, but not
from outside of the cluster

app=
web

External Service
Client [ClusterIP]

Pod

app= app=
web web

Pod Pod

Cluster

Copyright © Dell Inc. All Rights Reserved.


Imperative way to create service from a deployment

$ kubectl expose deployment web-deploy --type=ClusterIP --name=web-service --port=8080

service/web-service exposed

Expose the resource as a Service


The name of the Service

The type of Service to create The port number on which the Service
will be exposed

Copyright © Dell Inc. All Rights Reserved.


View the service

$ kubectl get service web-service

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE


web-service ClusterIP 10.43.216.111 <none> 8080/TCP 14m

This is the internal IP address of the


service for the clients within the cluster to
call

Copyright © Dell Inc. All Rights Reserved.


YAML manifest of the ClusterIP service

apiVersion: v1
The API version for Service is v1
kind: Service

metadata: This YAML describes a Service


name: web-service
The Service type is ClusterIP. If we omit this, it will still
spec: default to ClusterIP
type: ClusterIP

ports: The port this Service will be available on


- port: 8080
targetPort: 80
The container port the Service will forward to
selector:
app: web
All pods with this label will be part of the Service

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our service

$ vim nginx-clusterip-service.yaml

apiVersion: v1
selector:
kind: Service app: web

metadata:
name: web-service

spec:
type: ClusterIP

ports:
- port: 8080
targetPort: 80

Copyright © Dell Inc. All Rights Reserved.


Delete the service we previously created

$ kubectl delete service web-service

service "web-service" deleted

Copyright © Dell Inc. All Rights Reserved.


Create the service with the manifest

$ kubectl apply -f nginx-clusterip-service.yaml

service/web-service created

Copyright © Dell Inc. All Rights Reserved.


View the service

$ kubectl get service web-service

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE


web-service ClusterIP 10.43.216.111 <none> 8080/TCP 14m

Take note of this Cluster IP

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the ClusterIP service

$ kubectl run testpod --image=alpine -it -- sh

If you don't see a command prompt, try pressing enter.


/#

This command will run within the pod when the pod
We are using the alpine image to run a starts, in our case is the shell command
test container for testing

Open interactive session within pod

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the ClusterIP service

$ kubectl run testpod --image=alpine -it -- sh

If you don't see a command prompt, try pressing enter.


/ # curl
sh: curl: not found

We are going to use curl to test the connectivity from the testpod to the
ClusterIP service. But Alpine does not come pre-installed with curl

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the ClusterIP service

$ kubectl run testpod --image=alpine -it -- sh

/ # apk add curl


fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz
(1/8) Installing ca-certificates (20230506-r0)
(2/8) Installing brotli-libs (1.1.0-r1) We use apk, Alpine’s package manager to install curl into
(3/8) Installing c-ares (1.24.0-r1) the container for our testing
(4/8) Installing libunistring (1.1-r2)
(5/8) Installing libidn2 (2.3.4-r4)
(6/8) Installing nghttp2-libs (1.58.0-r0)
(7/8) Installing libcurl (8.5.0-r0)
(8/8) Installing curl (8.5.0-r0)
Executing busybox-1.36.1-r15.trigger
Executing ca-certificates-20230506-r0.trigger
OK: 12 MiB in 23 packages
/#

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the ClusterIP service

$ kubectl run testpod --image=alpine -it -- sh

/ # curl -s 10.43.225.11:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; } This is the Cluster IP service’s address. Take note of the port
body { width: 35em; margin: 0 auto; 8080 that we expose in the service YAML manifest
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to


<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>


</body>
</html>

Copyright © Dell Inc. All Rights Reserved.


View the endpoints

$ kubectl get endpoints web-service

NAME ENDPOINTS AGE


web-service 10.42.1.12:80,10.42.2.13:80 5m54s

Copyright © Dell Inc. All Rights Reserved.


Recap: ClusterIP Service

• Default service type


• Exposes the service on a cluster-internal IP address
• Can only be accessed from pods running inside of the
cluster, but not from outside of the cluster

app=
web

External Service
Client [ClusterIP]

Pod

app= app=
web web

Pod Pod

Cluster

Copyright © Dell Inc. All Rights Reserved.


NodePort Service

• Exposes the service through the worker node’s IP address


• Can be accessed from outside of the cluster
• The node ports are opened on every node in the cluster

app= app=
web web
External
NodePort app= Pod Pod
Client web

Service Worker Node-1


[NodePort]

app= app=
web db

NodePort Pod Pod

Worker Node-2

Cluster

Copyright © Dell Inc. All Rights Reserved.


Update the k3d cluster to map the ports

$ k3d cluster edit mycluster --port-add 8080:30001@server:0

INFO[0000] Renaming existing node k3d-mycluster-serverlb to k3d-mycluster-serverlb-EHtGV...


INFO[0000] Creating new node k3d-mycluster-serverlb... Mapping service port 8080 to all node
ports 30001
INFO[0000] Stopping existing node k3d-mycluster-serverlb-EHtGV...
INFO[0011] Starting new node k3d-mycluster-serverlb...
INFO[0011] Starting Node 'k3d-mycluster-serverlb'
INFO[0018] Deleting old node k3d-mycluster-serverlb-EHtGV...
INFO[0018] Successfully updated mycluster

Copyright © Dell Inc. All Rights Reserved.


YAML manifest of the NodePort service

apiVersion: v1

kind: Service

metadata: The Service type is NodePort


name: web-nodeport-service

spec: The port this Service will be available on


type: NodePort

ports: The container port the Service will forward to


- port: 8080
targetPort: 80
The port of the node to allow external clients to
nodePort: 30001
access
selector:
app: web All pods with this label will be part of the Service

Copyright © Dell Inc. All Rights Reserved.


Difference between ClusterIP and NodePort
service
apiVersion: v1 apiVersion: v1

kind: Service kind: Service

metadata: metadata:
name: web-nodeport-service name: web-service

spec: spec:
type: NodePort type: ClusterIP

ports: ports:
- port: 8080 - port: 8080
targetPort: 80 targetPort: 80
nodePort: 30001
selector:
selector: app: web
app: web

Copyright © Dell Inc. All Rights Reserved.


NodePort Mapping

targetPort
80
app= app=
web web
External NodePort port
30001 8080 app= Pod Pod
Client web

Service Worker Node-1


[NodePort]

app= app=
web db
NodePort
30001
Pod Pod

Worker Node-2

Cluster

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our service

$ vim nginx-nodeport-service.yaml

apiVersion: v1
selector:
kind: Service app: web

metadata:
name: web-nodeport-service

spec:
type: NodePort

ports:
- port: 8080
targetPort: 80
nodePort: 30001

Copyright © Dell Inc. All Rights Reserved.


Delete the service we previously created

$ kubectl delete service web-service

service "web-service" deleted

Copyright © Dell Inc. All Rights Reserved.


Create the service with the manifest

$ kubectl apply -f nginx-nodeport-service.yaml

service/web-nodeport-service created

Copyright © Dell Inc. All Rights Reserved.


View the service

$ kubectl get service web-nodeport-service

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE


web-nodeport-service NodePort 10.43.17.185 <none> 8080:30001/TCP 51m

NodePort service has an internal Cluster


IP too

In addition, it has the node ports 30001


of all nodes mapped to the service port
8080

Copyright © Dell Inc. All Rights Reserved.


List all pods

$ kubectl get pods -o wide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
web-deploy-7869bd98c5-5l9x5 1/1 Running 0 60m 10.42.2.4 k3d-mycluster-agent-2 <none> <none>
web-deploy-7869bd98c5-k5dnj 1/1 Running 0 60m 10.42.2.5 k3d-mycluster-agent-2 <none> <none>
web-deploy-7869bd98c5-sdpvv 1/1 Running 0 60m 10.42.0.6 k3d-mycluster-agent-0 <none> <none>
web-deploy-7869bd98c5-cx29q 1/1 Running 0 60m 10.42.1.5 k3d-mycluster-agent-1 <none> <none>
web-deploy-7869bd98c5-pt254 1/1 Running 0 60m 10.42.1.4 k3d-mycluster-agent-1 <none> <none>

All the pods are placed across all the 3


worker nodes

Copyright © Dell Inc. All Rights Reserved.


List all nodes

$ kubectl get nodes -o wide

NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k3d-mycluster-agent-1 Ready <none> 64m v1.27.4+k3s1 192.168.144.5 <none> K3s dev 4.15.0-213-generic containerd://1.7.1-k3s1
k3d-mycluster-server-0 Ready control-plane,master 64m v1.27.4+k3s1 192.168.144.2 <none> K3s dev 4.15.0-213-generic containerd://1.7.1-k3s1
k3d-mycluster-agent-0 Ready <none> 64m v1.27.4+k3s1 192.168.144.3 <none> K3s dev 4.15.0-213-generic containerd://1.7.1-k3s1
k3d-mycluster-agent-2 Ready <none> 64m v1.27.4+k3s1 192.168.144.4 <none> K3s dev 4.15.0-213-generic containerd://1.7.1-k3s1

Note the IP addresses of the worker nodes

Copyright © Dell Inc. All Rights Reserved.


Access the NodePort service directly from outside of cluster

$ curl -s http://192.168.144.3:30001

<!DOCTYPE html>
<html> Worker Node-1
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to


<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>


</body>
</html>

Copyright © Dell Inc. All Rights Reserved.


Access the NodePort service directly from outside of cluster

$ curl -s http://192.168.144.4:30001

<!DOCTYPE html>
<html> Worker Node-2
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to


<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>


</body>
</html>

Copyright © Dell Inc. All Rights Reserved.


Access the NodePort service directly from outside of cluster

$ curl -s http://192.168.144.5:30001

<!DOCTYPE html>
<html> Worker Node-3
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to


<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>


</body>
</html>

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the NodePort service

$ kubectl run testpod --image=alpine -it -- sh

If you don't see a command prompt, try pressing enter.


/#

This command will run within the pod when the pod
We are using the alpine image to run a starts, in our case is the shell command
test container for testing

Open interactive session within pod

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the NodePort service

$ kubectl run testpod --image=alpine -it -- sh

/ # apk add curl


fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz
(1/8) Installing ca-certificates (20230506-r0)
(2/8) Installing brotli-libs (1.1.0-r1) We use apk, Alpine’s package manager to install curl into
(3/8) Installing c-ares (1.24.0-r1) the container for our testing
(4/8) Installing libunistring (1.1-r2)
(5/8) Installing libidn2 (2.3.4-r4)
(6/8) Installing nghttp2-libs (1.58.0-r0)
(7/8) Installing libcurl (8.5.0-r0)
(8/8) Installing curl (8.5.0-r0)
Executing busybox-1.36.1-r15.trigger
Executing ca-certificates-20230506-r0.trigger
OK: 12 MiB in 23 packages
/#

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the NodePort service

$ kubectl run testpod --image=alpine -it -- sh

/ # curl -s http://192.168.144.4:30001
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; } This is one of the worker node’s IP address accessed
body { width: 35em; margin: 0 auto; through port 30001
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to


<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>


</body>
</html>

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the NodePort service

$ kubectl run testpod --image=alpine -it -- sh

/ # curl -s http://10.43.17.185:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; } We can access through its cluster IP address as well
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to


<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>


</body>
</html>

Copyright © Dell Inc. All Rights Reserved.


Recap: NodePort Service

• Exposes the service through the worker node’s IP address


• Can be accessed from outside of the cluster
• The node ports are opened on every node in the cluster

app= app=
web web
External
NodePort app= Pod Pod
Client web

Service Worker Node-1


[NodePort]

app= app=
web db

NodePort Pod Pod

Worker Node-2

Cluster

Copyright © Dell Inc. All Rights Reserved.


LoadBalancer

• Exposes the service externally using a cloud provider’s load


balancer
• An external IP address is provided by the external load balancer
• Network traffic is distributed across multiple nodes in the cluster

app=
web
app= app=
web web
Cloud Service
External NodePort Pod Pod
Provider Load
Client [LoadBalancer]
Balancer
Worker Node-1

app= app=
web db

NodePort Pod Pod

Worker Node-2

Cluster

Copyright © Dell Inc. All Rights Reserved.


YAML manifest of an AWS LoadBalancer
service
apiVersion: v1

kind: Service

metadata:
name: web-lb-service
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: external
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip AWS load balancer
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing annotations

spec:
type: LoadBalancer
The Service type is LoadBalancer
ports:
- port: 80
targetPort: 80

selector:
app: web
Copyright © Dell Inc. All Rights Reserved.
Comparison between Service Types

Type Description Accessible from Accessible from Ports


inside cluster? outside cluster?

ClusterIP Exposes service on an port,


Yes No
internal cluster IP targetPort

NodePort nodePort,
Exposes service on each
Yes Yes port,
worker node’s port
targetPort

LoadBalancer Exposes service nodePort,


externally via a cloud Yes Yes port,
provider’s load balancer targetPort

Copyright © Dell Inc. All Rights Reserved.


Recap

• Kubernetes assigns a unique IP address for every pod in the cluster


• Pods are ephemeral and therefore we cannot rely on the IP addresses to be
stable over time
• A service forwards network traffic to a group of pods based on label selection
and port mapping
• The three notable service types are ClusterIP, NodePort and LoadBalancer

Copyright © Dell Inc. All Rights Reserved.


Applying Deployment Strategies

Copyright © Dell Inc. All Rights Reserved.


Learning Objectives

At the end of this topic, you will be able to:


• understand the importance of zero downtime for critical applications in
Kubernetes
• understand how deployment strategies play a crucial role in ensuring
continuous availability and reliability by minimising service disruptions during
updates and maintenance activities
• apply the different deployment strategies in Kubernetes, and understand how
each strategy works and when to use it based on application requirements

Copyright © Dell Inc. All Rights Reserved.


Updating Applications

• In traditional software environments, deployments or upgrades


to applications may result in downtime and disruption of service
• In the modern software landscape, it is increasingly important,
even expected, that applications can be updated without
causing any downtime to the end users

! We’re Experiencing
Some Downtime

OK

Copyright © Dell Inc. All Rights Reserved.


Avoiding or Minimising Application Downtime

• Adopting the appropriate deployment strategy enables us to


create, upgrade or downgrade different versions of applications
with zero or minimal downtime
• Kubernetes provides deployment strategies that define the
application lifecycle and how updates to the application should
be applied

Copyright © Dell Inc. All Rights Reserved.


Deployment Strategies

• Common deployment strategies include


– Rolling Update (default)
– Recreate
– Blue/Green
– Canary

Copyright © Dell Inc. All Rights Reserved.


Rolling Update Deployment Strategy

Deployment

ReplicaSet Deployment complete! ReplicaSet

v1 v1 v1 v2 v2 v2

Copyright © Dell Inc. All Rights Reserved.


Rolling Update Deployment Strategy

• Ensures there is no downtime during the update process


• Creates a new ReplicaSet to replace the old containers with the
new ones
• We can control the range of available and excess pods through
the maxSurge and maxUnavailable fields in the Deployment
YAML manifest

Copyright © Dell Inc. All Rights Reserved.


RollingUpdate Deployment YAML
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-deploy

spec: This is the section to define the deployment strategy


replicas: 3

strategy:
type: RollingUpdate maxSurge determines the number of pods that can be run
rollingUpdate: temporarily in addition to the replicas specified in the update. In our
maxSurge: 1
case here, it will be maximum of 4 pods
maxUnavailable: 1

selector:
matchLabels:
app: web maxUnavailable determines the number of pods that may be
template: unavailable during the update. In our case here, it will be only 2
metadata: pods are available at a time
labels:
app: web
spec:
containers:
- name: web
image: nginx:alpine

Copyright © Dell Inc. All Rights Reserved.


RollingUpdate Deployment YAML
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-deploy

spec:
replicas: 3

strategy:
type: RollingUpdate Instead of absolute numbers, we can use percentage as well.
rollingUpdate: This is the default if we do not specify.
maxSurge: 25%
maxUnavailable: 25%

selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
spec:
containers:
- name: web
image: nginx:alpine

Copyright © Dell Inc. All Rights Reserved.


List our current deployment

$ kubectl get deployment web-deploy -o wide

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR


web-deploy 3/3 3 3 2d20h web nginx app=web

The current image is nginx

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our rollingupdate deployment

$ vim nginx-rollingupdate-deployment.yaml

apiVersion: apps/v1
selector:
kind: Deployment matchLabels:
app: web
metadata:
name: web-deploy template: We will update the deployment’s
metadata: image to nginx:alpine
spec: labels:
replicas: 3 app: web
spec:
strategy: containers:
type: RollingUpdate - name: web
rollingUpdate: image: nginx:alpine
maxSurge: 1
maxUnavailable: 1

Copyright © Dell Inc. All Rights Reserved.


Create the deployment with the manifest

$ kubectl apply -f nginx-rollingupdate-deployment.yaml

deployment.apps/web-deploy configured

Copyright © Dell Inc. All Rights Reserved.


List and watch the pods

$ kubectl get pods --watch

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
web-deploy-7869bd98c5-k5dnj 1/1 Running 0 2d20h 10.42.2.5 k3d-mycluster-agent-2 <none> <none>
web-deploy-7869bd98c5-pt254 1/1 Running 0 2d20h 10.42.1.4 k3d-mycluster-agent-1 <none> <none>
web-deploy-599fd76bc8-q49td 0/1 ContainerCreating 0 5s <none> k3d-mycluster-agent-1 <none> <none>
web-deploy-599fd76bc8-9vv6h 0/1 ContainerCreating 0 5s <none> k3d-mycluster-agent-0 <none> <none>
...

Copyright © Dell Inc. All Rights Reserved.


List our deployment

$ kubectl get deployment web-deploy -o wide

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR


web-deploy 3/3 3 3 2d20h web nginx:alpine app=web

The image is updated to


nginx:alpine

Copyright © Dell Inc. All Rights Reserved.


Recreate Deployment Strategy

Deployment

ReplicaSet Deployment complete! ReplicaSet

v1 v1 v1 v2 v2 v2

Copyright © Dell Inc. All Rights Reserved.


Recreate Deployment Strategy

• All old pods are deleted before the new ones are created
• Short period of downtime when the app becomes completely
unavailable
• Use this strategy if your app does not support multiple versions in
parallel

Copyright © Dell Inc. All Rights Reserved.


Recreate Deployment YAML
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-deploy

spec: No other configuration options need to be provided


replicas: 3

strategy:
type: Recreate

selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
spec:
containers:
- name: web
image: nginx:perl

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our recreate deployment

$ vim nginx-recreate-deployment.yaml

apiVersion: apps/v1
selector:
kind: Deployment matchLabels:
app: web
metadata:
name: web-deploy template: We will update the deployment’s
metadata: image to nginx:perl
spec: labels:
replicas: 3 app: web
spec:
strategy: containers:
type: Recreate - name: web
image: nginx:perl

Copyright © Dell Inc. All Rights Reserved.


Create the deployment with the manifest

$ kubectl apply -f nginx-recreate-deployment.yaml

deployment.apps/web-deploy configured

Copyright © Dell Inc. All Rights Reserved.


List and watch the pods

$ kubectl get pods --watch

NAME READY STATUS RESTARTS AGE


web-deploy-6585dd4b64-zdsd6 0/1 ContainerCreating 0 6s
web-deploy-6585dd4b64-8zkxw 0/1 ContainerCreating 0 6s
web-deploy-6585dd4b64-br8wb 0/1 ContainerCreating 0 6s
web-deploy-6585dd4b64-8zkxw 1/1 Running 0 10s
web-deploy-6585dd4b64-br8wb 1/1 Running 0 10s
web-deploy-6585dd4b64-zdsd6 1/1 Running 0 10s

3 pods are down and recreated

Copyright © Dell Inc. All Rights Reserved.


List our deployment

$ kubectl get deployment web-deploy -o wide

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR


web-deploy 3/3 3 3 3d1h web nginx:perl app=web

The image is updated to


nginx:perl

Copyright © Dell Inc. All Rights Reserved.


Let’s clean up first

$ kubectl delete deployment web-deploy

deployment.apps "web-deploy" deleted

Copyright © Dell Inc. All Rights Reserved.


Blue-Green Deployment Strategy

Service
[LoadBalancer]

v1 v1 v1 v2 v2 v2

Testing complete!
ReplicaSet Deployment complete! ReplicaSet

Deployment Deployment
(Old) (New)

Copyright © Dell Inc. All Rights Reserved.


Blue-Green Deployment Strategy

• Create a Deployment object for both application versions


• Use a specific label on the pod template of the “blue” deployment
• Use a different label on the pod template of the “green”
deployment
• Create a Service to route traffic to the replicas managed by the
specific deployment you want to direct to with the label selector
• Switch the traffic to point to the other deployment by simply
changing the label selection in the service

Copyright © Dell Inc. All Rights Reserved.


Blue-Green Deployment YAML
apiVersion: apps/v1 apiVersion: apps/v1
kind: Deployment kind: Deployment
metadata: metadata:
name: web-deploy-blue name: web-deploy-green
spec: spec:
replicas: 3 replicas: 3

selector: selector:
matchLabels: matchLabels:
app: blue app: green
template: template: green deployment
metadata: blue deployment label metadata: label
labels: labels:
app: blue app: green
spec: spec:
containers: containers:
- name: web - name: web
image: nginx image: httpd

Update the deployment from nginx to httpd

Copyright © Dell Inc. All Rights Reserved.


Blue-Green Service YAML

apiVersion: v1
kind: Service
metadata:
name: web-service

spec:
The label selector is currently pointing to the replicas
selector: managed by the blue deployment
app: blue

ports:
- port: 80
targetPort: 80

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our blue deployment

$ vim nginx-blue-deployment.yaml

apiVersion: apps/v1
selector:
kind: Deployment matchLabels:
app: blue
metadata:
name: web-deploy-blue template:
metadata:
spec: labels:
replicas: 3 app: blue
spec:
containers:
- name: web
image: nginx

Copyright © Dell Inc. All Rights Reserved.


Create the deployment with the manifest

$ kubectl apply -f nginx-blue-deployment.yaml

deployment.apps/web-deploy-blue created

Copyright © Dell Inc. All Rights Reserved.


List and watch the pods

$ kubectl get pods -L app

NAME READY STATUS RESTARTS AGE APP


web-deploy-blue-68f5bb5958-cdvjp 1/1 Running 0 4m36s blue
web-deploy-blue-68f5bb5958-gd2gl 1/1 Running 0 4m36s blue
web-deploy-blue-68f5bb5958-6wnlz 1/1 Running 0 4m36s blue

These 3 pods are from the blue deployment

Copyright © Dell Inc. All Rights Reserved.


List our deployment

$ kubectl get deployment web-deploy-blue -o wide

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR


web-deploy 3/3 3 3 3d1h web nginx app=blue

The image is nginx for the blue


deployment

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our green deployment

$ vim nginx-green-deployment.yaml

apiVersion: apps/v1
selector:
kind: Deployment matchLabels:
app: green
metadata:
name: web-deploy-green template:
metadata:
spec: labels:
replicas: 3 app: green
spec:
containers:
- name: web
image: httpd

Copyright © Dell Inc. All Rights Reserved.


Create the deployment with the manifest

$ kubectl apply -f nginx-green-deployment.yaml

deployment.apps/web-deploy-green created

Copyright © Dell Inc. All Rights Reserved.


List and watch the pods

$ kubectl get pods -L app

NAME READY STATUS RESTARTS AGE APP


web-deploy-blue-68f5bb5958-cdvjp 1/1 Running 0 4m36s blue
web-deploy-blue-68f5bb5958-gd2gl 1/1 Running 0 4m36s blue
web-deploy-blue-68f5bb5958-6wnlz 1/1 Running 0 4m36s blue
web-deploy-green-7d76f987f9-znkxc 1/1 Running 0 31s green
web-deploy-green-7d76f987f9-htn7h 1/1 Running 0 31s green
web-deploy-green-7d76f987f9-cph2c 1/1 Running 0 31s green

These 3 pods are from the green deployment

Copyright © Dell Inc. All Rights Reserved.


List our deployment

$ kubectl get deployment web-deploy-green -o wide

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR


web-deploy 3/3 3 3 3d1h web httpd app=green

The image is httpd for the


green deployment

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our service

$ vim nginx-bluegreen-service.yaml

apiVersion: v1

kind: Service

metadata:
name: web-service

spec: The label selector is currently pointing to the


selector: replicas managed by the blue deployment
app: blue
ports:
- port: 80
targetPort: 80

Copyright © Dell Inc. All Rights Reserved.


Create the service with the manifest

$ kubectl apply -f nginx-bluegreen-service.yaml

service/web-service created

Copyright © Dell Inc. All Rights Reserved.


View the service

$ kubectl get service web-service -o wide

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR


web-service ClusterIP 10.43.117.202 <none> 80/TCP 6m45s app=blue

The service is directing to


the blue deployment

Copyright © Dell Inc. All Rights Reserved.


View the endpoints

$ kubectl get endpoints web-service

NAME ENDPOINTS AGE


web-service 10.42.0.10:80,10.42.1.11:80,10.42.2.11:80 8m49s

The 3 endpoints are


● 10.42.0.10
● 10.42.1.11
● 10.42.2.11

Copyright © Dell Inc. All Rights Reserved.


List the pods

$ kubectl get pods -o wide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE


READINESS GATES

web-deploy-blue-68f5bb5958-cdvjp 1/1 Running 0 24m 10.42.1.11 k3d-mycluster-agent-1 <none> <none>


web-deploy-blue-68f5bb5958-gd2gl 1/1 Running 0 24m 10.42.0.10 k3d-mycluster-agent-0 <none> <none>
web-deploy-blue-68f5bb5958-6wnlz 1/1 Running 0 24m 10.42.2.11 k3d-mycluster-agent-2 <none> <none>

web-deploy-green-7d76f987f9-znkxc 1/1 Running 0 20m 10.42.2.12 k3d-mycluster-agent-2 <none> <none>


web-deploy-green-7d76f987f9-htn7h 1/1 Running 0 20m 10.42.1.12 k3d-mycluster-agent-1 <none> <none>
web-deploy-green-7d76f987f9-cph2c 1/1 Running 0 20m 10.42.0.11 k3d-mycluster-agent-0 <none> <none>

The service directs traffic to these 3 pods from the


blue deployment as the endpoints

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the service

$ kubectl run testpod --image=alpine -it --rm -- sh

If you don't see a command prompt, try pressing enter.


/ # curl
sh: curl: not found

We are going to use curl to test the connectivity from the testpod to
the service. But Alpine does not come pre-installed with curl

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the service

$ kubectl run testpod --image=alpine -it --rm -- sh

/ # apk add curl


fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz
(1/8) Installing ca-certificates (20230506-r0)
(2/8) Installing brotli-libs (1.1.0-r1) We use apk, Alpine’s package manager to install
(3/8) Installing c-ares (1.24.0-r1) curl into the container for our testing
(4/8) Installing libunistring (1.1-r2)
(5/8) Installing libidn2 (2.3.4-r4)
(6/8) Installing nghttp2-libs (1.58.0-r0)
(7/8) Installing libcurl (8.5.0-r0)
(8/8) Installing curl (8.5.0-r0)
Executing busybox-1.36.1-r15.trigger
Executing ca-certificates-20230506-r0.trigger
OK: 12 MiB in 23 packages
/ #

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the service

$ kubectl run testpod --image=alpine -it --rm -- sh

/ # curl -s http://10.43.117.202
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; } Access the service and we see
body { width: 35em; margin: 0 auto; the output returned by nginx
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to


<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>


</body>
</html>

Copyright © Dell Inc. All Rights Reserved.


Modify the service YAML manifest to switch to GREEN

$ vim bluegreen-service.yaml

apiVersion: v1

kind: Service

metadata:
name: web-service Update from `blue` to
`green` to switch
spec: deployment
selector:
app: blue
ports:
- port: 80
targetPort: 80

Copyright © Dell Inc. All Rights Reserved.


Update the service with the manifest

$ kubectl apply -f bluegreen-service.yaml

service/web-service configured

Copyright © Dell Inc. All Rights Reserved.


View the service

$ kubectl get service web-service -o wide

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR


web-service ClusterIP 10.43.117.202 <none> 80/TCP 16m app=green

The service is changed to


directing to the green
deployment

Copyright © Dell Inc. All Rights Reserved.


View the endpoints

$ kubectl get endpoints web-service

NAME ENDPOINTS AGE


web-service 10.42.0.11:80,10.42.1.12:80,10.42.2.12:80 8m49s

The 3 endpoints are updated


● 10.42.0.11
● 10.42.1.12
● 10.42.2.12

Copyright © Dell Inc. All Rights Reserved.


List the pods

$ kubectl get pods -o wide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE


READINESS GATES

web-deploy-blue-68f5bb5958-cdvjp 1/1 Running 0 24m 10.42.1.11 k3d-mycluster-agent-1 <none> <none>


web-deploy-blue-68f5bb5958-gd2gl 1/1 Running 0 24m 10.42.0.10 k3d-mycluster-agent-0 <none> <none>
web-deploy-blue-68f5bb5958-6wnlz 1/1 Running 0 24m 10.42.2.11 k3d-mycluster-agent-2 <none> <none>

web-deploy-green-7d76f987f9-znkxc 1/1 Running 0 20m 10.42.2.12 k3d-mycluster-agent-2 <none> <none>


web-deploy-green-7d76f987f9-htn7h 1/1 Running 0 20m 10.42.1.12 k3d-mycluster-agent-1 <none> <none>
web-deploy-green-7d76f987f9-cph2c 1/1 Running 0 20m 10.42.0.11 k3d-mycluster-agent-0 <none> <none>

The service directs traffic to these 3 pods from the


green deployment as the endpoints

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the service

$ kubectl run testpod --image=alpine -it --rm -- sh

If you don't see a command prompt, try pressing enter.


/ # curl
sh: curl: not found

We are going to use curl to test the connectivity from the testpod to the
service. But Alpine does not come pre-installed with curl

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the service

$ kubectl run testpod --image=alpine -it --rm -- sh

/ # apk add curl


fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz
(1/8) Installing ca-certificates (20230506-r0)
(2/8) Installing brotli-libs (1.1.0-r1) We use apk, Alpine’s package manager to
(3/8) Installing c-ares (1.24.0-r1) install curl into the container for our testing
(4/8) Installing libunistring (1.1-r2)
(5/8) Installing libidn2 (2.3.4-r4)
(6/8) Installing nghttp2-libs (1.58.0-r0)
(7/8) Installing libcurl (8.5.0-r0)
(8/8) Installing curl (8.5.0-r0)
Executing busybox-1.36.1-r15.trigger
Executing ca-certificates-20230506-r0.trigger
OK: 12 MiB in 23 packages
/ #

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the service

$ kubectl run testpod --image=alpine -it --rm -- sh

/ # curl -s http://10.43.117.202
<html><body><h1>It works!</h1></body></html>

Access the service and we see


the output returned by httpd
instead

Copyright © Dell Inc. All Rights Reserved.


Let’s clean up first

$ kubectl delete deployment web-deploy-blue

deployment.apps "web-deploy-blue" deleted

Copyright © Dell Inc. All Rights Reserved.


Let’s clean up first

$ kubectl delete deployment web-deploy-green

deployment.apps "web-deploy-green" deleted

Copyright © Dell Inc. All Rights Reserved.


Let’s clean up first

$ kubectl delete service web-service

service "web-service" deleted

Copyright © Dell Inc. All Rights Reserved.


Recap: Blue-Green Deployment Strategy

Service
[LoadBalancer]

v1 v1 v1 v2 v2 v2

Testing complete!
ReplicaSet Deployment complete! ReplicaSet

Deployment Deployment
(Old) (New)

Copyright © Dell Inc. All Rights Reserved.


Canary Deployment Strategy

Service
[LoadBalancer]

v1 v1 v1 v2

ReplicaSet Deployment complete! ReplicaSet

Deployment Deployment
(Old) (New)

Copyright © Dell Inc. All Rights Reserved.


Canary Deployment Strategy

• Similar to the blue-green deployment strategy


• Both versions of the application are running at the same time
• The new application version is available only to a small subset of users
• Useful to implement A/B testing of new features

Copyright © Dell Inc. All Rights Reserved.


Canary Deployment Strategy Approach

• Create a Deployment object for both application versions


• Use the same label on the pod template of the older version of the
deployment
• Use the same label on the pod template of the newer version of the
deployment. The number replicas should be lower than the replicas
of the older deployment
• Create a Service to route traffic to both the replicas with the same
label selector

Copyright © Dell Inc. All Rights Reserved.


Canary Deployment YAML
apiVersion: apps/v1 apiVersion: apps/v1
kind: Deployment kind: Deployment
metadata: metadata:
name: web-deploy-old name: web-deploy-new
spec: spec:
replicas: 3 replicas: 1

selector: selector:
matchLabels: matchLabels:
app: canary app: canary
template: Same label for both template:
metadata: deployments metadata:
labels: labels:
app: canary app: canary
spec: spec:
containers: containers:
- name: web - name: web
image: nginx image: httpd

3 replicas running the old version on nginx and 1 replica


running the new version on httpd
Copyright © Dell Inc. All Rights Reserved.
Canary Service YAML

apiVersion: v1
kind: Service
metadata:
name: web-service

spec:
The label selector is currently pointing to the replicas managed
selector: by both the old and new deployment
app: canary

ports:
- port: 80
targetPort: 80

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our old deployment

$ vim nginx-old-deployment.yaml

apiVersion: apps/v1
selector:
kind: Deployment matchLabels:
app: canary
metadata:
name: web-deploy-old template:
metadata:
spec: labels:
replicas: 3 app: canary
spec:
containers:
- name: web
image: nginx

Copyright © Dell Inc. All Rights Reserved.


Create the deployment with the manifest

$ kubectl apply -f nginx-old-deployment.yaml

deployment.apps/web-deploy-old created

Copyright © Dell Inc. All Rights Reserved.


List the pods

$ kubectl get pods -l app

NAME READY STATUS RESTARTS AGE


web-deploy-old-674b947d5-rqdjs 1/1 Running 0 23s
web-deploy-old-674b947d5-jqdgl 1/1 Running 0 23s
web-deploy-old-674b947d5-k9ls9 1/1 Running 0 23s

Copyright © Dell Inc. All Rights Reserved.


List our deployment

$ kubectl get deployment web-deploy-old -o wide

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR


web-deploy-old 3/3 3 3 106s web nginx app=canary

The image is nginx for the


old deployment

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our new deployment

$ vim httpd-new-deployment.yaml

apiVersion: apps/v1
selector:
kind: Deployment matchLabels:
app: canary
metadata:
name: web-deploy-new template:
metadata:
spec: labels:
replicas: 1 app: canary
spec:
containers:
- name: web
image: httpd

Copyright © Dell Inc. All Rights Reserved.


Create the deployment with the manifest

$ kubectl apply -f httpd-new-deployment.yaml

deployment.apps/web-deploy-new created

Copyright © Dell Inc. All Rights Reserved.


List the pods

$ kubectl get pods -l app

NAME READY STATUS RESTARTS AGE


web-deploy-old-674b947d5-rqdjs 1/1 Running 0 4m6s
web-deploy-old-674b947d5-jqdgl 1/1 Running 0 4m6s
web-deploy-old-674b947d5-k9ls9 1/1 Running 0 4m6s
web-deploy-new-7557c67ff7-b9bk6 1/1 Running 0 4m1s

3 pods from old version and


1 pod from new version

Copyright © Dell Inc. All Rights Reserved.


List our deployment

$ kubectl get deployment web-deploy-new -o wide

NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR


web-deploy-new 1/1 1 1 5m48s web httpd app=canary

The image is httpd for the


new deployment

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our service

$ vim canary-service.yaml

apiVersion: v1

kind: Service

metadata:
name: web-service

spec:
selector:
app: canary
ports:
- port: 80
targetPort: 80

Copyright © Dell Inc. All Rights Reserved.


Create the service with the manifest

$ kubectl apply -f canary-service.yaml

service/web-service created

Copyright © Dell Inc. All Rights Reserved.


View the service

$ kubectl get service web-service -o wide

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR


web-service ClusterIP 10.43.241.247 <none> 80/TCP 4s app=canary

Copyright © Dell Inc. All Rights Reserved.


View the endpoints

$ kubectl describe endpoints web-service

Name: web-service
Namespace: default
Labels: <none>
Annotations: endpoints.kubernetes.io/last-change-trigger-time: 2024-03-01T11:47:03Z
Subsets:
Addresses: 10.42.0.13,10.42.1.17,10.42.2.16,10.42.2.17
NotReadyAddresses: <none>
Ports:
Name Port Protocol
---- ---- --------
<unset> 80 TCP
The 4 endpoints are
● 10.42.0.13
● 10.42.1.17
● 10.42.2.16
● 10.42.2.17

Copyright © Dell Inc. All Rights Reserved.


List the pods

$ kubectl get pods -o wide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS


GATES
web-deploy-old-674b947d5-rqdjs 1/1 Running 0 10m 10.42.0.13 k3d-mycluster-agent-0 <none> <none>
web-deploy-old-674b947d5-jqdgl 1/1 Running 0 10m 10.42.2.16 k3d-mycluster-agent-2 <none> <none>
web-deploy-old-674b947d5-k9ls9 1/1 Running 0 10m 10.42.1.17 k3d-mycluster-agent-1 <none> <none>
web-deploy-new-7557c67ff7-b9bk6 1/1 Running 0 10m 10.42.2.17 k3d-mycluster-agent-2 <none> <none>

The service directs traffic to these 4 pods where 3 are


on nginx and 1 is on httpd

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the service

$ kubectl run testpod --image=alpine -it --rm -- sh

If you don't see a command prompt, try pressing enter.


/ # curl
sh: curl: not found

We are going to use curl to test the connectivity from the testpod to
the service. But Alpine does not come pre-installed with curl

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the service

$ kubectl run testpod --image=alpine -it --rm -- sh

/ # apk add curl


fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz
(1/8) Installing ca-certificates (20230506-r0)
(2/8) Installing brotli-libs (1.1.0-r1) We use apk, Alpine’s package manager to install
(3/8) Installing c-ares (1.24.0-r1) curl into the container for our testing
(4/8) Installing libunistring (1.1-r2)
(5/8) Installing libidn2 (2.3.4-r4)
(6/8) Installing nghttp2-libs (1.58.0-r0)
(7/8) Installing libcurl (8.5.0-r0)
(8/8) Installing curl (8.5.0-r0)
Executing busybox-1.36.1-r15.trigger
Executing ca-certificates-20230506-r0.trigger
OK: 12 MiB in 23 packages
/ #

Copyright © Dell Inc. All Rights Reserved.


Create a test pod to access the service

$ kubectl run testpod --image=alpine -it --rm -- sh

/ # curl -s 10.43.241.247
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style> Try running this a couple of times, and you will randomly
html { color-scheme: light dark; } see either the nginx or httpd output. Since there are 3
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; } nginx replicas and only 1 httpd replica, the chances of
</style>
</head>
getting the nginx output is higher
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to


<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>


</body>
</html>

Copyright © Dell Inc. All Rights Reserved.


Let’s clean up first

$ kubectl delete deployment web-deploy-old

deployment.apps "web-deploy-old" deleted

Copyright © Dell Inc. All Rights Reserved.


Let’s clean up first

$ kubectl delete deployment web-deploy-new

deployment.apps "web-deploy-new" deleted

Copyright © Dell Inc. All Rights Reserved.


Let’s clean up first

$ kubectl delete service web-service

service "web-service" deleted

Copyright © Dell Inc. All Rights Reserved.


Recap: Canary Deployment Strategy

Service
[LoadBalancer]

v1 v1 v1 v2

ReplicaSet Deployment complete! ReplicaSet

Deployment Deployment
(Old) (New)

Copyright © Dell Inc. All Rights Reserved.


Comparison between Deployment Strategies
Type Use Case Trade-offs

Rolling Update Rolling out a new version incrementally Old and new versions run at
with zero downtime the same time during the
upgrade. Slower deployment
process

Recreate Need to quickly roll out new version. Downtime is required


Suitable in developer environment for
testing

Blue-Green Complex upgrades with zero downtime Require twice the amount of
and reduced risk. Can easily rollback to resources to run
previous version

Canary Rolling out experimental features to a Require additional resources


smaller group of users. Good for A/B but not as much as
testing Blue-Green

Copyright © Dell Inc. All Rights Reserved.


Recap

• In today’s software environment, there is a growing emphasis and expectation


on updating applications seamlessly without interrupting end users’ access
• We explored deployment strategies in Kubernetes, including rolling updates,
recreate, blue-green deployments and canary deployments
• Each strategy comes with its own set of trade-offs that must be carefully
considered before adoption, with the priority being to ensure the best possible
user experience

Copyright © Dell Inc. All Rights Reserved.


Externalizing Configuration with ConfigMaps
and Secrets

Copyright © Dell Inc. All Rights Reserved.


Learning Objectives

At the end of this topic, you will be able to:


• understand the role of ConfigMaps and Secrets in Kubernetes and their
significance in managing configuration data and sensitive information
• Identify when to use ConfigMaps versus Secrets
• Explore practical examples of how ConfigMaps and Secrets can be integrated
to applications for handling configuration or credentials

Copyright © Dell Inc. All Rights Reserved.


Separating app configuration from source code
• Hard-coding app configuration in our source code is considered
bad practice and has many drawbacks such as lack of flexibility,
code maintenance challenges, difficulty in environment
management and security risks
• Using environment variables is one approach we can isolate
critical app configuration data from our applications
• This adheres to the design principle of separation of concern
• When we create a pod, we can set environment variables for the
containers that run in the pod

config config

App

Copyright © Dell Inc. All Rights Reserved.


Let’s create a pod running mysql

$ kubectl run mydatabase --image=mysql

pod/mydatabase created

Copyright © Dell Inc. All Rights Reserved.


List the pod

$ kubectl get pod mydatabase

NAME READY STATUS RESTARTS AGE


mydatabase 0/1 ContainerCreating 0 12s

Copyright © Dell Inc. All Rights Reserved.


List and watch the pod

$ kubectl get pod mydatabase --watch

NAME READY STATUS RESTARTS AGE


mydatabase 0/1 Error 5 (102s ago) 4m10s
mydatabase 0/1 CrashLoopBackOff 5 (12s ago) 4m13s

Copyright © Dell Inc. All Rights Reserved.


View the logs of the pod

$ kubectl logs mydatabase

2024-03-03 13:58:45+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server


8.3.0-1.el8 started.
2024-03-03 13:58:45+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2024-03-03 13:58:45+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server
8.3.0-1.el8 started.
2024-03-03 13:58:45+00:00 [ERROR] [Entrypoint]: Database is uninitialized and password
option is not specified
You need to specify one of the following as an environment variable:
- MYSQL_ROOT_PASSWORD
- MYSQL_ALLOW_EMPTY_PASSWORD We need to specify one of the following as
- MYSQL_RANDOM_ROOT_PASSWORD an environment variable to be injected to
the pod

Copyright © Dell Inc. All Rights Reserved.


Include environment variable(s) to the pod manifest

apiVersion: v1

kind: Pod

metadata:
name: mydatabase

spec:
containers:
- name: mydatabase
image: mysql
env:
- name: MYSQL_ROOT_PASSWORD We can inject the environment variables here
value: "secretpassword"

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our pod

$ vim mysql-pod.yaml

apiVersion: v1
kind: Pod
metadata:
name: mydatabase
spec:
containers:
- name: mydatabase
image: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: “secretpassword”

Copyright © Dell Inc. All Rights Reserved.


Delete the previous pod

$ kubectl delete pod mydatabase

pod "mydatabase" deleted

Copyright © Dell Inc. All Rights Reserved.


Create the pod with the manifest

$ kubectl apply -f mysql-pod.yaml

pod/mydatabase created

Copyright © Dell Inc. All Rights Reserved.


List and watch the pod

$ kubectl get pod mydatabase --watch

NAME READY STATUS RESTARTS AGE


mydatabase 1/1 Running 0 13s

Copyright © Dell Inc. All Rights Reserved.


View the logs of the pod

$ kubectl logs mydatabase


2024-03-03T14:18:40.745189Z 0 [System] [MY-011323] [Server] X Plugin ready for
connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2024-03-03T14:18:40.745520Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for
connections. Version: '8.3.0' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL
Community Server - GPL.

Copyright © Dell Inc. All Rights Reserved.


ConfigMaps
• Are used to store configuration data outside of the app
• The purpose is to decouple configuration from the pod lifecycle
• Configuration data are stored in key-value pairs
• These key-value pairs can then be injected into a container as environment
variables, or mounted as a volume and mapped to a filesystem path of the
container

key: value
key: value

Container
Injected as
environment
variables
ConfigMap Mounted as volume to Pod ConfigMap
filesystem path

Copyright © Dell Inc. All Rights Reserved.


Create the configmap the imperative way
$ kubectl create configmap mysql-configmap
--from-literal=MYSQL_ROOT_PASSWORD=secretpassword

configmap/mysql-configmap created

Copyright © Dell Inc. All Rights Reserved.


The ConfigMap YAML manifest

apiVersion: v1

kind: ConfigMap

metadata:
name: mysql-configmap
Note that the ConfigMap manifest does not
data:
have a spec section. All the key-value pairs
MYSQL_ROOT_PASSWORD: secretpassword
are specified under the data section

Copyright © Dell Inc. All Rights Reserved.


Delete the previous pod

$ kubectl delete pod mydatabase

pod "mydatabase" deleted

Copyright © Dell Inc. All Rights Reserved.


Consume the configmap as environment variables

apiVersion: v1

kind: Pod

metadata:
name: mydatabase

spec:
containers:
- name: mydatabase
image: mysql
envFrom: The key-value pair in the configmap is
- configMapRef: injected as an environment variable in the
name: mysql-configmap container

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our pod

$ vim mysql-cm-pod.yaml

apiVersion: v1
kind: Pod
metadata:
name: mydatabase
spec:
containers:
- name: mydatabase
image: mysql
envFrom:
- configMapRef:
name: mysql-configmap

Copyright © Dell Inc. All Rights Reserved.


Create the pod with the manifest

$ kubectl apply -f mysql-cm-pod.yaml

pod/mydatabase created

Copyright © Dell Inc. All Rights Reserved.


List and watch the pod

$ kubectl get pod mydatabase --watch

NAME READY STATUS RESTARTS AGE


mydatabase 0/1 ContainerCreating 0 5s
mydatabase 1/1 Running 0 8s

Copyright © Dell Inc. All Rights Reserved.


We can also create a configmap based on file

$ vim database.json

{
“database”: { We create a database configuration file
that can be read by the app in the
“host”: “mysql-service”, container
“user”: “dba”
}

Copyright © Dell Inc. All Rights Reserved.


Create the configmap the imperative way

$ kubectl create configmap mysql-json-configmap --from-file=database.json

configmap/mysql-json-configmap created

Location of the file to be loaded as a


configmap

Copyright © Dell Inc. All Rights Reserved.


The ConfigMap YAML manifest

apiVersion: v1

kind: ConfigMap

metadata:
name: mysql-json-configmap

data:
db.json: |-
{
“database”: {
Database configuration content from the
“host”: “mysql-service”,
file db.json
“user”: “dba”
}

Copyright © Dell Inc. All Rights Reserved.


Mount the configmap as a volume
apiVersion: v1

kind: Pod

metadata:
name: mydatabase

spec:
volumes:
- name: mysql-config-volume We create a volume and reference it to the
configMap: configmap
name: mysql-json-configmap

containers:
- name: mydatabase
image: mysql
volumeMounts: We mount the volume to the filepath /etc/config
- name: mysql-config-volume of the container
mountPath: /etc/config
envFrom:
- configMapRef:
name: mysql-configmap

Copyright © Dell Inc. All Rights Reserved.


Delete the previous pod

$ kubectl delete pod mydatabase

pod "mydatabase" deleted

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our pod

$ vim mysql-volumemount-pod.yaml

apiVersion: v1 containers:
- name: mydatabase
kind: Pod image: mysql

metadata: volumeMounts:
name: mydatabase - name: mysql-config-volume
mountPath: /etc/config
spec:
volumes: envFrom:
- name: mysql-config-volume - configMapRef:
configMap: name: mysql-configmap
name: mysql-json-configmap

Copyright © Dell Inc. All Rights Reserved.


Create the pod with the manifest

$ kubectl apply -f mysql-volumemount-pod.yaml

pod/mydatabase created

Copyright © Dell Inc. All Rights Reserved.


List and watch the pod

$ kubectl get pod mydatabase --watch

NAME READY STATUS RESTARTS AGE


mydatabase 0/1 ContainerCreating 0 5s
mydatabase 1/1 Running 0 8s

Copyright © Dell Inc. All Rights Reserved.


Access the pod to validate the mounted volume

$ kubectl exec -it mydatabase -- /bin/sh

sh-4.4#

Copyright © Dell Inc. All Rights Reserved.


Access the pod to validate the mounted volume

$ kubectl exec -it mydatabase -- /bin/sh

sh-4.4# cat /etc/config/database.json

{
“database”: {
“host”: “mysql-service”,
“user”: “dba”
}

Copyright © Dell Inc. All Rights Reserved.


What we have done

MYSQL_ROOT_PASSWORD:
key: value secretpassword key: value

json
Container
Injected as
environment variables

ConfigMap Mounted as volume to filesystem Pod ConfigMap


path /etc/config
mysql-json-configmap mysql-configmap
mydatabase

Copyright © Dell Inc. All Rights Reserved.


Secrets
• ConfigMaps are designed to store non-sensitive configuration data as the data
is stored as plain text
• To store sensitive data such as passwords, API keys, TLS certificates etc, we
can use Secrets to do that
• Similar to ConfigMaps, Secrets store data as key-value pairs
• The data is stored in Base64-encoded format
• Take note that the data is encoded and not encrypted

Copyright © Dell Inc. All Rights Reserved.


Create the secret the imperative way

$ kubectl create secret generic mysql-secret --from-literal=MYSQL_ROOT_PASSWORD=secretpassword

secret/mysql-secret created

We need to specify the secret type here. The type generic is


for creating secret from a file or literal value. There are other
types we will cover later in the topic.

Copyright © Dell Inc. All Rights Reserved.


View the secret YAML output

$ kubectl get secret mysql-secret -o yaml

apiVersion: v1
data:
MYSQL_ROOT_PASSWORD: c2VjcmV0cGFzc3dvcmQ=
The password is automatically encoded
kind: Secret in base64 format when we create the
metadata: secret using literals the imperative way
creationTimestamp: "2024-03-05T01:16:19Z"
name: mysql-secret
namespace: default
resourceVersion: "38324"
uid: 63340e3a-97cb-46d2-8f72-b16711fd4895
type: Opaque The type Opaque is for generic sensitive data
when we create the secret as generic in the
kubectl command

Copyright © Dell Inc. All Rights Reserved.


We can decode the password easily

$ echo c2VjcmV0cGFzc3dvcmQ= | base64 --decode

secretpassword

Copyright © Dell Inc. All Rights Reserved.


The Secret YAML manifest

apiVersion: v1

kind: Secret

metadata:
name: mysql-secret
If we are creating the secret declaratively
data:
using the YAML manifest, we have to
MYSQL_ROOT_PASSWORD: c2VjcmV0cGFzc3dvcmQ=
encode the value and define here
type: Opaque

Copyright © Dell Inc. All Rights Reserved.


We can encode the password in any Unix-based CLI

$ echo secretpassword | base64

c2VjcmV0cGFzc3dvcmQK

Copyright © Dell Inc. All Rights Reserved.


Alternatively, we can use the stringData attribute

apiVersion: v1

kind: Secret

metadata:
name: mysql-secret

stringData: If we use stringData attribute instead of


data, we can assign plain-text values
MYSQL_ROOT_PASSWORD: secretpassword

type: Opaque

Copyright © Dell Inc. All Rights Reserved.


Delete the previous pod

$ kubectl delete pod mydatabase

pod "mydatabase" deleted

Copyright © Dell Inc. All Rights Reserved.


Consume the secret as environment variables

apiVersion: v1 apiVersion: v1

kind: Pod kind: Pod

metadata: metadata:
name: mydatabase name: mydatabase

spec: spec:
containers: containers:
- name: mydatabase - name: mydatabase
image: mysql image: mysql
envFrom: envFrom:
- secretRef: - configMapRef:
name: mysql-secret name: mysql-configmap

The key-value pair in the secret is injected as


an environment variable in the container

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our pod

$ vim mysql-secret-pod.yaml

apiVersion: v1
kind: Pod
metadata:
name: mydatabase
spec:
containers:
- name: mydatabase
image: mysql
envFrom:
- secretRef:
name: mysql-secret

Copyright © Dell Inc. All Rights Reserved.


Create the pod with the manifest

$ kubectl apply -f mysql-secret-pod.yaml

pod/mydatabase created

Copyright © Dell Inc. All Rights Reserved.


List and watch the pod

$ kubectl get pod mydatabase --watch

NAME READY STATUS RESTARTS AGE


mydatabase 0/1 ContainerCreating 0 5s
mydatabase 1/1 Running 0 8s

Copyright © Dell Inc. All Rights Reserved.


Access the pod to check the environment variables

$ kubectl exec mydatabase -- env

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=mydatabase
GOSU_VERSION=1.16
MYSQL_MAJOR=innovation The secret value is automatically decoded
MYSQL_VERSION=8.3.0-1.el8 and injected as an environment variable in
MYSQL_SHELL_VERSION=8.3.0-1.el8 the container
MYSQL_ROOT_PASSWORD=secretpassword
KUBERNETES_PORT=tcp://10.43.0.1:443
KUBERNETES_PORT_443_TCP=tcp://10.43.0.1:443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_ADDR=10.43.0.1
KUBERNETES_SERVICE_HOST=10.43.0.1
KUBERNETES_SERVICE_PORT=443
KUBERNETES_SERVICE_PORT_HTTPS=443
HOME=/root

Copyright © Dell Inc. All Rights Reserved.


We can also mount a secret as a volume

$ vim database.json

{
“database”: { We create a database configuration
file that can be read by the app in
“host”: “mysql-service”, the container
“user”: “dba”
}

Copyright © Dell Inc. All Rights Reserved.


Create the secret the imperative way

$ kubectl create secret generic mysql-json-secret --from-file=database.json

secret/mysql-json-secret created

Location of the file to be loaded as a


secret

Copyright © Dell Inc. All Rights Reserved.


View the secret YAML output

$ kubectl get secret mysql-json-secret -o yaml

apiVersion: v1
data:
database.json:
ewogIOKAnGRhdGFiYXNl4oCdOiB7CiAgICDigJxob3N04oCdOiDigJxteXNxbC1zZXJ2aWNl4oC
dLAogICAg4oCcdXNlcuKAnTog4oCcZGJh4oCdCn0KCg==
kind: Secret
metadata:
creationTimestamp: "2024-03-05T02:11:15Z"
name: mysql-json-secret
namespace: default
resourceVersion: "40325"
uid: 0e2f8f43-86b5-4795-adda-6ef672a2e297
type: Opaque

Copyright © Dell Inc. All Rights Reserved.


Mount the secret as a volume
apiVersion: v1

kind: Pod

metadata:
name: mydatabase

spec:
volumes: We create a volume and reference it to the secret. Take note
- name: mysql-secret-volume that we use secretName to reference the secret, unlike
secret:
ConfigMap which uses name
secretName: mysql-json-secret

containers:
- name: mydatabase
image: mysql
volumeMounts: We mount the volume to the filepath /etc/config of the
- name: mysql-secret-volume container. Take note that we set readOnly to true as files
mountPath: /etc/config provided by secret mounted as volume cannot be modified
readOnly: true
envFrom:
- secretRef:
name: mysql-secret

Copyright © Dell Inc. All Rights Reserved.


Delete the previous pod

$ kubectl delete pod mydatabase

pod "mydatabase" deleted

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our pod

$ vim mysql-volumemount-secret-pod.yaml

apiVersion: v1 containers:
- name: mydatabase
kind: Pod image: mysql

metadata: volumeMounts:
name: mydatabase - name: mysql-secret-volume
mountPath: /etc/config
spec: readOnly: true
volumes:
- name: mysql-secret-volume envFrom:
secret: - secretRef:
secretName: mysql-json-secret name: mysql-secret

Copyright © Dell Inc. All Rights Reserved.


Create the pod with the manifest

$ kubectl apply -f mysql-volumemount-secret-pod.yaml

pod/mydatabase created

Copyright © Dell Inc. All Rights Reserved.


List and watch the pod

$ kubectl get pod mydatabase --watch

NAME READY STATUS RESTARTS AGE


mydatabase 0/1 ContainerCreating 0 5s
mydatabase 1/1 Running 0 8s

Copyright © Dell Inc. All Rights Reserved.


Access the pod to validate the mounted volume

$ kubectl exec -it mydatabase -- /bin/sh

sh-4.4#

Copyright © Dell Inc. All Rights Reserved.


Access the pod to validate the mounted volume

$ kubectl exec -it mydatabase -- /bin/sh

sh-4.4# cat /etc/config/database.json

{
“database”: {
“host”: “mysql-service”,
“user”: “dba”
}

Copyright © Dell Inc. All Rights Reserved.


Let’s clean up

$ kubectl delete pod mydatabase

pod "mydatabase" deleted

Copyright © Dell Inc. All Rights Reserved.


What we have done

MYSQL_ROOT_PASSWORD:
key: value secretpassword key: value

json
Container
Injected as
environment variables

Secret Mounted as volume to filesystem Pod Secret


path /etc/config
mysql-json-configmap mysql-secret
mydatabase

Copyright © Dell Inc. All Rights Reserved.


Recap

• ConfigMaps and Secrets provide a centralized method for managing your


application's configuration.
• Both ConfigMaps and Secrets can be injected into application containers,
enabling environment variable injection and volume mounting.
• ConfigMaps store configurations in plaintext, while Secrets use a Base64
encoded format to obscure sensitive values.
• While Secrets encode values, additional encryption methods are recommended
for enhanced security in production environments.

Copyright © Dell Inc. All Rights Reserved.


Persisting Data with Volumes

Copyright © Dell Inc. All Rights Reserved.


Learning Objectives

At the end of this topic, you will be able to:


• Learn the importance of persisting data through volumes
• Gain insights into the various volume types available in Kubernetes
• Develop a practical understanding of how to configure and use these volumes
within your Kubernetes deployments to effectively manage application data

Copyright © Dell Inc. All Rights Reserved.


Decoupling Data from Pods

• Containers and Pods are ephemeral in nature


• Each container in a pod has its own isolated filesystem
• Any data that is written to the container filesystem is temporary
• If the pod, node or cluster is shut down, the data is lost forever
• We need a persistent storage that can survive container restarts

Container
Persistent
Storage

Pod

Copyright © Dell Inc. All Rights Reserved.


PersistentVolumes and PersistentVolumeClaims

• PersistentVolume is the storage resource in a Kubernetes cluster


• Using PersistentVolumes, we can attach stateful storage to any
pods
• This can help us preserve data beyond the lifespan of the pod
• PersistentVolumes can be provided by a Kubernetes administrator
or assigned dynamically by mapping to a StorageClass
• PersistentVolumeClaim requests the resources of the
PersistentVolume

Copyright © Dell Inc. All Rights Reserved.


Managing Volumes in a platform-agnostic way

• Disk resources are represented in Kubernetes as


PersistentVolumes (similar to how physical/virtual machines
are represented as Nodes)

App

PersistentVolumeClaim (PVC)

PersistentVolume (PV)
Cluster

Physical Storage

Copyright © Dell Inc. All Rights Reserved.


Creating PersistentVolumeClaims
• After we have a PersistentVolume, we need to create a
PersistentVolumeClaim
• We then bind the PersistentVolumeClaim to the Pod by
mounting it as a volume

External Storage
Container

VolumeMount PersistentVolumeClaim
Volume

Pod PersistentVolume

Copyright © Dell Inc. All Rights Reserved.


The PersistentVolume YAML manifest

apiVersion: v1

kind: PersistentVolume

metadata:
name: myvolume

spec:
capacity: Allocate storage capacity size to 100Mi
storage: 100Mi

accessModes: Read/write access by a single node


- ReadWriteOnce

hostPath:
path: /data Mounts the directory /data from worker node’s
filesystem

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our persistentvolume

$ vim persistentvolume.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
name: myvolume
spec:
capacity:
storage: 100Mi
accessModes:
- ReadWriteOnce
hostPath:
path: /data

Copyright © Dell Inc. All Rights Reserved.


Create the persistentvolume with the manifest

$ kubectl apply -f persistentvolume.yaml

persistentvolume/myvolume created

Copyright © Dell Inc. All Rights Reserved.


List the persistentvolume

$ kubectl get persistentvolume myvolume

NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
myvolume 100Mi RWO Retain Available 6s

Reclaim policy for the volume.


ReadWriteOnce access mode
Retain is the default

Copyright © Dell Inc. All Rights Reserved.


Common Types of Volumes
Type Description

emptyDir A simple empty directory used for storing transient data

hostPath Used for mounting directories from worker node’s filesystem to the pod

gitRepo A volume initialized by checking out the contents of a Git repository

nfs A NFS share mounted to the filesystem of the pod

configMap, secret Kubernetes configuration resources mounted to the filesystem of the pod

persistentVolumeClaim Using pre or dynamically provisioned persistent storage

Copyright © Dell Inc. All Rights Reserved.


PersistentVolume Access Modes
Type Short Form Description

ReadWriteOnce RWO Read/write access by a single node

ReadOnlyMany ROX Read-only access by many nodes

ReadWriteMany RWX Read/write access by many nodes

ReadWriteOncePod RWOP Read/write access mounted by a single pod

Copyright © Dell Inc. All Rights Reserved.


PersistentVolume Reclaim Policies
Type Description

Retain Default. When a PersistentVolumeClaim is deleted, the PersistentVolume is not


automatically deleted. Instead, it remains within the cluster and retains its data.

Requires manual intervention to reclaim the volume for future use

Delete When the PersistentVolumeClaim is deleted, remove the PersistentVolume and its
associated storage

Recycle Deprecated

Copyright © Dell Inc. All Rights Reserved.


StorageClass
apiVersion: storage.k8s.io/v1

kind: StorageClass
Determines where and how this storage
Administrator Developer metadata: is set up. Different storage has its own
name: ebs-sc provisioner.

provisioner: ebs.csi.aws.com

volumeBindingMode: WaitForFirstConsumer

parameters:
csi.storage.k8s.io/fstype: xfs
type: io1
iopsPerGB: “50”
...

Copyright © Dell Inc. All Rights Reserved.


The PersistentVolumeClaim YAML manifest

apiVersion: v1

kind: PersistentVolumeClaim

metadata:
name: myvolumeclaim

spec: Specifying an empty string as the storageClassName


accessModes: ensures the persistent volume claim binds to a
- ReadWriteOnce pre-provisioned persistent volume

storageClassName: ""

resources:
requests:
storage: 100Mi

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our persistentvolumeclaim

$ vim persistentvolumeclaim.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: myvolumeclaim
spec:
accessModes:
- ReadWriteOnce
storageClassName: ""
resources:
requests:
storage: 100Mi

Copyright © Dell Inc. All Rights Reserved.


Create the persistentvolumeclaim with the manifest

$ kubectl apply -f persistentvolumeclaim.yaml

persistentvolumeclaim/myvolumeclaim created

Copyright © Dell Inc. All Rights Reserved.


List the persistentvolumeclaim

$ kubectl get persistentvolumeclaim myvolumeclaim

NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE


myvolumeclaim Bound myvolume 100Mi RWO 7s

Bound means binding to the


ReadWriteOnce access mode
PersistentVolume is successful

Copyright © Dell Inc. All Rights Reserved.


Describe the persistentvolumeclaim

$ kubectl describe persistentvolumeclaim myvolumeclaim

Name: myvolumeclaim
Namespace: default
StorageClass:
Status: Bound
Volume: myvolume
Labels: <none>
Annotations: pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 100Mi
Access Modes: RWO
VolumeMode: Filesystem
Used By: <none> This indicates that the persistentvolumeclaim is not
Events: <none> used by any pods yet

Copyright © Dell Inc. All Rights Reserved.


Mount the persistentvolumeclaim as a volume
apiVersion: v1

kind: Pod

metadata:
name: web

spec:
volumes:
- name: mystorage We create a volume and reference it to the
persistentVolumeClaim: persistentvolumeclaim
claimName: myvolumeclaim

containers:
- name: web
image: nginx
volumeMounts: We mount the volume to the filepath
- name: mystorage
/mnt/data of the container
mountPath: /mnt/data

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our pod

$ vim nginx-pvc-pod.yaml

apiVersion: v1 containers:
- name: web
kind: Pod image: nginx

metadata: volumeMounts:
name: web - name: mystorage
mountPath: /mnt/data
spec:
volumes:
- name: mystorage
persistentVolumeClaim:
claimName: myvolumeclaim

Copyright © Dell Inc. All Rights Reserved.


Create the pod with the manifest

$ kubectl apply -f nginx-pvc-pod.yaml

pod/web created

Copyright © Dell Inc. All Rights Reserved.


List and watch the pod

$ kubectl get pod web --watch

NAME READY STATUS RESTARTS AGE


web 1/1 Running 0 24s

Copyright © Dell Inc. All Rights Reserved.


Access the pod to validate the mounted volume

$ kubectl exec -it web -- /bin/sh

Copyright © Dell Inc. All Rights Reserved.


Create a file message to /mnt/data

$ kubectl exec -it web -- /bin/sh

# echo “I love pvc!” > /mnt/data/message

Copyright © Dell Inc. All Rights Reserved.


Describe the persistentvolumeclaim

$ kubectl describe persistentvolumeclaim myvolumeclaim

Name: myvolumeclaim
Namespace: default
StorageClass:
Status: Bound
Volume: myvolume
Labels: <none>
Annotations: pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 100Mi
Access Modes: RWO
VolumeMode: Filesystem
Used By: web This indicates that the persistentvolumeclaim is
Events: <none> being mounted and used by one pod named web

Copyright © Dell Inc. All Rights Reserved.


Delete the pod

$ kubectl delete pod web

pod "web" deleted

Copyright © Dell Inc. All Rights Reserved.


Create the pod with the manifest again

$ kubectl apply -f nginx-pvc-pod.yaml

pod/web created

Copyright © Dell Inc. All Rights Reserved.


List and watch the pod

$ kubectl get pod web --watch

NAME READY STATUS RESTARTS AGE


web 1/1 Running 0 24s

Copyright © Dell Inc. All Rights Reserved.


Access the pod to validate the mounted volume

$ kubectl exec -it web -- /bin/sh

Copyright © Dell Inc. All Rights Reserved.


View the message in /mnt/data

$ kubectl exec -it web -- /bin/sh

# cat /mnt/data/message

“I love pvc!”

Copyright © Dell Inc. All Rights Reserved.


Recap

• PersistentVolumes (PVs) provide persistent storage solutions that outlive the


lifespan of pods, ensuring data durability and availability
• StorageClasses automate PV provisioning, eliminating the need for manual
admin intervention for each PVC.
• PVs support access modes like ReadWriteOnce, ReadOnlyMany, and
ReadWriteMany, controlling how nodes and pods access volumes.

Copyright © Dell Inc. All Rights Reserved.


Managing Namespaces and
Resource Quotas

Copyright © Dell Inc. All Rights Reserved.


Learning Objectives

At the end of this topic, you will be able to:


• Understand namespaces and their role in organising and partitioning resources
within a cluster
• Explore resource quotas as a mechanism for controlling and allocating
resources within Kubernetes namespaces
• Understand how resource quotas define usage limits for CPU, memory, storage
and other resources

Copyright © Dell Inc. All Rights Reserved.


Namespaces

• Kubernetes uses namespaces to organise resources in the cluster


• Namespaces provide scopes for the resources such as pods,
replicasets, deployments, services etc
• A common use case of namespaces is to segregate different
software environments such as development, testing and production
• By default, Kubernetes resources are deployed into the default
namespace if we do not explicitly specify the namespace

Copyright © Dell Inc. All Rights Reserved.


Namespaces

• To specify the namespace, we pass kubectl the --namespace or -n


flag
• To specify all the namespaces in the cluster, we pass kubectl the
--all-namespaces or -A flag

Copyright © Dell Inc. All Rights Reserved.


List all resources in the default namespace

$ kubectl get all

NAME READY STATUS RESTARTS AGE


pod/web 1/1 Running 0 2m26s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE


service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 5d10h

Copyright © Dell Inc. All Rights Reserved.


List all resources in the default namespace

$ kubectl get all --namespace default

NAME READY STATUS RESTARTS AGE


pod/web 1/1 Running 0 2m26s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE


service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 5d10h

Copyright © Dell Inc. All Rights Reserved.


List all resources in the default namespace

$ kubectl get all -n default

NAME READY STATUS RESTARTS AGE


pod/web 1/1 Running 0 2m26s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE


service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 5d10h

Copyright © Dell Inc. All Rights Reserved.


Let’s create a namespace

$ kubectl create namespace dev

namespace/dev created

Copyright © Dell Inc. All Rights Reserved.


List all namespaces

$ kubectl get namespaces

NAME STATUS AGE


default Active 15d
kube-system Active 15d
kube-public Active 15d
kube-node-lease Active 15d
dev Active 42s

Copyright © Dell Inc. All Rights Reserved.


A Namespace YAML manifest

apiVersion: v1

kind: Namespace

metadata:
name: dev

Copyright © Dell Inc. All Rights Reserved.


Let’s deploy a pod in the dev namespace

$ kubectl run web --image=nginx -n dev

pod/web created

Copyright © Dell Inc. All Rights Reserved.


List the pod in the dev namespace

$ kubectl get pod web -n dev

NAME READY STATUS RESTARTS AGE


web 1/1 Running 0 42s

Copyright © Dell Inc. All Rights Reserved.


What happens when we deploy a pod with the same name in
the dev namespace?

$ kubectl run web --image=nginx -n dev

Error from server (AlreadyExists): pods "web" already exists

Copyright © Dell Inc. All Rights Reserved.


Let’s create another namespace

$ kubectl create namespace test

namespace/test created

Copyright © Dell Inc. All Rights Reserved.


Let’s deploy a pod in the test namespace

$ kubectl run web --image=nginx -n test

pod/web created

We can create resources with the same name in different namespaces.


Namespaces help us to avoid naming collisions and represent a scope for
object names

Copyright © Dell Inc. All Rights Reserved.


List the pod in the test namespace

$ kubectl get pod web -n test

NAME READY STATUS RESTARTS AGE


web 1/1 Running 0 42s

Copyright © Dell Inc. All Rights Reserved.


Set a namespace preference

$ kubectl config set-context --current --namespace=test

Context "k3d-mycluster" modified.

Copyright © Dell Inc. All Rights Reserved.


List the pod in the current namespace

$ kubectl get pod web

NAME READY STATUS RESTARTS AGE


web 1/1 Running 0 42s

Copyright © Dell Inc. All Rights Reserved.


Switch back to default namespace

$ kubectl config set-context --current --namespace=default

Context "k3d-mycluster" modified.

Copyright © Dell Inc. All Rights Reserved.


Delete a namespace

$ kubectl delete namespace dev

namespace "dev" deleted

Deleting a namespace will automatically delete all the objects in that


namespace

Copyright © Dell Inc. All Rights Reserved.


List the pod in the dev namespace

$ kubectl get pod web -n dev

No resources found in dev namespace.

Copyright © Dell Inc. All Rights Reserved.


YAML manifest of a Deployment running nginx
apiVersion: apps/v1

kind: Deployment

metadata:
name: web-deploy
namespace: test To create the deployment in the namespace,
we just need to add this under the metadata
spec: section
replicas: 3

selector:
matchLabels:
app: web

template:
metadata:
labels:
app: web
spec:
containers:
- name: web
image: nginx
Copyright © Dell Inc. All Rights Reserved.
Create a YAML manifest for our deployment

$ vim nginx-deployment-test-namespace.yaml

apiVersion: apps/v1
selector:
kind: Deployment matchLabels:
app: web
metadata:
name: web-deploy template:
namespace: test metadata:
labels:
spec: app: web
replicas: 3 spec:
containers:
- name: web
image: nginx

Copyright © Dell Inc. All Rights Reserved.


Create the deployment with the manifest

$ kubectl apply -f nginx-deployment-test-namespace.yaml

deployment.apps/web-deploy configured

Copyright © Dell Inc. All Rights Reserved.


List all the objects in the test namespace

$ kubectl get all -n test

NAME READY STATUS RESTARTS AGE


pod/web 1/1 Running 0 14d
pod/web-deploy-7869bd98c5-nztf7 1/1 Running 0 14d
pod/web-deploy-7869bd98c5-wfqrj 1/1 Running 0 14d
pod/web-deploy-7869bd98c5-5k68c 1/1 Running 0 14d

NAME READY UP-TO-DATE AVAILABLE AGE


deployment.apps/web-deploy 3/3 3 3 14d

NAME DESIRED CURRENT READY AGE


replicaset.apps/web-deploy-7869bd98c5 3 3 3 14d

Copyright © Dell Inc. All Rights Reserved.


Delete a namespace

$ kubectl delete namespace test

namespace "test" deleted

Deleting a namespace will automatically delete all the objects in that


namespace

Copyright © Dell Inc. All Rights Reserved.


List all the objects in the test namespace

$ kubectl get all -n test

No resources found in test namespace.

Copyright © Dell Inc. All Rights Reserved.


Defining Resource Utilisation

• Application workloads executed in pods consume resources


such as CPU and memory
• Kubernetes allows us to define the minimum amount of
resources needed to run the application and the maximum
amount of resources the application is allowed to consume at
the container level
• We can use the ResourceQuota to define and aggregate
resource constraints on a namespace level

Copyright © Dell Inc. All Rights Reserved.


ResourceQuota

• The ResourceQuota can be created to define the usable,


maximum amount of resources on the namespace level
• The Kubernetes scheduler will ensure and take care of
enforcing those defined rules
• Examples of rules are setting
▪ an upper limit for a specific object type that can be created (e.g.
maximum of 3 pods)
▪ the total sum of compute resources (e.g. 1024Mi of RAM)

Copyright © Dell Inc. All Rights Reserved.


YAML manifest of a ResourceQuota
apiVersion: v1

kind: ResourceQuota
Limit the number of pods that can be
metadata: created in the namespace to 2
name: web-quota
namespace: staging

spec:
hard:
pods: 2

Copyright © Dell Inc. All Rights Reserved.


Create a staging namespace

$ kubectl create namespace staging

namespace/staging created

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our resourcequota

$ vim web-resourcequota.yaml

apiVersion: v1

kind: ResourceQuota

metadata:
name: web-quota
namespace: staging

spec:
hard:
pods: 2

Copyright © Dell Inc. All Rights Reserved.


Create the resourcequota with the manifest

$ kubectl apply -f web-resourcequota.yaml

resourcequota/web-quota created

Copyright © Dell Inc. All Rights Reserved.


View the resourcequota details

$ kubectl describe resourcequota web-quota -n staging

Name: web-quota
Namespace: staging
Resource Used Hard
-------- ---- ----
pods 0 2

Copyright © Dell Inc. All Rights Reserved.


Let’s create a deployment with more than 2 replicas
$ kubectl create deployment test-deploy --image=nginx --replicas=3
-n staging

deployment.apps/test-deploy created

Copyright © Dell Inc. All Rights Reserved.


List all pods

$ kubectl get pods -n staging

NAME READY STATUS RESTARTS AGE


test-deploy-859f95ffcc-77vkx 1/1 Running 0 50s
test-deploy-859f95ffcc-s6fwp 1/1 Running 0 50s

Only 2 pods are created in this deployment

Copyright © Dell Inc. All Rights Reserved.


List our deployment

$ kubectl get deployment test-deploy -n staging

NAME READY UP-TO-DATE AVAILABLE AGE


test-deploy 2/3 2 2 5s

2 pods are created in this deployment

Copyright © Dell Inc. All Rights Reserved.


Describe our deployment

$ kubectl describe deployment test-deploy -n staging

Name: test-deploy
Namespace: staging
CreationTimestamp: Fri, 22 Mar 2024 01:24:53 +0000
Labels: app=test-limit
Annotations: deployment.kubernetes.io/revision: 1
Selector: app=test-limit
Replicas: 3 desired | 2 updated | 2 total | 2 available | 1 unavailable
...

Copyright © Dell Inc. All Rights Reserved.


Describe our deployment

$ kubectl describe deployment test-deploy -n staging

...

Conditions:
Type Status Reason
---- ------ ------
Available False MinimumReplicasUnavailable
ReplicaFailure True FailedCreate
Progressing True ReplicaSetUpdated

...

Copyright © Dell Inc. All Rights Reserved.


View events

$ kubectl get events -n staging

...
3m25s Normal Pulled pod/test-deploy-6d9f8d87cf-nvd5d Successfully pulled
image "nginx" in 1.943434388s (1.943443685s including waiting)
3m25s Normal Created pod/test-deploy-6d9f8d87cf-xkndn Created container
nginx
3m25s Normal Created pod/test-deploy-6d9f8d87cf-nvd5d Created container
nginx
3m25s Normal Started pod/test-deploy-6d9f8d87cf-xkndn Started container
nginx
3m25s Normal Started pod/test-deploy-6d9f8d87cf-nvd5d Started container
nginx
2m1s Warning FailedCreate replicaset/test-deploy-6d9f8d87cf (combined from similar
events): Error creating: pods "test-deploy-6d9f8d87cf-cnr2c" is forbidden: exceeded quota: web-quota,
requested: pods=1, used: pods=2, limited: pods=2
...

Copyright © Dell Inc. All Rights Reserved.


Include min and max resources in ResourceQuota
apiVersion: v1

kind: ResourceQuota

metadata:
name: web-quota
namespace: staging

spec: The minimum resources requested


hard: across all pods
pods: 2

requests.cpu: ”1”
requests.memory: 1024Mi The maximum resources that can be
used by all pods
limits.cpu: ”2”
limits.memory: 2048Mi

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our new resourcequota

$ vim web-resourcequota-limit.yaml

apiVersion: v1

kind: ResourceQuota
We keep the same name to override the
metadata: resourcequota we have created previously
name: web-quota
namespace: staging

spec:
hard:
pods: 2

requests.cpu: ”1”
requests.memory: 1024Mi

limits.cpu: ”2”
limits.memory: 2048Mi

Copyright © Dell Inc. All Rights Reserved.


Update the resourcequota with the manifest

$ kubectl apply -f web-resourcequota-limit.yaml

resourcequota/web-quota configured

Copyright © Dell Inc. All Rights Reserved.


View the resourcequota details

$ kubectl describe resourcequota web-quota -n staging

Name: web-quota
Namespace: staging
Resource Used Hard
-------- ---- ----
limits.cpu 0 2
limits.memory 0 2Gi
pods 0 2
requests.cpu 0 1
requests.memory 0 1Gi

Copyright © Dell Inc. All Rights Reserved.


Delete the deployment

$ kubectl delete deployment test-deploy -n staging

deployment.apps "test-deploy" deleted

Copyright © Dell Inc. All Rights Reserved.


Create a new deployment

$ kubectl create deployment test-deploy --image=nginx -n staging

deployment.apps/test-deploy created

Copyright © Dell Inc. All Rights Reserved.


List our deployment

$ kubectl get deployment test-deploy -n staging

NAME READY UP-TO-DATE AVAILABLE AGE


test-deploy 0/1 0 0 5s

No pods are created in this deployment

Copyright © Dell Inc. All Rights Reserved.


Describe our deployment

$ kubectl describe deployment test-deploy -n staging

Name: test-deploy
Namespace: staging
CreationTimestamp: Mon, 25 Mar 2024 00:01:22 +0000
Labels: app=test-deploy
Annotations: deployment.kubernetes.io/revision: 1
Selector: app=test-deploy
Replicas: 1 desired | 0 updated | 0 total | 0 available | 1 unavailable
...

Copyright © Dell Inc. All Rights Reserved.


Describe our deployment

$ kubectl describe deployment test-deploy -n staging

...

Conditions:
Type Status Reason
---- ------ ------
Progressing True NewReplicaSetCreated
Available False MinimumReplicasUnavailable
ReplicaFailure True FailedCreate
OldReplicaSets: <none>
NewReplicaSet: test-deploy-6d9f8d87cf (0/1 replicas created)

...

...

Copyright © Dell Inc. All Rights Reserved.


View events

$ kubectl get events -n staging

quota: web-quota: must specify limits.cpu for: nginx; limits.memory for: nginx; requests.cpu for: nginx;
requests.memory for: nginx
117s Warning FailedCreate replicaset/test-deploy-6d9f8d87cf Error creating: pods
"test-deploy-6d9f8d87cf-qdq6q" is forbidden: failed quota: web-quota: must specify limits.cpu for: nginx;
limits.memory for: nginx; requests.cpu for: nginx; requests.memory for: nginx
116s Warning FailedCreate replicaset/test-deploy-6d9f8d87cf Error creating: pods
"test-deploy-6d9f8d87cf-6wnwh" is forbidden: failed quota: web-quota: must specify limits.cpu for: nginx;
limits.memory for: nginx; requests.cpu for: nginx; requests.memory for: nginx
116s Warning FailedCreate replicaset/test-deploy-6d9f8d87cf Error creating: pods
"test-deploy-6d9f8d87cf-vljng" is forbidden: failed quota: web-quota: must specify limits.cpu for: nginx;
limits.memory for: nginx; requests.cpu for: nginx; requests.memory for: nginx
35s Warning FailedCreate replicaset/test-deploy-6d9f8d87cf (combined from similar
events): Error creating: pods "test-deploy-6d9f8d87cf-jrk42" is forbidden: failed quota: web-quota: must
specify limits.cpu for: nginx; limits.memory for: nginx; requests.cpu for: nginx; requests.memory for:
nginx

Copyright © Dell Inc. All Rights Reserved.


Delete the deployment

$ kubectl delete deployment test-deploy -n staging

deployment.apps "test-deploy" deleted

Copyright © Dell Inc. All Rights Reserved.


Include requests and limits in Deployment YAML
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-deploy
namespace: staging

spec:
replicas: 1

selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web

spec:
containers: Include the requests and limits for the resources
- name: web
image: nginx
resources:
requests:
cpu: "1"
memory: "1Gi"
limits:
cpu: "2"
memory: "2Gi"

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our deployment

$ vim nginx-deployment-test-namespace-limit.yaml

apiVersion: apps/v1 template:


metadata:
kind: Deployment labels:
app: web
metadata: spec:
name: web-deploy containers:
namespace: staging - name: web
image: nginx
spec: resources:
replicas: 1 requests:
cpu: “1”
selector: memory: “1Gi”
matchLabels: limits:
app: web cpu: “2”
memory: “2Gi”

Copyright © Dell Inc. All Rights Reserved.


Create the deployment with the manifest

$ kubectl apply -f nginx-deployment-test-namespace-limit.yaml

deployment.apps/web-deploy configured

Copyright © Dell Inc. All Rights Reserved.


Recap

• Namespaces organise Kubernetes resources by providing separate scopes


within each namespace
• ResourceQuota enforces the maximum resource usage for each namespace,
ensuring efficient and balanced resource allocation
• Once ResourceQuota is set, all Pods and Deployments in that namespace must
specify their required resources to stay within the defined limits

Copyright © Dell Inc. All Rights Reserved.


Best Practices and Troubleshooting

Copyright © Dell Inc. All Rights Reserved.


Learning Objectives

At the end of this topic, you will be able to:


• Understand and implement best practices to ensure the successful operation,
management, and scalability of your Kubernetes cluster
• Identify and troubleshoot Kubernetes-related issues effectively
• Understand the importance of resource requests and limits, namespace usage,
proper labeling and annotations, and maintaining security best practices

Copyright © Dell Inc. All Rights Reserved.


Best Practices

• Use of labels and annotations


• Resource requests and limits
• Deployments over pods
• Immutable containers
• Namespace isolation

Copyright © Dell Inc. All Rights Reserved.


Use of labels and annotations

• Tag Kubernetes resources with labels for better organisation


and management
• This allows efficient querying and grouping of resources by
using label selectors to filter resources based on specific
criteria
• Annotations are similar to labels but is used to provide
additional context by attaching non-identifying metadata to
Kubernetes resources

Copyright © Dell Inc. All Rights Reserved.


Resource requests and limits

• Set resource requests and limits for CPU and memory to ensure fair
resource allocation and avoid resource contention
• Resource requests specify the minimum amount of CPU and
memory that a container requires to run. Kubernetes can schedule
pods on nodes that have sufficient capacity to meet these
requirements
• Resource limits protect the overall health of the Kubernetes cluster
by preventing individual pods from consuming excessive resources

Copyright © Dell Inc. All Rights Reserved.


Favour deployments over pods

• Prefer using deployments over directly managing pods to have


features like rolling updates and rollbacks
• Kubernetes automatically manages the rollout process by
gradually replacing old pods with new ones to ensure smooth
transitions and minimising disruption to the services
• Deployments support horizontal scaling by adjusting the number
of replicas based on resource utilisation or custom metrics

Copyright © Dell Inc. All Rights Reserved.


Immutable containers

• Treat your containers as immutable and avoid making changes


directly in production
• Immutable containers ensure consistency and predictability in your
application environment
• It enhances security by minimising the attack surface and reducing
the risk of vulnerabilities
• Rebuild and redeploy the containers when updates are needed

Copyright © Dell Inc. All Rights Reserved.


Namespace isolation

• Use namespaces to isolate resources and logically separate


different environments such as development, staging and
production
• Kubernetes Role-Based Access Control (RBAC) can be applied at
the namespace level to control access to resources within each
namespace
• Namespace level RBAC enhances security and compliance by
enforcing the principle of least privilege by ensuring that only
authorised users can interact with resources in a given namespace

Copyright © Dell Inc. All Rights Reserved.


Troubleshooting Approach

• Identify the root cause of an issue


• Inspect resource status and events
• Examine pod and container logs
• Review resource quotas and limits
• Take action and correct the failure accordingly

Copyright © Dell Inc. All Rights Reserved.


Troubleshooting

• kubectl
• logs
• exec -it
• cp
• get events
• top nodes
• top pods

Copyright © Dell Inc. All Rights Reserved.


No Available Nodes for Scheduling

• When a pod is created, the scheduler assigns it to a node


• Sometimes, the scheduler is not able to find an available node
for scheduling
• This could be due to reasons like
– the cluster does not have any nodes configured
– all the nodes have failed

Copyright © Dell Inc. All Rights Reserved.


Pod Definition with Node Selector

apiVersion: v1

kind: Pod

metadata:
name: web

spec: We simulate a scenario in which no nodes are


containers: available for assignment by specifying a node
- name: web selector `tier: backend`. This means that the pod
image: nginx can only be assigned to a node with this node
label, which in our case does not match any of
the current nodes we have
nodeSelector:
tier: backend

Copyright © Dell Inc. All Rights Reserved.


Create a YAML manifest for our pod

$ vim nginx-nodeselector-pod.yaml

apiVersion: v1
kind: Pod
metadata:
name: web
spec:
containers:
- name: web
image: nginx
nodeSelector:
tier: backend

Copyright © Dell Inc. All Rights Reserved.


Create the pod with the manifest

$ kubectl apply -f nginx-nodeselector-pod.yaml

pod/web created

Copyright © Dell Inc. All Rights Reserved.


List the pod

$ kubectl get pod web -o wide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
web 0/1 Pending 0 3m40s <none> <none> <none> <none>

Pod is in Pending state for some time No node has been assigned to

Copyright © Dell Inc. All Rights Reserved.


View detailed status of the pod

$ kubectl describe pod web

Name: web
Namespace: default
Priority: 0
Service Account: default
Node: <none>
Labels: <none>
Annotations: <none> Failed scheduling due to no available nodes for
Status: Pending scheduling
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 5m34s default-scheduler 0/4 nodes are available: 1 node(s) had untolerated taint
{node-role.kubernetes.io/master: }, 3 node(s) didn't match Pod's node affinity/selector. preemption: 0/4 nodes are
available: 4 Preemption is not helpful for scheduling..
Warning FailedScheduling 27s default-scheduler 0/4 nodes are available: 1 node(s) had untolerated taint
{node-role.kubernetes.io/master: }, 3 node(s) didn't match Pod's node affinity/selector. preemption: 0/4 nodes are
available: 4 Preemption is not helpful for scheduling..

Copyright © Dell Inc. All Rights Reserved.


List all nodes in our cluster

$ kubectl get nodes --show-labels

NAME STATUS ROLES AGE VERSION LABELS

k3d-mycluster-agent-1 Ready <none> 5d15h v1.27.4+k3s1


beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=k3s,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.
io/hostname=k3d-mycluster-agent-1,kubernetes.io/os=linux,node.kubernetes.io/instance-type=k3s

k3d-mycluster-agent-0 Ready <none> 5d15h v1.27.4+k3s1


beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=k3s,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.
io/hostname=k3d-mycluster-agent-0,kubernetes.io/os=linux,node.kubernetes.io/instance-type=k3s

k3d-mycluster-agent-2 Ready <none> 5d15h v1.27.4+k3s1


beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=k3s,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.
io/hostname=k3d-mycluster-agent-2,kubernetes.io/os=linux,node.kubernetes.io/instance-type=k3s

k3d-mycluster-server-0 Ready control-plane,master 5d15h v1.27.4+k3s1


beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=k3s,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.
io/hostname=k3d-mycluster-server-0,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=true,node-role.kubernetes.io/master
=true,node.kubernetes.io/instance-type=k3s

Note that there is no label `tier: backend`

Copyright © Dell Inc. All Rights Reserved.


Add the label to one of the nodes

$ kubectl label nodes k3d-mycluster-agent-1 tier=backend

node/k3d-mycluster-agent-1 labeled

Copyright © Dell Inc. All Rights Reserved.


List the pod

$ kubectl get pod web -o wide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
web 1/1 Running 0 16m 10.42.1.10 k3d-mycluster-agent-1 <none> <none>

Pod has been assigned to the node we


Pod is in Running state now have labelled previously

Copyright © Dell Inc. All Rights Reserved.


List all nodes in our cluster

$ kubectl get nodes --show-labels

NAME STATUS ROLES AGE VERSION LABELS

k3d-mycluster-agent-1 Ready <none> 5d15h v1.27.4+k3s1


beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=k3s,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.
io/hostname=k3d-mycluster-agent-1,kubernetes.io/os=linux,node.kubernetes.io/instance-type=k3s,tier=backend

k3d-mycluster-agent-0 Ready <none> 5d15h v1.27.4+k3s1


beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=k3s,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.
io/hostname=k3d-mycluster-agent-0,kubernetes.io/os=linux,node.kubernetes.io/instance-type=k3s

k3d-mycluster-agent-2 Ready <none> 5d15h v1.27.4+k3s1


beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=k3s,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.
io/hostname=k3d-mycluster-agent-2,kubernetes.io/os=linux,node.kubernetes.io/instance-type=k3s

k3d-mycluster-server-0 Ready control-plane,master 5d15h v1.27.4+k3s1


beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=k3s,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.
io/hostname=k3d-mycluster-server-0,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=true,node-role.kubernetes.io/master
=true,node.kubernetes.io/instance-type=k3s

The label `tier: backend`has been added to the node

Copyright © Dell Inc. All Rights Reserved.


Clean up by deleting our pod

$ kubectl delete pod web

pod "web" deleted

Copyright © Dell Inc. All Rights Reserved.


Clean up by removing the label from our node

$ kubectl label nodes k3d-mycluster-agent-1 tier-

node/k3d-mycluster-agent-1 unlabeled

We remove the label by appending a


minus sign to the label key

Copyright © Dell Inc. All Rights Reserved.


Recap

• Adopting Kubernetes best practices is essential for maximizing full potential of


its features
• Key practices include labeling, using Deployments over Pods, and using
immutable containers.
• Use kubectl commands like logs, exec, and get events to identify root causes of
errors
• Labeling nodes allows pods to directly schedule onto specific nodes

Copyright © Dell Inc. All Rights Reserved.

You might also like