Deploying Kubernetes on GCP with Kubespray
5 likes39,288 views
The document outlines a presentation on deploying Kubernetes using Kubespray, detailing the pros and cons of various tools such as kops, kubeadm, and kubo. It provides a step-by-step guide for deploying a Kubernetes cluster on Google Cloud Platform (GCP) using Terraform and includes commands for key tasks like generating SSH keys and running Ansible playbooks. Recommendations include using a release version of Kubespray and employing specific flags to enhance deployment reliability.
![Step 1: Create ssh keys
# Create new rsa keys
if [ ! -f cust_id_rsa ] ; then
ssh-keygen -t rsa -b 2048 -f cust_id_rsa -q -N ""
fi
# Load keys
eval $(ssh-agent)
ssh-add -D
ssh-add cust_id_rsa](https://image.slidesharecdn.com/deployingkubernetesongcpwithkubespray-180622040719/85/Deploying-Kubernetes-on-GCP-with-Kubespray-9-320.jpg)
![Step 3: Get cluster ips
bastion_ip=$(terraform output bastion_ip)
lb_ip=$(terraform output lb_ip)
master_ips=$(gcloud compute instances list --filter="${res_prefix}-masters" --format=json |jq -r
'.[].networkInterfaces[].networkIP'|tr "n" " "| sed -e "s/ {1,}$//")
node_ips=$(gcloud compute instances list --filter="${res_prefix}-workers" --format=json |jq -r
'.[].networkInterfaces[].networkIP'|tr "n" " "| sed -e "s/ {1,}$//")](https://image.slidesharecdn.com/deployingkubernetesongcpwithkubespray-180622040719/85/Deploying-Kubernetes-on-GCP-with-Kubespray-13-320.jpg)
![Step 5: Generate inventory
[all]
master1 ansible_ssh_host=10.0.0.8 ip=10.0.0.8
...
[kube-master]
master1
master2
master3
[etcd]
master1
master2
master3
...](https://image.slidesharecdn.com/deployingkubernetesongcpwithkubespray-180622040719/85/Deploying-Kubernetes-on-GCP-with-Kubespray-16-320.jpg)
Deploying Kubernetes on GCP with Kubespray
- 1. June 12, 2018 | 5:00 pm Silicon Valley, CA, USA Deploying Kubernetes with kubespraySergey Matykevich Cloud Foundry Engineer Altoros
- 2. Kubernetes automation ● kops ● kubeadm ● kubo ● kubespray
- 3. kops Pros: ● Automates both infrastructure creation and cluster startup ● Uses cloud providers native features ● Support updates and upgrades ● Is maintained by kubernetes team Cons: ● Limited infrastructure support (AWS, GCP) ● Still in alpha
- 4. kubeadm Pros: ● The most flexible tool ● Suitable for bare metal installations Cons: ● Don’t provision infrastructure. ● Require a lot of manual work
- 5. kubo Pros: ● Uses BOSH ● Support multiple cloud providers ● Monitors health of your cluster Cons: ● Uses BOSH ● Require BOSH specific knowledge ● Might be difficult to customize
- 6. Kubespray Pros: ● Supports multiple deployment ● Reasonable security by default ● Automatic HA ● Supports upgrades ● Under the hood uses kubeadm ● If you know ansible you know Kubespray ● Easy to customize for exposed items Cons: ● If you pull straight from master you might be in trouble fast ● Uses Ansible ● Can be slow ● Not everything is idempotent (cert changes) ● Still some lacking items, like l7 load balancers in gcp ● Can get complicated for customizations not exposed
- 7. Kubespray A couple of recommendations: Use a release version to base your work off of KubeSpray uses ansible so one thing I would recommend if you are experimenting or just deploying to different environments is use --flush-cache What am I showing today? ● Deploy to GCP ● Use terraform to follow infra as code ● Cookie cutter templates for multiple environments
- 9. Step 1: Create ssh keys # Create new rsa keys if [ ! -f cust_id_rsa ] ; then ssh-keygen -t rsa -b 2048 -f cust_id_rsa -q -N "" fi # Load keys eval $(ssh-agent) ssh-add -D ssh-add cust_id_rsa
- 10. Step 2: Run terraform terraform init terraform plan -out cust.plan terraform apply cust.plan
- 11. Private version GCP network Nat gateway Load balancer Private subnet jumpbox Master instance group Worker instance group
- 12. Public version GCP network Load balancer Public subnet jumpbox Master instance group Worker instance group
- 13. Step 3: Get cluster ips bastion_ip=$(terraform output bastion_ip) lb_ip=$(terraform output lb_ip) master_ips=$(gcloud compute instances list --filter="${res_prefix}-masters" --format=json |jq -r '.[].networkInterfaces[].networkIP'|tr "n" " "| sed -e "s/ {1,}$//") node_ips=$(gcloud compute instances list --filter="${res_prefix}-workers" --format=json |jq -r '.[].networkInterfaces[].networkIP'|tr "n" " "| sed -e "s/ {1,}$//")
- 14. Step 4: Clone kubespay repo git clone https://github.com/kubernetes-incubator/kubespray.git cd kubespray && git checkout ${kubespray_version}
- 15. Step 5: Apply customizations # bring in modifications, in particular security changes rsync -avz $git_root/mods/roles/* roles/
- 16. Step 5: Generate inventory [all] master1 ansible_ssh_host=10.0.0.8 ip=10.0.0.8 ... [kube-master] master1 master2 master3 [etcd] master1 master2 master3 ...
- 17. Step 6: Run the playbook ansible-playbook -vvv -i inventory/hosts.ini cluster.yml --flush-cache
- 18. Step 7: Connect to the cluster #change the admin.conf file to the current lb_ip sed -i -e "s/lb-apiserver.kubernetes.local/$lb_ip/g" inventory/artifacts/admin.conf ## echo out set KUBECONFIG echo "Set your KUBECONFIG" echo "export KUBECONFIG=$(pwd)/inventory/artifacts/admin.conf"
- 20. Questions?
Editor's Notes
- #2: San Francisco
- #3: San Francisco
- #4: San Francisco
- #5: San Francisco
- #6: San Francisco
- #7: San Francisco
- #8: San Francisco
- #9: San Francisco
- #10: San Francisco
- #11: San Francisco
- #12: San Francisco
- #13: San Francisco
- #14: San Francisco
- #15: San Francisco
- #16: San Francisco
- #17: San Francisco
- #18: San Francisco
- #19: San Francisco
- #20: San Francisco
- #21: San Francisco