GitOps - Operation By Pull Request
6 likes1,451 views
Presentation given at Cloud Native Copenhagen, Cloud Native Aalborg, and Cloud Native Aarhus in December 2020
Ad
More Related Content
What's hot (20)
Similar to GitOps - Operation By Pull Request (20)
Ad
More from Kasper Nissen (11)
Ad
Recently uploaded (20)
GitOps - Operation By Pull Request
- 1. Operations by Pull Request
- 2. $ whoami Kasper Nissen (@phennex) Cloud Architect / Site Reliability Engineer at Lunar CNCF Ambassador Certified Kubernetes Administrator Cloud Native Aarhus (Cloud Native Copenhagen) Cloud Native Nordics Occasional speaker at Meetups, Conferences Blog: kubecloud.io
- 4. AgendaWhat is Gitops? Flavours of Gitops GitOps at LUNAR Chaos Engineering GitOps at Cloud Native Nordics Progressive Delivery nda
- 11. Banking products Value added services & third party partnerships Tech stack Business Premium Cards Accounts Installments Rethinking the banking experience
- 16. noun UK /ˌrek.ənˌsɪl.iˈeɪ.ʃən/ US /ˌrek.ənˌsɪl.iˈeɪ.ʃən/ the process of making two people or groups of people friendly again after they have argued seriously or fought and kept apart from each other, or a situation in which this happens the process of making two opposite beliefs, ideas, or situations agree Source: https://dictionary.cambridge.org/dictionary/english/reconciliation
- 17. apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.17.5 ports: - containerPort: 80 nginx-1 nginx-2 nginx-3
- 20. noun UK /ɡɪt/ɒp/ US /git/ɑːp/ GitOps is a way to do Kubernetes cluster management and application delivery. It works by using Git as a single source of truth for declarative infrastructure and applications. With Git at the center of your delivery pipelines, developers can make pull requests to accelerate and simplify application deployments and operations tasks to Kubernetes. Source : https://www.weave.works/technologies/gitops/
- 24. kubectl apply -f deploy/
- 25. ● CI/CD needs write access to your clusters ● How to track rollout failures? ● No audit trail of the kubernetes resources ● No single source of truth of the state in the cluster It’s imperative.
- 27. Flavours ● Decentralized One-way flow ● Decentralized Two-way flow ● Centralized flow
- 32. Where should service config live? service repo config repo
- 35. Features ● of applications to specified target environments ● Support for multiple config management/templating tools (Kustomize, Helm, Ksonnet, Jsonnet, plain-YAML) ● Ability to and to ● SSO Integration (OIDC, OAuth2, LDAP, SAML 2.0, GitHub, GitLab, Microsoft, LinkedIn) ● Multi-tenancy and RBAC policies for authorization ● -anywhere to any application configuration committed in Git repository ● Health status analysis of application resources ● Automated configuration and
- 36. Features ● of applications to its ● which provides real-time view of application activity ● for automation and CI integration ● Webhook integration (GitHub, BitBucket, GitLab) ● Access tokens for automation ● PreSync, Sync, PostSync hooks to support complex application rollouts (e.g.blue/green & canary upgrades) ● Audit trails for application events and API calls ● Prometheus metrics ● Parameter overrides for overriding ksonnet/helm parameters in Git
- 39. What is Flux CD? “Flux is a tool that automatically ensures that the state of your Kubernetes cluster matches the configuration you’ve supplied in Git. It uses an operator in the cluster to trigger deployments inside Kubernetes, which means that you don’t need a separate continuous delivery tool.” Source: www.fluxcd.io
- 40. Why Flux CD
- 41. The Flux CD workflow Source: https://fluxcd.io/
- 43. Argo Flux Source: https://github.com/argoproj/gitops-engine ● gitops-engine ○ ○ ○ ○ ○
- 45. Why GitOps? ● Audit trail of deployments ● Limit access to clusters ● Make Disaster Recovery an unpainful event
- 48. Our solution controlled by release-manager pushed by Jenkins
- 50. release-manager ● 4 components ○ ■ The server itself, this is where the magic happens ○ ■ Kubernetes controller listens for updates on resources and reports back to release-server ○ ■ CLI for developers to control releases ○ ■ Tool for generating metadata object; artifact.json ■ Handles slack communication from CI as well Source: https://github.com/lunarway/release-manager
- 52. hamctl Source: https://github.com/lunarway/release-manager # implicit order between envs: master -> dev -> staging -> prod $ hamctl promote --env prod # choose which brand to deploy where $ hamctl release --branch hotfix --env dev # setup auto-release policy $ hamctl policy apply auto-release --branch master --env dev
- 54. Secret Sealed Secret kubeseal --cert cert.crt sealed-secrets- controller Sealed Secret Secret sealed-secrets Source: https://github.com/bitnami-labs/sealed-secrets
- 56. Audit
- 59. Gameday
- 61. How to restore the environment? volume #1 volume #2 volume #X elb #1 elb #2 elb #X secret #1..X dns kubernetes configuration etcd external partner #1 external partner #2
- 62. Clusters as cattle herds instead of pets
- 63. Clusters as cattle herds instead of pets bob bob ht998a m8i7h3 pet instance pet clusters lpq8qr
- 68. Decentralized two-way flow kubernetes flux-cd helmstats-api website prometheus-operator nginx-ingress -controller cert- manager sealed- secrets node-exporters Kube-state- metrics prometheus grafana https://github.com/cloud-native-nordics/k8s-config-repo
- 70. HelmRelease apiVersion: flux.weave.works/v1beta1 kind: HelmRelease metadata: name: nginx-ingress namespace: default annotations: flux.weave.works/automated: "false" spec: releaseName: nginx-ingress chart: repository: https://kubernetes-charts...googleapis.com/ name: nginx-ingress version: 1.22.1 values: controller: publishService: enabled: true
- 73. Progressive delivery is the process of pushing changes to a product iteratively, first to a small audience and then to increasingly larger audiences to maintain quality control (QC). The goal of progressive delivery is to improve delivery times for new product features and mitigate risk by controlling who is able to see them. Source: https://searchitoperations.techtarget.com/definition/progressive-delivery
- 75. Canary Canary release is a technique to reduce the risk of introducing a new software version in production by slowly rolling out the change to a small subset of users before rolling it out to the entire infrastructure and making it available to everybody. apiVersion: flagger.app/v1alpha3 kind: Canary metadata: name: podinfo spec: targetRef: kind: Deployment name: podinfo progressDeadlineSeconds: 60 service: targetPort: 9898 canaryAnalysis: interval: 1m threshold: 10 maxWeight: 50 stepWeight: 5 metrics: - name: request-success-rate threshold: 99 interval: 1m - name: request-duration threshold: 500 interval: 30s webhooks: - name: load-test metadata: cmd: "hey -z 1m -q 10 -c 2 http://podinfo.test:9898/"
- 78. Everything in Git ● ○ ■ ■ ○ ● ○ apiVersion: lunar.bank/v1 kind: PostgreSQLUser metadata: name: kni spec: name: kni read: - host: value: some.host.com reason: "I am a developer" write: - host: valueFrom: configMapKeyRef: name: database key: db.host database: value: user schema: value: user reason: "Related to support ticket LW-1234" start: 2019-09-16T10:00:00Z end: 2019-09-16T14:00:00Z
- 81. cluster-api apiVersion: "cluster.k8s.io/v1alpha1" kind: MachineDeployment metadata: name: nodes namespace: kube-system spec: replicas: 5 selector: matchLabels: foo: bar template: metadata: labels: foo: bar spec: providerSpec: value: cloudProvider: "aws" cloudProviderSpec: region: "eu-central-1" availabilityZone: "eu-central-1a" vpcId: "vpc-819f62e9" subnetId: "subnet-2bff4f43" instanceType: "t2.micro" instanceProfile: "kubernetes-v1" diskSize: 50 .. operatingSystem: "coreos" operatingSystemSpec: disableAutoUpdate: true
- 84. Contact: