Migrating from Self-Managed Kubernetes on EC2 to a GitOps Enabled EKS
0 likes720 views
Did your company start down the path of building a cloud native platform using Kubernetes with the goal of enabling developers to innovate faster and increase productivity, but then run into challenges keeping it operating in an optimal way? In this session, Weaveworks will discuss how to migrate from self-managed Kubernetes on EC2 to a GitOps managed Shared Services Platform (SSP) on EKS. A SSP built on EKS and managed with Weave GitOps provides developers and operators with common workflows to update both applications and infrastructure. With every change in version control, full audit trails are available, and security is enforced. While at the same time enabling easier rollbacks and faster mean-time-to-recovery (MTTR). In short, a Weave GitOps managed SSP increases developer velocity while boosting stability. How to operate a hybrid Kubernetes architecture, using managed EKS in the AWS Cloud and EKS-Distro on premises. How to structure your infrastructure repository to efficiently manage multiple teams. How to use Kubernetes RBAC to provide secure cluster multi-tenancy. How to use GitOps to promote releases across a hybrid set of independent clusters. How to accomplish data and operational sovereignty.
Ad
More Related Content
What's hot (20)
Similar to Migrating from Self-Managed Kubernetes on EC2 to a GitOps Enabled EKS (20)
Ad
More from Weaveworks (20)
Ad
Recently uploaded (20)
Migrating from Self-Managed Kubernetes on EC2 to a GitOps Enabled EKS
- 1. Migrating from Self-Managed Kubernetes (KOPS) on EC2 to GitOps Enabled EKS
- 2. Today’s Speakers Daniel Lizio-Katzen Head of Strategy & Partnerships Weaveworks Leonardo Murillo Principal Partner Solution Architect Weaveworks 3
- 3. • Founding chair of the CNCF technical oversight committee (TOC) • Coined the term GitOps, and created the open source tools that make it work • Creator of eksctl, the most used way to work with AWS EKS • Invented open source solutions to run Kubernetes at scale for our own Weave Cloud SaaS product Team Thought Leadership • Alexis Richardson, CEO • Cornelia Davis, CTO • Steve George, COO • Global Presence: – US East, Central, West – Europe – India, Japan Notable Facts • Founded in 2014 • Investors include: Accel, AWS, Deutsche Telekom, Ericsson, Google Ventures, Orange and Redline • Top 10 contributor to the CNCF • Multiple - thousand plus star open source projects About Weaveworks 4
- 4. Weaveworks is a Leader in Open Source & Cloud Native Kubernetes We led the creation of the flagship Kubernetes installer Kubeadm We created EKSctl – the official EKS CLI enabling GitOps on AWS Weave Net - the original container SDN & Firewall LibCNI Kubernetes network model - work with CoreOS (now RH/IBM) Weave Ignite - the container VM for secure, fast Kubernetes anywhere Observability We made Prometheus scalable with Weave Cortex & launched 1st Prom-aaS, powering EA’s global games. Cortex is now a CNCF project. Weave Scope is is one of the ”top tools for monitoring Kubernetes” providing management and monitoring and visualization for <20,000 users CD and GitOps tooling CNCF Flux is a Kubernetes-native CD tool for GitOps - also Flux-Helm. Weave Flagger for progressive delivery JKcfg for templating, policy and actions Kubediff - diff alerting for Kubernetes to enable GitOps Kured - Cluster Reboot Tool Grafanalib - GitOps dashboarding for Grafana Integrations Weaveworks for: Istio, Kubeflow, OpenFaaS, Cloud Foundry 5
- 5. Selected Customers Financial Services Technology Other Industries 6 Telecom
- 7. Git Delivery Controllers GitOps – An Operating Model for Cloud Native Build GIT Test IDE “Immutability Firewall” Operational Controllers Continuous Integration Deployment (clusters, apps) Monitoring Logging (Observability) Management (operations) Compiled declarative state 8
- 8. Software agents ensure correctness and alert on divergence Approved changes to the desired state are automatically applied to the system The canonical desired system state is versioned (with Git) 9 GitOps Principles 1 The entire system is described declaratively. 2 3 4
- 9. Software agents ensure correctness and alert on divergence Approved changes to the desired state are automatically applied to the system The canonical desired system state is versioned (with Git) The entire system is described declaratively. 10 GitOps Principles 1 2 3 4 Includes cluster specifications, components and workloads Clusters, apps and infra stack is versioned Includes cluster, workload, infrastructure CRUD Divergence from cluster/app/infra spec is detected & corrected automatically
- 10. 11 App is developed & tested locally Built automatically using CI of your choice Container Image pushed automatically to a container registry Deployed automatically using Weave deployment manager... ...to an Execution Environment of your choice Development on Kubernetes ABCDE’s
- 11. CD: GitOps is a technology evolution of DevOps Imperative Automation DevOps GitOps Infrastructure as Code Platform as Code Mutation Single State Deployment Declarative Automation Immutability Deployment and Operations Reconciliation / Convergence 12
- 12. 13
- 13. 14
- 14. Deployment Speed 50% faster Deployment Frequency 65% increase Time Spent Coding 75% more ● Increased developer productivity through reduced complexity ● Improved security posture ● Greater uptime and faster mean-time-to-recovery (MTTR) ● Higher infrastructure utilization through better platform design and orchestration ● Reduced licensing costs by eliminating unnecessary tooling How Weave GitOps Shared Services Platforms creates value for our joint customers? 15
- 15. So, You’re Sick of Managing Kubernetes on EC2 and You Want to Migrate to EKS
- 16. Common Issues with Self Managed K8s ● Clusters that were built using imperative VM patterns and tools ● Long lived “unicorn” clusters that have unique configurations ● The team that originally built and supported the existing clusters has moved on ● The cluster configurations don’t match the configurations in source control ● Clusters are running antiquated versions of K8s without an easy way to upgrade
- 17. Cluster Sprawl Cluster Inconsistency Inefficient utilization Day-2 Operations Overhead Redundant Services Platform Operators Application Developers Infrastructure Concerns Redundant efforts across teams Reliability and Availability
- 18. The Shared Services Platform Cluster Sprawl Cluster Inconsistency Inefficient utilization Day-2 Operations Overhead Redundant Services SSP abstracts the complexities of cloud-native Kubernetes infrastructure from developers. Workloads are consolidated into large scale, production-ready clusters. Platform operators centrally apply security and monitoring best practices, and services are centrally managed and offered to all teams. What about Developer Teams? SSP provides self-service and autonomy for teams, while maintaining workload segregation and reducing operations and infrastructure efforts from dev teams.
- 19. Migrating Path from KOPS to EKS with GitOps Path to Migration / POC: Providing a Canonical Template ● Path to an EKS Shared Services Platform managed by Weave GitOps ○ EKSctl ■ First capture cluster config and state from Kops state store, map to EKSctl ClusterConfig ○ Velero ■ Backup Kubernetes API resources and persistent volumes to have same workloads running in the EKS cluster ○ Evict Workloads from KOPS ■ Then restore via Velero backups to EKS ● POC ○ Creating a reproducible EKS cluster based on standard configs
- 22. K8s is quite complex if you look under the hood Networking Storage Worker Scaling Control Plane High Availability Upgrades and Patches Worker Recovery Security and Authentication Amazon Elastic Kubernetes Service (Amazon EKS) Managed K8s abstracts all that away
- 24. How do operate our SSP? GitOps! An Operating Model for Cloud Native 1. Desired state of the full system is declared in code 2. Desired state is versioned, versions are immutable 3. Agents reconcile the running state with the desired state 4. Reconciliation happens continuously Principles
- 25. Services Team Base Integration Development Production Environments prod int dev Team Blue Git://team.blue prod int dev Full codebase at: https://github.com/weaveworks-gitops-demo/team-ssp
- 26. Success Let’s look at the benefits
- 27. Kubernetes without the management complexity Secured, compliant developer autonomy Multi-stage deployment off the shelf DRY Reproducibility Faster recovery ( < MTTR ) Common Tooling and Lifecycle Auditability and Traceability Benefits
- 28. Benefits of Migrating to EKS Why EKS? ● Managed control plane, Kubernetes upgrades, cluster maintenance, more here ● EKSctl created at Weavework with a close partnership with AWS ● AWS SLAs for outages:
- 29. Weaveworks Offerings TECHNOLOGY & CONSULTING SERVICES 30
- 30. Professional Services Overview Weaveworks can engage with you from Cloud Native readiness review, architecture & design to long term dedicated expertise ● Provide organizations with detailed readiness review of your current or planned cloud native initiatives ● Architect, Designing and Building the platform alongside your team. ● Provide long term stability with Dedicated SRE expertise ● Work with your team to integrate and develop new ways of working that use GitOps to full advantage 31
- 31. TECHNOLOGY: Weave GitOps Enterprise Workload Workload Workload Workload Container Control Release Management Visualisation Monitoring & Metrics Alerting Cluster audits Deployment Policy Dashboards Kubernetes Cluster configuration Fleet management Cluster components Logging and Tracing Networking Storage Infrastructure Automation Security • Continuous Delivery, observability and monitoring • Consistent developer workflows across multiple deployments • Team workspaces for multi-tenanted usage • Extend Kubernetes to managed platform using GitOps model • An Open Source Kubernetes platform for on-premise deployment • Additive to manage Kubernetes (e.g. EKS, AKS or GKE) • Upgrades to new versions • Extensible controls to implement security and policy controls Developer Experience Operator Experience 32
- 32. Education Enablement Platform Modern Ops Weaveworks Consulting, Training and SRE Service • Guided technology choices • Cloud native reference architecture designs • Cloud native technology options and selection Modern App Platform w/EKS • Configuration management for the whole platform • Integrated governance, risk & compliance • Seamlessly integrated metrics • 24/7 worldwide support Faster Delivery, Lower TCO • Automation, management and Continuous Delivery • CloudWatch monitoring and alerting • Increased developer self service capabilities • Training for cluster operators, application operators and developers • Delivery of POCs and experimental environments The steps to cloud native in production ... GET STARTED FAST TEACH AND MANIFEST EKS SKILLS DELIVER A PRODUCTION READY APP PLATFORM ON EKS GITOPS TO ENABLE AN AGILE DELIVERY MODEL 1 2 3 4 33
- 33. AWS Partner Dev Day - free workshop When: Thursday, September 16 at 10am ET | 3pm BST | 4pm CST (120min) What: Creating a Shared Services Platform on EKS and Managed with Weave GitOps Topics: ● How to setup and configure a SSP using EKS and Weave GitOps ● How to structure your infrastructure repository to efficiently manage multiple teams ● How to use Kubernetes RBAC to provide secure cluster multi-tenancy ● How to use GitOps to promote releases across a set of independent clusters Registration: Look for an email later today
- 34. Thank you! Contact us to discuss how to get your customers operating EKS at scale quickly! Daniel Lizio-Katzen Head of Strategy & Partnerships [email protected] Leonardo Murillo Principal Partner Solution Architect [email protected]