AWS reinvent 2019 recap - Riyadh - Containers and Serverless - Paul Maddox
1 like661 views
This document provides an overview and agenda for an AWS storage, compute, containers, serverless, and management tools presentation. It includes summaries of several upcoming AWS services and features related to EBS, S3, EC2, EKS, Fargate, Lambda, and AWS Cost Optimizer. The speaker is introduced as Paul Maddox, Principal Architect at AWS, with a background in development, SRE, and systems architecture.
![[AWS Dev Day] 앱 현대화 | AWS Fargate를 사용한 서버리스 컨테이너 활용 하기 - 삼성전자 개발자 포털 사례 - 정영준...](https://cdn.slidesharecdn.com/ss_thumbnails/appmodernizationawsfargate-190930044252-thumbnail.jpg?width=560&fit=bounds)
![AWS Technical Day Riyadh Nov 2019 [Migration]](https://cdn.slidesharecdn.com/ss_thumbnails/aws-tech-day-riydh-migration-191102193614-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
Similar to AWS reinvent 2019 recap - Riyadh - Containers and Serverless - Paul Maddox (15)
More from AWS Riyadh User Group (17)
Ad
Recently uploaded (20)
Ad
AWS reinvent 2019 recap - Riyadh - Containers and Serverless - Paul Maddox
- 1. About me Paul Maddox Principal Architect Amazon Web Services • 18 years of dev, SRE, and systems architecture background • Developer (Rust, Go, Java, C, NodeJS) • 5.5 years at Amazon, working with internal and external development teams. • 99% of my time spent on containers, serverless and devops Twitter: @paulmaddox Email: [email protected]
- 2. Agenda • Storage Updates • Compute Updates • New Instance Types • Savings Plans • EC2 Image Builder • Containers • ECS Capacity Providers & Auto Scaling • ECS CLI v2 • Amazon EKS Managed Worker Nodes & Fargate • Serverless • Lambda Provisioning • Amplify (iOS, Android, Datastore) • Event Bridge
- 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 4. EBS Direct APIs for Snapshots Introducing A simple set of APIs that provide access to directly read EBS snapshot data, enabling backup providers to achieve up to 70% faster backups for EBS volumes at lower costs. Up to 70% faster backup times More granular recovery point objectives (RPOs) Lower cost backups Storage Easily track incremental block changes on EBS volumes to achieve: General Availability – December 3 ListSnapshotBlocks , ListChangedBlocks , GetSnapshotBlock
- 5. Amazon S3 Access Points Introducing Simplify managing data access at scale for applications using shared data sets on Amazon S3. Easily create hundreds of access points per bucket, each with a unique name and permissions customized for each application. DRAFT General Availability – December 3 Storage
- 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 7. Amazon EC2 Inf1 Instances Introducing The fastest and lowest cost machine learning inference in the cloud Featuring AWS Inferentia, the first custom ML chip designed by AWS Up to 3X higher throughput and 40% lower cost per inference, compared to GPU-powered G4 instances Compute General Availability – December 3 Natural language processing PersonalizationObject detection Speech recognition Image processing Fraud detection
- 8. AWS Graviton2 Processor Introducing Enabling the best price/performance for your cloud workloads 64 vCPUs 20 Gbps 14 Gbps EBS Graviton1 Processor Graviton2 Processor DRAFTCompute Preview – December 3
- 9. AWS Graviton2 Based Instances Introducing Up to 40% better price-performance for general purpose, compute intensive, and memory intensive workloads. l M6g C6g R6g DRAFT Built for: General-purpose workloads such as application servers, mid-size data stores, and microservices Instance storage option: M6gd Built for: Compute intensive applications such as HPC, video encoding, gaming, and simulation workloads Instance storage option: C6gd Built for: Memory intensive workloads such as open-source databases, or in-memory caches Instance storage option: R6gd Compute Preview – December 3
- 10. AWS Compute Optimizer Introducing Identify optimal EC2 instances and Auto Scaling group with a ML- powered recommendation engine. Integrated with AWS Organizations. DRAFTManagement Tools General Availability – December 3
- 12. Receive lower rates automatically. Easy to use with recommendations in AWS Cost Explorer Significant savings up to 72% Flexible across instance family, size, OS, tenancy or Region Compute/Cost Management Announced – November 6 Simplify purchasing with a flexible pricing model that offers savings on Amazon EC2/ECS & AWS Fargate. AWS Lambda coming soon! Savings Plans
- 13. Build, maintain, and share secure OS images more quickly & easily. Both Linux and Windows, with automation for components and tests. Introducing DRAFTCompute General Availability – December 3 EC2 Image Builder
- 14. 14
- 15. 15
- 16. Spare capacity with savings up to 70% off of Fargate standard pricing Improved scalability, reduced operational cost to run containers Containers New Features Accelerating momentum for AWS container services
- 17. Building 👉 Growing 👉 Releasing 👉 Debugging
- 18. Building 👉 Growing 👉 Releasing 👉 Debugging
- 19. “I want to add a backend.” “I want my Backend to be able to talk to my Frontend.” Service Discovery New Service Building 👉 Growing 👉 Releasing 👉 Debugging
- 20. Building 👉 Growing 👉 Releasing 👉 Debugging
- 21. “I want to deploy to staging.” “I want to deploy to production.” “I just want to git push.” Staging Environment Production Environment Continuous Delivery Building 👉 Growing 👉 Releasing 👉 Debugging
- 22. archer env init VPC, Cluster, LoadBalancer, ECR Repo, etc Building 👉 Growing 👉 Releasing 👉 Debugging
- 23. Building 👉 Growing 👉 Releasing 👉 Debugging
- 24. archer env init production --prod VPC, Cluster, LoadBalancer, ECR Repo, etc Additional Protections - termination protection, more AZs, etc. Building 👉 Growing 👉 Releasing 👉 Debugging
- 25. archer app deploy --env production Building 👉 Growing 👉 Releasing 👉 Debugging
- 26. “I just want to git push.” Continuous Delivery Building 👉 Growing 👉 Releasing 👉 Debugging
- 27. archer pipeline init Building 👉 Growing 👉 Releasing 👉 Debugging
- 29. Deploying Kubernetes with Amazon EKS $ eksctl create cluster --managed ∙ exciting auto-generated name, e.g. “fabulous-mushroom-1527688624” ∙ 2x m5.large nodes (this instance type suits most common use-cases, and is good value for money) ∙ use official AWS EKS AMI ∙ us-west-2 region ∙ dedicated VPC (check your quotas)
- 30. Production Workloads Native and upstream Seamless integrations OSS Contribution Managed Kubernetes, with Amazon EKS
- 31. Production workloads Single tenant Multi-AZ and highly available architecture by default 99.9% Service Level Agreement for every cluster
- 32. Native and upstream Upstream conformant Integration testing with Kubernetes tooling APIs and existing tooling just work
- 33. OSS contributions AWS contributes bug fixes, security patches, and tooling improvements Open-source components Contribute to or maintain over 30 OSS projects on GitHub for Kubernetes
- 35. Deploying Kubernetes with Amazon EKS $ eksctl create cluster --managed ∙ exciting auto-generated name, e.g. “fabulous-mushroom-1527688624” ∙ 2x m5.large nodes (this instance type suits most common use-cases, and is good value for money) ∙ use official AWS EKS AMI ∙ us-west-2 region ∙ dedicated VPC (check your quotas)
- 36. Deploying Kubernetes with Amazon EKS $ cat > cluster.yml apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: basic-cluster region: me-south-1 managedNodeGroups: - name: ng-1 instanceType: m5.large desiredCapacity: 10 - name: ng-2 instanceType: m5.xlarge desiredCapacity: 2 $ eksctl create cluster –f cluster.yml • Desired capacity • Min / Max sizes • Disk volume size • SSH configuration • Private/public subnets • Security Groups • Labels & Tags • IAM policies When defining a cluster with YAML, you get repeatability, but also the option to configure:
- 37. Architecture Overview Your VPC AWS Cloud Availability Zone (AZ) 1 Kubernetes Node Group Availability Zone (AZ) 2 K8s Worker AWS VPC K8s Worker Kubernetes Node Group K8s Worker K8s Worker AZ 1 AZ 2 AZ 3 Highly scalable and available, multi AZ managed Kubernetes control plane Amazon Elastic Kubernetes Service
- 38. Management of the Kubernetes control plane Phase 1
- 39. Management of the Kubernetes control plane Phase 1 Phase 2 Management of the Kubernetes data plane
- 40. We started with Managed Node Groups
- 42. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved. You should be able to write your code and have it run, without having to worry about configuring complex management tools. This is the vision behind AWS Fargate. CTO, Amazon.com Dr. Werner Vogels
- 43. Amazon EKS on Fargate Bring existing pods Production ready Rightsized and integrated You don’t need to change your existing pods. Fargate works with existing workflows and services that run on Kubernetes. Launch pods quickly. Easily run pods across multiple AZs for high availability. Each pod runs in an isolated compute environment. Only pay for the resources you need to run your pods. Includes native AWS integrations for networking and security.
- 44. What matters for Fargate Fargate is a serverless compute platform for containers on AWS The differences between using EKS and ECS with Fargate are driven by the orchestration system
- 45. Architecture Overview Your VPC AWS Cloud Availability Zone (AZ) 1 Kubernetes Node Group Availability Zone (AZ) 2 K8s Worker AWS VPC K8s Worker Kubernetes Node Group K8s Worker K8s Worker AZ 1 AZ 2 AZ 3 Highly scalable and available multi AZ managed Kubernetes control plane Amazon Elastic Kubernetes Service
- 46. Your VPC AWS Cloud Availability Zone (AZ) 1 Availability Zone (AZ) 2 AWS VPC AZ 1 AZ 2 AZ 3 Highly scalable and available multi AZ managed Kubernetes control plane Amazon Elastic Kubernetes Service Container Elastic network interface Elastic network interface Container AWS Fargate Architecture Overview
- 47. DEMO
- 48. Things you can’t do (for now) Deploy Daemonsets Use service type LoadBalancer (CLB/NLB) Running privileged containers Run stateful workloads Recap: EKS for Fargate introduces UX changes Things you no longer need to do Manage Kubernetes worker nodes Pay for unused capacity Use K8s Cluster Autoscaler (CA) Things you get out of the box VM isolation at pod level Pod level billing Easy chargeback in multi-tenant scenarios
- 49. Limits: Things to keep in mind AWS accounts have a soft limit of 100 Fargate tasks/pods per region You increase this limit Due to the nature of the solution, there’s a limit of 5,000 pods per cluster K8s tests up to 5,000 workers per cluster
- 50. Scalability: Things to keep in mind Single individual pod start time may be longer on Fargate than on EC2 Each pod deployment sources a virtual node first from the Fargate fleet Pod deployments at scale may be faster due to Fargate parallelism E.g., think of the delay that Cluster Autoscaler can introduce in sourcing new EC2 capacity
- 51. Pricing Standard EKS cluster pricing $0.10 per hour Standard Fargate Pricing for vCPU and memory
- 52. Next Steps Deploy your first cluster with eksctl: https://eksctl.io Up your Kubernetes knowledge with our extensive, free online workshop: https://eksworkshop.com Get familiar with the AWS Containers roadmap: https://github.com/aws/containers-roadmap/projects/1
- 53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 54. Provisioned Concurrency on AWS Lambda New Feature • Keeps functions initialized and warm, ensuring start times stay in the milliseconds • Full control over when provisioned concurrency is set • No code changes are required to provision concurrency on functions in production • Can be combined with AWS Auto Scaling DRAFTServerless General Availability – December 3
- 55. Achieve up to 67% cost reduction and 50% latency reduction compared to REST APIs. HTTP APIs are also easier to configure than REST APIs, allowing customers to focus more time on building applications. Reduce application costs by up to 67% Reduce application latency by up to 50% Configure HTTP APIs easier and faster than before HTTP APIs for Amazon API Gateway Introducing DRAFTMobile Services Preview – December 4
- 56. Demo!
- 57. AWS Step Functions Express Workflows Introducing Orchestrate AWS compute, database, and messaging services at rates greater than 100,000 events/second, suitable for high-volume event processing workloads such as IoT data ingestion, streaming data processing and transformation. DRAFTApp Integration General Availability – December 3
- 58. 59
- 59. Amazon EventBridge Schema Registry Introducing Store event structure - or schema - in a shared central location, so it’s faster and easier to find the events you need. Generate code bindings right in your IDE to represent an event as an object in code. DRAFTApp Integration Preview – December 3
- 60. Amplify for iOS & Android Introducing DRAFTMobile Services General Availability – December 3 Open source libraries and toolchain that enable mobile developers to build scalable and secure cloud powered serverless applications.
- 61. Adding Amplify to your iOS / Android App $ cd <root of your iOS/Android app folder> $ amplify init $ amplify add storage ? Please select from one of the below mentioned services: Content (Images, audio, video, etc.) ? You need to add auth (Amazon Cognito) to your project in order to add storage for user files. Do you want to add auth now? Yes Do you want to use the default authentication and security configuration? Default configuration Warning: you will not be able to edit these selections. How do you want users to be able to sign in? Username Do you want to configure advanced settings? No, I am done. Successfully added auth resource ? Please provide a friendly name for your resource that will be used to label this category in the project: mybucket ? Please provide bucket name: mybucket ? Who should have access: Auth and guest users ? What kind of access do you want for Authenticated users? create/update, read, delete ? What kind of access do you want for Guest users? create/update, read, delete ? Do you want to add a Lambda Trigger for your S3 Bucket? No 62
- 62. 63
- 63. Amplify DataStore New Feature DRAFTMobile Services General Availability – December 3 Multi-platform (iOS/Android/React Native/Web) on-device persistent storage engine that automatically synchronizes data between mobile/web apps and the cloud using GraphQL.
- 64. Go Build! Here to help you build
- 65. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! Paul Maddox Principal Solutions Architect Amazon Web Services