Amazon EC2 Spot Instances Workshop
0 likes487 views
This document provides an overview of using Amazon EC2 Spot Instances for compute workloads. It discusses EC2 Spot pricing and purchase options, features like interruptions and orchestration with Auto Scaling Groups and Spot Fleet. Use cases where Spot is well-suited include stateless, fault-tolerant workloads. Integrations with container and big data services like ECS, EKS and EMR are also covered. The presentation emphasizes flexibility, automation and diversification to maximize cost savings from Spot while minimizing risks of interruptions.
![© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
aws ec2 describe-spot-price-history --start-time 2018-05-06T07:08:09 --end-time 2018-05-06T08:08:09 --
instance-types c4.2xlarge --availability-zone eu-west-1a --product-description "Linux/UNIX (Amazon VPC)“
{
"SpotPriceHistory": [
{
"Timestamp": "2018-05-06T06:30:30.000Z",
"AvailabilityZone": "eu-west-1a",
"InstanceType": "c4.2xlarge",
"ProductDescription": "Linux/UNIX (Amazon VPC)",
"SpotPrice": "0.122300"
}
]
}
EC2 Spot pricing history – API access](https://image.slidesharecdn.com/spotworkshopforawsugs-190903070027/85/Amazon-EC2-Spot-Instances-Workshop-18-320.jpg)
![© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
API Parameters
"MixedInstancesPolicy": {
"LaunchTemplate": {
"LaunchTemplateSpecification": {
"LaunchTemplateName": ”MyLaunchTemplate"
},
"Overrides": [
{ "InstanceType": "c5.large" },
{ "InstanceType": "c4.large" }
]
},
"InstancesDistribution": {
"OnDemandAllocationStrategy": "prioritized",
"OnDemandBaseCapacity": 10,
"OnDemandPercentageAboveBaseCapacity": 50,
"SpotAllocationStrategy": ”capacity-optimized"
}
}
AZ1 and
AZ2
Desired
Min
Max
On-Demand Base
50% On-Demand
50% Spot
Minimum On-Demand (10)](https://image.slidesharecdn.com/spotworkshopforawsugs-190903070027/85/Amazon-EC2-Spot-Instances-Workshop-39-320.jpg)
Amazon EC2 Spot Instances Workshop
- 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. [email protected] Sunday, 25 Aug 2019 Amazon EC2 Spot Instances Compute at up to 90% off. Scale more. Get faster results. Build resilient services. Chakravarthy Nagarajan Specialist Solution Architect, EC2 Spot Sridhar Bharadwaj Business Development Manager, EC2 Spot
- 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda • EC2 Spot Instances overview • Pricing model • Major features and functionality • Interruption details • Spot orchestration options (Auto Scaling Groups, Fleet) • Console Demo • Monitoring price and usage • Use Cases – where to use Spot • Main takeaways for success with Spot
- 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 Spot – cool Cost Savings
- 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon EC2 purchase options Spot Instances Spare EC2 capacity at off On-Demand prices Fault-tolerant, flexible, stateless workloads Reserved Instances Make a 1 or 3-year commitment and receive a off On-Demand prices Committed & steady-state usage On-Demand Pay for compute capacity with no long-term commitments Spiky workloads, to define needs
- 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. To optimize Amazon EC2, combine purchase options for fault- tolerant, flexible, stateless workloads
- 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Spare capacity at scale Clemson university – 1.1 Million cores https://tinyurl.com/clemson-spot
- 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Spot Instances - basics Price changes infrequently based on long term supply and demand of spare capacity in each pool independently Just request capacity and pay the current rate. No Bidding Interruptions only happen when OD needs capacity. No outbidding
- 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Large customer base
- 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon EC2 Spot integrations
- 10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Flexibility is key to successful adoption Instance flexible Time flexible AZ flexible
- 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. $0.27 $0.29$0.50 1b 1c1a 8XL $0.30 $0.16$0.214XL $0.07 $0.08$0.082XL $0.05 $0.04$0.04XL $0.01 $0.04$0.01L C4 $1.76 On Demand $0.88 $0.44 $0.22 $0.11 EC2 Spot pools- instance type flexibility Each instance family Each instance size Each Availability Zone (60) In every region (20) Is a separate Spot pool R5 M4 C5 I3 M5d R4 D2 C4
- 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Time flexibility examples • Model training • Genomics • Development • Testing • One-time queries Time sensitive workloadsTime insensitive workloads examples • Web services • APIs • Analytics • Grid computing • Containers
- 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Spot Blocks • Defined duration workload without interruptions • Commit for 1-6 hours. Instances terminate after duration. • Lower discounts compared to Spot • No Auto-Scaling and No Instance Diversification
- 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What about interruptions?
- 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Interruption behaviors Terminate HibernateStop
- 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Spot Instance Advisor https://aws.amazon.com/ec2/spot/instance-advisor/
- 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 Spot pricing history
- 18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. aws ec2 describe-spot-price-history --start-time 2018-05-06T07:08:09 --end-time 2018-05-06T08:08:09 -- instance-types c4.2xlarge --availability-zone eu-west-1a --product-description "Linux/UNIX (Amazon VPC)“ { "SpotPriceHistory": [ { "Timestamp": "2018-05-06T06:30:30.000Z", "AvailabilityZone": "eu-west-1a", "InstanceType": "c4.2xlarge", "ProductDescription": "Linux/UNIX (Amazon VPC)", "SpotPrice": "0.122300" } ] } EC2 Spot pricing history – API access
- 19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Spot orchestration options comparison Auto Scaling Groups Spot Fleet Maintains target capacity (upon interruptions or failures) Instance type diversification Availability zone diversification Allocation strategy (N Lowest Pools, Lowest Price) Autoscaling (target tracking, stepped with custom metrics) ELB integration *Detach on interruption notification requires automation On-demand capacity mixed with Spot Lifecycle hooks, termination policies, protection, detach, processes Weights Roadmap
- 20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Monitoring Spot usage – Cost Explorer Y e a r b y m o n t h – l o n g t e r m t r e n d s
- 21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Monitoring Spot usage – Cost Explorer M o n t h b y d a y – s h o r t t e r m c h a n g e s a n d a n o m a l y d e t e c t i o n
- 22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Monitoring Spot usage • Filter and group by: account, instance type, region, tags • Data is available via API • If you need: - Deeper insights - Hour-level data resolution or one hour data freshness - Resource ids Use Cost and Usage Reports or Spot Instance Data Feed (easily query with Athena or visualize with Quicksight / Tableau / Looker /…)
- 23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Monitoring Spot usage – Savings Summary
- 24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Is my workload Spot Ready? Stateless Fault-Tolerant Flexible: Multi- AZ and Instance Flexibility Loosely Coupled Looks familiar?
- 25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Main takeaways for success with Spot • Be instance type agnostic and let ASG/Fleet provide the required capacity at the lowest price • Adopt Launch Templates to benefit from new ASG and Fleet features • New instance families generally have higher interruption rates • Architect for fault-tolerance to be Spot compatible and increase your availability
- 26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you https://aws.amazon.com/ec2/spot [email protected]
- 27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Usage of EC2 Spot Instances with specific Workloads
- 28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Stateless web application or API frontend • Spot diversification before all else • To mitigate risk, launch On-Demand in a different pool if Spot capacity is insufficient • Availability and performance should not be impacted, ensure low bootstrap time & some over-provisioning https://tinyurl.com/SpotAppnextBlog M o s t l y i d e n t i c a l f o r q u e u e w o r k e r s
- 29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon EC2 Auto Scaling
- 30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What is Amazon EC2 Auto Scaling? Amazon EC2 Auto Scaling
- 31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Auto Scaling Group (ASG) introduction Logical group of instances for your service Minimum and maximum bound for the number of instances that can be in the ASG Launch or terminate instances to meet the desired capacity Desired Min Max
- 32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Scheduled scaling Dynamic scaling Predictive scaling Manual scaling Amazon EC2 Auto Scaling
- 33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Predictive scaling in Amazon EC2 Auto Scaling Machine learning technology behind the scenes Machine learning model Load metric and forecasts next two days based on the pre-trained model Performs regression analysis between load metric and scaling metric Schedules scaling actions for the next two days, hourly Repeats every day Capacity provisioning on-premises Capacity provisioning with dynamic scaling Capacity provisioning with predictive scaling and dynamic scaling Time Load/Capacity Time Load/Capacity Time Load/Capacity Provisioned capacity Actual capacity demand
- 34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Auto Scaling Groups with Multiple Purchase Options and Instance Types
- 35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Before: Multiple ASGs to use Spot, On-Demand, and RIs together m4.large Spot ASG m5.large Spot ASG c4.large On-Demand ASG Availability Zone 1 Availability Zone 2 Availability Zone3 One ASG for each purchase option and instance type
- 36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. After: Include Spot, On-Demand, and RIs in a single ASG m4.large Spot m5.large Spot c4.large On-Demand Availability Zone 1 Availability Zone 2 Availability Zone3 A single ASG combines purchase options and instance types
- 37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Save up to 90% using EC2 Auto Scaling and EC2 Fleet Automatically provision and scale instances across instance families and purchase models in a single Auto Scaling group Lowest cost Specify what percentage of your Auto Scaling group capacity should be fulfilled by On-Demand Instances and Spot Instances to optimize cost Prioritized list Use a prioritized list for On-Demand Instance types to scale capacity during an urgent, unpredictable event to optimize performance Reduce operational overhead
- 38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon EC2 Fleet and Allocation strategies Amazon EC2 Fleet • Provisions capacity across multiple instance types according to allocation strategies Allocation strategies • On-Demand prioritized list of instance types • Spot instances across the N lowest priced instance pools • Capacity Optimized • Allocation strategies determine which instance types are launched and terminated
- 39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. API Parameters "MixedInstancesPolicy": { "LaunchTemplate": { "LaunchTemplateSpecification": { "LaunchTemplateName": ”MyLaunchTemplate" }, "Overrides": [ { "InstanceType": "c5.large" }, { "InstanceType": "c4.large" } ] }, "InstancesDistribution": { "OnDemandAllocationStrategy": "prioritized", "OnDemandBaseCapacity": 10, "OnDemandPercentageAboveBaseCapacity": 50, "SpotAllocationStrategy": ”capacity-optimized" } } AZ1 and AZ2 Desired Min Max On-Demand Base 50% On-Demand 50% Spot Minimum On-Demand (10)
- 40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Instance type overrides and allocation strategies • ASG adjusts to new configuration as scale up and down • As ASG scales up • Launch instances according to the new configuration • As ASG scales down • Prioritize terminating instances not matching the new configuration • New termination policy: AllocationStrategy Instance type overrides: m4.large, m5.large m4.large m5.large Instance type overrides: m5.large, c5.large m4.large m5.large c5.large Instance type overrides: m5.large, c5.large m4.large m5.large c5.large
- 41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Recommendations on Mixed Instances Policy Choose at least 2 instance type overrides • Improves availability for On-Demand and Spot Instances Diversify across at least N = 2 Spot Instance pools • Reduces risk from fluctuations in Spot capacity and prices Choose instance types of same size across families • Maintains stability as dynamically scale up and down Use default spot max price • Leverages spot cost savings while defaulting to on-demand price as maximum price to pay
- 42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 Spot with Amazon ECS
- 43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Spot and Containerized workloads • Best practices overlap: cattle, not pets • Cluster instances (worker nodes) are conceptually redundant and ephemeral in containerized workloads • Instance type flexibility is easy
- 44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Integrated directly into ECS console, or use CloudFormation, Terraform ECS creates a Spot Fleet in your account, Or – use an Auto Scaling Group to bootstrap instances Autoscaling ECS service: CPU/MEM, HTTP requests Spot Fleet / ASG on reserved metric - CPU, MEM Interruptions are handled automatically via scripts installed in User Data ECS: automatically Drain the instance Spot Fleet / ASG: automatically replace the instance ECS - Provisioning and scaling
- 45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 Spot with Amazon EKS
- 46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubernetes and EKS considerations • EKS is a managed control plane, Spot is only relevant for the worker nodes • CA (cluster-autoscaler) & ASG: Use one ASG with a diversified set of instance types • Run a DaemonSet on every worker to catch the Spot interruption and drain the node • Use labels to identify Spot nodes (for the DaemonSet, and other purposes – schedule non-prod?)
- 47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubernetes and EKS scaling • HPA (horizontal pod autoscaler) Autoscales the number of pods in a Deployment/ReplicaSet • CA (cluster-autoscaler) Autoscales the number of worker nodes in the cluster when: o Pods cannot be scheduled due to lack of compute resources o Nodes are underutilized and important pods can be rescheduled elsewhere
- 48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Availability Zone 1 etcd Master etcd Master Availability Zone 2 Availability Zone 3 etcd Master EKS – Master & etcd vs Worker nodes Worker nodes Your AWS account Managed master & etcd Not visible in your AWS account
- 49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 Spot with Amazon EMR
- 50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use Spot in EMR, unless heavily time-constrained Spot best practice – diversify and be instance type agnostic Use Instance Fleets - up to 5 instance types Improved chances of getting capacity, decreased impact from interruptions Spark will automatically recover from instance failures/interruptions Enable Dynamic allocation of executors (default in EMR) Decouple storage from compute (from HDFS to S3 EMRFS) Defined duration (Spot blocks) and fallback to On-Demand If you need auto-scaling (i.e for Presto), use uniform groups EMR – provisioning and instance types r3.4xlarge i3.4xlarge r4.4xlarge m4.10xlarge
- 51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. # Parallelized nodes Time # Parallelized nodes Time Job running time: 1 hourJob running time: 10 hours Parallelization with Spot Instances
- 52. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EMR Nodes – Long running clusters On- demand Master node EMR cluster Task nodesCore nodes On- demand Spot Spot Spot SpotHDFS HDFS Core nodes can be added and removed gracefully Master Node must keep running Cluster can tolerate loss of task nodes.
- 53. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EMR Nodes – Data driven workloads On- demand Master node EMR cluster Task nodesCore nodes On- demand On- demand Spot Spot Spot SpotHDFS HDFS • On-demand for Master • On-demand for Core, so no loss of data from these nodes • Spot for task nodes
- 54. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EMR Nodes – Cost-driven & Transient workloads Spot Master node EMR cluster Task nodesCore nodes Spot Spot Spot Spot Spot SpotHDFS HDFS • All node types use EC2 Spot • Can use any EC2 Spot instance choices.
- 55. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EMR Choices for EC2 Spot - Compute flexibility • Instance groups • Instance fleets
- 56. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EMR Instance groups
- 57. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EMR Instance fleets
- 58. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Instance fleets – Target capacity • Target capacity is a mix of OD and EC2 Spot • Can specify # of units each of these is contributing to target capacity • If no spot • Can specify provisioning timeout AND • Switch to OD
- 59. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. When to choose instance fleets • Widest variety of provisioning options • Choose a mix of instance types • Choose mix of on-demand/spot • Specify Target capacity as mix of on-demand or spot • Auto-scaling not available
- 60. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you https://aws.amazon.com/ec2/spot [email protected]