Scaling horizontally on AWS

Scaling horizontally on AWS
Bozhidar Bozhanov, LogSentinel

About me
• Senior software engineer and architect
• Founder & CEO @ LogSentinel
• Blog: techblog.bozho.net
• Twitter: @bozhobg
• Stackoverflow top 50

Why?
• Why high availability?
• Why scalability?
• To account for increased load
• If you have decent HA, you’re likely scalable
• Don’t overdesign
• Why AWS (or any cloud provider)?

AWS
• IaaS (Infrastructure as a service) originally (EC2)
• Virtual machines
• Load balancers
• Security groups
• PaaS services ontop
• Multiple regions – US, EU, Asia, etc.
• Each region has multiple availability zones (roughly equal to “data centers”)
• Cross-availability zone is easy
• Cross-region is harder
• Similar to Azure, Google Cloud, etc.

Rule of thumb: stateless applications
• No persistent state on the application nodes
• Caches and temporary files are okay
• Distributed vs local cache
• Session state: distributed vs no session state (e.g. JWT)
• Makes the application layer horizontally scalable
• Application nodes are disposable

Executing only once in a cluster
• Sometimes you need to execute a scheduled piece of code only once in a cluster
• Database-backed schedule job management
• Distributed locks (Hazelcast)
• Using queues (SQS, AmazonMQ, RabbitMQ)?

Scaling
• Autoscaling groups
• Groups of virtual machines (instances) with identical configuration
• Scale-up - configure criteria for launching new virtual machines – e.g. “more than 5
minutes of CPU utilization over 80%”
• Scale-down – configure criteria for destroying virtual machines
• Allows for handling spikes, or gradual increase of load
• Spot instances
• Cheap instances you “bid” for. Can be reclaimed at any time
• Useful for heavy background processes.
• Useful for test environments.

Data stores
• Managed
• RDBMS (AWS RDS) – MySQL, MariaDB, Postgres, Oracle, MS SQL
• Search engines – Elasticsearch
• Caches – Elasticache (Redis and memcached)
• Custom:
• Amazon Aurora
• CloudSearch
• S3, SimpleDB, Dynamo
• Own installation: spin VMs, install anything you like (e.g. Cassandra, Hbase, own
Postgres, own Elasticsearch, own caching solution)

Scaling data stores
• The custom ones are automatically scaled (S3, SimpleDB)
• The managed ones are scaled by configuration
• Own deployments are scaled via auto-scaling groups
• Data sharding vs replication with consistent hashing
• Resharding is not trivial
• Replication with consistent hashing can handle scaling up automatically *

Elastic load balancer
• AWS-provided software load balancer
• Points to specified target machines or group of machines (roughly ASGs)
• Configurable: protocols, ports, healthcheck, monitoring metrics
• TLS termination
• AWS-managed certificates
• Load balancer in front of application nodes
• Load balancer in front of data store nodes
• vs application-level load-balancing (configuration vs fetching db nodes dynamically)

Things to automate
• Hardware and network resources (CloudFormation)
• Application and database configuration (OpsWorks: Puppet, Chef, S3+bash, Capistrano)
• Instances
• launch configurations + bash
• docker containers + bash (Elastic Container Service vs Fargate, Kubernetes)
• Why automate?
• because autoscaling benefits from automated instance creation

Scripted stacks
• You can create all instances, load balancers, auto-scaling groups, launch configurations,
security groups, domains, elasticsearch domains, etc., etc.. manually
• But CloudFormation is way better
• JSON or YAML
• CloudFormation manages upgrade
• Stack parameters (instance types, number of nodes, domains used, s3 buckets, etc.)

"DatabaseLaunchConfiguration": {
"Type": "AWS::AutoScaling::LaunchConfiguration",
"Properties": {
"AssociatePublicIpAddress": true,
"IamInstanceProfile": {
"Ref": "InstanceRoleInstanceProfile"
},
"ImageId": {
"Fn::FindInMap": [
{
"Ref": "DatabaseStorageType"
},
{
"Ref": "AWS::Region"
},
"Linux"
]
},
"InstanceType": {
"Ref": "DatabaseInstanceType"
},
"SecurityGroups": [
{
"Ref": "DatabaseSecurityGroup"
}
]
}

"WebAppLoadBalancer": {
"Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
"Properties": {
"Scheme": "internet-facing",
"Type": "application",
"Subnets": [
{
"Ref": "PublicSubnetA"
},
{
"Ref": "PublicSubnetB"
},
{
"Ref": "PublicSubnetC"
}
],
"SecurityGroups": [
{
"Ref": "WebAppLoadBalancerSecurityGroup"
}
]
}
},

"WebAppTargetGroup": {
"Type": "AWS::ElasticLoadBalancingV2::TargetGroup",
"Properties": {
"HealthCheckIntervalSeconds": 30,
"HealthCheckProtocol": "HTTP",
"HealthCheckTimeoutSeconds": 10,
"HealthyThresholdCount": 2,
"HealthCheckPath": "/healthcheck",
"Matcher": {
"HttpCode": "200"
},
"Port": 8080,
"Protocol": "HTTP",
"TargetGroupAttributes": [
{
"Key":
"deregistration_delay.timeout_seconds",
"Value": "20"
}
],
"UnhealthyThresholdCount": 3,
"VpcId": {
"Ref": "VPC"
}
}
},

Why CloudFormation?
• Replicable stacks
• Used for different customers
• Used for different environments
• Used for disaster recovery
• Having a clear documentation of your entire infrastructure
• DevOps friendly
• Not that hard to learn
• Drawbacks: slow change-and-test cycles, proprietary
• Alternatives: Terraform
• Tries to abstract stack creation independent of provider, but you still depend on
proprietary concepts like ELB, security groups, etc.

Configuration provisioning
• OpsWorks – hosted Puppet or Chef
• Capistrano – “login to all machines and do x, y, z”
• S3 – simple, no learning curve
• Instance launch configuration includes files to fetch from S3 (app.properties,
db.properties, cassandra.conf, mysql.conf, etc.)
• CloudFormation can write dynamic values to conf files (e.g. ELB address)

"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash -xn",
"yum update -y aws-cfn-bootstrapn",
"yum install -y aws-clin",
"cat <<EOF >> /var/app/app.propertiesn",
{
"Fn::Join": [
"",
[
"n",
“db.host=",
{
"Ref": "DatabaseELBAddress"
},
"n",
"elasticsearch.url=https://",
{
"Ref": "ElasticSearchDomainName"
},
"n",
"root.url=https://",
{
"Ref": "DomainName"
]
]
},
"EOF“

Automated instance setup
• Elastic Container Services
• Deploy docker containers on EC2 instances
• Fargate abstracts the need to manage the underlying EC2 instance
• Kubernetes – vendor-independent
• But don’t rush into using kubernetes (or Docker for that matter).
• Packer – creates images
• Manual
• Launch configuration to fetch and execute setup.sh
• Allows for easy zero downtime blue-green deployment
• Instance setup changed? Destroy the it and launch a new one
• Simple. Simple is good.

Blue-green deployment
• Two S3 “folders” – blue and green
• Shared database
• Two autoscaling groups – blue (currently active) and green (currently passive)
• Upload new release artifact (e.g. fat jar) to s3://setup-bucket/green
• Activate the green ASG (increase required number of instances)
• Wait for nodes to launch
• Execute acceptance tests
• Switch DNS record (Route53) from blue ELB to green ELB
• Turquoise (intermediate deployment in case of breaking database changes)
• Can be automated via script that uses AWS CLI or APIs

Other useful services
• IAM – user and role management (each instance knows its role, no need for passwords)
• S3 – distributed storage / key-value store / universally applicable
• CloudTrail – audit trail of all infrastructure changes
• CloudWatch – monitoring of resources
• KMS – key management
• Glacier – cold storage
• Lambda – “serverless” a.k.a. function execution

General best practices
• Security groups
• Only open ports that you need
• Bastion host – entry point to the stack via SSH
• VPC (virtual private cloud)
• your own virtual network, private address space, subnets (per e.g. availability zone),
etc.
• Multi-factor authentication

Conclusion
• Scalability is a function of your application first and infrastructure second
• AWS is pretty straightforward to learn
• You can have scalable, scripted infrastructure without big investments
• New services appear often – check them out
• Vendor lock-in is almost inevitable
• But concepts are (almost) identical across cloud providers
• If something can be done easily without an AWS-specific service, prefer that
• Bash is inevitable

Scaling horizontally on AWS

Recommended

More Related Content

What's hot (19)

Similar to Scaling horizontally on AWS (18)

More from Bozhidar Bozhanov (20)

Recently uploaded (20)

Scaling horizontally on AWS