AWS Primer and Quickstart

AWS PRIMER AND QUICKSTART Manish Pandit
02/01/2017

SETTING THE STAGE
The purpose of this talk is to introduce the most popular, relevant AWS offerings
Each one of these slides can be an hour long talk on their own.
Some of the topics can be forward referencing, but it all comes together at the end.
Best way to learn is to do – sign up for the free tier.

EVOLUTION
Distributed Computing
The Internet
Virtualization and APIs
“XYZ as a service”

SOFTWARE AS A SERVICE
Abbreviated as SaaS
Provides a “Service” or “Services” to customers in a multi-tenant manner
Namely for HRMS
Salesforce for CRM
Pingdom to monitor site availability and performance
Twillio to manage SMS
Almost always exposes APIs for custom integration

PLATFORM AS A SERVICE
Abbreviated as PaaS
A Virtualized Environment for clients to run their applications without having to worry about the
underlying details – Operating System, Application Servers, Hardware, Networking
Heroku for Rails Applications
Google App Engine for Python, Java, PHP and Go

INFRASTRUCTURE AS A SERVICE
Abbreviated as IaaS
A Virtualized Environment for clients to have “almost” full control of the underlying hardware,
operating system, networking without having to worry about building these components from scratch.
Think of IaaS as lego blocks which you can assemble based on what you need, and how many you
need. These blocks can be CPU, Memory, Storage, Network IO, Operating Systems, Application
Servers….
IaaS provides the building blocks for PaaS and SaaS
AWS
Microsoft Azure
Google Cloud Platform

(HIGH) AVAILABILITY
Minimal Disruption
Failover (DR)
Measured in 9s

FAULT TOLERANCE
Zero Service Disruption
Usually achieved via Redundancy
Costs more, due to redundancy
(Very) Hard to achieve (Think of the database J)

SCALABILITY
Vertical
Add more CPU/Memory/Storage
Always limited by the max capacity
Horizontal
Adding more components and connecting them to perform as a single unit
Usually referred to as clustering
Can be done on-demand (Elastic!)

AWS
Provides an easy way to build Highly available, Fault Tolerant, Elastic Architectures

BRIEF HISTORY
Evolved out of Amazon’s own infrastructure – rent out server capacity
Officially launched in August 2006 with EC2 (Elastic Compute Cloud)
Textbook example of Innovation – Small teams, Focused, Independent

AWS COMPONENTS
If you have an AWS account, please log in and follow along!

WAYS TO ACCESS AWS
AWS Console – Simplest, web-based tool
AWS CLI (and brand new, AWS shell), python based
AWS REST APIs
AWS SDKs for Java, C++, Python, Ruby, PHP, Go, Node, iOS, JS, Android….
Tip: Use Eclipse, it has a well supported AWS Plugin

AWS – HIGH AVAILABILITY & FAULT TOLERANCE
Regions
Geographically isolated, independent hosting centers.
Offered to provide fault tolerance
14 plus govcloud
Availability Zones
Within a region, they provide low-latency links between each other, but are isolated
Can be anywhere from 3-5 depending on the region

REGIONS & ZONES
Region – us-east-1
AZs – us-east-1a, us-east-1b, us-east-1c, us-east1d,
us-east-1e

CALLING AWS SERVICES
Every user can get an access key and a secret
There can be at-most 2 key-secret pairs associated with a user (why?)
Do not confuse this with the AWS instance’s keypair
These credentials are used in AWS SDK, AWS REST APIs, and AWS CLI to consume
services via code or scripts.
(Does this mean if your application needs to talk to S3, you will add these keys in
application config and launch an instance, assuming the user has S3 access policy tied
to him?)

AWS RESOURCES
AWS resources are uniquely identified via an ARN (Amazon Resource Name)
arn:aws:iam::141849685257:user/mpandit
arn:aws:iam::141849685257:group/admins
arn:aws:sqs:us-east-1:141849685257:mpandit_test_queue
arn:aws:s3:::api-docs
arn:aws:dynamodb:us-east-1:141849685257:table/mpandit_test
arn:aws:ec2:us-east-1::image/ami-1a2b3c4d
AWS resources can be at global level, account level, or region level
Some services support IAM resource level permissions

COMPUTE
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable
compute capacity in the cloud. It is designed to make web-scale cloud
computing easier for developers.

EC2
Allows you to launch instances
The instances are created from AMIs
The AMIs can be created from Instances
AMIs are specific to a region – you can however copy them to another region
The instances can have local storage, or EBS-backed storage (Root device types)
Are accessed via keypairs, which are generated when the instance is launched.
The keypairs can be re-used.

EC2 VIRTUALIZATION
AWS supports two types of virtualization
HVM (relies on the host machine’s hardware optimized for virtualization)
Para-virtual (relies on the host machine with no special virtualization extensions)
The AMI will tell you the virtualization type
HVM is usually preferred for GPU and enhanced networking type applications
The virtualization software used by AWS is a super customized version of Xen
Hypervisor

AMI
AMIs are Templates defining the Operating System, Storage Type, Application
Servers and components, Virtualization Type, Root Device Type
They’re always tied to a region in which they’re created
They can be shared publically, or within accounts, or kept private
Supported Operating Systems are RHEL, Ubuntu, Microsoft Windows Server, Amazon
Linux, Centos, Debian, SUSE, etc.

EC2 INSTANCE FAMILIES
Instance Families define the performance and/or capacity of the instance
General Purpose – t2, m4, m3
Compute Optimized – c4, c3
Memory Optimized – r3, r4, x1
Storage Optimized – i2, d2
Accelerated/GPU Computing – p2, g2
Each instance family has sizes, specified as nano, micro, small, medium, large, xlarge,
2xlarge, 4xlarge, 8xlarge, 10xlarge, 16xlarge, 32xlarge (based on the family).

EC2 INSTANCE PROVISIONING
On-Demand
What we usually do, create an instance on the fly
Reserved
We reserve the capacity and can (and usually do) pay upfront to save costs
They’re tied to a specific AZ, but you can reserve them across AZs
Reservation terms are 1-3 years
Spot
You bid, but AWS will terminate your instance if the bid price goes up
Scheduled
Same as reserved, except are provisioned on a recurring basis

STORAGE
AWS S3, Simple Storage Service is an object storage
You cannot run a database off of S3 (since it operates on objects, not files)
Objects are stored in S3 buckets
Standard S3 provides 11 9s of durability and 4 9s of availability over a given year
S3 has multiple storage classes to fit the use case (and budget)
Any S3 bucket can be made public and turned into a website
S3 objects can be encrypted via the AES-256 encryption (SSE-S3), or via KMS
managed keys (SSE-KMS), or via customer provided keys (SSE-C).

STORAGE
EBS (Elastic Block Storage) is a block level storage provided by AWS
EBS Volumes are virtual disks to store files, and are fixed in size (when they’re
created) and are tied to an AZ
Can be provisioned as HDD (cheapest), or SSD General Purpose or SSD PIOPS
Can be attached to an instance, and can be unmounted
Can be “snapshotted”. These snapshots are stored in S3.
The snapshots can be converted to volumes. (why?)
The snapshots are also used to resize the volume (why?)
EBS volumes can be encrypted with 256-AES and CMK (Customer Master Keys which
are managed with KMS, transparently to us).

DATABASES
AWS supports Relational Database Service (RDS) for MySQL, Postgres, Oracle, MS-
SQL, MariaDB, and Aurora
Amazon Aurora is an optimized, MySQL syntax compatible database
AWS supports NoSQL Database via DynamoDB (both a Document and key-value
based store).
RDS does automatic backups, has a defined maintenance window for patches etc.
(How is access to RDS controlled?)

MESSAGING
AWS supports both push and pull models for messaging.
SQS, Simple Queue Service, is one of the oldest AWS offering
Standard SQS allows creating Queues with guaranteed “at least once” delivery.
However, given the high availability architecture, the order of the messages is not
guaranteed
FIFO SQS is new, and allows for ordered messages but has a limit of 300 TPS
SNS, Simple Notification Service is a push messaging service that is used to send SMS
notifications, HTTP notifications, lamba triggering, or emails.
SNS can also post messages to SQS, used for Fan-out messaging
SES, Simple Email Service is used for email messages like campaigns, etc.

LAMBDA
Code as a Service
I believe this is the next wave of cloud computing – so get on board!
Is a piece of code that can be run when trigged
Can be triggered via many AWS events (API call, S3, DynamoDB, Cron,
Cloudwatch…)
Pricing is calculated based on memory*compute time

SECURITY, IDENTITY & COMPLIANCE
IAM (Identity and Access Management) is the foundation of all AWS account security.
IAM allows users to be created within an AWS account
These users can be added to groups (no nesting)
The permissions are managed as policies, and a user/group can be tied to
permissions. Examples can be a policy to allow read access to a particular S3 bucket,
or a policy to allow all S3 operations.
AWS recommends restrictive permissions, in that permissions have to be explicitly
attached to a user.
IAM Roles are attached to an instance to allow that instance to use AWS services
specified by the permissions in that role without the application having to use any
credentials. There is no such thing as a user-role.

REDSHIFT
Redshift is a petabyte scale, fully managed, data warehouse.
Supports standard SQL (Postgres) query model
Has custom JDBC drivers that can be downloaded for use

ANALYTICS
EMR – Elastic Map Reduce, a hosted Hadoop cluster
Kinesis – Used for processing and analyzing streaming data
ML – Amazon’s Machine Learning
Data Pipeline – Used for ETL type operations, where we can define data-driven
workflows and transformations

CLOUDWATCH
Used to monitor AWS resources
All AWS services support CloudWatch, which is a collection of metrics
The metrics depend on the AWS resource being monitored – every resource comes
with a predefined set of them.
The metrics can be used to create alarms, which can send emails, send SNS messages,
invoke Lambda functions, or trigger autoscaling actions.
Each CloudWatch alarm can have one of 3 states – INSUFFICIENT_DATA, OK, ALARM

ROUTE 53
Route 53 is Amazon’s Highly Available DNS service.
(Why is it called Route 53?)
Can be used to host domains, create and manage (many types of) DNS records.
Supports 5 routing policies applicable to highly available, fault tolerant systems
Simple
Weighted
Latency
Failover (used for DR)
Geolocation

CLOUDFORMATION
AWS CloudFormation provides JSON Documents which describe a “stack”, or a
collection of related AWS resources.
Often referred to as IaC (Infrastructure as Code)
A stack can be deleted, updated, or created
Plenty of open source CFTs for generic deployments
Terraform makes it quite a bit easier to deal with CFTs

ELB - ELASTIC LOAD BALANCERS
Software load balancers, to provide Fault Tolerance
Highly Available, Elastic
Enabler for Auto-Scaling
Support cross-zone load balancing for HTTP, HTTPS, TCP and SSL traffic
Supports SSL Termination
Rely on a healthcheck (TCP or HTTP) check to determine backend state
TCP:22
HTTP:80/index.html
A failed healthcheck marks the instance as “Out of Service”

AUTO SCALING GROUPS
An autoscaling group (ASG) is a collection of similar EC2 Instances (AMIs, type,
Subnets, Security Groups…)
An ASG has an initial size or desired capacity (number of EC2 instances in it)
An ASG can be associated with an ELB (almost always!)

AUTOSCALING POLICIES
Use CloudWatch Alarms, or SQS to trigger capacity changes
Example can be, if Average CPU utilization is > 80%, launch 2 new instances
Can be dynamic, manual (update desired capacity of ASG), or scheduled
The ASG will always have the number of instances specified in the group configuration
(desired capacity)
Cool Down policies govern the scale-down of the ASG

AWS NETWORKING
Instances are launched within a VPC, Virtual Private Cloud
A VPC is a logically isolated network within an AWS Region
VPC has a CIDR range (/16 for default)
Every subnet has a Route Table associated with it
You can create subnets within a VPC (/20 for default)
Subnets are tied to an AZ, cannot span across AZs (Why?)
Subnets can be private, or public

AWS NETWORKING
Private subnets have no connectivity to the internet
Public subnets have a route to the internet via an Internet Gateway (provided by
AWS)
Private subnets may need outbound connections to the internet (why?), which is
achieved via a NAT instance, or a NAT gateway
There can only be 1 Internet Gateway, and 1 NAT gateway in a VPC

AWS NETWORKING - SECURITY
Every VPC has a default NACL (Network Access Control List) that controls all inbound
and outbound traffic
A subnet is tied to one and only one NACL, but 1 NACL can be shared across subnets
NACLs are stateless, in that you do not have to specify the rules to allow for the
response traffic.
For example, you can create rules to allow in/out SSH traffic on port 22 on a given
source (could be the internet 0.0.0.0/0, or a specific IP, or a specific CIDR block).
With these rules, and all other traffic set to DENY, any instance launched in a subnet
tied to this NACL will allow SSH from the given source (not so fast, we got security
groups!)

AWS NETWORKING - SECURITY
Every instance can be associated with multiple security groups.
A security group is like an instance level firewall.
Security groups are stateful, in that response traffic is automatically allowed when a
request is made out from the instance
Security groups can be shared across instances within the VPC
Tip : We use the security groups to create a bastion host in a private subnet, and all
instances in that subnet can allow SSH only from this host.

AWS NETWORKING
VPCs can be connected via VPC peering
Data centers can be connected to VPCs via VPN Gateways
VPN Gateways carry traffic over the internet.
For dedicated, highly available, high speed connections, it is recommended to use
AWS DirectConnect. Think of it as a private, direct connection between the data
center and AWS.

TOO MUCH?
Your AWS account comes with a default VPC with a public subnet in it.
/16 CIDR
Comes with a default subnet in each AZ (/20)
Attached an Internet Gateway to the VPC
Adds a route to the Internet Gateway in the VPC’s route table
Has a default security group (allow all traffic on all ports)
Has a default NACL (Allows all traffic inbound and outbound)
You an launch an instance from the get-go and everything will work as expected of a public
instance.
DO NOT DELETE YOUR DEFAULT VPC!

BILLING
Resources are billed even when not used
It is very tempting to leave instances running, or leave ELBs with no back end servers
to send traffic to – but it all adds up
You pay for every bit that moves on the network, every bit that sits on storage
Consolidated billing helps with aggregating multiple accounts (Prod, QA, Dev) to take
advantage of tiered pricing of many services
Can set up billing alerts

THINGS I DID NOT COVER
EFS Batch WAF
ElasticSearch Beanstalk KMS
CodeDeploy Glacier CodeCommit
CodePipeline Storage Gateways SWF
CloudTrail ElasticCache IoT
Trusted Advisor OpsWorks
Certificate Management CloudHSM
CloudFront API Gateway ….many, many more!

RESOURCES
AWS Whitepapers
AWS Re:Invent Videos on Youtube (Look for the 2016 ones)
AWS FAQs for each product

LAB
Using the default VPC, build a simple Hello World website using any server you want
Create a Security Group to allow inbound 80 from 0.0.0.0/0
Create an ASG from the instance with desired capacity as 3.
Create an ELB and attach it to the ASG
Watch AutoScaing happen as new instance(s) spin up to match the desired capacity
Delete the ASG
Delete the ELB

AWS Primer and Quickstart

Recommended

More Related Content

What's hot (7)

Similar to AWS Primer and Quickstart (20)

More from Manish Pandit (20)

Recently uploaded (20)

AWS Primer and Quickstart