NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.

com
EC2 Instance Storage Section

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
What’s an EBS Volume?
• An EBS (Elastic Block Store) Volume is a network drive you can attach
to your instances while they run
• It allows your instances to persist data, even after their termination
• They can only be mounted to one instance at a time (at the CCP
level)
• They are bound to a specific availability zone

• Analogy: Think of them as a “network USB stick”

• Free tier: 30 GB of free EBS storage of type General Purpose (SSD) or
Magnetic per month

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
EBS Volume
• It’s a network drive (i.e. not a physical drive)
• It uses the network to communicate the instance, which means there might be a bit of
latency
• It can be detached from an EC2 instance and attached to another one quickly

• It’s locked to an Availability Zone (AZ)

• An EBS Volume in us-east-1a cannot be attached to us-east-1b
• To move a volume across, you first need to snapshot it

• Have a provisioned capacity (size in GBs, and IOPS)

• You get billed for all the provisioned capacity
• You can increase the capacity of the drive over time

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
EBS Volume - Example
US-EAST-1A US-EAST-1B

EBS EBS EBS EBS EBS

(10 GB) (100 GB) (50 GB) (50 GB) (10 GB)
unattached

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
EBS – Delete on Termination attribute

• Controls the EBS behaviour when an EC2 instance terminates

• By default, the root EBS volume is deleted (attribute enabled)
• By default, any other attached EBS volume is not deleted (attribute disabled)
• This can be controlled by the AWS console / AWS CLI
• Use case: preserve root volume when instance is terminated

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
EBS Snapshots
• Make a backup (snapshot) of your EBS volume at a point in time
• Not necessary to detach volume to do snapshot, but recommended
• Can copy snapshots across AZ or Region

US-EAST-1A US-EAST-1B

EBS Snapshot

EBS snapshot restore EBS

(50 GB) (50 GB)

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
EBS Snapshots Features
EBS Snapshot EBS Snapshot
Archive
• EBS Snapshot Archive
• Move a Snapshot to an ”archive tier” that is archive
75% cheaper
• Takes within 24 to 72 hours for restoring
the archive

• Recycle Bin for EBS Snapshots EBS Snapshot Recycle Bin

• Setup rules to retain deleted snapshots so
you can recover them after an accidental delete
deletion
• Specify retention (from 1 day to 1 year)

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
AMI Overview
• AMI = Amazon Machine Image
• AMI are a customization of an EC2 instance
• You add your own software, configuration, operating system, monitoring…
• Faster boot / configuration time because all your software is pre-packaged
• AMI are built for a specific region (and can be copied across regions)
• You can launch EC2 instances from:
• A Public AMI: AWS provided
• Your own AMI: you make and maintain them yourself
• An AWS Marketplace AMI: an AMI someone else made (and potentially sells)

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
AMI Process (from an EC2 instance)
• Start an EC2 instance and customize it
• Stop the instance (for data integrity)
• Build an AMI – this will also create EBS snapshots
• Launch instances from other AMIs

Custom AMI
US-EAST-1A US-EAST-1B
Launch
Create AMI from AMI

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
EC2 Image Builder
• Used to automate the creation of Virtual Machines or container images
• => Automate the creation, maintain, validate and test EC2 AMIs
• Can be run on a schedule (weekly, whenever packages are updated, etc…)
• Free service (only pay for the underlying resources)

Build Components applied Test suite is run

(customize software on instance) (is the AMI working, secure?)

create create
AMI is distributed
(can be multiple regions)
EC2 Image Builder Builder EC2 Instance New AMI Test EC2 Instance

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
EC2 Instance Store
• EBS volumes are network drives with good but “limited” performance
• If you need a high-performance hardware disk, use EC2 Instance
Store

• Better I/O performance

• EC2 Instance Store lose their storage if they’re stopped (ephemeral)
• Good for buffer / cache / scratch data / temporary content
• Risk of data loss if hardware fails
• Backups and Replication are your responsibility

© Stephane Maarek
NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
Very high IOPS
Local EC2 Instance Store

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
EFS – Elastic File System
• Managed NFS (network file system) that can be mounted on 100s of EC2
• EFS works with Linux EC2 instances in multi-AZ
• Highly available, scalable, expensive (3x gp2), pay per use, no capacity planning
us-east-1a us-east-1b us-east-1c

EC2 Instances EC2 Instances EC2 Instances

Security Group

EFS FileSystem

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
EBS vs EFS
Availability Zone 1 Availability Zone 2 Availability Zone 1 Availability Zone 2

EBS EBS
EFS EFS
Mount Mount
Target Target
snapshot restore

EBS Snapshot
EFS
© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
EFS Infrequent Access (EFS-IA)
• Storage class that is cost-optimized for files not
accessed every day
• Up to 92% lower cost compared to EFS Standard
• EFS will automatically move your files to EFS-IA no access
for 60 days
based on the last time they were accessed EFS Standard

• Enable EFS-IA with a Lifecycle Policy move Lifecycle Policy

• Example: move files that are not accessed for 60
days to EFS-IA
• Transparent to the applications accessing EFS
EFS IA

Amazon EFS File System

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
Shared Responsibility Model for EC2 Storage

• Infrastructure • Setting up backup / snapshot

procedures
• Replication for data for EBS
volumes & EFS drives • Setting up data encryption
• Replacing faulty hardware • Responsibility of any data on
the drives
• Ensuring their employees • Understanding the risk of
cannot access your data using EC2 Instance Store

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
Amazon FSx – Overview
• Launch 3rd par ty high-performance file systems on AWS
• Fully managed service

FSx for Lustre FSx for FSx for

Windows File NetApp ONTAP
Server

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
Amazon FSx for Windows File Server
• A fully managed, highly reliable, and Region

scalable Windows native shared Availability Zone 1 Availability Zone 2

file system
• Built on Windows File Server FSx for Windows FSx for Windows
File Server File Server
• Supports SMB protocol & EC2 instance
\\fs-0123456789abcdef0.example.com\share
Windows NTFS
• Integrated with Microsoft Active
Directory over SMB

• Can be accessed from AWS or Corporate data center

your on-premise infrastructure

Windows client

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
Amazon FSx for Lustre
• A fully managed, high-performance, scalable file storage for High Performance
Computing (HPC)
• The name Lustre is derived from “Linux” and “cluster”
• Machine Learning, Analytics, Video Processing, Financial Modeling, …
• Scales up to 100s GB/s, millions of IOPS, sub-ms latencies
Region Corporate data center

link access your data

Amazon S3

Amazon FSx for Lustre

Server

access your data

Compute instances

© Stephane Maarek
 NOT FOR DISTRIBUTION © Stephane Maarek www.datacumulus.com
EC2 Instance Storage - Summary
• EBS volumes:
• network drives attached to one EC2 instance at a time
• Mapped to an Availability Zones
• Can use EBS Snapshots for backups / transferring EBS volumes across AZ
• AMI: create ready-to-use EC2 instances with our customizations
• EC2 Image Builder : automatically build, test and distribute AMIs
• EC2 Instance Store:
• High performance hardware disk attached to our EC2 instance
• Lost if our instance is stopped / terminated
• EFS: network file system, can be attached to 100s of instances in a region
• EFS-IA: cost-optimized storage class for infrequent accessed files
• FSx for Windows: Network File System for Windows servers
• FSx for Lustre: High Performance Computing Linux file system

© Stephane Maarek