Cloud

Tutorial: AWS EC2 and AWS IoT

TA for class CSE 520S, Fall, Aug/30/2017

Haoran Li
Agenda
Ø  AWS EC2: IaaS
q  Project 0: Part 1

Ø  AWS IoT: A Platform-as-a-Service example

q  Project 0: Part 2

2
Pointers
Ø  Amazon EC2
q  http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/
EC2_GetStarted.html
Ø  Amazon IoT
q  http://docs.aws.amazon.com/iot/latest/developerguide/what-is-
aws-iot.html

Ø  Resource list for course projects

q  http://cps.cse.wustl.edu/index.php/List_of_Projects

Ø  Apply for $40 credits for Amazon AWS

q  https://aws.amazon.com/education/awseducate/apply/

3
AWS EC2
 Today’s “Mainframe”

Ø  Mainframe Computer (1970s) Ø  Datacenter (2016)

q  Multiuser shares one Computers Ø  Multiuser shares multiple
computers (A datacenter)
Service Provider Handles the Hardware.
You consume the computational resource.
[1] source: hBp://www.alamy.com/stock-photo-icl-2900-series-mainframe-computer-installaLon-and-programmers-at-92958461.html 5
[2] source: hBps://www.stratacore.com/the-advisor/data-center-m-a-recap-2016
Infrastructure as a Service (IaaS)
Ø  IaaS
APP
q  "physical server box” Web
q  Virtual Machine Service Data

You Manage
•  Memory
•  Storage Runtime
•  CPU
Middleware
•  Network
OS
Ø  Example

Service Provider Manages

Virtualization
q  AWS EC2
q  AWS HPC Server

Ø  Usecase Storage
q  Build up you VM cluster Network

6
Tutorial: Big Picture
Ø  Create
q  Two EC2 Ubuntu Instances
•  T2.micro
q  Ensure the connectivity

Ø  Micro benchmark
q  Latency
•  ping
q  Bandwidth
•  iperf3 EC2 t2.micro EC2 t2.micro

AWS

7
 Step 1: Create/Launch Instances
Ø  USE AWS Console WebUI
Ø  Create t2.micro Instance EC2: t2.micro
q  1 Xeon vCPU @ 2.5GHz
q  1GB Memory
q  8GB EBS Storage
q  ??? Network

Ø  Create Operating System

q  AMI: Ubuntu 16.04 Server

Ø  Get its Public DNS
(HostName)
Ø  Get the Private Key

Source: hBps://www.shareicon.net/tag/network?&&cl=darkslategray&&s=glyph 8
Access AWS EC2 Service
Ø  Sign Up and Sign In

Ø  Choose EC2 Service

Source: hBps://aws.amazon.com/console 9
Launch instances
Ø  Launch instance in EC2 Dashboard

10
Choose AMI
Ø  Choose Your Image Type (Template Operating System)

11
Instance Type (“Hardware” of VM)
Ø  Choose your Virtual CPU number and Memory
q  T2.micro 1 vCPU + 1GB @ $0.012 per Hour (running state)

ATTENTION: {CPU, Mem, Net} and Storage(EBS) are charged Separately!

12
Storage (“Hard Drive”)

Mount Point Default 8GB: $0.10 per GB-month

13
Review & Launch: Generate Private Key

Private Key:
Use it to login
your VM

14
 Check the Dashboard
Ø  Check the State: Running
Ø  Check the Public DNS

We will ssh into the

instance via this address

15
Step2: SSH into the VM
Ø  Use SSH and Key to login the VM

SSH EC2 t2.micro EC2 t2.micro

AWS

16
Change the Permission of the Private Key
Ø  Change it to “owner read only”, i.e. Permission 400
q  sudo chmod 400 {Your_Private_Key}.pem

17
 Login Into it
Ø  Host Address: Public DNS Windows User: Use PuWy as a SSH client[2]

Ø  Username: ubuntu[1]
q  ssh –i {Your_Private_Key} ubuntu@{Pub_DNS_Name}

[1] Manage User Account: hBp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/managing-users.html 18

[2] Using puBy to login: hBp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/puBy.html
Step3: Latency and Bandwidth Test
What’s the RTT delay?
What’s the Throughput?

EC2 t2.micro EC2 t2.micro

AWS

19
ConnecYvity SeZng
Ø  Security Group Settings
Ø  Edit Inbound Rules

20
Allow Inbound ConnecYons
Ø  Rules
q  Enable All TCP/UDP/ICMP

21
 Test the latency
Ø  Using Ping
q  Get RTT Min/Avg/Max/Dev

Ping staLsLcs

22
Test the Bandwidth
Ø  Using iperf3
q  Install: sudoapt-get install iperf3
q  Server: iperf3 –s
q  Client: iperf3 –c {server_ip}

1Gbps Bandwidth

23
Step 4: Remember to Stop/Terminate
Ø  All about Money
q  E.g. t2.micro 1-VCPU + 1GB Mem @ $0.012 per Hour (running
state)[1]

[1] EC2 Pricing: hBps://aws.amazon.com/ec2/pricing/on-demand/ 24

Remember to Stop Your VM
Ø  Either shutdown from CLI:
q  sudo shutdown –h now
Ø  Or Stop your instances from WebUI:

25
 Stopped: However, they will sYll charge you
Ø  Charge for EBS Storage[1]

Ø  Pricing
q  E.g. gp2 volume: $0.10 per GB-month
[1] Why charged by EBS: hBps://aws.amazon.com/premiumsupport/knowledge-center/ebs-charge-stopped-instance/
[2] EBS Pricing: hBps://aws.amazon.com/ebs/pricing/

26
 Stopped V.S. Terminated
Ø  “Terminate" your instance: PERMANENTLY DELETES the
instance, and free the EBS storage.
Ø  “Stop" the instance: Release the run time computational
resource, but keep the disk.
Ø  Note: Frequently committing your work to your repository.
State Stop Terminate
Root volume The volume(EBS) is preserved The volume(EBS) is deleted.
(e.g. /dev/sda1)
Billing You stop incurring charges for You stop incurring charges for
an instance as soon as its state an instance as soon as its state
changes to stopping. changes to shutting-down.

Ø  More info: Read “Differences Between Reboot, Stop,

and Terminate” in [1]
[1] Life Cycle: hBp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-lifecycle.html 27
[2] EBS Pricing: hBps://aws.amazon.com/ebs/pricing/
Project 0: Part 1 AWS EC2
Ø  Create Your AWS account
Ø  Build up two EC2 instances and Setting up the Connectivity
Ø  Run ping / iperf3 and get latency / bandwidth statistics
q  1. Repeat this experiment (inbound bandwidth test)
q  2. Create Instance in different zone (e.g. us-west-2a/b/c), test the
bandwidth between your PC and the EC2 Instance
•  Which Zone shows the best bandwidth / latency?
q  3. Pick up one “Zone”, write a script to repeat test every 2 hours (or
more frequent, if you want) in a consecutive 24 hours
•  Plot a figure to show the latency against time
•  Plot a figure to show the bandwidth against time
q  4. Run a single test for 10 minute, will the stats change?
•  Note: iperf3 –c {server_ip} –t {seconds}
Ø  Email your results (inline, you don’t need to write a report) to
[email protected]

28
AWS IoT: A PaaS Example
Pla`orm as a Service (PaaS)

You Manage
Ø  PaaS APP
q  You get a framework
Data
q  Host Application
q  Tools Runtime

Middleware

Service Provider Manages

Ø  Example
OS
q  AWS IoT
Virtualization

Ø  Usecase Server
q  Build up you’re smart A/C
Storage
controller
Network

8/30/17 30
Internet-of-Things
Ø  Things (Devices)
q  Many of them
•  Different Types
•  Isolated Systems

q  Data and Command

•  Sensing the world
•  Give Response

q  Challenge
•  United: Connected + Communication
•  Smart: Data Analytics + Strategy

8/30/17 Source: hBps://aws.amazon.com/iot-plakorm/ 31

hBp://www.brain-smart.net/smart-brain-health-blog/page/2/#axzz4W4oSp8a6
SoluYon: AWS IoT

United: Connect + CommunicaLon Smart: Other Cloud Service

Data Storage
Machine Learning

8/30/17 Source: hBps://aws.amazon.com/iot-plakorm/ 32

Tutorial: Hello AWS IoT!

Random
Integer
[1, 100]

Publish Forward

EC2 t2.micro

AWS Amazon SNS

8/30/17 Source: hBps://aws.amazon.com/iot-plakorm/ 33

Step 1: Create a Virtual "Thing”

AWS IoT

Virtual
“Thing” /
Shadow

8/30/17 34
Get into AWS Manage Console
Ø  Create your own AWS account
Ø  Sign In IoT Manage Console
q  https://aws.amazon.com/iot/

8/30/17 35
Create a thing
Ø  1. AWS IoT Menu
q  Registry
•  Things è Create
Ø  2. Give a name

8/30/17 36
Basic Interact: Publish
Ø  Using Embedded MQTT Client to Test

Ø  Check the Things Shadow

8/30/17 37
Basic Interact: Subscribe

8/30/17 38
Step 2: Connect a “Physical” Device

AWS IoT
Random
Integer
[1, 100]
CerLficate
MQTT Client

Virtual ABach
“Thing” /
Shadow

EC2 t2.micro

AWS
Copy Policy

8/30/17 39
Create and get CerYficates
Ø  Create Certificates
q  Security è Certificates è Create

Ø  Download Cert Files

•  1. public & private key
•  2. thing cert
•  3. Root CA for AWS

8/30/17 40
Create Policy and aWach it to cert
Ø  Create Policy

Ø  Attach Policy to Certificates

8/30/17 41
Connect your Device
Ø  Copy certificates to your EC2 Instance
q  Note: through scp utility

Ø  Choose your AWS SDK (support MQTT)

q  Node JS
q  Python
q  Java

Ø  You can also use third party MQTT tools

q  Python (paho-mqtt library)

8/30/17 42
Some Notes
Ø  1.You will need these certification when setting up the TLS1.2
verification

Ø  2.You will need the endpoint and port (8883) when connect
to AWS IoT Gateway

8/30/17 43
More: Rule Engine, Link with SNS services
Ø  Simple Notification Service
AWS IoT

Publish
Virtual
EC2 t2.micro “Thing” /
Shadow
AWS
Forward

Subscribe
Topic:
CSE520_Tutorial

Subscribe Amazon SNS

8/30/17 44
Create a Rule in Amazon IoT
Ø  Add a query to filter your inteseting topic (event)

Ø  Add an Action:
q  Forward this message to SNS
q  Specify Dest ARN
q  Enable Rule

8/30/17 45
NoYficaYon on SMS & Email

8/30/17 46
AWS IoT: A PaaS Example

Publish Subscribe

EC2 t2.micro EC2 t2.micro

AWS AWS
AWS IoT as a MQTT Message Broker
Forward
AWS IoT storages Things States

AWS IoT provides the capability to link

Against other services
Amazon SNS
Choose your Project topic:
Pick up the service you interest: Try it first.
And can you build your own service by using open source components?

8/30/17 Source: hBps://aws.amazon.com/iot-plakorm/ 47

Project 0 Part2: AWS IoT
Ø  Create a Thing in AWS IoT
Ø  Use Web MQTT tool to update Thing Shadow State
q  i.e. publish to shadow/update topic
Ø  Setup MQTT client in EC2 instance
Ø  Subscribe the shadow/update topic in EC2 instance
Ø  Email the screenshots to [email protected]

Subscribe

EC2 t2.micro

AWS

48
Project0: Part 2
Ø  Sample Screenshot:
q  1. Updated Shadow State
in WebUI
q  2. Printed message on EC2
terminal

49
Project 0:
Ø  Part 1 AWS EC2
q  Bandwidth Test

Ø  Part 2 AWS IoT

q  Subscribe to shadow/update

Ø  Due on: 10AM Sep/6 (Wednesday)

50
Pointers
Ø  Amazon EC2
q  http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/
EC2_GetStarted.html
Ø  Amazon IoT
q  http://docs.aws.amazon.com/iot/latest/developerguide/what-is-
aws-iot.html

Ø  Resource list for course projects

q  http://cps.cse.wustl.edu/index.php/List_of_Projects

Ø  Apply for $40 credits for Amazon AWS

q  https://aws.amazon.com/education/awseducate/apply/

51
Thanks!

Haoran Li
Aug/30/2017
Project 0:
Ø  Part 1 AWS EC2
q  Bandwidth Test

Ø  Part 2 AWS IoT

q  Subscribe to shadow/update

Ø  Due on: 10AM Sep/6 (Wednesday)

53
Project 0: Part 1 AWS EC2
Ø  Create Your AWS account
Ø  Build up two EC2 instances and Setting up the Connectivity
Ø  Run ping / iperf3 and get latency / bandwidth statistics
q  1. Repeat this experiment (inbound bandwidth test)
q  2. Create Instance in different zone (e.g. us-west-2a/b/c), test the
bandwidth between your PC and the EC2 Instance
•  Which Zone shows the best bandwidth / latency?
q  3. Pick up one “Zone”, write a script to repeat test every 2 hours (or
more frequent, if you want) in a consecutive 24 hours
•  Plot a figure to show the latency against time
•  Plot a figure to show the bandwidth against time
q  4. Run a single test for 10 minute, will the stats change?
•  Note: iperf3 –c {server_ip} –t {seconds}
Ø  Email your results (inline, you don’t need to write a report) to
[email protected]

54
Project 0 Part2: AWS IoT
Ø  Create a Thing in AWS IoT
Ø  Use Web MQTT tool to update Thing Shadow State
q  i.e. publish to shadow/update topic
Ø  Setup MQTT client in EC2 instance
Ø  Subscribe the shadow/update topic in EC2 instance
Ø  Email the screenshots to [email protected]

Subscribe

EC2 t2.micro

AWS

55
Project0: Part 2
Ø  Sample Screenshot:
q  1. Updated Shadow State
in WebUI
q  2. Printed message on EC2
terminal

56