100% found this document useful (1 vote)
6 views

Practical Machine Learning with AWS : Process, Build, Deploy, and Productionize Your Models Using AWS Himanshu Singh pdf download

The document is about the book 'Practical Machine Learning with AWS' by Himanshu Singh, which covers processes for building, deploying, and productionizing machine learning models using AWS. It includes various chapters on AWS services, machine learning concepts, and practical applications. The book is available for download and includes supplementary materials on GitHub.

Uploaded by

bltvzbxgzg265
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
100% found this document useful (1 vote)
6 views

Practical Machine Learning with AWS : Process, Build, Deploy, and Productionize Your Models Using AWS Himanshu Singh pdf download

The document is about the book 'Practical Machine Learning with AWS' by Himanshu Singh, which covers processes for building, deploying, and productionizing machine learning models using AWS. It includes various chapters on AWS services, machine learning concepts, and practical applications. The book is available for download and includes supplementary materials on GitHub.

Uploaded by

bltvzbxgzg265
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 56

Practical Machine Learning with AWS : Process,

Build, Deploy, and Productionize Your Models


Using AWS Himanshu Singh download

https://textbookfull.com/product/practical-machine-learning-with-
aws-process-build-deploy-and-productionize-your-models-using-aws-
himanshu-singh/

Download full version ebook from https://textbookfull.com


We believe these products will be a great fit for you. Click
the link to download now, or visit textbookfull.com
to discover even more!

Practical Machine Learning with AWS Process Build


Deploy and Productionize Your Models Using AWS 1st
Edition Himanshu Singh

https://textbookfull.com/product/practical-machine-learning-with-
aws-process-build-deploy-and-productionize-your-models-using-
aws-1st-edition-himanshu-singh/

Biota Grow 2C gather 2C cook Loucas

https://textbookfull.com/product/biota-grow-2c-gather-2c-cook-
loucas/

Learning AWS Design build and deploy responsive


applications using AWS Cloud components Aurobindo
Sarkar

https://textbookfull.com/product/learning-aws-design-build-and-
deploy-responsive-applications-using-aws-cloud-components-
aurobindo-sarkar/

Practical Machine Learning and Image Processing: For


Facial Recognition, Object Detection, and Pattern
Recognition Using Python Himanshu Singh

https://textbookfull.com/product/practical-machine-learning-and-
image-processing-for-facial-recognition-object-detection-and-
pattern-recognition-using-python-himanshu-singh/
Beginning MLOps with MLFlow: Deploy Models in AWS
SageMaker, Google Cloud, and Microsoft Azure Sridhar
Alla

https://textbookfull.com/product/beginning-mlops-with-mlflow-
deploy-models-in-aws-sagemaker-google-cloud-and-microsoft-azure-
sridhar-alla/

Deploy Containers on AWS With EC2, ECS, and EKS Shimon


Ifrah

https://textbookfull.com/product/deploy-containers-on-aws-with-
ec2-ecs-and-eks-shimon-ifrah/

Deploy Machine Learning Models to Production: With


Flask, Streamlit, Docker, and Kubernetes on Google
Cloud Platform Pramod Singh

https://textbookfull.com/product/deploy-machine-learning-models-
to-production-with-flask-streamlit-docker-and-kubernetes-on-
google-cloud-platform-pramod-singh/

Beginning MLOps with MLFlow : Deploy Models in AWS


SageMaker, Google Cloud, and Microsoft Azure 1st
Edition Sridhar Alla

https://textbookfull.com/product/beginning-mlops-with-mlflow-
deploy-models-in-aws-sagemaker-google-cloud-and-microsoft-
azure-1st-edition-sridhar-alla/

AI as a Service: Serverless machine learning with AWS


1st Edition Peter Elger

https://textbookfull.com/product/ai-as-a-service-serverless-
machine-learning-with-aws-1st-edition-peter-elger/
Practical
Machine Learning
with AWS
Process, Build, Deploy, and Productionize
Your Models Using AWS

Himanshu Singh
Practical Machine
Learning with AWS
Process, Build, Deploy,
and Productionize Your Models
Using AWS

Himanshu Singh
Practical Machine Learning with AWS
Himanshu Singh
ALLAHABAD, Uttar Pradesh, India

ISBN-13 (pbk): 978-1-4842-6221-4 ISBN-13 (electronic): 978-1-4842-6222-1 


https://doi.org/10.1007/978-1-4842-6222-1

Copyright © 2021 by Himanshu Singh


This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with
every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an
editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the
trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not
identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to
proprietary rights.
While the advice and information in this book are believed to be true and accurate at the date of publication,
neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or
omissions that may be made. The publisher makes no warranty, express or implied, with respect to the
material contained herein.
Managing Director, Apress Media LLC: Welmoed Spahr
Acquisitions Editor: Celestin Suresh John
Development Editor: Laura Berendson
Coordinating Editor: Aditee Mirashi
Cover designed by eStudioCalamar
Cover image designed by Freepik (www.freepik.com)
Distributed to the book trade worldwide by Springer Science+Business Media New York, 1 New York Plaza,
Suite 4600, New York, NY 10004-1562, USA. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@
springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is a California LLC and the sole
member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc
is a Delaware corporation.
For information on translations, please e-mail [email protected]; for reprint,
paperback, or audio rights, please e-mail [email protected].
Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and
licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales
web page at www.apress.com/bulk-sales.
Any source code or other supplementary material referenced by the author in this book is available to
readers on GitHub via the book’s product page, located at www.apress.com/978-1-4842-6221-4. For more
detailed information, please visit www.apress.com/source-code.
Printed on acid-free paper
Table of Contents
About the Author����������������������������������������������������������������������������������������������������� xi

About the Technical Reviewer������������������������������������������������������������������������������� xiii


Acknowledgments���������������������������������������������������������������������������������������������������xv

Introduction�����������������������������������������������������������������������������������������������������������xvii

Part I: Introduction to Amazon Web Services���������������������������������������� 1


Chapter 1: Cloud Computing and AWS���������������������������������������������������������������������� 3
What Is the Cloud?������������������������������������������������������������������������������������������������������������������������ 3
Control of Cloud Systems�������������������������������������������������������������������������������������������������������������� 4
Public Cloud����������������������������������������������������������������������������������������������������������������������������� 5
Private Cloud��������������������������������������������������������������������������������������������������������������������������� 5
Community Cloud�������������������������������������������������������������������������������������������������������������������� 5
Hybrid Cloud���������������������������������������������������������������������������������������������������������������������������� 5
Cloud Services������������������������������������������������������������������������������������������������������������������������������ 5
Infrastructure as a Service������������������������������������������������������������������������������������������������������ 6
Platform as a Service�������������������������������������������������������������������������������������������������������������� 6
Software as a Service������������������������������������������������������������������������������������������������������������� 6
Anything as a Service�������������������������������������������������������������������������������������������������������������� 6
Introduction to Amazon Web Services������������������������������������������������������������������������������������������ 7
AWS Management Console����������������������������������������������������������������������������������������������������������� 8
AWS Command-Line Interface���������������������������������������������������������������������������������������������������� 11
AWS Storage Services���������������������������������������������������������������������������������������������������������������� 12
Amazon S3���������������������������������������������������������������������������������������������������������������������������� 13
Amazon Elastic File System�������������������������������������������������������������������������������������������������� 14
AWS Storage Gateway����������������������������������������������������������������������������������������������������������� 15

iii
Table of Contents

AWS Compute Services��������������������������������������������������������������������������������������������������������������� 18


Amazon EC2�������������������������������������������������������������������������������������������������������������������������� 19
Other Services����������������������������������������������������������������������������������������������������������������������� 22
Amazon Elastic Container Registry��������������������������������������������������������������������������������������������� 22
AWS Networking and Content Delivery Services������������������������������������������������������������������������ 23
Amazon VPC��������������������������������������������������������������������������������������������������������������������������� 23
Amazon API Gateway������������������������������������������������������������������������������������������������������������� 25
Amazon CloudFront��������������������������������������������������������������������������������������������������������������� 26
Conclusion���������������������������������������������������������������������������������������������������������������������������������� 28

Chapter 2: AWS Pricing and Cost Management������������������������������������������������������ 29


Understanding the Pricing of AWS���������������������������������������������������������������������������������������������� 29
AWS Free Tier������������������������������������������������������������������������������������������������������������������������������ 31
Factors Affecting Pricing in AWS������������������������������������������������������������������������������������������������� 31
AWS Cost Optimization���������������������������������������������������������������������������������������������������������������� 32
Right-Sizing��������������������������������������������������������������������������������������������������������������������������� 32
Using Reserved Instances����������������������������������������������������������������������������������������������������� 36
Using Spot Instances������������������������������������������������������������������������������������������������������������� 37
Using the Cost Explorer��������������������������������������������������������������������������������������������������������� 38
AWS Trusted Advisor������������������������������������������������������������������������������������������������������������������� 41
Pricing of AWS Services�������������������������������������������������������������������������������������������������������������� 43
Conclusion���������������������������������������������������������������������������������������������������������������������������������� 44

Chapter 3: Security in Amazon Web Services�������������������������������������������������������� 45


The SSR Model of AWS��������������������������������������������������������������������������������������������������������������� 45
Compliance���������������������������������������������������������������������������������������������������������������������������� 47
Physical and Environmental Security������������������������������������������������������������������������������������ 48
Business Continuity Management����������������������������������������������������������������������������������������� 48
Network Security������������������������������������������������������������������������������������������������������������������� 49
AWS Account Security Features�������������������������������������������������������������������������������������������������� 50
Passwords for Authentication������������������������������������������������������������������������������������������������ 51
Multifactor Authentication����������������������������������������������������������������������������������������������������� 51

iv
Table of Contents

Access Keys for API Authentication��������������������������������������������������������������������������������������� 51


X.509 Certificates������������������������������������������������������������������������������������������������������������������ 52
AWS Identity and Access Management��������������������������������������������������������������������������������������� 52
Federation of Users in AWS��������������������������������������������������������������������������������������������������� 54
How Access Management Is Done in AWS���������������������������������������������������������������������������� 55
Attribute-Based Access Control��������������������������������������������������������������������������������������������� 57
AWS Web Application Firewall����������������������������������������������������������������������������������������������������� 59
AWS Shield���������������������������������������������������������������������������������������������������������������������������������� 60
AWS Firewall Manager���������������������������������������������������������������������������������������������������������������� 61
Conclusion���������������������������������������������������������������������������������������������������������������������������������� 62

Part II: Machine Learning in AWS�������������������������������������������������������� 63


Chapter 4: Introduction to Machine Learning��������������������������������������������������������� 65
Introduction to Machine Learning and Artificial Intelligence������������������������������������������������������ 65
Supervised Learning�������������������������������������������������������������������������������������������������������������� 66
Unsupervised Learning���������������������������������������������������������������������������������������������������������� 67
Reinforcement Learning�������������������������������������������������������������������������������������������������������� 68
Deep Learning����������������������������������������������������������������������������������������������������������������������������� 70
Machine Learning in AWS����������������������������������������������������������������������������������������������������������� 71
Amazon SageMaker�������������������������������������������������������������������������������������������������������������������� 72
Understanding How SageMaker Works��������������������������������������������������������������������������������� 74
Preprocessing of Data in SageMaker������������������������������������������������������������������������������������ 76
Model Training in SageMaker������������������������������������������������������������������������������������������������ 77
Model Deployment in SageMaker������������������������������������������������������������������������������������������ 79
Built-in SageMaker Algorithms���������������������������������������������������������������������������������������������� 81
Custom Algorithms in SageMaker����������������������������������������������������������������������������������������� 84
Other Machine Learning Services by AWS���������������������������������������������������������������������������������� 85
Amazon Comprehend������������������������������������������������������������������������������������������������������������ 85
Amazon Polly������������������������������������������������������������������������������������������������������������������������� 85
Amazon Rekognition�������������������������������������������������������������������������������������������������������������� 86
Amazon Translate������������������������������������������������������������������������������������������������������������������ 87

v
Table of Contents

Amazon Transcribe���������������������������������������������������������������������������������������������������������������� 87
Amazon Textract�������������������������������������������������������������������������������������������������������������������� 88
Conclusion���������������������������������������������������������������������������������������������������������������������������������� 88

Chapter 5: Data Processing in AWS������������������������������������������������������������������������ 89


Preprocessing in Jupyter Notebook�������������������������������������������������������������������������������������������� 89
Preprocessing Using SageMaker’s Scikit-Learn Container��������������������������������������������������������� 98
Creating Your Own Preprocessing Code Using ScriptProcessor����������������������������������������������� 105
Creating a Docker Container������������������������������������������������������������������������������������������������ 105
Building and Pushing the Image������������������������������������������������������������������������������������������ 106
Using a ScriptProcessor Class��������������������������������������������������������������������������������������������� 107
Using Boto3 to Run Processing Jobs���������������������������������������������������������������������������������������� 108
Installing Boto3�������������������������������������������������������������������������������������������������������������������� 109
Initializing Boto3������������������������������������������������������������������������������������������������������������������ 110
Making Dockerfile Changes and Pushing the Image����������������������������������������������������������� 110
Creating a Processing Job��������������������������������������������������������������������������������������������������� 111
Monitoring Processing Jobs Using CloudWatch������������������������������������������������������������������������ 115
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 117

Chapter 6: Building and Deploying Models in SageMaker������������������������������������ 119


Exploring the Linear Learner Algorithm������������������������������������������������������������������������������������ 119
Overview of Linear Regression�������������������������������������������������������������������������������������������� 119
Overview of Logistic Regression����������������������������������������������������������������������������������������� 120
SageMaker Application of Linear Learner��������������������������������������������������������������������������� 121
Exploring the XGBoost Algorithm���������������������������������������������������������������������������������������������� 126
Gradient Boosting Algorithm������������������������������������������������������������������������������������������������ 126
XGBoost Algorithm��������������������������������������������������������������������������������������������������������������� 127
SageMaker Application of XGBoost������������������������������������������������������������������������������������� 128
Exploring the Blazing Text Algorithm����������������������������������������������������������������������������������������� 133
Skip Gram Architecture of Word Vectors Generation����������������������������������������������������������� 133
Continuous Bag of Words Architecture of Word Vectors Generation������������������������������������ 134
SageMaker Application of Blazing Text�������������������������������������������������������������������������������� 135

vi
Table of Contents

Exploring the Image Classification Algorithm��������������������������������������������������������������������������� 137


ResNet��������������������������������������������������������������������������������������������������������������������������������� 138
SageMaker Application of Image Classification������������������������������������������������������������������ 140
Exploring the SeqToSeq Algorithm�������������������������������������������������������������������������������������������� 145
Recurrent Neural Networks������������������������������������������������������������������������������������������������� 145
Encoder-Decoder Architecture�������������������������������������������������������������������������������������������� 146
SageMaker Application of SeqToSeq����������������������������������������������������������������������������������� 147
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 154

Chapter 7: Using CloudWatch with SageMaker���������������������������������������������������� 155


Amazon CloudWatch����������������������������������������������������������������������������������������������������������������� 155
CloudWatch Logs���������������������������������������������������������������������������������������������������������������������� 157
Training Jobs����������������������������������������������������������������������������������������������������������������������� 157
Processing Jobs������������������������������������������������������������������������������������������������������������������ 160
CloudWatch Metrics������������������������������������������������������������������������������������������������������������������ 162
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 165

Chapter 8: Running a Custom Algorithm in SageMaker��������������������������������������� 167


The Problem Statement������������������������������������������������������������������������������������������������������������ 167
Running the Model�������������������������������������������������������������������������������������������������������������������� 168
Transforming Code to Use SageMaker Resources�������������������������������������������������������������������� 169
Creating the Training Script������������������������������������������������������������������������������������������������� 169
Creating the Inference Script���������������������������������������������������������������������������������������������� 173
Configuring the Endpoint Generation Files�������������������������������������������������������������������������� 175
Setting Up the Dockerfile����������������������������������������������������������������������������������������������������� 176
Pushing the Docker Image to ECR��������������������������������������������������������������������������������������� 179
Training the Model�������������������������������������������������������������������������������������������������������������������� 182
Deploying the Model����������������������������������������������������������������������������������������������������������������� 183
Doing Real-Time Inference������������������������������������������������������������������������������������������������������� 183
Doing Batch Transformation������������������������������������������������������������������������������������������������������ 184
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 188

vii
Table of Contents

Chapter 9: Making an End-to-End Pipeline in SageMaker������������������������������������ 189


Overview of Step Functions������������������������������������������������������������������������������������������������������ 189
Upgrading Step Functions��������������������������������������������������������������������������������������������������������� 190
Defining the Required Parameters�������������������������������������������������������������������������������������������� 191
Setting Up the Required Roles�������������������������������������������������������������������������������������������������� 192
Adding a Policy to the Existing SageMaker Role����������������������������������������������������������������� 192
Creating a New IAM Role for Step Functions����������������������������������������������������������������������� 193
Setting Up the Training Step����������������������������������������������������������������������������������������������������� 196
Setting Up the Endpoint Configuration Step������������������������������������������������������������������������������ 198
Setting Up the Endpoint Step���������������������������������������������������������������������������������������������������� 198
Creating a Chain of the Steps��������������������������������������������������������������������������������������������������� 199
Defining the Workflow and Starting Operation�������������������������������������������������������������������������� 199
Exploring the Jobs in Step Functions���������������������������������������������������������������������������������������� 200
Exploring the JSON File That Can Be Passed as Input�������������������������������������������������������������� 203
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 203

Part III: Other AWS Services�������������������������������������������������������������� 205


Chapter 10: Machine Learning Use Cases in AWS������������������������������������������������ 207
Use Case 1: Natural Language Processing Using Amazon Comprehend����������������������������������� 207
Analysis of Text�������������������������������������������������������������������������������������������������������������������� 207
Custom Classification���������������������������������������������������������������������������������������������������������� 210
Use Case 2: Sales Forecasting Using Amazon Forecast����������������������������������������������������������� 215
Creating a Dataset Group���������������������������������������������������������������������������������������������������� 216
Defining Column Attributes�������������������������������������������������������������������������������������������������� 216
Importing Data��������������������������������������������������������������������������������������������������������������������� 217
Making Predictions�������������������������������������������������������������������������������������������������������������� 218
Use Case 3: Image Text Extraction Using Amazon Textract������������������������������������������������������� 222
Extracting Tabular Information�������������������������������������������������������������������������������������������� 222
Extracting Form Data����������������������������������������������������������������������������������������������������������� 224
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 225

viii
Table of Contents

 ppendix A: Creating a Root User Account to Access the


A
Amazon Management Console����������������������������������������������������������������������������� 227

Appendix B: Creating an IAM Role������������������������������������������������������������������������ 229

Appendix C: Creating an IAM User������������������������������������������������������������������������ 231

Appendix D: Creating an S3 Bucket���������������������������������������������������������������������� 233

Appendix E: Creating a SageMaker Notebook Instance���������������������������������������� 235

Index��������������������������������������������������������������������������������������������������������������������� 237

ix
About the Author
Himanshu Singh is a technology lead and senior NLP
engineer at Legato Healthcare (an Anthem company). He
has seven years of experience in the AI industry, primarily
in computer vision and natural language processing. He has
authored three books on machine learning. He has an MBA
from Narsee Monjee Institute of Management Studies, and a
postgraduate diploma in applied statistics.

xi
About the Technical Reviewer
Anindita Basak is a cloud architect and DevOps engineer.
With more than a decade of experience, she helps
enterprises to enable their digital transformation journey
empowered with multicloud, DevOps, advanced analytics,
and AI. She co-authored the books Stream Analytics with
Microsoft Azure and Hands-on Azure Machine Learning and
was a technical reviewer of seven books on Azure along
with two video courses on Azure data analytics. She has also
worked extensively with AWS Infra, DevOps, and analytics.

xiii
Acknowledgments
I’d like to thank my parents and brother for their unbounded support and the
Apress-­Springer team.

xv
Introduction
This book is structured into three parts. The first part of the book covers the concepts of
cloud computing and gives an overview of how AWS works. The second part of the book
takes on AWS in detail and covers SageMaker, Step Functions, S3 buckets, ECR, etc. The
last part talks about the use cases for AWS services. Different services such as Amazon
Comprehend and Extract are discussed here.
Specifically, Part I starts by covering cloud terminologies. It helps you understand
the cloud concepts required to use AWS. Then the book discusses the various AWS
services that Amazon provides and how they help users in different ways. It discusses
the different functionalities of AWS that are categorized under storage-based, compute-­
based, security-based, etc. By end of the chapters in this part, you will have an overview
of how AWS works.
Part II discusses SageMaker in detail. The part starts by running a basic
preprocessing script in SageMaker and ends with building a complete end-to-end
pipeline of machine learning in it. It covers how SageMaker talks with different services
such as ECR, S3, Step Functions, etc., to build the final model.
Part III discusses three use cases of machine learning using some of the other
services of AWS. The book discusses how to extract text using Amazon Textract, how
to use Amazon Comprehend, and how to make a time-series model using Amazon
Forecast.
This book was written to give people who know Python and machine learning some
experience with AWS. It teaches you how to use the power of AWS to build your heavy
models and how AWS provides you with services to make super models or deploy your
custom code with the same AWS support.

xvii
PART I

Introduction to Amazon
Web Services
CHAPTER 1

Cloud Computing
and AWS
This chapter covers the different components of cloud computing and of Amazon Web
Services (AWS). After reading the chapter, you’ll understand the different important
components of AWS, which will make it easier to understand the machine learning
components of AWS.

What Is the Cloud?


So, what is the cloud? If you look at memes shared across the internet, you might think
the cloud is nothing but someone else’s computer that you can use from your own
computing device, for your own personal use. Then the question arises, why do we
need the other computer when we have our own? It’s because our computer may not
have things that the other system has. Maybe your budget when buying a system was
less than the other person’s, and he therefore has more computational power to use. So,
instead of buying a new system with more computational power, you can just access the
other system for some amount of time and then return to your own system. This is the
benefit that the cloud provides. And, by the way, we all know the other system is not just
any normal system. Cloud systems are provided by big companies such as Amazon and
Google. So even if you are trying to buy a new system with as much computational power
as cloud systems, you will not be able to afford it.
Formally speaking, the cloud is a particular computing service that is present at a
different remote location that we can access using networking or the internet. Cloud
services may include storage services, infrastructure services, software services, or
any other specific services that you need. Figure 1-1 shows how different devices are
connected to cloud systems at a remote location.

3
© Himanshu Singh 2021
H. Singh, Practical Machine Learning with AWS, https://doi.org/10.1007/978-1-4842-6222-1_1
Chapter 1 Cloud Computing and AWS

Figure 1-1. How different devices are connected to cloud systems at a remote
location

If we are able to access any of the services present at the remote location using the
internet or networking, then we call this cloud computing.

Control of Cloud Systems


Obviously, if someone is allowing access to use their personal system over the internet,
then they may want to restrict access in some ways. Or, they may want one group of
people to have full access, but another group to have limited access. This is done to avoid
security issues and not expose the vulnerabilities present in the system. To solve this
problem, cloud computing comes in four types.

• Public cloud

• Private cloud
• Hybrid cloud

• Community cloud

4
Chapter 1 Cloud Computing and AWS

Public Cloud
When the entire cloud infrastructure is open for public consumption, then it is called a
public cloud. Examples are the email services provided by Google or Yahoo.

Private Cloud
When only a specific group of people can access the services provided by a cloud, then
it is called a private cloud. An example is when people in an organization can access the
resources present in the organization’s cloud, but no one from outside the organization
can access the same resources.

Community Cloud
When a cloud service is accessible to a group of organizations, then it is called a
community cloud. For example, different organizations can access the services of AWS
or the Google Cloud Platform by registering. So, the same services are available to all the
organizations that have paid for it, but not to anyone else.

Hybrid Cloud
When a cloud service provides both options (i.e., services of a public and private cloud),
then it is termed a hybrid cloud. An example is using two services of AWS. When we train
a model using SageMaker training, it is a private task for specific organizations because
it contains sensitive data and other things, but when we train a model and then share
the endpoint publicly, it is a public cloud because whoever has the link can access that
endpoint. (You’ll learn more about SageMaker in later chapters.)

Cloud Services
Now that you have learned about the different ways of accessing a cloud, let’s dive
deeper and look at the services that a cloud platform may provide. We can group these
services into four domains.
• Infrastructure as a service (IaaS)
• Platform as a service (PaaS)
• Software as a service (SaaS)

• Anything as a service (XaaS)


5
Chapter 1 Cloud Computing and AWS

Infrastructure as a Service
As the name suggests, when a cloud service provider gives access to users to the
infrastructure that it has built, it is considered an IaaS. For example, a cloud provider
may give access to virtual machines, physical machines, storage devices, etc. For
example, we can use Google Drive to store information on the cloud, since Google is
providing its hard drives as a service. AWS also provides machines called EC2 instances
that individuals can use to do operations that require higher computational power.

Platform as a Service
Sometimes, instead of requiring an entire infrastructure, we want only a specific
development runtime where we can write our code or make games or websites. This
way the cost of building an entire infrastructure can be reduced. This type of service is
a PaaS. For example, we can use Google Colaboratory for writing Python or R code. In
addition, we can use AWS SageMaker to train and put a machine learning model into
production. There are other service providers as well such as Microsoft Azure, Google
Cloud Platform, IBM Cloud, etc.

Software as a Service
When we don’t want the runtime, but we want to use a specific software application
with its built-in runtime, we don’t need a PaaS, which would give us the runtime as well
as dependencies and software we’d need to install. Hence, there are cloud services that
provide specific software for specific uses, called SaaS. Examples of SaaS are Amazon
Ground Truth, which is used for data management, and Office 365 by Microsoft.

Anything as a Service
The first three types of services have been on the market for quite some time, but now,
because of the advancement in technology, cloud service providers are providing almost
anything as a cloud service. For example, we can now draw sketches of web pages and
give them to Azure, which converts them into HTML pages. In addition, you can play
online songs by just talking to Alexa, which is connected to AWS. All this comes under
the umbrella of XaaS.

6
Chapter 1 Cloud Computing and AWS

Let’s now dive deeper into a specific cloud service provider, called Amazon Web
Services (AWS).

Introduction to Amazon Web Services


AWS provides global cloud computing services across many countries and is currently
responsible for handling the infrastructure of many companies, including small and
large enterprises. According to the AWS documentation, currently AWS caters to
hundreds of thousands of businesses in 190 countries.
AWS provides more than 150 services that can be used on demand and can be paid
for based on the time used. Currently, AWS has data servers in a lot of regions, and you
can choose to use only one region’s server that is closest to your users. The following is
the list of data servers across the globe:

• North America

• Ohio (US East)

• Oregon (US West)

• Northern California (US West)

• Northern Virginia (US West)

• Gov Cloud (US East and US West)

• Canada (Central)

• South America
• Sao Paulo

• Europe/Middle East/Africa

• London (Europe)

• Stockholm (Europe)

• Frankfurt (Europe)
• Paris (Europe)

• Bahrain (Middle East)

• Ireland (Europe)

7
Chapter 1 Cloud Computing and AWS

• Asia Pacific

• Singapore (Asia Pacific)

• Beijing (Mainland China)

• Sydney (Asia Pacific)

• Tokyo (Asia Pacific)

• Seoul (Asia Pacific)

• Ningxia (Mainland China)

• Osaka (Asia Pacific)

• Mumbai (Asia Pacific)

• Hong Kong (Asia Pacific)

As I mentioned, AWS has more than 150 services. The question is, how do you access
them? Is there a single centralized place from where they can be accessed? Well, yes!
This place is called the AWS Management Console. Let’s look at some of the features of
this console and how it is really helpful to users.

AWS Management Console


With the AWS Management Console (AMC), not only can you access the services, but it
provides some other cool features as well. Some of them are as follows:
• Once you have created an account on AWS and logged in to AMS,
then your session remains active only for 12 hours. After that, you
need to log in again. Obviously, this time limit is customizable. This
feature is provided for security reasons.

• Not only can you access AMS from the Web, but you can use the
mobile app as well. The AMS app is present both on IOS and on
Android devices.

8
Chapter 1 Cloud Computing and AWS

• AMS provides access to different learning resources, articles,


documentation, videos, etc., which help us in understanding the
different services of AWS.

• You can even customize and personalize AMS based on your usage
and needs.

After logging in to AWS, you will see the following features:

• Search button to find specific services

• Recently visited services by a user

• List of all the services

• Links to automated workflows

• Link to learning resources

Figures 1-2, 1-3, 1-4, and 1-5 show the different screens of AMS.

Figure 1-2. Find Services feature and recently visited services on AMS

9
Chapter 1 Cloud Computing and AWS

Figure 1-3. Learning resources on AMS

Figure 1-4. Automation on AMS

10
Chapter 1 Cloud Computing and AWS

Figure 1-5. List of all services on AMS

We will look at how to log in to AWS and visit AMS in detail in the next section about
machine learning. Now, let’s move to the next feature of AWS called the AWS Command-­Line
Interface (AWS CLI).

AWS Command-Line Interface


If AMC gives you a visual interface to access the AWS services, the CLI gives you some
advanced power to access the same services through the console. It is used by advanced
developers who have spent some time with AWS. You just need to download a single
tool, and then you can use it to control different services, write scripts, and have control
over the automation of services.

11
Chapter 1 Cloud Computing and AWS

AWS provides a lot of resources such as reference documents, GitHub repositories,


forums, etc., for understanding AWS CLI. Though one can use AWS CLI from the default
console such as the command prompt of Windows or terminals of Linux and Mac, there
is a dedicated AWS shell that provides some advanced functionalities. Some of them are
as follows:

• Autocompletion support

• Inline documentation of commands

• OS shell commands, which can also be executed from the same shell

We will be using the CLI a lot when we will cover machine learning in detail.
Therefore, we will look at its practical aspects directly in that section.
Because AWS provides so many services, covering all of them is not possible in
one book. Moreover, this book is about machine learning, so it doesn’t make any sense
to cover every service here. But, we will discuss three services that I think are really
important and commonly used. The following are the services that we are going to
discuss here:

• AWS Storage Services

• AWS Compute Services

• AWS Networking and Content Delivery Services

Let’s start the discussion with the first one, Storage Services.

AWS Storage Services


When we work on a cloud platform and use its services, obviously we’ll have a lot of
data depending on the requirements. For example, if we are building a website, then we
will have images, videos, and lots of other things to store. If we have a machine learning
model, then we will have terabytes of data to handle. This data can be both structured
and unstructured. Similarly, for business purposes, we can have multiple Excel sheets
or presentations. All these data types must be stored somewhere in the cloud, and the
cloud platform should provide this facility.

12
Chapter 1 Cloud Computing and AWS

AWS provides a lot of options for data storage, and we’ll discuss three of them in this
section.

• Amazon S3

• Amazon Elastic File System (EFS)

• AWS Storage Gateway

Amazon S3
One of the most used services of AWS is Amazon Simple Storage System (S3). It provides
you with an interface where you can store your data in a similar way to how you store it
in your local file system. You can create folders and multiple subdirectories to organize
your data. The following are some of the basic features Amazon S3 provides:

• It provides scalability, which is currently leading in the industry.

• It provides real-time data availability.

• It provides security and optimized performance.

• It has a durability of 99.9999999… percent (11 nines).

S3 is really simple to use. First let’s understand some of the naming conventions used
by Amazon S3.

Buckets
A bucket is just like a folder in your local file system. It is a container used for storing
your files.

Objects
The files that you store in S3 are termed objects. All the objects are stored inside the
buckets.

Keys
Every object that you store will be given a unique identifier called a key. Also, not only
objects but buckets are provided with unique keys.

13
Chapter 1 Cloud Computing and AWS

Does S3 only provide simple storage, as its name suggests? I will say yes and no. Yes,
because its main use is storage only, and it is really simple. No, because it has lots of
other features revolving around the storage feature that make it a go-to service for every
customer. Let’s see what those features are that make S3 so powerful.

• Based on how frequently data is being used, S3 provides different


types of storage classes.

• S3 STANDARD: Data that needs to be frequently accessed

• S3 STANDARD_IA: Data that needs to be less frequently accessed

• S3 GLACIER: Data that we want to archive

• Storage without security is nothing. AWS provides access control to


the buckets that you have created. You can accomplish this using
policies. The following are the three levels of control based on policies
that we can apply:

• Who can access which bucket?

• From which network can the buckets be accessed?

• At what time should the buckets be accessed?

• You can also create versions of your objects. For example, if the
same Excel sheet is updated five times, then five versions of it can be
created.

In this entire book, Amazon S3 is the service that we will be using continuously with
machine learning services. We will discuss the services in detail in the next section.

Amazon Elastic File System


Amazon Elastic File System (EFS) is an elastic network file system that most of the
AWS cloud services are compatible with. It is called elastic because it is scalable as
well as shrinkable. If you upload a smaller amount of data, then it shrinks its size to
accommodate that data. But if you upload a larger amount of data, then it can scale up
its size. Scaling up can be in the petabytes as well. EFS works with the latest version of
NFS, which is NFSv4.1. Hence, it is compatible with almost everything that you want to
develop.

14
Chapter 1 Cloud Computing and AWS

Tip Using Network File System (NFS), you can store, edit, delete, and perform
other operations similar to how you perform them in your local system. It is a
kind of distributed file system that uses network-attached storage (NAS). The
current version of NFS provides advanced features such as strong authentication,
file caching, and support for Windows File System. NFS can be accessed now on
global WANs.

Just like S3, EFS provides two kinds of file storage.


• Standard Access
• Infrequent Access
When we want to access data frequently, we use Standard Access, while infrequently
used data can be stored in Infrequent Access EFS. Also, just like S3, you can authenticate
and authorize data in EFS and encrypt it further. Finally, you can add policies, just like
S3, for maintaining access control.

AWS Storage Gateway


AWS Storage Gateway is a hybrid infrastructure provided by AWS. If you want to use your
on-premise infrastructure for all your storage needs but still you want some functionality
by which you can use the cloud storage services of AWS, then Storage Gateway is the best
solution.
Storage Gateway provides three kinds of solutions.
• File Gateway
• Volume Gateway
• Tape Gateway

File Gateway
Using this service, all the files are stored in S3. It gives you a virtual application with
which you can manage all your files in S3. Retrieving/storing files is done using protocols
such as Network File System or Server Message Block. The virtual software that we are
talking about is nothing but a virtual machine with which you manage your files. This
can be with VMware ESXi or Microsoft Hyper-V.

15
Chapter 1 Cloud Computing and AWS

V
 olume Gateway
Instead of files, you can directly store volumes in the cloud that you can later mount as
Internet Small Computer System Interface (iSCSI). Again, the software that is deployed
on-premise is a virtual machine. The following kinds of volumes are supported:

• Cached volumes

• Stored volumes

Having cached volumes means storing the data entirely in S3, and then the
frequently used data is cached in the local system. Figure 1-6 shows the cached volume
gateway architecture provided by AWS.

Figure 1-6. Cached volumes, Storage Gateway architecture

Figure 1-6 is divided into three parts. The left part shows the actual users using the
local architecture. The middle component is the local infrastructure of an organization.
The right component has an S3 connection for the data backup.
When you store your entire data locally and then back up the snapshot versions of
this data on the cloud, then it is the stored volume support of Volume Gateway. We can
use this in the case of disaster recovery. For example, if you lose your local data, you can
download the latest snapshot from the cloud. Again, we use S3 as the storage service
here. Figure 1-7 shows the architecture of a storage volume.

16
Random documents with unrelated
content Scribd suggests to you:
was found another manner of thing than was expected; for our men
could see no enemy to shoot at, but yet felt their bullets out of the
thick bushes where they lay in ambush. The English wanted not
courage or resolution, but could not discover nor find an enemy to
fight with, yet were galled by the enemy.” In the arts of ambush and
surprise, with which the Indians were so familiar, the colonists were
without practice. It is to the want of this experience, purchased at a
very dear rate in the course of the war, that we must ascribe the
numerous surprises and defeats from which the colonists suffered at
its commencement.
Driven to the necessity of defensive warfare, those in command
on the river determined to establish a magazine and garrison at
Hadley. Captain Lathrop, who had been dispatched from the
eastward to the assistance of the river towns, was sent with eighty
men, the flower of the youth of Essex County, to guard the wagons
intended to convey to Hadley three thousand bushels of unthreshed
wheat, the produce of the fertile Deerfield meadows. Just before
arriving at Deerfield, near a small stream still known as Bloody
Brook, under the shadow of the abrupt conical Sugar Loaf, the
southern termination of the Deerfield mountain, Lathrop, on
September 18, fell into an ambush, and, after a brave resistance,
perished there with all his company. Captain Moseley, stationed at
Deerfield, marched to his assistance, but arrived too late to help
him. Deerfield was abandoned, and burned by the Indians.
Springfield, about the same time, was set on fire, but was partially
saved by the arrival, with troops from Connecticut, of Major Treat,
successor to the lately deceased Mason in the chief command of the
Connecticut forces. An attack on Hatfield was vigorously repelled by
the garrison.
Meanwhile, hostilities were spreading; the Indians on the
Merrimac began to attack the towns in their vicinity, and the whole
of Massachusetts was soon in the utmost alarm. Except in the
immediate neighborhood of Boston, the country still remained an
immense forest dotted by a few openings. The frontier settlements
could not be defended against a foe familiar with localities, scattered
in small parties, skilful in concealment, and watching with patience
for some unguarded or favorable moment. Those settlements were
mostly broken up, and the inhabitants, retiring toward Boston,
spread everywhere dread and intense hatred of “the bloody
heathen.” Even the praying Indians, and the small dependent and
tributary tribes, became objects of suspicion and terror. They had
been employed at first as scouts and auxiliaries, and to good
advantage; but some few, less confirmed in the faith, having
deserted to the enemy, the whole body of them were denounced as
traitors. Eliot the apostle, and Gookin, superintendent of the subject
Indians, exposed themselves to insults, and even to danger, by their
efforts to stem this headlong fury, to which several of the
magistrates opposed but a feeble resistance. Troops were sent to
break up the praying villages at Mendon, Grafton, and others in that
quarter. The Natick Indians, “those poor despised sheep of Christ,”
as Gookin affectionately calls them, were hurried off to Deer Island,
in Boston harbor, where they suffered excessively from a severe
winter. A part of the praying Indians of Plymouth colony were
confined, in like manner, on the islands in Plymouth harbor.
Not content with realities sufficiently frightful, superstition, as
usual, added bugbears of her own. Indian bows were seen in the
sky, and scalps in the moon. The northern lights became an object
of terror. Phantom horsemen careered among the clouds or were
heard to gallop invisible through the air. The howling of wolves was
turned into a terrible omen. The war was regarded as a special
judgment in punishment of prevailing sins. Among these sins, the
General Court of Massachusetts, after consultation with the elders,
enumerated neglect in the training of the children of church-
members; pride, in men’s wearing long and curled hair; excess in
apparel; naked breasts and arms, and superfluous ribbons; the
toleration of Quakers; hurry to leave meeting before blessing asked;
profane cursing and swearing; tippling-houses; want of respect for
parents; idleness; extortion in shopkeepers and mechanics; and the
riding from town to town of unmarried men and women, under
pretence of attending lectures—“a sinful custom, tending to
lewdness.” Penalties were denounced against all these offences; and
the persecution of the Quakers was again renewed. A Quaker
woman had recently frightened the Old South congregation in
Boston by entering that meeting-house clothed in sackcloth, with
ashes on her head, her feet bare, and her face blackened, intending
to personify the smallpox, with which she threatened the colony, in
punishment for its sins.
About the time of the first collision with Philip, the Tarenteens, or
Eastern Indians, had attacked the settlements in Maine and New
Hampshire, plundering and burning the houses, and massacring
such of the inhabitants as fell into their hands. This sudden diffusion
of hostilities and vigor of attack from opposite quarters made the
colonists believe that Philip had long been plotting and had gradually
matured an extensive conspiracy, into which most of the tribes had
deliberately entered, for the extermination of the whites. This belief
infuriated the colonists, and suggested some very questionable
proceedings. It seems, however, to have originated, like the war
itself, from mere suspicions. The same griefs pressed upon all the
tribes; and the struggle once commenced, the awe which the
colonists inspired thrown off, the greater part were ready to join in
the contest. But there is no evidence of any deliberate concert; nor,
in fact, were the Indians united. Had they been so, the war would
have been far more serious. The Connecticut tribes proved faithful,
and that colony remained untouched. Uncas and Ninigret continued
friendly; even the Narragansets, in spite of so many former
provocations, had not yet taken up arms. But they were strongly
suspected of intention to do so, and were accused by Uncas of
giving, notwithstanding their recent assurances, aid and shelter to
the hostile tribes.
An attempt had lately been made to revive the union of the New
England colonies. At a meeting of commissioners, on September 9,
1675, those from Plymouth presented a narrative of the origin and
progress of the present hostilities. Upon the strength of this
narrative the war was pronounced “just and necessary,” and a
resolution was passed to carry it on at the joint expense, and to
raise for that purpose a thousand men, one-half to be mounted
dragoons. If the Narragansets were not crushed during the winter, it
was feared they might break out openly hostile in the spring; and at
a subsequent meeting a thousand men were ordered to be levied to
co-operate in an expedition specially against them.
The winter was unfavorable to the Indians; the leafless woods
no longer concealed their lurking attacks. The frozen surface of the
swamps made the Indian fastnesses accessible to the colonists. The
forces destined to act against the Narragansets—six companies from
Massachusetts, under Major Appleton; two from Plymouth, under
Major Bradford; and five from Connecticut, under Major Treat—were
placed under the command of Josiah Winslow, Governor of Plymouth
since Prince’s death—son of that Edward Winslow so conspicuous in
the earlier history of the colony. In December the Massachusetts and
Plymouth forces marched to Petasquamscot, on the west shore of
Narragansett Bay, where they made some forty prisoners. Being
joined by the troops from Connecticut, and guided by an Indian
deserter, after a march of fifteen miles through a deep snow, they
approached a swamp in what is now the town of South Kingston,
one of the ancient strongholds of the Narragansets. Driving the
Indian scouts before them, and penetrating the swamp, the colonial
soldiers soon came in sight of the Indian fort, built on a rising
ground in the morass, a sort of island of two or three acres, fortified
by a palisade, and surrounded by a close hedge a rod thick. There
was but one entrance, quite narrow, defended by a tree thrown
across it, with a block-house of logs in front and another on the
flank. It was the “Lord’s day,” but that did not hinder the attack. As
the captains advanced at the heads of their companies, the Indians
opened a galling fire, under which many fell. But the assailants
pressed on, and forced the entrance. A desperate struggle ensued.
The colonists were once driven back, but they rallied and returned to
the charge, and, after a two hours’ fight, became masters of the
fort. Fire was put to the wigwams, near six hundred in number, and
all the horrors of the Pequot massacre were renewed. The corn and
other winter stores of the Indians were consumed, and not a few of
the old men, women, and children perished in the flames. In this
bloody contest, long remembered as the “Swamp Fight,” the colonial
loss was terribly severe. Six captains, with two hundred and thirty
men, were killed or wounded; and at night, in the midst of a snow-
storm; with a fifteen miles’ march before them, the colonial soldiers
abandoned the fort, of which the Indians resumed possession. But
their wigwams were burned; their provisions destroyed; they had no
supplies for the winter; their loss was irreparable. Of those who
survived the fight, many perished of hunger.
Even as a question of policy, this attack on the Narragansets was
more than doubtful. The starving and infuriated warriors, scattered
through the woods, revenged themselves by attacks on the frontier
settlements. On February 10, 1676, Lancaster was burned, and forty
of the inhabitants killed or taken; among the rest, Mrs. Rolandson,
wife of the minister, the narrative of whose captivity is still
preserved. Groton, Chelmsford, and other towns in that vicinity were
repeatedly attacked. Medfield, twenty miles from Boston, was
furiously assaulted, and, though defended by three hundred men,
half the houses were burned. Weymouth, within eighteen miles of
Boston, was attacked a few days after. These were the nearest
approaches which the Indians made to that capital. For a time the
neighborhood of the Narraganset country was abandoned. The
Rhode Island towns, though they had no part in undertaking the
war, yet suffered the consequences of it. In March, Warwick was
burned, and Providence was partially destroyed. Most of the
inhabitants sought refuge in the islands, but the aged Roger Williams
accepted a commission as captain for the defence of the town he
had founded. Walter Clarke was presently chosen governor in
Coddington’s place, the times not suiting a Quaker chief magistrate.
The whole colony of Plymouth was overrun. Houses were burned
in almost every town, but the inhabitants, for the most part, saved
themselves in their garrisons, a shelter with which all the towns now
found it necessary to be provided. On March 26 Captain Pierce, with
fifty men and some friendly Indians, while endeavoring to cover the
Plymouth towns, fell into an ambush and was cut off. That same day,
Marlborough was set on fire; two days after Rehoboth was burned.
The Indians seemed to be everywhere. On April 18 Captain
Wadsworth, marching to the relief of Sudbury, fell into an ambush,
and perished with fifty men. The alarm and terror of the colonists
reached again a great height. But affairs were about to take a turn.
The resources of the Indians were exhausted; they were now
making their last efforts.
A body of Connecticut volunteers, under Captain Denison, and of
Mohegan and other friendly Indians, Pequots and Niantics, swept the
entire country of the Narragansets, who suffered, as spring
advanced, the last extremities of famine. Canonchet, the chief
sachem, said to have been a son of Miantonimoh, but probably his
nephew, had ventured to his old haunts to procure seed-corn with
which to plant the rich intervals on the Connecticut, abandoned by
the colonists. Taken prisoner, he conducted himself with all that
haughty firmness esteemed by the Indians the height of
magnanimity. Being offered his life on condition of bringing about a
peace, he scorned the proposal. His tribe would perish to the last
man rather than become servants to the English. When ordered to
prepare for death, he replied, “I like it well; I shall die before my
heart is soft, or I shall have spoken anything unworthy of myself.”
Two Indians were appointed to shoot him, and his head was cut off
and sent to Hartford.
The colonists had suffered severely. Men, women, and children
had perished by the bullets of the Indians, or fled naked through the
wintry woods by the light of their blazing houses, leaving their goods
and cattle a spoil to the assailants. Several settlements had been
destroyed, and many more had been abandoned; but the oldest and
wealthiest remained untouched. The Indians, on the other hand, had
neither provisions nor ammunition. On May 12, while attempting to
plant corn and catch fish at Montague Falls, on the Connecticut
River, they were attacked with great slaughter by the garrison of the
lower towns, led by Captain Turner, a Boston Baptist, and at first
refused a commission on that account, but as danger increased,
pressed to accept it. Yet this enterprise was not without its
drawbacks. As the troops returned, Captain Turner fell into an
ambush and was slain, with thirty-eight men. Hadley was attacked
on a lecture day, June 12, while the people were at meeting; but the
Indians were repulsed by the bravery of Goffe, one of the fugitive
regicides, long concealed in that town. Seeing this venerable
unknown man come to their rescue, and then suddenly disappear,
the inhabitants took him for an angel.
Major Church, at the head of a body of two hundred volunteers,
English and Indians, energetically hunted down the hostile bands in
Plymouth colony. The interior tribes about Mount Wachusett were
invaded and subdued by a force of six hundred men, raised for that
purpose. Many fled to the north to find refuge in Canada—guides
and leaders, in after years, of those French and Indian war parties
by which the frontiers of New England were so terribly harassed.
Just a year after the fast at the commencement of the war, a
thanksgiving was observed for success in it.
No longer sheltered by the River Indians, who now began to
make their peace, and even attacked by bands of the Mohawks,
Philip returned to his own country, about Mount Hope, where he was
still faithfully supported by his female confederate and relative,
Witamo, squaw sachem of Pocasset. Punham, also, the Shawomet
vassal of Massachusetts, still zealously carried on the war, but was
presently killed. Philip was watched and followed by Church, who
surprised his camp on August 1st, killed upward of a hundred of his
people, and took prisoners his wife and boy. The disposal of this
child was a subject of much deliberation. Several of the elders were
urgent for putting him to death. It was finally resolved to send him
to Bermuda, to be sold into slavery—a fate to which many other of
the Indian captives were subjected. Witamo shared the disasters of
Philip. Most of her people were killed or taken. She herself was
drowned while crossing a river in her flight; but her body was
recovered, and the head, cut off, was stuck upon a pole at Taunton,
amid the jeers and scoffs of the colonial soldiers, and the tears and
lamentations of the Indian prisoners.
Philip still lurked in the swamps, but was now reduced to
extremity. Again attacked by Church, he was killed by one of his own
people, a deserter to the colonists. His dead body was beheaded and
quartered, the sentence of the English law upon traitors. One of his
hands was given to the Indian who had shot him, and on August 17,
the day appointed for a public thanksgiving, his head was carried in
triumph to Plymouth.
The popular rage against the Indians was excessive. Death or
slavery was the penalty for all known or suspected to have been
concerned in shedding English blood. Merely having been present at
the “Swamp Fight” was adjudged by the authorities of Rhode Island
sufficient foundation for sentence of death, and that, too,
notwithstanding they had intimated an opinion that the origin of the
war would not bear examination. The other captives who fell into the
hands of the colonists were distributed among them as ten-year
servants. Roger Williams received a boy for his share. Many chiefs
were executed at Boston and Plymouth on the charge of rebellion;
among others, Captain Tom, chief of the Christian Indians at Natick,
and Tispiquin, a noted warrior, reputed to be invulnerable, who had
surrendered to Church on an implied promise of safety. A large body
of Indians, assembled at Dover to treat of peace, were treacherously
made prisoners by Major Waldron, who commanded there. Some
two hundred of these Indians, claimed as fugitives from
Massachusetts, were sent by water to Boston, where some were
hanged, and the rest shipped off to be sold as slaves. Some
fishermen of Marblehead having been killed by the Indians at the
eastward, the women of that town, as they came out of meeting on
a Sunday, fell upon two Indian prisoners who had just been brought
in, and murdered them on the spot. The same ferocious spirit of
revenge which governed the contemporaneous conduct of Berkeley
in Virginia toward those concerned in Bacon’s rebellion, swayed the
authorities of New England in their treatment of the conquered
Indians. By the end of the year the contest was over in the South,
upward of two thousand Indians having been killed or taken. But
some time elapsed before a peace could be arranged with the
Eastern tribes, whose haunts it was not so easy to reach.
In this short war of hardly a year’s duration the Wampanoags
and Narragansets had suffered the fate of the Pequots. The Niantics
alone, under the guidance of their aged sachem, Ninigret, had
escaped destruction. Philip’s country was annexed to Plymouth,
though sixty years afterward, under a royal order in council, it was
transferred to Rhode Island. The Narraganset territory remained as
before, under the name of King’s Province, a bone of contention
between Connecticut, Rhode Island, the Marquis of Hamilton, and
the Atherton claimants. The Niantics still retained their ancient seats
along the southern shores of Narragansett Bay. Most of the surviving
Narragansets, the Nipmucks, and the River Indians, abandoned their
country, and migrated to the North and West. Such as remained,
along with the Mohegans and other subject tribes, became more
than ever abject and subservient.
The work of conversion was now again renewed, and, after such
overwhelming proofs of Christian superiority, with somewhat greater
success. A second edition of the Indian Old Testament, which seems
to have been more in demand than the New, was published in 1683,
revised by Eliot, with the assistance of John Cotton, son of the
“great Cotton,” and minister of Plymouth. But not an individual exists
in our day by whom it can be understood. The fragments of the
subject tribes, broken in spirit, lost the savage freedom and rude
virtues of their fathers, without acquiring the laborious industry of
the whites. Lands were assigned them in various places, which they
were prohibited by law from alienating. But this very provision,
though humanely intended, operated to perpetuate their indolence
and incapacity. Some sought a more congenial occupation in the
whale fishery, which presently began to be carried on from the
islands of Nantucket and Martha’s Vineyard. Many perished by
enlisting in the military expeditions undertaken in future years
against Acadie and the West Indies. The Indians intermarried with
the blacks, and thus confirmed their degradation by associating
themselves with another oppressed and unfortunate race. Gradually
they dwindled away. A few sailors and petty farmers, of mixed blood,
as much African as Indian, are now the sole surviving
representatives of the aboriginal possessors of southern New
England.
On the side of the colonists the contest had also been very
disastrous. Twelve or thirteen towns had been entirely ruined, and
many others partially destroyed. Six hundred houses had been
burned, near a tenth part of all in New England. Twelve captains and
more than six hundred men in the prime of life had fallen in battle.
There was hardly a family not in mourning. The pecuniary losses and
expenses of the war were estimated at near a million of dollars.
Massachusetts was burdened with a heavy debt. No aid nor relief
seems to have come from abroad, except a contribution from Ireland
of £500 for the benefit of the sufferers by the war, chiefly collected
by the efforts of Nathaniel Mather, lately successor to his brother
Samuel as minister of the non-conformist congregation at Dublin.

SYNOPSIS OF THE PRINCIPAL EVENTS,


CHIEFLY
MILITARY, BETWEEN THE DEFEAT OF KING
PHILIP, 1676, AND THE CAPTURE
OF QUEBEC, 1759
1676. Bacon’s Rebellion in Virginia against the government of Sir
William Berkeley.
1679. The Scottish Covenanters are defeated by the Duke of
Monmouth at Bothwell Bridge.
1681. William Penn obtains his patent from the English Crown.
1682. Purchase of East Jersey by William Penn. He takes
possession of New Castle (Delaware) and founds the Colony of
Pennsylvania. La Salle descends the Mississippi to its mouth.
1684. The charter of the Massachusetts Bay Company is
declared forfeited to the English Crown.
1685. James II. succeeds his brother, Charles II., as King of
England. Insurrection of the Earl of Argyll and the Duke of
Monmouth. Defeat of Monmouth at Sedgemoor; his execution.
1686. Sir Edmund Andros is made Governor of New England.
1688. William of Orange lands in England; flight of James II.
1689. William and Mary are proclaimed King and Queen of
England. England declares war against France. Victory of the
Scottish Jacobites at Killiecrankie. Overthrow of Andros in New
England. Beginning of King William’s War in America.
1690. The Orangemen in Ireland win the battle of the Boyne.
Destruction of Schenectady by the French and Indians. Sir William
Phips, commanding a New England expedition, captures Port Royal,
and later makes a fruitless demonstration against Quebec.
1691. The Jacobites are overcome in Scotland. Surrender of
Limerick, the last stronghold of James II. in Ireland.
1692. Union of the Plymouth and Massachusetts colonies.
Witchcraft delusion at Salem.
1693. The French Admiral Tourville defeats the English fleet off
Cape St. Vincent.
1697. France makes peace at Ryswick with Holland, Spain, and
England. Close of King William’s War in America.
1699. The French begin the settlement of Louisiana.
1701. Beginning of the War of the Spanish Succession.
1702. Death of William III. and accession of Queen Anne.
Successful campaign of Churchill (Marlborough) in the Netherlands.
Naval triumph of the English and Dutch over the Spanish and French
at Vigo. Queen Anne’s War in America. French settlement in
Alabama.
1704. The English are victorious over the French at the battle of
Blenheim. Capture of Gibraltar by the English. Massacre of white
settlers by the Indians at Deerfield, Massachusetts.
1706. Marlborough defeats the French and Bavarians at the
battle of Ramillies.
1708. Victory of Marlborough and Prince Eugene, at Oudenarde,
over the Dukes of Burgundy and Vendôme.
1711. Unsuccessful expedition of the English and New England
forces under Walker against Canada.
1713. Treaty of Utrecht. Close of Queen Anne’s War in America.
Acadia (Nova Scotia, etc.) ceded to England by France, which also
restores the Hudson Bay region. The power of the Tuscarora Indians
broken by the Carolinians.
1714. George I., Elector of Hanover, succeeds to the English
Crown.
1715. Rebellion in Scotland and in the North of England in favor
of James Edward Stuart, the Jacobite pretender.
1718. French settlement of New Orleans.
1720. Failure of Law’s Mississippi scheme in France.
1722. Establishment of the Moravian settlement in Pennsylvania
under Count Zinzendorf.
1727. Accession of George II.
1728. Discovery of Behring’s Strait.
1729. Carolina, purchased by the English Crown, is divided into
the royal provinces of North and South Carolina.
1730. Baltimore is laid out.
1732. Oglethorpe embarks from England to establish a
settlement in Georgia.
1733. Founding of Savannah.
1741. New Hampshire is finally separated from Massachusetts.
1744. Beginning of King George’s War in America. The French
capture Canseau (afterward Canso), and are repulsed at Annapolis.
1745. Jacobite rising in Scotland. Charles Edward, the young
Pretender, is victorious at Prestonpans. The New England troops,
under Sir William Pepperell, reduce the French fortress of Louisburg.
1746. Jacobite defeat at Culloden.
1748. The Peace of Aix-la-Chapelle terminates the War of the
Austrian Succession and King George’s War in America. Louisburg
restored to France.
1749. The Ohio Company receives its grant from George II.
1753. Friction between French and Americans on tributaries of
the Alleghany, along American western frontier. Washington’s vain
protest against the French seizure of Venango.
1754. Beginning of the French and Indian War in America.
Washington’s attack upon Jumonville, near Great Meadows, the first
action. The French compel Washington to capitulate at Fort
Necessity.
1755. Braddock’s expedition against Fort Duquesne and his
disastrous defeat. Abortive expeditions by the English against
Niagara and Crown Point.
1756. Formal declaration of hostilities between France and
England, and beginning of the Seven Years’ War. Capture of Oswego
by the French.
1757. Montcalm takes Fort William Henry on Lake George.
1758. Victory of Montcalm at Ticonderoga. Reduction of
Louisburg, and capture of Forts Frontenac and Duquesne by the
English.
V

THE FALL OF QUEBEC, 1759

[The visits of Breton fishermen to Newfoundland in the


early sixteenth century, the voyages of Cartier to the St.
Lawrence in 1534 and 1541–43, the foundation of Port Royal
in Acadia in 1605, and of Quebec by Champlain in 1608, were
the beginnings of a French occupancy of the northern and
central portions of North America which led inevitably to
conflict with England and the American colonists. The title
based upon Marquette’s discovery of the Mississippi in 1673,
and La Salle’s exploration and claim to the whole vast valley
in 1682, would have confined the English to the Atlantic
seaboard. The contact between the wholly different types
represented in English and French colonization caused friction
which became acute when King William’s War broke out in
1689. The eight years of that war, with its profitless capture
of Port Royal, Nova Scotia, were followed by Queen Anne’s
War, 1702–13, and King George’s War, 1744–48, and the
interval after the Treaty of Utrecht was a truce rather than
peace. The French were strengthening their hold along the
western frontier of the English colonists, at Fort Duquesne,
and elsewhere. Braddock’s defeat in 1755, and attacks upon
Crown Point and Niagara, preceded the formal declaration of
hostilities between France and England in 1756, the
beginning of the Seven Years’ War, involving nearly all
Europe, with England and Prussia facing Russia, France,
Austria, Sweden, and Saxony. In America, in 1756–57, the
incompetency of Loudon and Abercrombie, the dilatory
preparations to attack Louisburg, and Montcalm’s capture of
Fort William Henry, made the first stage of the war a gloomy
one. But Pitt’s entrance into the British cabinet as Secretary of
State brought an intelligent and active prosecution of the war.
The next year, 1758, witnessed the capture of Fort Frontenac
on Ontario, Fort Duquesne, and Louisburg by the English and
American forces.—Editor.]

T HE British Parliament met late in November, 1758, at a time


when the nation was aglow with enthusiasm over the
successes of the year—Louisburg and Frontenac in North
America, and the driving of the French from the Guinea coast as the
22
result of battles at Sénégal (May) and Gorée (November). The war
was proving far more costly than had been anticipated, yet Pitt
rigidly held the country to the task; but not against its will, and the
necessary funds were freely voted. Walpole wrote to a friend: “Our
unanimity is prodigious. You would as soon hear ‘No’ from an old
maid as from the House of Commons.” The preparations for the new
year were on a much larger scale than before; both by land and sea
France was to be pushed to the uttermost, and the warlike spirit of
Great Britain seemed wrought to the highest pitch.

The new French premier, Choiseul, was himself not lacking in


activity. He renewed with vigor the project of invading Great Britain,
preparations therefor being evident quite early in the year 1759.
Fifty thousand men were to land in England, and twelve thousand in
Scotland, where the Stuart cause still lingered. But as usual the
effort came to naught. The Toulon squadron was to co-operate with
one from Brest; Boscawen, who now commanded the Mediterranean
fleet, apprehended the former while trying to escape through the
Straits of Gibraltar in a thick haze (August 17), and after destroying
several of the ships dispersed the others; while Sir Edward Hawke
annihilated the Brest fleet in a brilliant sea-fight off Quiberon Bay
23
(November 20). Relieved of the possibility of insular invasion, the
Channel and Mediterranean squadrons were now free to raid French
commerce, patrol French ports, and thus intercept communication
with New France, and to harry French—and, later, Spanish—colonies
overseas.

PROGRESS OF FRENCH DISCOVERY IN THE INTERIOR 1600–1762


(FULL SIZE)

In 1757 Clive had regained Calcutta and won Bengal at the


famous battle of Plassey. Two years thereafter the East Indian seas
were abandoned by the French after three decisive actions won by
Pitt’s valiant seamen, and India thus became a permanent
24
possession of the British empire. In January, 1759, also, the British
25
captured Guadeloupe, in the West Indies. Lacking sea power, it
was impossible for France much longer to hold her colonies; it was
but a question of time when the remainder should fall into the
clutches of the mistress of the ocean.
Notwithstanding all this naval activity, Pitt’s principal operations
were really centred against Canada. The movement thither was to
be along two lines, which eventually were to meet in co-operation.
First, a direct attack was to be made upon Quebec, headed by
Wolfe, who was to be convoyed and assisted by a fleet under the
command of Admiral Saunders; second, Amherst—now commander-
in-chief in America, Abercrombie having been recalled—was to
penetrate Canada by way of Lakes George and Champlain. He was
to join Wolfe at Quebec, but was authorized to make such diversions
as he found practicable—principally to re-establish Oswego and to
relieve Pittsburg (Fort Duquesne) with reinforcements and supplies.
Wolfe’s selection as leader of the Quebec expedition occasioned
general surprise in England. Yet it was in the natural course of
events. He had been the life of the Louisburg campaign of the year
before, and when Amherst was expressing the desire of attacking
Quebec after the reduction of Cape Breton he wrote to the latter:
“An offensive, daring kind of war will awe the Indians and ruin the
French. Block-houses and a trembling defensive encourage the
meanest scoundrels to attack us. If you will attempt to cut up New
26
France by the roots, I will come with pleasure to assist.”
Wolfe, whose family enjoyed some influence, had attained a
captaincy at the age of seventeen and became a major at twenty. He
was now thirty-two, a major-general, and with an excellent fighting
record both in Flanders and America. Quiet and modest in demeanor,
although occasionally using excitable and ill-guarded language, he
was a refined and educated gentleman; careful of and beloved by
his troops, yet a stern disciplinarian; and although frail in body, and
often overcome by rheumatism and other ailments, capable of great
strain when buoyed by the zeal which was one of his characteristics.
The majority of his portraits represent a tall, lank, ungainly form,
with a singularly weak facial profile; but it is likely that these belie
him, for he had an indubitable spirit, a profound mind, quick
intuition, a charming manner, and was much thought of by women.
Indeed, just before sailing, he had become engaged to the beautiful
and charming Katharine Lowther, sister of Lord Lonsdale, and
27
afterward the Duchess of Bolton.
On February 17 Wolfe departed with Saunders’ fleet of twenty-
one sail, bearing the king’s secret instructions to “carry into
execution the said important operation with the utmost application
28
and vigor.” The voyage was protracted, and after arrival at
Louisburg he was obliged to wait long before the promised troops
appeared. He had expected regiments from Guadeloupe, but these
could not yet be spared, owing to their wretched condition; and the
Nova Scotia garrisons had also been weakened by disease, so that of
the twelve thousand agreed upon he finally could muster somewhat
29
under nine thousand. These were of the best quality of their kind;
although the general still entertained a low opinion of the value of
the provincials, who, it must be admitted, were, however serviceable
in bush-ranging, far below the efficiency of the regulars in a
campaign of this character. The force was divided into three
brigades, under Monckton, Townsend, and Murray, young men of
ability; although Townsend’s supercilious manner—the fruit of a
superior social connection—did not endear him either to his men or
his colleagues.
On June 1 the fleet began to leave Louisburg. There were thirty-
nine men-of-war, ten auxiliaries, seventy-six transports, and a
hundred and sixty-two miscellaneous craft, which were manned by
thirteen thousand naval seamen and five thousand of the mercantile
marine—an aggregate of eighteen thousand, or twice as many as
30
the landsmen under Wolfe. While to the latter is commonly given
credit for the result, it must not be forgotten that the victory was
quite as much due to the skilful management of the navy as to that
of the army, the expedition being in all respects a joint enterprise,
into which the men of both branches of the service entered with
intense enthusiasm.
The French had placed much reliance on the supposed
impossibility of great battle-ships being successfully navigated up the
St. Lawrence above the mouth of the Saguenay without the most
careful piloting. This portion of the river, a hundred and twenty miles
in length, certainly is intricate water, being streaked with perplexing
currents created by the mingling of the river’s strong flow with the
flood and ebb of the tide; the great stream is diverted into two
parallel channels by reefs and islands, and there are numerous
shoals—moreover, the French had removed all lights and other aids
to navigation. But British sailors laughed at difficulties such as these,
and, while they managed to capture a pilot, had small use for him,
preferring their own cautious methods. Preceded by a crescent of
sounding-boats, officered by Captain James Cook, afterward of
glorious memory as a pathfinder, the fleet advanced slowly but
safely, its approach heralded by beacons gleaming nightly to the
fore, upon the rounded hill-tops overlooking the long thin line of
riverside settlement which extended eastward from Quebec to the
31
Saguenay.
The French had at first expected attacks only from Lake Ontario
and from the south. But receiving early tidings of Wolfe’s expedition,
through convoys with supplies from France that had escaped
Saunders’ patrol of the gulf, general alarm prevailed, and Montcalm
decided to make his stand at Quebec. To the last he appears to have
shared in the popular delusion that British men-of-war could not
ascend the river; nevertheless, he promptly summoned to the capital
the greater part of the militia from all sections of Canada, save that
a thousand whites and savages were left with Pouchot to defend
Niagara, twelve hundred men under De la Corne to guard Lake
Ontario, and Bourlamaque, with upward of three thousand, was
ordered to delay Amherst’s advance and thus prevent him from
joining Wolfe. The population of Canada at the time was about
eighty-five thousand souls, and of these perhaps twenty-two
32
thousand were capable of bearing arms. The force now gathered
in and about Quebec aggregated about seventeen thousand, of
whom some ten thousand were militia, four thousand regulars of the
line, and a thousand each of colonial regulars, seamen, and Indians;
of these two thousand were reserved for the garrison of Quebec,
under De Ramezay, while the remainder were at the disposal of
33
Montcalm for the general defence.
The “rock of Quebec” is the northeast end of a long, narrow
triangular promontory, to the north of which lies the valley of the St.
Charles and to the south that of the St. Lawrence. The acclivity on
the St. Charles side is lower and less steep than the cliffs fringing
the St. Lawrence, which rise almost precipitously from two to three
hundred feet above the river—the citadel cliff being three hundred
and forty-five feet, almost sheer. Either side of the promontory was
easily defensible from assault, the table-land being only reached by
steep and narrow paths. Surmounting the cliffs, at the apex of the
triangle, was Upper Town, the capital of New France. Batteries,
largely manned by sailors, lined the cliff-tops within the town, and
the western base, fronting the Plains of Abraham, was protected by
fifteen hundred yards of insecure wall—for, after all, Quebec had,
despite the money spent upon it, never been scientifically fortified,
its commanders having from the first relied chiefly upon its natural
position as a stronghold.
At the base of the promontory, on the St. Lawrence side, is a
wide beach occupied by Lower Town, where were the market, the
commercial warehouses, a large share of the business
establishments, and the homes of the trading and laboring classes. A
narrow strand, little more than the width of a roadway, extended
along the base of the cliffs westward, communicating with the up-
river country; another road led westward along the table-land above.
Thus the city obtained its supplies from the interior both by highway
and by river.
THE PLAINS OF ABRAHAM ON THE MORNING OF THE BATTLE

Entrance to the St. Charles side of the promontory had been


blocked by booms at the mouth of that river, protected by strong
redoubts; and off Lower Town was a line of floating batteries.
Beyond the St. Charles, for a distance of seven miles eastward to the
gorge of the Montmorenci, Montcalm disposed the greater part of his
forces, his position being a plain naturally protected by a steep slope
descending to the meadow and tidal flats which here margin the St.
Lawrence. This plain rises gradually from the St. Charles, until at the
Montmorenci cataract it attains a height of three hundred feet, and
along the summit of the slope were well-devised trenches. The
gorge furnished a strong natural defence to the left wing, for it could
be forded only in the dense forest at a considerable distance above
the falls, and to force this approach would have been to invite an
ambuscade. Wolfe contented himself, therefore, with intrenching a
considerable force along the eastern bank of the gorge, and thence
issuing for frontal attacks on the Beauport Flats—so called from the
name of the village midway. Montcalm had chosen this as the chief
line of defence, on the theory that the approach by the St. Charles
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and


personal growth!

textbookfull.com

You might also like