Machine Learning and its Applications 1st

Edition Peter Wlodarczak download

https://textbookfull.com/product/machine-learning-and-its-
applications-1st-edition-peter-wlodarczak/

Download more ebook from https://textbookfull.com

We believe these products will be a great fit for you. Click
the link to download now, or visit textbookfull.com
to discover even more!

Mathematics for Machine Learning 1st Edition Marc Peter

Deisenroth

https://textbookfull.com/product/mathematics-for-machine-
learning-1st-edition-marc-peter-deisenroth/

Industrial Applications of Machine Learning Pedro

Larrañaga

https://textbookfull.com/product/industrial-applications-of-
machine-learning-pedro-larranaga/

AI as a Service: Serverless machine learning with AWS

1st Edition Peter Elger

https://textbookfull.com/product/ai-as-a-service-serverless-
machine-learning-with-aws-1st-edition-peter-elger/

Machine learning refined foundations algorithms and

applications Second Edition Borhani

https://textbookfull.com/product/machine-learning-refined-
foundations-algorithms-and-applications-second-edition-borhani/
Machine Learning Refined: Foundations, Algorithms, and
Applications Second Edition Jeremy Watt

https://textbookfull.com/product/machine-learning-refined-
foundations-algorithms-and-applications-second-edition-jeremy-
watt/

Mathematical Theories of Machine Learning - Theory and

Applications Bin Shi

https://textbookfull.com/product/mathematical-theories-of-
machine-learning-theory-and-applications-bin-shi/

Blockchain, big data and machine learning: trends and

applications First Edition Kumar

https://textbookfull.com/product/blockchain-big-data-and-machine-
learning-trends-and-applications-first-edition-kumar/

Teaching Learning Based Optimization Algorithm And Its

Engineering Applications 1st Edition R. Venkata Rao

https://textbookfull.com/product/teaching-learning-based-
optimization-algorithm-and-its-engineering-applications-1st-
edition-r-venkata-rao/

Machine Learning Algorithms for Industrial Applications

Santosh Kumar Das

https://textbookfull.com/product/machine-learning-algorithms-for-
industrial-applications-santosh-kumar-das/
Machine Learning and
its Applications
Peter Wlodarczak
University of Southern Queensland
Toowoomba, Queensland, Australia

p,
p,
A SCIENCE PUBLISHERS BOOK
A SCIENCE PUBLISHERS BOOK
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2020 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Printed on acid-free paper
Version Date: 20190821
International Standard Book Number-13: 978-1-138-32822-8 (Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the
validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the
copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to
publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let
us know so we may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted,
or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, includ-
ing photocopying, microfilming, and recording, or in any information storage or retrieval system, without written
permission from the publishers.

For permission to photocopy or use material electronically from this work, please access www.copyright.com
(http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers,
MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety
of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment
has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.

Library of Congress Cataloging-in-Publication Data

Names: Wlodarczak, Peter, author.
Title: Machine learning and its applications / Peter Wlodarczak, University
of Southern Queensland, Toowoomba, Queensland, Australia.
Description: First edition. | Boca Raton, FL : CRC Press/Taylor & Francis
Group, [2020] | Includes bibliographical references and index. |
Summary: “This book describes Machine Learning techniques and algorithms
that have been used in recent real-world application. It provides an
introduction to Machine Learning, describes the most widely used
techniques and methods. It also covers Deep Learning and related areas
such as function approximation or. The book gives real world examples
where Machine Learning techniques are applied and describes the basic
math and the commonly used learning techniques”-- Provided by publisher.
Identifiers: LCCN 2019033419 | ISBN 9781138328228 (hardcover ; acid-free
paper)
Subjects: LCSH: Machine learning.
Classification: LCC Q325.5 .W635 2020 | DDC 006.3/1--dc23
LC record available at https://lccn.loc.gov/2019033419

Visit the Taylor & Francis Web site at

http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
For my parents
Preface

Artificial intelligence or AI, and in particular, machine learning has

received a lot of attention in media due to the rapid advances we
have seen in this area over the past few years. Progress in artificial
intelligence has been slow in the 1990’s and a certain disillusionment
was noticeable in research since artificial intelligence did not seem
to live up to the expectation. Since the beginning of the new century,
machine learning has seen rapid progress due to new approaches, in
particular, deep learning. Deep learning goes further back, however,
due to recent advances in other areas, in particular, the development
of graphics processing units that provide the necessary computing
power, and the Internet, that made an unprecedented amount of data
publicly available, machine learning has gained a lot of momentum and
we have seen an incredible amount of new applications in many areas
of research and in practice. Machine learning is used for spam filtering,
for medical image analysis, for voice commands, for autonomously
driving cars and to analyze sensor data for the Internet of Things
just to name a few. Every high-end smart phone is now AI enabled.
Deep learning is used to predict earthquakes, for automatic language
translation, for automatic coloring of black and white movies and
for forecasting in finance. The Mars rover Curiosity utilizes artificial
intelligence to autonomously select inspection-worthy soil and rock
samples with high accuracy. The list goes on.

Recently, machine learning also attracted a lot of attention due to

visions of jobs being lost to machines, dangers of autonomous systems,
vi < Machine Learning and its Applications

such as robots who could turn evil, loss of control over existential
infrastructure or even a war starting due to artificial intelligence.
However, it should be noted that artificial intelligence, as the name
suggests, is artificial and has nothing to do with human intelligence. A
machine may be capable of recognizing whether an animal in a picture
is a lion or a tiger, but it is not able to understand the concept behind
what it recognized, i.e., it does not understand the concept of a living
being. A deep learner is, in essence, a huge mathematical formula. It
has nothing to do with how the human brain works, where biochemical
processes are executed. An artificial neural network is inspired by
nature. It has a grain of how we think the brain works and a lot of math
thrown at it. Visions of machines replacing doctors in diagnosing a
patient or lawyers in sentencing a perpetrator are probably premature.
It is more likely that we will see artificial intelligence supporting a
doctor or a lawyer in his daily work instead of replacing her or him in
the near future.

However, we will likely see many more applications of artificial

intelligence in areas that we have not thought of yet. Artificial
intelligence and machine learning pose many interesting problems in
practice and in research. The fact that we try to imitate nature gives
us a better understanding of what problems nature had to solve during
evolution. It also helps us to better understand how it solved them.
However, how human learning works is still largely unknown. Before
we have a better understanding of how the brain works, it is unlikely
that we will be able to replace it, no matter how much math we apply.

When I started to work with machine learning there were excellent

books explaining the math used in machine learning algorithms. There
were also excellent books about data mining and knowledge discovery.
Some of them are listed in the reference section. They can be used for
studying and for reference. However, there were no books explaining
the basic concepts behind machine learning and how they are applied
in practice. It is important to understand the math and the algorithms
used in machine learning. However, it is not necessary to know all
the details about the algorithms to be able to effectively use them for
real world problems. Machine learning has evolved over the past years
to the point where a lot of the complexity is hidden in frameworks
such as TensorFlow or Apache Mahout. A data scientist can use these
 Preface < vii

frameworks without knowing the implementation details. Many popular

programming languages have libraries with implementations that are
ready to be used in programs. R, Python, Java or Scala have many
different implementations of machine learning methods and the data
scientist can continue using his preferred programming language by
including the libraries that fit the purpose best. There are also tools, such
as RapidMiner, Weka or Knime to name a few, where no programming
skills are required and machine learning workflows can be created
graphically. Obviously, next to the open source implementations, there
are also commercial tools offered by different vendors to choose from.

This book aims to explain the basic concepts behind machine learning
and the machine learning methods. It is the work of several years of
experience in applying these methods in practice in various projects. It
tries to give a concise description of the algorithms and the math behind
them to the level where it helps explain the inner workings. However,
it is not intended to give an exhaustive mathematical description with
all the derivations. Also, machine learning goes back to the 1940’s
and has evolved since then and many different techniques have been
developed, too many to describe in one single book. Also, for virtually
every machine learning method, variations have been proposed that
might be more suitable for certain problems than others. Nevertheless,
some of the basic concepts apply to many of these methods and
understanding them makes it easy to familiarize oneself with new
methods that have not been used before. This book is written in such
a way that each chapter can be read individually with the caveat that
some redundancy exists in the chapters. This book will hopefully make
it easier for the reader to get started with machine learning and support
them in the fascinating journey through the world of data science and
machine learning.
Contents

Preface v
List of Figures xiii
List of Tables xv

SECTION I: INTRODUCTION

1. Introduction 3
1.1 Data mining 5
1.2 Data mining steps 6
1.3 Data collection 7
1.4 Data pre-processing 8
1.5 Data analysis 10
1.5.1 Supervised learning 10
1.5.2 Unsupervised learning 11
1.5.3 Semi-supervised learning 12
1.5.4 Machine learning and statistics 13
1.6 Data post-processing 15
2. Machine Learning Basics 17
2.1 Supervised learning 19
2.1.1 Perceptron 21
2.2 Unsupervised learning 25
2.2.1 k-means clustering 27
x < Machine Learning and its Applications

2.3 Semi-supervised learning 28

2.4 Function approximation 29
2.5 Generative and discriminative models 32
2.6 Evaluation of learner 32
2.6.1 Stochastic gradient descent 34
2.6.2 Cluster evaluation 37

SECTION II: MACHINE LEARNING

3. Data Pre-processing 43
3.1 Feature extraction 44
3.2 Sampling 46
3.3 Data transformation 47
3.4 Outlier removal 47
3.5 Data deduplication 48
3.6 Relevance filtering 48
3.7 Normalization, discretization and aggregation 49
3.8 Entity resolution 50
4. Supervised Learning 53
4.1 Classification 56
4.1.1 Artificial neural networks 57
4.1.2 Bayesian models 67
4.1.3 Decision trees 69
4.1.4 Support vector machines 74
4.1.5 k-nearest neighbor 79
4.2 Regression analysis 82
4.2.1 Linear regression 85
4.2.2 Polynomial regression 91
4.3 Logistic regression 92
5. Evaluation of Learner 97
5.1 Evaluating a learner 97
5.1.1 Accuracy 99
5.1.2 Precision and recall 99
5.1.3 Confusion matrix 101
5.1.4 Receiver operating characteristic 103
 Contents < xi

6. Unsupervised Learning 107

6.1 Types of clustering 109
6.1.1 Centroid, medoid and prototype-based 110
clustering
6.1.2 Density-based clustering 110
6.2 k-means clustering 110
6.3 Hierarchical clustering 113
6.4 Visualizing clusters 115
6.5 Evaluation of clusters 116
6.5.1 Silhouette coefficient 117
7. Semi-supervised Learning 119
7.1 Expectation maximization 120
7.2 Pseudo labeling 123

SECTION III: DEEP LEARNING

8. Deep Learning 127

8.1 Deep learning basics 128
8.1.1 Activation functions 129
8.1.2 Feature learning 132
8.2 Convolutional neural networks 133
8.3 Recurrent neural networks 137
8.4 Restricted Boltzmann machines 141
8.5 Deep belief networks 143
8.6 Deep autoencoders 144

SECTION IV: LEARNING TECHNIQUES

9. Learning Techniques 149

9.1 Learning issues 150
9.1.1 Bias-variance tradeoff 150
9.2 Cross-validation 154
9.3 Ensemble learning 155
9.4 Reinforcement learning 156
9.5 Active learning 157
xii < Machine Learning and its Applications

9.6 Machine teaching 158

9.7 Automated machine learning 159

SECTION V: MACHINE LEARNING

APPLICATIONS

10. Machine Learning Applications 163

10.1 Anomaly detection 164
10.1.1 Security 165
10.1.2 Predictive maintenance 166
10.2 Biomedicale applications 167
10.2.1 Medical applications 167
10.3 Natural language processing 168
10.3.1 Text mining 169
10.4 Other applications 171
11. Future Development 175
11.1 Research directions 177
References 181
Index 185
List of Figures

1.1 Data mining steps 6

1.2 Email pre-processing 9
2.1 Machine learning areas 18
2.2 Likert scale 20
2.3 Perceptron 22
2.4 Linearly separable and inseparable data points 24
2.5 Activation functions 25
2.6 Linear and nonlinear fitting 25
2.7 k-means clustering 28
2.8 Function approximation 30
2.9 Gradient descent 35
2.10 Gradient descent issues 36
3.1 Feature vector 45
4.1 Neuron 58
4.2 Artificial neural network 59
4.3 Recurrent neural network 60
4.4 Perceptron 61
4.5 Activation functions 62
4.6 Hyperbolic tangent 65
4.7 Decision tree 70
4.8 Random forest 74
4.9 Support vector machine 75
4.10 Support vector machine hyperplanes 76
4.11 Kernel trick 78
4.12 k-nearest neighbor 81
xiv < Machine Learning and its Applications

4.13 Regression analysis 83

4.14 Regression analysis steps 85
4.15 An overfitted regression curve 86
4.16 Linear regression 87
4.17 Logistic regression curve 93
5.1 Precision recall 100
5.2 Precision and recall 101
5.3 Multiclass confusion matrix 102
5.4 Receiver operating characteristic 104
6.1 Exclusive and non-exclusive clusters 109
6.2 Agglomerative and divisive clustering 114
6.3 Single, complete and average linkage 114
6.4 Dendrogram clustering 116
7.1 Maximum likelihood estimation 121
7.2 Maximum likelihood initial estimation 122
7.3 Maximum likelihood approximation 123
8.1 Rectified linear unit 130
8.2 Softplus function 132
8.3 Convolutional neural network layers 135
8.4 Convolutional neural network 136
8.5 Recurrent network types 138
8.6 Unfold through time 140
8.7 Restricted Boltzmann machine 142
8.8 Deep autoencoder 145
9.1 Linear fitting 152
9.2 The bias-variance tradeoff 153
9.3 Reinforcement learning 157
List of Tables

1.1 Machine learning and statistical terminology 14

5.1 Confusion matrix for the cancer tissue example 102
INTRODUCTION I
Chapter 1

Introduction

CONTENTS
1.1 Data mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Data mining steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4 Data pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5 Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5.1 Supervised learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5.2 Unsupervised learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5.3 Semi-supervised learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.5.4 Machine learning and statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.6 Data post-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Machine Learning (ML) is a sub-area of artificial intelligence, or AI.

The central scientific goal of artificial intelligence is to understand the
principles that make intelligent behavior possible in natural or artifi-
cial systems [31]. Early artificial intelligence research showed a lot
of interest in making computers reason and deduce facts [25]. In re-
cent research, the focus of artificial intelligence has shifted towards
designing and building agents that act intelligently. An agent can be
a living object, such as a human or an animal, or it can be a robot, a
sensor or a car. In artificial intelligence, we are interested in computa-
tional agents whose decisions can be explained. Artificial intelligence
is founded on mathematics, logic, philosophy, probability theory, lin-
4  Machine Learning and its Applications

guistics, neuroscience, and decision theory. It has many applications

in computer vision, robotics, natural language processing (NLP) and
machine learning.

The goal of machine learning is to enable computers to learn on their

own. Machine learning is about making computers modify or adapt
their actions (whether these actions are making predictions, or con-
trolling a robot) so that these actions become more accurate, where
accuracy is measured by how well the chosen actions reflect the cor-
rect ones [25]. Using machine learning, a computer can learn rules
by itself without the need of a programmer to develop them manu-
ally. This is particularly useful when the rules get too complex for a
developer to implement, or when there are too many rules to be pro-
grammed manually. An example where machine learning is used are
spam filters. Spam filters classify mails into legitimate and unsolicited
mails, i.e., spam. There are constantly new forms of spam appearing
and there are simply too many different types of spam mails for it to be
possible to program the rules to recognize them manually. Spam filters
learn the rules from past spam mails and learn new rules as new types
of spam appear. How computers learn to solve problems without being
explicitly programmed is the central question this book tries to answer.

Machine learning is nothing new. The term “Machine Learning” was

coined by Arthur Samuel in 1959 while working at IBM. He defined
machine learning as a field of study that gives computers the ability
to learn without being explicitly programmed [2]. In his seminal work
“Some Studies in machine learning Using the Game of Checkers” [35],
he probably describes the first program with the capability to learn. His
checkers program learned to improve its playing by itself. A computer
can be programmed so that it will learn to play a better game of check-
ers than can be played by the person who wrote the program [35].
However, machine learning goes further back, to the 1940s, with Mc-
Culloch and Pitts’ theories of biological learning [26] and Rosenblatt’s
perceptron, the first artificial neuron, described in 1958 [34].

Machine learning has gained momentum in the past years due to big
progresses, in particular using deep learners, that have been made in
object and speech recognition and autonomous systems, such as au-
 Introduction  5

tonomously driving cars, automatic machine translation, image caption

generation or coloring black and white images.

Machine learning is an interdisciplinary area of research based on

statistics, probability, neuroscience, psychology and physics. Machine
learning is a mathematical formalization of learning. As such, there is
a discrepancy between natural learning in humans and animals, which
is a biochemical process, and learning on computers, which is a math-
ematical abstraction. In humans, learning is carried out in wetware, in
computers it is carried out in hardware.

1.1 Data mining

Data mining (DM) is defined as the process of discovering patterns in
data [40]. There is no clear distinction in literature between data min-
ing and machine learning. In some publications, data mining focuses
on extracting data patterns and finding relationships between data [10],
whereas machine learning focuses on making predictions [36]. Data
mining is the process of discovering useful patterns and trends in large
data sets. Predictive analytics is the process of extracting information
from large data sets in order to make predictions and estimates about
future outcomes [17]. Therefore the distinction is in the aim. Data min-
ing aims to interpret data, to find patterns that can explain some phe-
nomenon. Machine learning on the other hand, aims to make predic-
tions by building models that can foresee some future outcome. How-
ever, clustering is a machine learning technique that aims to understand
the underlying structure of the data and has similar goals to data min-
ing. Here, we treat machine learning as a subarea of data mining where
the rules are learned automatically.

Humans learn from experience, machines learn from data. Data is the
starting point for all machine learning projects. Machine learning tech-
niques learn the rules from historic data in order to create an inner rep-
resentation, an abstraction, that is often difficult to interpret. Program-
ming computers to learn from experience should eventually eliminate
the need for much of this detailed programming effort [35].
6  Machine Learning and its Applications

1.2 Data mining steps

A typical data mining workflow goes through several steps. These steps
include data collection, cleansing, transformation, aggregation, model-
ing, predictive analysis, visualization and dissemination. However, de-
pending on the problem at hand, those steps might vary or additional
steps might be necessary. Data mining requires domain specific knowl-
edge. If a data mining project is supposed to detect fraud and money
laundering in financial transactions or group news articles into cate-
gories such as “Politics”, “Business”, “Science” etc., different tech-
niques apply. Getting familiar with the domain of the application and
setting the goals of the data mining project are necessary preliminary
tasks in order for the data mining project to complete successfully.
Domain knowledge is also necessary to evaluate the performance of a
trained learner. Figure 1.1 summarizes the data mining steps.

Figure 1.1: Data mining steps.

Data mining is a highly iterative process and typically goes through

many iterations until satisfactory results are achieved. Data mining
projects have to be executed with great care. The growth of data in the
past years has fostered the development of new data mining applica-
tions where the internal workings often go undocumented. The black-
box approach is dangerous since the results can be difficult to explain,
or lead to erroneous conclusions. For instance, cancer tissue in medical
images that was undetected or a traffic situation that was misinterpreted
by an autonomously driving car. The ease with which these applica-
tions can manipulate data, combined with the power of the formidable
data mining algorithms embedded in the black-box software, make
their misuse proportionally more hazardous [17]. Ultimately, one can
 Introduction  7

find anything in data and if a machine learning project is carried out

without proficiency it can lead to expensive failures.

1.3 Data collection

Data collection is the process of tapping into data sources and extract-
ing the data needed for training and testing a model. Data sources in-
clude databases, data warehouses, transaction data, the Internet, sensor
data from the Internet of Things or streaming data, but many more
sources exist. If the data is stored in its original format, it is called a
data lake. The data can be historic or real-time, streaming data. For
instance, training a model for spam filtering needs historic, labeled le-
gitimate and spam mails for training and testing. Spam mails are art-
fully crafted in order to avoid elimination by spam filters, making spam
filtering a tricky task. Spam filtering is one of the most widely-used ap-
plications of machine learning.

Often there is more than one data source and multiple data sources
need to be combined, a process called data integration. As a general
technology, data mining can be applied to any kind of data as long as
the data are meaningful for a target application [10].

It is often difficult to collect enough training data. Data is sometimes

not publicly available or cannot be accessed for privacy or security
reasons. To mitigate the problem of sparse data sets, synthetic data can
be created, or training techniques such as cross-validation can be ap-
plied. Also, to train a learner, labeled data is needed. Raw data, such
as emails, are not labeled, and producing labeled training sets can be a
laborious task. Semi-supervised techniques such as active learning can
be used in these situations. Active learning is a form of online learn-
ing in which the agent acts to acquire useful examples from which to
learn [31]. In offline learning, all training data is available beforehand,
whereas in online learning the training data arrive while the learner is
trained. The learning algorithm can actively ask the agent to label data
while it arrives. Data availability, whether labeled or not, is crucial for
the success of a machine learning project and should be clarified before
a machine learning project is initiated.
8  Machine Learning and its Applications

1.4 Data pre-processing

A problem that plagues practical machine learning is poor quality of
the data [40]. Real world data is often noisy and inconsistent and can-
not be used as is for practical machine learning applications. Also, real
world data is seldom in a format that can be directly used as input for a
machine learning scheme. That is why data pre-processing is needed.
Data pre-processing is usually where most of the time is spent. It is not
unusual that it takes up to 70% of the effort in a data mining project.

Whereas machine learning techniques are domain independent, data

pre-processing is highly domain specific. For instance, depending on
whether text data is analyzed or images, different pre-processing steps
apply. The pre-processing tasks also depend on the learning algorithm
applied. Some algorithms can handle noise better than others. For in-
stance, linear regression is very sensitive to outliers, which makes out-
lier removal a mandatory pre-processing step.

There are many different pre-processing techniques. Typical data pre-

processing tasks next to outlier removal include relevance filtering,
data deduplication, data transformation, entity resolution and data en-
richment. For instance, going back to the spam filter example, spam
mails typically contain words or phrases such as “buy online”, “online
pharmacy” or hyperlinks more often than legitimate mails. The fre-
quency of certain words or phrases gives an indication of whether the
mail is spam or legitimate. Irrelevant words, characters or symbols are
first removed from the mail, a process called stop word removal. There
is no authoritative list of stop words and they depend on what is being
mined for. Stop words are the most common words in a language, e.g.,
“the”, “who”, “that”. After stop word removal, the frequencies of the
remaining words are counted, to create a word list with their frequen-
cies. The resulting list is called a bag-of-words. The bag-of-words is
then used as input for a machine learning scheme. Figure 1.2 shows
an email before and after pre-processing.

Since machine learning algorithms usually do not take text as input,

creating a bag-of-words with their frequencies is a very common pre-
processing step in many text analysis tasks. It is simple but effective
 Introduction  9

Subject: CONGRATULATIONS, YOU JUST WON ~ CONGRATULATIONS, ¥91d' Jld'5T UIQN

THE NATIONAllOTIERY THE NATIONAllOTIERY

From: Fulfillment officer <[email protected]> FreA'I:: F~:~lfiiiR'IeRt effieer det:tept @gRiail.eeA'I:I'

To: [email protected] le: leEiarsiah@gA'Iaii.EBA'I:

CONGRATULATIONS, you just won 3,5 million CONGRATULATIONS~ won 3,5 million
Dollars. Click on the link below to claim your 9ellaf!r.. Click eA-ttte link &elew4& claim your
3,5 million Dollar reward: 3,5 million 9eJ.Ia.f reward:

http://natlottery.claimreward.ru http://natlottery.claimreward.ru

Congratulations from the National lottery Congratulations ~ National lottery

again agaH>

To unsubstribe, ditk followinc link: ~ ditk followinc link:

http://natlottery.dalmreward.ru http://natlottery.dalmreward.ru

Figure 1.2: Email pre-processing.

for tasks such as opinion mining, text classification or information re-

trieval.

Extracting the relevant information from a data set is called feature ex-
traction. It selects the relevant observation points from raw data. Some-
times the feature set is reduced, a process called feature selection. Fea-
ture selection helps to avoid overfitting since it reduces the complexity,
but it should still describe the data with sufficient accuracy. The result-
ing set of features is represented as a feature vector. A feature vector is
what many machine learning algorithms use as input. Features can be
numerical, e.g., the age of a person, or categorical, e.g., the job title of
a person. The input for a machine learning algorithm can be a tensor
with any number of dimensions. If the tensor has one dimension it is a
vector, if it has two dimensions it is a matrix.

Data transformation is another common data pre-processing step. It

converts data into a format that can be used as input for machine learn-
ing algorithms. For instance, converting feet into meters, strings into
numeric values or continuous into discrete values are common trans-
formation tasks.

In 1997, IBM’s Deep Blue defeated chess champion Garry Kasparov.

This was hailed as one of the milestones of machine learning. From
a pre-processing point of view, chess is a simple task since chess has
clearly defined rules and a move such as “Bishop c3 - d4” is unam-
10  Machine Learning and its Applications

biguous. In other words, chess data has no noise and a limited set of
rules.

1.5 Data analysis

Data analysis is the process of knowledge discovery. During the data
analysis phase, the predictive model is created. There are many data
analysis methods that do not use machine learning techniques. How-
ever, this book focuses on data analysis using machine learning, other
methods are beyond the scope of this book.

Machine learning is divided into supervised, semi-supervised and un-

supervised learning. Some widely-used learning methods include en-
semble learning, reinforcement learning and active learning. Ensem-
ble learning combines several supervised methods to form a stronger
learner. Reinforcement learning are reward-based algorithms which
learn how to attain a complex objective, the goal. Active learning is
a special form of semi-supervised learning.

1.5.1 Supervised learning

Supervised learning techniques are applied when labeled data is
present. The labeled data is used for training and testing. Labeling is
often a manual process and can be time consuming and expensive. Ev-
ery training data record is associated with the correct label. For spam
filtering, labeled data will mean a data set of spam and of legitimate
mails. Here, the labels are “spam” and “legitimate”. During training,
the machine learning algorithm learns the relationship between the
email and the associated label, “spam” or “legitimate”. The learned
relationship is then used for classifying new emails that the learner has
not seen before into their corresponding category.

Supervised methods can be used for classification and regression. Clas-

sification groups data into categories. Spam filtering is a classification
problem since mails are classified into spam and legitimate mails. The
classes are the labels. Since there are two categories, it it a binary clas-
 Introduction  11

sification problem. If there are more than two classes, it is called a

multi-class or multi-label classification problem.

Regression analysis is used for estimating the relationship among vari-

ables. It tries to determine the strength of the relationship between a se-
ries of changing variables, the independent variables, usually denoted
by X, and the dependent variable, usually denoted by Y. If there is one
dependent variable, it is called simple linear regression, if there is more
than one dependent variable, it is called multiple linear regression. In
classification, you are looking for a label, in regression for a number.
Predicting if it is going to rain tomorrow is a classification problem
where the labels are “rainy” or “sunny”, predicting how many millime-
ters it is going to rain is a regression problem. The target or dependent
variable y is a continuous variable. Contrarily, discrete variables take
on a finite number of values. Typical supervised methods are Bayesian
models, artificial neural networks, support vector machines, k-nearest
neighbor, regression models and decision tree induction.

1.5.2 Unsupervised learning

Unsupervised learning techniques are used when no labeled data is
present. In other words, there is no y. One of the most popular unsu-
pervised approaches is clustering. The goal of clustering is to under-
stand the data by finding the underlying structure of data. Clustering
groups data based on some similarity measure. For instance, a com-
pany groups it’s online users into customer groups with similar pur-
chasing behavior and demographics in order to better target them with
a marketing campaign. In this example the similarities are the purchas-
ing habits and the age group. Since there is no labeled data, evaluating
a cluster is not an obvious task to do. There is no ground truth that the
result of a clustering task can be compared with.

Clustering can also be used to represent data in a more compressed for-

mat (dimensionality reduction), for data summarization, while keeping
it’s structure and usefulness. Clustering can also be used as a prelim-
inary step for supervised machine learning algorithms, for instance,
12  Machine Learning and its Applications

to reduce dimensionality to improve performance of the supervised

learner.

Typical clustering algorithms include k-means clustering, hierarchical

clustering or principal component analysis (PCA). In k-means cluster-
ing, the algorithm groups data points into k groups where k is the center
of a group and is called centroid. The centroid is also called geometric
center or barycenter and represents the mean position of all the data
points of a cluster group. In k-means clustering, the data points are
grouped around the centroids. The points closest to the centroid k are
added to the cluster. For instance, a warehouse inventory is grouped by
sales activity or sensor data in an Internet of Things (IoT) application
is grouped into normal and deviant sensor data for anomaly detection.

In k-means clustering, the number of k has to be defined beforehand.

Selecting the number of clusters is not always an obvious task to do.
For instance, grouping images of animals into their biological fami-
lies such as felines, canines, etc., requires prior knowledge about how
many families there are going to be in the images to be clustered. k-
means is easy and converges quickly, which often makes it a good
starting point for a machine learning project.

Hierarchical clustering or hierarchical cluster analysis (HCA), groups

data by creating a tree or dendrogram. There are two approaches, a
bottom up or agglomerative approach, where each data point starts in
it’s own cluster, and the top down or divisive approach, where all ob-
servations are put into one cluster and splits are performed in order to
create a hierarchy that is usually presented as a dendrogram.

Human and animal learning is largely unsupervised. Humans learn

from observing the world, not by being told the label of every object,
making unsupervised learning more biologically plausible.

1.5.3 Semi-supervised learning

Semi-supervised learning is typically used when a small amount of la-
belled data and a large amount of unlabelled data is present. Labeling
Discovering Diverse Content Through
Random Scribd Documents
Now Mrs Mackinnon had what is called “an ill tongue,” and
she did not spare poor Alaister as she turned over his torn
garments; but he was well accustomed to her attacks, and
had learnt that silence was his only safety, so he took one
child on his knee as he sat by the fire, and rocked the
cradle with his foot, in hopes of softening his wife’s temper.
As the evening advanced, she became pretty tired of having
all the talk to herself, so sat down opposite him, and with a
cross face, and in a sharp voice, asked what made him sit
there without speaking,—could not he tell her any news
after being sae long away from his gude wife and the
weans?

When this question was put, Alaister was always sure the
scold was over, however cross the voice was in which it was
asked; so he began at once to tell all the events of a
harvest home at which he said he had been the night
before, but he was at once stopped by an angry “Hout!”
from his wife, and then followed a storm of abuse for telling
her about things which had happened three years before;
then, pointing to the fields of green oats that were to be
seen all around, she asked him what sort of harvest home
there could be at that time of year. Alaister was sorely
puzzled, for certainly the corn was still green; but yet he
felt sure it was only yesterday he had been at the harvest
feast, and if not at that—where had he been? He could
remember nothing of the wedding, and stared at his wife,
who at last began to be alarmed at his perfectly stupid look,
and said, “Is the man fey?” As soon as she said this, his
night’s adventure returned to his mind, and looking on the
ground, he saw it alive with fairies, laughing and mocking
him. Had it been earlier in the day, he would have run out
of the house, but it was nearly dark, and the uncomfortable
Will-of-the-Wisp came into his mind, so he sank down again
in his chair, and shut his eyes, fully determined not to
speak; but he could not keep this resolution. Again and
again he was impelled to begin stories, and as often was he
told that these things had happened years before. He then
tried to play, but could remember none but the very oldest
tunes, such as had been out of date for many years, and
when, wearied in mind and body, he fell asleep, he dreamed
of fairies and discomforts all night long.

Next day he set out again on his wanderings, hoping that it

was only in his own house that the fairies would haunt him;
but no—go where he would they were by him, nor could he
tell any story which was not at least three years old. His
former admirers, the women, now asked him, jeeringly, for
“three-year-old news;” when he was seen coming towards a
farm, he was treated almost as a beggar, and was sent to
the back door, where he got a piece of oat-cake and a drink
of milk, but was never asked into the house. Occasionally
the servants asked him why he did not carry a wallet like
other “puir bodies;” but Alaister, though often really in
want, never would condescend to a wallet. By degrees he
became more and more impoverished; he was thin, and had
a look of great unhappiness. His hose hung over the heels
of his worn shoes, from which the silver buckles had long
since disappeared; his second-best kilt was very much the
worse for wear, nor had he money to buy a new one; and as
to the one he had worn on the night from which his woes
dated, it had even beat the thrifty Mrs Mackinnon to get it
into tolerable repair again.

In all the country side it had become the common

expression, when any old story was told, “Hout! that’s
Piper’s news;” and at last Alaister, feeling that he was
despised where he had been respected, and laughed at by
those at whom he had laughed, without even having a
comfortable house in which to hide himself, for Mrs
Mackinnon’s tongue was more abusive than ever,
determined to retire from the world.
Being in low spirits, of course he chose the most dismal
spot he could find; it was a bleak glen, down which the
north wind howled in winter, and in summer the sun hardly
reached its depths; for the bare rocks were high and near
each other, so that it was always cold and damp. But this
suited Alaister’s frame of mind. One chill day in autumn he
crept into a sort of hollow in the rock; there was a constant
trickle, trickle, trickle, down the sides of this hole, and the
water soaked through blackened patches of liver-wort and
moss; the floor was damp and slippery, and on it Alaister
sat down to think how very uncomfortable he was, and to
abuse the fairies as the cause of all his misfortunes.

It grew colder and colder, and darker and darker, and

Alaister began half to repent of his determination to die in a
cave, when a flash of light shone into the hollow, and in an
instant his old acquaintances, the three Will-of-the-Wisps,
were dancing round him in a more frenzied way than ever;
now they were up in the roof, now out in the open air, now
far back in the darkness where he thought there was only
rock. But the cave seemed to become larger every moment,
and the water dried up as the Will-of-the-Wisps darted
along the sides, and then Alaister saw the well-remembered
tod’s-tail moss hang where liver-wort had been before, and
stag’s-horn moss again covered the dark floor. The air felt
dry and warm, and a comfortable sleepy peace crept over
the heart of the distressed piper; he began to think that, on
the whole, it was more enjoyable to be in the fairies’ cave
than in a hay-loft on a gusty autumn night; and when the
glittering band sparkled into their hall he smiled, and
offered to play to them again, and soon they were all
dancing merrily on the moss, for it was now too cold, even
for fairies, to spend the whole night in the woods.

Then came the feast, and this time Alaister was given on
acorn cup full of brightest mountain dew; and though he
thought it a small allowance for a full-grown man, still he
knew that the little creatures had no larger cups; and not to
disappoint them or fail in his manners, he nodded to the
king, and with a “Here’s your very gude health, sir,” emptied
his cup. Immediately he sunk back on the floor and slept,
for the dew that had been given him has, it is said,
wonderful powers, making mortals forget their homes and
former lives, and desire only to be with the fairies.

How long he slept no one can tell; he never more was seen:
but on calm summer nights his pipes can be heard droning
under ground, or in the sweet birch wood. From their being
heard to this day it is supposed that those who enter the
service of the fairies become immortal; but no one has
ventured to watch the gambols of the “gude fouk,” so as to
ascertain whether it is Alaister himself who still leads their
march, or whether another has succeeded him; indeed, the
glen is more shunned than ever, and the cave goes by the
name of the Piper’s Cave in all that district, while the
expression “Piper’s news” is known over the whole world.
 Story 9--Chapter I.
STORY NINE—The Genius of the Atmosphere.
High up on the side of a lofty mountain, overlooking the
wide ocean, several boys were seated together on the moss
and lichens which clothed the ground, and were the only
vegetable productions of that elevated region. The bright
sea sparkled in sunshine, far, far down below their feet,
though hidden at times from their sight by the dark clouds
which came rolling on, sometimes enveloping them in mist,
and at others breaking asunder and floating away far inland
towards other ranges of distant hills. High above their heads
rose a succession of rugged peaks, black, barren, and
fantastic in form, which the foot of man had never trod. The
boys on a party of pleasure had climbed up from a town by
the sea-side, and had brought with them, in knapsacks and
baskets, a supply of provisions, which they now sat down to
discuss. The keen pure air, and the exercise they had
undergone, sharpened their appetites and raised their
spirits, and they sat laughing and talking, and apparently
enjoying themselves to the utmost. Far below their feet
sea-fowl were skimming rapidly through the air, wheeling
and circling, now descending to the bright water below, and
then rising again up into the clear expanse of ether,
rejoicing in their freedom. On a crag below them, near
where she had built her nest, stood an osprey. With wings
expanding she prepared to take her flight; then off with a
cry of joy she flew, darting through the atmosphere, away,
away, over the ocean, looking down upon the tall ships
which sailed along slow and sluggishly compared to her
rapid progress. The boys eagerly watched her till she was
lost to sight in the distance.

“Oh, how I wish that I could fly, that I might skim over the
world like that sea eagle!” cried one, clapping his hands;
“what glorious fun would it not be? I should never consent
to walk again. All other amusements would be tame and
tasteless in comparison. Truly yes, it mast be a fine thing to
be able to fly like a bird. To fly!—to fly! Away!—away!” The
speaker as he uttered these words rose and stretched out
his arms over the ocean, as if in imagination at all events
he was about to spring off from his lofty perch, and to
follow the course of the osprey.

His enthusiasm inspired his companions. One after the

other exclaimed—

“Yes, indeed, it would be grand to be able to fly. Glorious to

mount up into the sky, without having tediously to climb up
a hill as we have done to-day; or to plunge down beneath
the waves, like those wild fowl; or to skim, as they can,
over the crests of the raging seas when storms blow
furiously, or to float in sunshine on the calm bosom of the
ocean.”

“Ay, of all things I would rather be a bird,” cried another.

“An eagle, a hawk, an albatross; any bird which can fly far
and swiftly. That is what I should like,—to fly, to fly, to fly!”
Thus one after the other they all expressed themselves.

Suddenly, as they were speaking, a loud crashing noise was

heard, and as, alarmed, they turned their heads, the rocks
behind them opened, disclosing a vast and glittering cavern,
out of which was seen slowly to advance, a lady, whose
garments shone with a dazzling radiance. Her form was
commanding, her face beautiful and benignant. The
astonished and bewildered boys scarcely dared to gaze at
her; but trembling and holding on to each other, they kept
their eyes cost on the ground. She spoke, and her voice
reassured them.

“You were all of you just now expressing a wish that you
could fly,” she said, in a sweet silvery tone. “Why do you
thus with to possess a power for which your All-wise
Creator has not designed you? Even could you by any
means secure wings to your body, of size sufficient to lift
you from the ground, your muscular powers are totally
inadequate to work them; your senses are not adapted to
the existence of a fast-flying bird; your brain would grow
dizzy, your eyes dim, you would be unable to draw breath in
the upper regions, through which your ambition would
induce you to wing your flight; you would speedily destroy
all your other senses. Be content with your lot. Still, if you
have a good object for your wishes, perhaps under certain
limitations they may be granted. Let me hear why you wish
to enjoy the power of flying?”

The boys looked at each other, and then up at the face of

the lady, and finding nothing in its calm expression to alarm
them, one after the other replied, the eldest speaking first:
—

“Because I should like to see what people are doing in the

world,” said he; “what nations are fighting with each other,
and how the hostile armies are drawn up. I have read of
fine processions, where priests walk with their sacred
images, when kings come to be crowned, and when their
subjects assemble to do them homage.”

“You need not say more,” observed the lady, and pointed to
another boy.

“I should like to follow all those ships I see sailing out

there,” he answered; “I should like to visit the strange lands
to which they are going, and to examine the curious things
they bring back.”

“You can accomplish thus much without flying,” answered

the lady; and passed on to another boy.

“I should like to fly, because it would be so curious to hover

about over cities, to look into houses, and to watch what
the inmates are doing,” said the boy.
The lady shook her head. “Such an employment is utterly
unworthy of an intelligent being,” she answered; “you would
make but an ill use of the power if you possessed it. What
have you to urge as a reason for obtaining the power you
wish for?” she inquired of a fourth boy.

“Oh! it would be so delightful to feel oneself floating up and

down in the air; now rising high, high up like a lark, now
skimming along over the smooth sea,” he answered, giving
expression to his words by the movement of his body.

“You evidently place the gratification of the senses above

the employment of the higher powers of your nature. Such
is but a bad claim for the possession of a new one.”

In this manner the lady questioned several other boys, but

she did not appear satisfied with any of their replies. At last
she asked a slight and thoughtful boy, who had been sitting
a little apart from the rest, why he had wished to possess
the power of flying?

“That I may better comprehend the glories of nature, and

understand what now appear the mysteries of the
universe,” he answered quietly, yet promptly; “whence the
rains, and mists, and winds come, and whither they go. I
would fly far away on the wings of the wind. I would visit
distant lands, to observe their conformation, to discover
new territories fit for the habitation of man. I would bear
messages of comfort and consolation from those in one
place to relatives far away. Oh! if I could fly, I am certain
that I should never weary of the work I had to do.”

“Well and wisely answered,” replied the lady. “I am the

Genius of the Atmosphere. The power you ask I cannot give
you: but follow me; I may be able to afford you some of the
gratification you so laudably desire.”
The boy, without hesitation, followed the lady towards the
rock from which she had emerged. It closed round him, and
he found himself in a cavern of vast size, and glittering with
gems of every hue, and of the richest water. The Genius
cast on him a smiling look, when she saw that his attention
was but little engrossed by these appearances.

“I cannot enable you to fly,” she remarked, “but I can

render you invisible, and bear you with me whither I go,
even to the uttermost parts of the earth. Come, note well
what you see. You may never again have the some
opportunity of observing the wonders of nature.”

As the Genius spoke, the boy found himself borne buoyantly

from off the earth. He passed close by his companions, who
were thoughtlessly laughing and talking as before, and on
he rapidly floated, they neither observing him nor the
Genius of the Atmosphere.

“Child of Earth, follow me,” said the Genius; and the boy
floated gently on, till he found himself in a region of perfect
calms. Below him, as he looked towards the earth, he saw
mountains of snow, and fields of ice glittering gloriously in
the slanting rays of the sun.

“We are at the north-pole of the earth,” said the Genius;

“you desire to know the course of the winds, and how they
are created—observe and learn.” As she spoke, she shook
from her robes a shower of silvery particles, which floated
buoyantly in the air. “See, at this point the silvery cloud
does not partake of the diurnal motion of the globe, but a
slight current of air, scarcely perceptible, is sending it
forward. We will follow it towards the southern pole. You
can scarcely see the earth, we are so high up. Lower down
are currents rushing towards the pole, which would impede
the progress of this silvery cloud.”
On, on, on, rapidly the Genius flew. A golden cloud
appeared. The two clouds met, but so softly, that there was
no commotion. Attracted by the globe, probably, they both
descended, slowly followed by the Genius and the boy, till
once more the earth appeared in sight, clothed with the
palm-tree, the orange, the pomegranate, the vine, and
numberless tropical fruits and flowers.

“We have reached a calm region, the tropic of Cancer,” said

the Genius. “Now watch the earth. It is turning from west to
east, while we move on in the direct line in which we
started, so that we appear to be crossing the globe
diagonally, and to the inhabitants of the earth that silvery
cloud appears to be coming from the north-east, and going
to the south-west. That silvery cloud is merely a portion,
made visible to your eye, of a great mass of air, which is
continually blowing, and which the inhabitants of the earth,
from the facilities it affords their commerce, call the north-
east trade-wind. Now see a golden cloud approaching us;
that is a mass of air coming from the southern pole. We are
arriving near the Equator. See, the two clouds meet. They
have an equal impetus; neither can give way, but, gently
and noiselessly pressed together, they rise to a higher
stratum of the atmosphere.”

On floated the boy and his guide, far up above the globe,
still on, in rather a less direct line than before, till again a
golden cloud was met, and gently that, and the cloud they
followed, descended till the earth was seen once more.

“We have reached the tropic of Capricorn, where these two

opposing currents form a calm, almost continuous, except
when certain interposing causes break it, and which I may
hereafter explain to you.” Passing out of the calm region,
away they floated towards the southern pole.
“Remark,” observed the Genius. “The silvery cloud, having
been pressed down by that other current from above, has a
south-eastern direction given to it, and therefore appears to
the people on earth to be coming, not from the north, but
from the north-west.”

A wide extent of ocean was seen beneath their feet. On

they floated. Then fields of ice and icebergs, and wide
extended lands covered with snow, and vast mountains of
ice. Once more they moved on, slowly as before.

“We are at the antarctic pole,” said the Genius. “See, our
cloud of silver meets another of gold, pressing gently.” Up,
up, they mount. “Once more we will move towards the
tropic of Capricorn, high up above the globe. Now we
descend in that calm region; and now close to the earth we
are moving on. But see, coming from the southern pole, the
globe moves as before, from west to east; and thus this
mass of air, of which our silvery cloud, remember, is but a
portion, seems to those on the earth to be coming from the
south-east. As this wind is always blowing, and as ships by
getting within its influence are borne easily forward, and it
thus facilitates commerce, it is called the south-east trade-
wind.”

On they went, till again the calms of the equator were

reached, or rather, till the air, exhausted by its long course,
met another gentle current, and the two pressing together
rose upwards, the silvery cloud going on towards the tropic
of Cancer, till forced by another current, known by its
golden hue, to descend, it went on close to the earth
towards the northern pole, where a calm, caused by
another gentle current meeting it, was created. Gently
pressed up, however, the silvery cloud finally reached the
higher region, whence the Genius and the boy had started
with it on its long journey.
“Had we started with the golden cloud, or rather with the
mass of air which that cloud represents, from the southern
pole, we should have seen precisely the same effects
produced,” said the Genius. “You now understand what
mortals call the theory of the trade-winds. You read in the
sacred word of God, which in his mercy and goodness he
gave to men to guide them in their passage through life,
that, ‘The wind goeth toward the south, and turneth about
unto the north; it whirleth about continually, and the wind
returneth again according to his circuits’ (Eccles. i. 6). Now,
boy, you have seen how true and beautiful is that account
written by the wise king of Israel.” The boy listened
attentively. “We will fly back to the equatorial calms,” said
the Genius; “see what effect the direct rays of the sun have
on the earth, or that portion of its surface. They affect the
air likewise; heat expands it, and then makes it rise; and it
also changes its specific gravity. Cold contracts it, and also
changes its specific gravity. These two causes are
unceasingly at work to produce the currents of air whose
courses we have been observing. The heat of the sun at the
equator expands the air, and thus it rises and flows north
and south; having arrived once more at the tropics, owing
to the counter current it meets, it descends, as we saw, and
flowing along near the earth, receives from it a rotatory
motion, which increases as it approaches the pole, where,
contracted by the cold, it masses into a dense body, and
ultimately is whirled upwards, forming an ascending
column, when it once more commences its never-ceasing
journey.”

As they flew towards the mountain whence they set out, the
boy expressed his thanks to the Genius; if he did not
comprehend all that she had shown him and told him, he
knew more about the matter than he had before done. She
saw by the expression of his countenance the gratification
he had enjoyed. “’Tis well,” she continued; “as a drop of
water is to the ocean which lies beneath us, so is the
knowledge you may obtain in a lifetime to the wonders
nature has to reveal. You desire to know more; gladly will I
show you more. Whenever you climb up to this rocky height
I will meet you, as I have done to-day, and each time unfold
new wonders to your view. Ah, you think that I might
descend to you, without making you toil up the mountain;
but know that knowledge will not come to you; you must
exert yourself, you must labour to attain it. You say that you
will willingly climb the height. That is well. That is the spirit
which ensures success. Return to your companions. They
will not have missed you.”

Suddenly the boy found himself as he had been before,

sitting a little apart from his friends. He was silent and
thoughtful as he descended the mountain, resolving to
return as soon as possible, to learn from the Genius more of
the wondrous mysteries of nature.
 Story 10--Chapter I.
STORY TEN—A Terrible Blanket.
Well, we were on the continent when I met with my terrible
blanket. We were going up one of the passes on foot, and
somehow I, as I usually do, lagged behind. I, of course, had
an Alpine stock in my hand, and I went swinging it away,
until at last it struck against a lump of rock overhanging a
precipice, so deep that, sailor as I am, I trembled as I
looked down. Well, the stick bounded from the granite
against my shin, and so I made a vow that the lump of
granite should take a run, or my name was not Theophilus.

But it was a tough job, for the stone was very big, and well
set in the rock; but after a deal of straining and pushing,
down it went with dull thuds, as it fell from rock to rock,
and at last it splashed into the water, which seethed up as
though trying to get at and drown me.

The job must have taken me longer than I thought for, for
when I looked before me I could see no one, and as I
looked I began to see that twilight was coming on.

Now, I don’t know whether you have been much among our
own high hills in Scotland or Wales; but, if you have, you
must know how rapidly night comes on. It is day one
moment and night the next, so to speak.

Now I knew this, and made haste forward.

I do not think I had gone twenty yards when I knew, by the

great wuthering sound about me, that a storm was brewing,
and it was on me in no time; and as the snow came down a
great curtain seemed to be drawn over the sky, it grew dark
so quickly.

Well, I groped on, but I didn’t like it. If it had been a storm
at sea now, I should not have cared much; if the mountains
about me had only been of water, I should not have cared
at all; but when I knew that a false step might send me
toppling down as the rock had toppled before me, I don’t
mind owning that I grew to like it all less and less.

I stooped down to look at the path, as well as I was able in

the little remaining light, and I found I was in no path at all.

As the last rays of light died out, and as the snow whirled
about me, I remember, as though it would be glad to make
my winding-sheet, I turned cautiously towards a slope of
rock, feeling with my stick before I took a step, for the
snow will fill up a crevice in no time, and you may sink
twenty feet before you know where you are; and at last I
touched the rock.

There was still an atom of light left, and by it I just

discerned a black part of the rock, which I took, and rightly,
to be a cave. So I crept towards it, into it, and crouched
down on the ground to leeward; and I can tell you the wind
was getting up.

Well, I hadn’t lain there three minutes when it was as dark

as you could wish it. I don’t know whether any of you have
ever been in the dark when full of anxiety; but if you have,
you will believe me when I say every precious minute
seemed an hour.

Suddenly I thought of my fusee-box, and I believe shouted

as I thought of it, for a second idea came into my head.
Suppose I struck the fusees about one a minute, they would
not only help me through the darkness, but, luck willing,
they might answer the purpose of a revolving light, and
guide those who were looking for me to my place of shelter,
or the light might be seen at the convent, from which I
knew by the guide we were not far when I stopped to upset
the rock.
And I give you my honest word that not for one second did
I feel any ill-will against my companions for leaving me
behind; I somehow knew it was all right.

So out came the fusee-box, and the next moment I had

struck a light. Why I looked round the cave I can’t tell, but I
did, and I caught my breath, as you may suppose, when
away in the dark I saw two great yellowish-green balls of
fire.

I don’t think I moved for a moment, and then I began to

question myself as to whether it was not all fancy.

So I thought I would strike another light; but the box had

fallen amongst the snow, and when I felt for the matches
they were all mixed up with the powder, which is about the
only name you can give the snow in those places; it is very
different from the clammy snow we see here.

Now, what was I to do? If I went out of the cavern I should

be frozen to death, while to remain in the cave, and near
those dreadful lights, was maddening.

Well, one way or the other, I determined not to go either

backwards or forwards; so I curled myself up as small as
possible, and lay shivering. I had only lain for what I now
know to be a very short time, but which I took to be hours,
when something soft came up against my knees and
elbows.

You may believe I dashed out my fist, and felt it sink a foot
deep in the soft snow, which I rightly guessed had drifted
up against the opposite side of the cavern till it fell over and
rolled up against me.

Good, so I was being snowed up, and I saw I must either go

nearer those dreadful balls, which by this time I was sure
were no fancy, and which I felt certain were looking towards
me through the darkness, or I must stay where I was to be
buried alive.

I don’t know how I came to the decision; but I did at last

decide to go further into the cavern, and so I shuffled out of
the way of the snow.

And then I lay still again, waiting.

In a moment or so, surrounded by danger as I was, I began

to find myself actually going quietly to sleep. I had no idea
then that that sleep might have been the sleep of death.

Well, in another minute or so, I felt a warm air on my face;

but I was too sleepy to move, and so I lay still.

And then, believe me I do not exaggerate, I felt four

weights press, one after the other, upon my body, and then
a soft, heavy weight sunk down upon me. I had no doubt it
was an animal of some kind; I felt quite sure of this when a
muzzle was placed as near my mouth as possible.

I dare say you will hardly believe it, but in a few moments
all my fear had gone, and I found myself growing grateful to
this creature, for he made me so good a blanket that the
heat came back into my body, and I felt no longer that dull
sleepiness of which I have spoken.

I do not at all know how long I had thus lain, when a bark
was heard, which disturbed the regular breathings of my
hairy friend, and I felt his big heart beat above me. Again
there was a bark, the broad loud bark of a big dog, and it
sounded much nearer than the first.

As my blanket heard it, he uttered a harsh sound, and leapt

from off my back.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com