Get ebook downloads in full at ebookmeta.

com

Machine Learning for Practical Decision Making: A

Multidisciplinary Perspective with Applications
from Healthcare, Engineering and Business
Analytics Christo El Morr
https://ebookmeta.com/product/machine-learning-for-
practical-decision-making-a-multidisciplinary-perspective-
with-applications-from-healthcare-engineering-and-business-
analytics-christo-el-morr/

OR CLICK BUTTON

DOWNLOAD NOW

Explore and download more ebook at https://ebookmeta.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Machine Learning for Business Analytics: Concepts,

Techniques and Applications with JMP Pro, 2nd Edition
Galit Shmueli
https://ebookmeta.com/product/machine-learning-for-business-analytics-
concepts-techniques-and-applications-with-jmp-pro-2nd-edition-galit-
shmueli/
ebookmeta.com

Machine Learning for Business Analytics: Concepts,

Techniques and Applications with JMP Pro, 2nd Edition
Galit Shmueli
https://ebookmeta.com/product/machine-learning-for-business-analytics-
concepts-techniques-and-applications-with-jmp-pro-2nd-edition-galit-
shmueli-2/
ebookmeta.com

Business Analytics A Data Driven Decision Making Approach

for Business Volume I 1st Edition Amar Sahay

https://ebookmeta.com/product/business-analytics-a-data-driven-
decision-making-approach-for-business-volume-i-1st-edition-amar-sahay/

ebookmeta.com

Fashion and Its Multi Cultural Facets 1st Edition Patricia

Hunt-Hurst

https://ebookmeta.com/product/fashion-and-its-multi-cultural-
facets-1st-edition-patricia-hunt-hurst/

ebookmeta.com
Ergodic Theoretic Methods in Group Homology A Minicourse
on L2 Betti Numbers in Group Theory Clara Löh

https://ebookmeta.com/product/ergodic-theoretic-methods-in-group-
homology-a-minicourse-on-l2-betti-numbers-in-group-theory-clara-loh/

ebookmeta.com

No Mercy (Valerie Law FBI Suspense Thriller #1) 1st

Edition Blake Pierce

https://ebookmeta.com/product/no-mercy-valerie-law-fbi-suspense-
thriller-1-1st-edition-blake-pierce/

ebookmeta.com

The Definitive Guide to Hotwire and Django 1st Edition

Michael Yin

https://ebookmeta.com/product/the-definitive-guide-to-hotwire-and-
django-1st-edition-michael-yin/

ebookmeta.com

Film and the Anarchist Imagination 2nd Edition Richard

Porton

https://ebookmeta.com/product/film-and-the-anarchist-imagination-2nd-
edition-richard-porton/

ebookmeta.com

The Vanquishers The Vanquishers 1 1st Edition Kalynn

Bayron

https://ebookmeta.com/product/the-vanquishers-the-vanquishers-1-1st-
edition-kalynn-bayron/

ebookmeta.com
Deep Learning Approaches to Cloud Security Deep Learning
Approaches for Cloud Security 1st Edition Pramod Singh
Rathore Editor Vishal Dutt Editor Rashmi Agrawal Editor
Satya Murthy Sasubilli Editor Srinivasa Rao Swarna Editor
https://ebookmeta.com/product/deep-learning-approaches-to-cloud-
security-deep-learning-approaches-for-cloud-security-1st-edition-
pramod-singh-rathore-editor-vishal-dutt-editor-rashmi-agrawal-editor-
satya-murthy-sasubilli-editor-sr/
ebookmeta.com
International Series in
Operations Research & Management Science

Christo El Morr
Manar Jammal
Hossam Ali-Hassan
Walid EI-Hallak

Machine Learning
for Practical
Decision Making
A Multidisciplinary Perspective
with Applications from Healthcare,
Engineering and Business Analytics
International Series in Operations Research &
Management Science
Founding Editor
Frederick S. Hillier, Stanford University, Stanford, CA, USA

Volume 334

Series Editor
Camille C. Price, Department of Computer Science, Stephen F. Austin State Uni-
versity, Nacogdoches, TX, USA

Editorial Board Members

Emanuele Borgonovo, Department of Decision Sciences, Bocconi University,
Milan, Italy
Barry L. Nelson, Department of Industrial Engineering & Management Sciences,
Northwestern University, Evanston, IL, USA
Bruce W. Patty, Veritec Solutions, Mill Valley, CA, USA
Michael Pinedo, Stern School of Business, New York University, New York, NY, USA
Robert J. Vanderbei, Princeton University, Princeton, NJ, USA

Associate Editor
Joe Zhu, Foisie Business School, Worcester Polytechnic Institute, Worcester, MA, USA
The book series International Series in Operations Research and Management
Science encompasses the various areas of operations research and management
science. Both theoretical and applied books are included. It describes current
advances anywhere in the world that are at the cutting edge of the field. The series
is aimed especially at researchers, advanced graduate students, and sophisticated
practitioners.
The series features three types of books:
• Advanced expository books that extend and unify our understanding of partic-
ular areas.
• Research monographs that make substantial contributions to knowledge.
• Handbooks that define the new state of the art in particular areas. Each
handbook will be edited by a leading authority in the area who will organize a
team of experts on various aspects of the topic to write individual chapters. A
handbook may emphasize expository surveys or completely new advances (either
research or applications) or a combination of both.
The series emphasizes the following four areas:
Mathematical Programming: Including linear programming, integer program-
ming, nonlinear programming, interior point methods, game theory, network opti-
mization models, combinatorics, equilibrium programming, complementarity
theory, multiobjective optimization, dynamic programming, stochastic program-
ming, complexity theory, etc.
Applied Probability: Including queuing theory, simulation, renewal theory,
Brownian motion and diffusion processes, decision analysis, Markov decision
processes, reliability theory, forecasting, other stochastic processes motivated by
applications, etc.
Production and Operations Management: Including inventory theory, produc-
tion scheduling, capacity planning, facility location, supply chain management,
distribution systems, materials requirements planning, just-in-time systems, flexible
manufacturing systems, design of production lines, logistical planning, strategic
issues, etc.
Applications of Operations Research and Management Science: Including
telecommunications, health care, capital budgeting and finance, economics, market-
ing, public policy, military operations research, humanitarian relief and disaster
mitigation, service operations, transportation systems, etc.
This book series is indexed in Scopus.
Christo El Morr • Manar Jammal •
Hossam Ali-Hassan • Walid El-Hallak

Machine Learning
for Practical Decision Making
A Multidisciplinary Perspective with
Applications from Healthcare, Engineering
and Business Analytics
Christo El Morr Manar Jammal
School of Health Policy and Management School of Information Technology
York University York University
Toronto, ON, Canada Toronto, ON, Canada

Hossam Ali-Hassan Walid El-Hallak

Department of International Studies Ontario Health
York University, Glendon Campus Toronto, ON, Canada
Toronto, ON, Canada

ISSN 0884-8289 ISSN 2214-7934 (electronic)

International Series in Operations Research & Management Science
ISBN 978-3-031-16989-2 ISBN 978-3-031-16990-8 (eBook)
https://doi.org/10.1007/978-3-031-16990-8

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland
AG 2022
This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether
the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by
similar or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, expressed or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
To our families for their love and support. To
our students, they are our inspiration.
Preface

This book fills a gap in the machine learning literature. It presents the machine
learning concepts in a very simple and practical way starting with a tangible example
and building on it the theory. Each machine learning chapter starts with a presenta-
tion of the problem to solve, followed by a practical example on how the algorithm
that solves the problem works, then comes the presentation of the machine learning
algorithm theory and closing remarks (e.g., pitfalls, advantages limitations). This is
followed by a set of key terms, a set of questions to test your understanding of the
chapter material, a set of references to read more about the subject and hands-on lab
exercises. The lab exercises allow readers to apply the chapter’s concepts and gain
needed skills. To maximize the benefit for readers and to expose them to a myriad of
machine learning languages and frameworks, the lab exercises (and sometimes the
chapter’s material) are built around Python mainly, followed by R (and R Studio),
and Weka. One of the strengths of this book is that it can be used by people who are
exposed to programming or would like to learn how to program and those who prefer
not to program but to solve decision-making problems with machine learning using
simple graphical user interface. The former can use Python, the machine learning
language par excellence, throughout all chapters, or R (limited to Chaps. 1–4, and 6).
The use of Tableau, a visual analytics platform, is reserved to Chap. 5, Data
Visualization, while Weka is explained and used in Chaps. 4 and 6–12; given the
simplicity of Weka, we believe that there was no necessity to add Weka-based lab
exercises for Chaps. 13–15.
This introductory textbook to machine learning for decision making can be used
by students in Computer Science, Information Technology, Health Informatics, and
Business fields. Depending on the students’ level of study and exposure to technol-
ogy, either Weka or Python can be used. However, given the pervasive use of Python
in the market we advise students of all sorts to get exposure to Python and how it
works. Teaching the whole Python language is beyond the scope of this book;
however, we cover Python’s libraries related to machine leaning (e.g., Scikit-
Learn, TensorFlow, Keras) and many Python programming concepts.

vii
viii Preface

Another strength of the book is its focus on the necessary content for an
introductory course to machine learning while providing enough complexity without
being complex or introducing heavy mathematical formulation; the exception being
neural networks where we considered that providing less simple mathematical
formulations was necessary as an illustration but skipping them would not be a
problem to understand the algorithm. There is no mathematical knowledge needed to
read and use this book. Chapter 2 provides, in a simple manner coupled with many
examples, the main mathematical concepts needed to understand the chapters.
A final strength of this book is the use of a variety of datasets from several
domains (e.g., health, business, social media, census, survey) which provide a good
exposure to the myriad types of applications in which machine learning can be used.
The book is organized in three parts: Part 1 is an introduction section that
encompasses Chaps. 1–5, it introduces machine learning fundamentals and allows
for installing the different software tools and the introduction to Python, R, and
Weka. The machine learning algorithms and corresponding lab exercises are covered
in Chaps. 6–15, and future perspectives are provided in Chap. 16.
Professors who adopt this book have flexibility in the way they want to teach the
material; it all depends on the objectives of their course. Some can use to teach
machine learning using Python and hence need to cover Chaps. 1, 4 and 6–16; this
could be true for courses related to practical machine learning. Professors who are
interested in teaching Analytics and (some) machine learning can cover Chaps. 1–5
and then some of the Chaps. 6–16. It is still feasible to cover all chapters in 12 weeks,
Chaps. 1 and 2 are an introduction and a quick overview that can be covered with
their labs in one session; Chaps. 3 (or some of it) and Chap. 4 are possible to
combine in one session. Chapters 5 and 16 can be covered in one session; then each
Chaps. 6–14 in one session; and final Chaps. 15 and 16 in one session. This is not to
impose a single way of approaching the textbook but to provide examples of
alternatives and demonstrate flexibility. The book provides you with flexibility to
be adopted in several contexts. The datasets used in the lab exercise touch upon
many domains: health, information technology, business, and engineering.
We hope that this first edition of the book will enrich the readers’ knowledge and
skills and we welcome their comments and suggestions. Readers can access the
Python code for the Chap. 6 to Chap. 15 lab exercises on GitHub https://github.com/
christoelmorr/ML-4-Practical-DM.git.

Toronto, ON, Canada Christo El Morr

Manar Jammal
Hossam Ali-Hassan
Walid El-Hallak
Contents

1 Introduction to Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Introduction to Machine Learning . . . . . . . . . . . . . . . . . . . . . 1
1.2 Origin of Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Growth of Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 How Machine Learning Works . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 Machine Learning Building Blocks . . . . . . . . . . . . . . . . . . . . 4
1.5.1 Data Management and Exploration . . . . . . . . . . . . . 4
1.5.2 The Analytics Landscape . . . . . . . . . . . . . . . . . . . . 13
1.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.7 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.8 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.9 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.10 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.10.1 Introduction to R . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.10.2 Introduction to RStudio . . . . . . . . . . . . . . . . . . . . . . 27
1.10.3 Introduction to Python and Jupyter Notebook IDE . . . 33
1.10.4 Do It Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2 Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.1 Overview of the Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.2 Definition of General Terms . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.3 Types of Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.3.1 Measures of Central Tendency . . . . . . . . . . . . . . . . . 46
2.4 Inferential Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.4.1 Data Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.4.2 Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.4.3 Type I and II Errors . . . . . . . . . . . . . . . . . . . . . . . . 48
2.4.4 Steps for Performing Hypothesis Testing . . . . . . . . . 48
2.4.5 Test Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

ix
x Contents

2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.6 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.7 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.8 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.9 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.9.1 Working Example in R . . . . . . . . . . . . . . . . . . . . . . 55
2.9.2 Working Example in Python . . . . . . . . . . . . . . . . . . 57
2.9.3 Do It Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.9.4 Do More Yourself (Links to Available Datasets for
Use) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3 Overview of Machine Learning Algorithms . . . . . . . . . . . . . . . . . . . 61
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.2 Data Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.3 Analytics and Machine Learning . . . . . . . . . . . . . . . . . . . . . . 63
3.3.1 Terminology Used in Machine Learning . . . . . . . . . 64
3.3.2 Machine Learning Algorithms: A Classification . . . . 65
3.4 Supervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.4.1 Multivariate Regression . . . . . . . . . . . . . . . . . . . . . 68
3.4.2 Decision Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.4.3 Artificial Neural Networks . . . . . . . . . . . . . . . . . . . 72
3.4.4 Naïve Bayes Classifier . . . . . . . . . . . . . . . . . . . . . . 75
3.4.5 Random Forest . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.4.6 Support Vector Machines (SVM) . . . . . . . . . . . . . . . 78
3.5 Unsupervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.5.1 K-Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3.5.2 K-Nearest Neighbors (KNN) . . . . . . . . . . . . . . . . . . 82
3.5.3 AdaBoost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
3.6 Applications of Machine Learning . . . . . . . . . . . . . . . . . . . . . 85
3.6.1 Machine Learning Demand Forecasting and Supply
Chain Performance . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.6.2 A Case Study on Cervical Pain Assessment with
Motion Capture . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.6.3 Predicting Bank Insolvencies Using Machine
Learning Techniques . . . . . . . . . . . . . . . . . . . . . . . . 89
3.6.4 Deep Learning with Convolutional Neural Network
for Objective Skill Evaluation in Robot-Assisted
Surgery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.8 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.9 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.10 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.11 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.11.1 Machine Learning Overview in R . . . . . . . . . . . . . . 94
Contents xi

3.11.2 Supervised Learning Overview . . . . . . . . . . . . . . . . 98

3.11.3 Unsupervised Learning Overview . . . . . . . . . . . . . . 104
3.11.4 Python Scikit-Learn Package Overview . . . . . . . . . . 106
3.11.5 Python Supervised Learning Machine (SML) . . . . . . 107
3.11.6 Unsupervised Machine Learning (UML) . . . . . . . . . 111
3.11.7 Do It Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
3.11.8 Do More Yourself . . . . . . . . . . . . . . . . . . . . . . . . . 113
References . . . . ....................................... . 113
4 Data Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.2 Data Preprocessing Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.2.1 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.2.2 Data Profiling, Discovery, and Access . . . . . . . . . . . 119
4.2.3 Data Cleansing and Validation . . . . . . . . . . . . . . . . 119
4.2.4 Data Structuring . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.2.5 Feature Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.2.6 Data Transformation and Enrichment . . . . . . . . . . . . 120
4.2.7 Data Validation, Storage, and Publishing . . . . . . . . . 121
4.3 Feature Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
4.3.1 Feature Creation . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
4.3.2 Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
4.3.3 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . 123
4.4 Feature Engineering Techniques . . . . . . . . . . . . . . . . . . . . . . . 123
4.4.1 Imputation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
4.4.2 Discretizing Numerical Features . . . . . . . . . . . . . . . 124
4.4.3 Converting Categorical Discrete Features to Numeric
(Binarization) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
4.4.4 Log Transformation . . . . . . . . . . . . . . . . . . . . . . . . 125
4.4.5 One-Hot Encoding . . . . . . . . . . . . . . . . . . . . . . . . . 125
4.4.6 Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
4.4.7 Reduce the Features Dimensionality . . . . . . . . . . . . . 128
4.5 Overfitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
4.6 Underfitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
4.7 Model Selection: Selecting the Best Performing Model of an
Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
4.7.1 Model Selection Using the Holdout Method . . . . . . . 133
4.7.2 Model Selection Using Cross-Validation . . . . . . . . . 134
4.7.3 Evaluating Model Performance in Python . . . . . . . . . 135
4.8 Data Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.9 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.10 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.11 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.12 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.12.1 Working Example in Python . . . . . . . . . . . . . . . . . . 138
xii Contents

4.12.2 Working Example in Weka . . . . . . . . . . . . . . . . . . . 150

4.12.3 Do It Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
4.12.4 Do More Yourself . . . . . . . . . . . . . . . . . . . . . . . . . 160
References . . . . ....................................... . 161
5 Data Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
5.2 Presentation and Visualization of Information . . . . . . . . . . . . . 168
5.2.1 A Taxonomy of Graphs . . . . . . . . . . . . . . . . . . . . . 168
5.2.2 Relationships and Graphs . . . . . . . . . . . . . . . . . . . . 169
5.2.3 Dashboards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
5.2.4 Infographics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
5.3 Building Effective Visualizations . . . . . . . . . . . . . . . . . . . . . . 184
5.4 Data Visualization Software . . . . . . . . . . . . . . . . . . . . . . . . . . 185
5.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
5.6 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
5.7 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
5.8 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
5.9 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
5.9.1 Working Example in Tableau . . . . . . . . . . . . . . . . . 189
5.9.2 Do It Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
5.9.3 Do More Yourself . . . . . . . . . . . . . . . . . . . . . . . . . 191
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
6 Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
6.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
6.2 A Practical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
6.3 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
6.3.1 Modeling the Linear Regression . . . . . . . . . . . . . . . 198
6.3.2 Gradient Descent . . . . . . . . . . . . . . . . . . . . . . . . . . 199
6.3.3 Gradient Descent Example . . . . . . . . . . . . . . . . . . . 202
6.3.4 Batch Versus Stochastic Gradient Descent . . . . . . . . 203
6.3.5 Examples of Error Functions . . . . . . . . . . . . . . . . . . 204
6.3.6 Gradient Descent Types . . . . . . . . . . . . . . . . . . . . . 205
6.4 Final Notes: Advantages, Disadvantages, and Best Practices . . 207
6.5 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
6.6 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
6.7 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
6.8 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
6.8.1 Working Example in R . . . . . . . . . . . . . . . . . . . . . . 211
6.8.2 Working Example in Python . . . . . . . . . . . . . . . . . . 215
6.8.3 Working Example in Weka . . . . . . . . . . . . . . . . . . . 222
6.8.4 Do It Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
6.8.5 Do More Yourself . . . . . . . . . . . . . . . . . . . . . . . . . 229
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
Contents xiii

7 Logistic Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

7.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
7.2 A Practical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
7.3 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
7.4 Final Notes: Advantages, Disadvantages, and Best Practices . . 234
7.5 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
7.6 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
7.7 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
7.8 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
7.8.1 Working Example in Python . . . . . . . . . . . . . . . . . . 237
7.8.2 Working Example in Weka . . . . . . . . . . . . . . . . . . . 242
7.8.3 Do It Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
7.8.4 Do More Yourself . . . . . . . . . . . . . . . . . . . . . . . . . 247
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
8 Decision Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
8.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
8.2 A Practical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
8.3 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
8.3.1 Tree Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
8.3.2 Training Decision Trees . . . . . . . . . . . . . . . . . . . . . 259
8.3.3 A Generic Algorithm . . . . . . . . . . . . . . . . . . . . . . . 261
8.3.4 Tree Pruning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
8.4 Final Notes: Advantages, Disadvantages, and Best Practices . . 267
8.5 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
8.6 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
8.7 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
8.8 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
8.8.1 Working Example in Python . . . . . . . . . . . . . . . . . . 270
8.8.2 Working Example in Weka . . . . . . . . . . . . . . . . . . . 273
8.8.3 Do It Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
8.8.4 Do More Yourself . . . . . . . . . . . . . . . . . . . . . . . . . 278
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
9 Naïve Bayes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
9.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
9.2 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
9.2.1 Bayes Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
9.2.2 The Naïve Bayes Classifier (NBC): Dealing with
Categorical Variables . . . . . . . . . . . . . . . . . . . . . . . 281
9.2.3 Gaussian Naïve Bayes (GNB): Dealing with
Continuous Variables . . . . . . . . . . . . . . . . . . . . . . . 283
9.3 A Practical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
9.3.1 Naïve Bayes Classifier with Categorical Variables
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
xiv Contents

9.3.2 Gaussian Naïve Bayes Example . . . . . . . . . . . . . . . . 285

9.4 Final Notes: Advantages, Disadvantages, and Best Practices . . 287
9.5 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
9.6 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
9.7 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
9.8 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
9.8.1 Working Example in Python . . . . . . . . . . . . . . . . . . 291
9.8.2 Working Example in Weka . . . . . . . . . . . . . . . . . . . 293
9.8.3 Do It Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
9.8.4 Do More Yourself . . . . . . . . . . . . . . . . . . . . . . . . . 298
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
10 K-Nearest Neighbors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
10.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
10.2 A Practical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
10.2.1 A Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
10.2.2 Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303
10.3 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
10.3.1 Distance Function . . . . . . . . . . . . . . . . . . . . . . . . . . 305
10.3.2 KNN for Classification . . . . . . . . . . . . . . . . . . . . . . 307
10.3.3 KNN for Regression . . . . . . . . . . . . . . . . . . . . . . . . 308
10.4 Final Notes: Advantages, Disadvantages, and Best Practices . . 308
10.5 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
10.6 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
10.7 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
10.8 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
10.8.1 Working Example in Python . . . . . . . . . . . . . . . . . . 311
10.8.2 Working Example in Weka . . . . . . . . . . . . . . . . . . . 314
10.8.3 Do It Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
10.8.4 Do More Yourself . . . . . . . . . . . . . . . . . . . . . . . . . 317
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
11 Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
11.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
11.2 A Practical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
11.2.1 Example 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
11.3 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
11.3.1 The McCulloch–Pitts Neuron . . . . . . . . . . . . . . . . . 322
11.3.2 The Perceptron . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
11.3.3 The Perceptron as a Linear Function . . . . . . . . . . . . 326
11.3.4 Activation Functions . . . . . . . . . . . . . . . . . . . . . . . . 327
11.3.5 Training the Perceptron . . . . . . . . . . . . . . . . . . . . . . 333
11.3.6 Perceptron Limitations: XOR Modeling . . . . . . . . . . 337
11.3.7 Multilayer Perceptron (MLP) . . . . . . . . . . . . . . . . . . 337
11.3.8 MLP Algorithm Overview . . . . . . . . . . . . . . . . . . . . 340
Contents xv

11.3.9 Backpropagation . . . . . . . . . . . . . . . . . . . . . . . . . . . 342

11.3.10 Backpropagation Algorithm . . . . . . . . . . . . . . . . . . 348
11.4 Final Notes: Advantages, Disadvantages, and Best Practices . . 349
11.5 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
11.6 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
11.7 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
11.8 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352
11.8.1 Working Example in Python . . . . . . . . . . . . . . . . . . 352
11.8.2 Working Example in Weka . . . . . . . . . . . . . . . . . . . 355
11.8.3 Do it Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
11.8.4 Do More Yourself . . . . . . . . . . . . . . . . . . . . . . . . . 360
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
12 K-Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
12.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
12.2 A Practical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
12.3 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
12.4 Inertia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
12.5 Minibatch K-Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374
12.6 Final Notes: Advantages, Disadvantages, and Best Practices . . 374
12.7 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374
12.8 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
12.9 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
12.10 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
12.10.1 Working Example in Python . . . . . . . . . . . . . . . . . . 376
12.10.2 Do It Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
12.10.3 Do More Yourself . . . . . . . . . . . . . . . . . . . . . . . . . 384
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
13 Support Vector Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
13.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
13.2 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
13.2.1 Important Concepts . . . . . . . . . . . . . . . . . . . . . . . . . 388
13.2.2 Margin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
13.2.3 Types of Support Vector Machines . . . . . . . . . . . . . 393
13.2.4 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
13.2.5 Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
13.2.6 Tuning Parameters . . . . . . . . . . . . . . . . . . . . . . . . . 397
13.2.7 Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
13.3 Advantages, Disadvantages, and Best Practices . . . . . . . . . . . . 399
13.4 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
13.5 Test Your Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
13.6 Read More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
13.7 Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402
13.7.1 Working Example in Python . . . . . . . . . . . . . . . . . . 402
Other documents randomly have
different content
 Ada did not move from her position on the bed, and as
the morning wore away, Nellie began to feel anxious about
her. She went in once and covered her over with a shawl,
but received no word from the poor heart-broken girl. She
asked their father what was to be done for her, and he said,
"Let her alone, dear."

So the long morning passed slowly away.

It was nearly dinner-time when Nellie once more re-

entered their room.

Ada did not seem to have moved, and Nellie busied

herself with her toilet, anxiously considering what she ought
to do. At last she went to her side, and said very low, "Dear
Ada, you will try to come down to dinner?"

Ada moved slightly and moaned.

"It is so sad for papa," resumed Nellie.

"I can't," answered Ada.

"We ought to try to do all we can for papa," persisted

Nellie.

"Oh, let me be, Nellie!" irritably exclaimed the miserable

girl. "I must be let alone. I cannot bear it."

"Very well, dearest; I will not trouble you any more."

She stooped and kissed the sad, hopeless face, and went
downstairs.

As she passed the nursery, Isabel peeped out. "Nellie,

can't you find us anything to do?" she said, whispering; "we
are so miserable, and nurse doesn't like us to play."
 Nellie entered, and found the nurse—whose eyes were
swollen with crying—holding the baby on her lap, and
rocking him backwards and forwards in the most forlorn
way.

"Poor dear children," she said, speaking in a hushed

voice; "we must find something to do. Supposing I bring
you some work that I have ready cut-out for the missionary
basket?"

Mary looked surprised. "Oh, Miss Nellie," she said, "how

can they bear to?"

"Mamma would have wished them to be employed,"

answered Nellie, gravely and firmly; "she would have been
so sorry for them, Mary."

Mary burst into tears, and hugged the baby, while Nellie
went for the work.

"You can think and talk of where dear mamma is,

darlings, just the same," she said when she came back.
"And all the better that you are carrying out one of her
wishes, to see the basket filled."

The little girls looked up comforted, and she continued,

"We cannot but be sad for many a long day, but we must all
try to be busy in doing what she would wish us; must we
not, darlings?"

"Can I have some work?" asked Dolly, "to help

mamma's basket?"

"Yes; there is some just right for you; see."

And so it was called from that time "Mamma's Basket."

 CHAPTER XVI.
"THINE EYES SHALL SEE THE KING."

THE afternoon of that long and sorrowful day wore away

slowly to Nellie.

She was sitting in the drawing room, Arthur and Tom

being in the nursery, with some writing paper before her,
but unable to begin the dreaded letter to her grandmamma,
when, after a light tap, the door opened, and she looked up
in the face of Christina.

Nellie threw herself into her arms and was strained in a

close embrace, and she found it impossible longer to keep
up the self-command which she had imposed upon herself
all day.
 Christina placed her on the sofa, and sat down by her
side, only whispering, "Poor Nellie, what you must have
gone through."

Nellie held her hand, and felt it a comfort to be able to

cry, which Christina understood; for she made no effort to
talk, but smoothed back her hair and stroked her hand in
silence.

So they remained for nearly an hour, till Nellie had wept

all her tears away. Then she sat up and looked round the
room with a shiver.

"Where is Ada?" asked Christina.

"On her bed; she has not moved or eaten to-day."

"Oh, Nellie!"

"It is so dreadful; but papa says 'let her alone.'"

"I will go to her. Do you think I could have a cup of tea

to take with me?"

"I will ring for it."

Christina turned to the somewhat disordered room, and

began putting the chairs in their places, and making it look
as usual. Nellie watched her in silence, feeling too miserable
to do anything else.

When Simmons came in answer to the bell, Christina

asked if she might have a cup of tea to take to Miss Ada.

"Oh yes, ma'am," said Simmons, "it is ready, for we

have just made our kitchen tea."
 She hastened away, and in a minute brought back three
cups of hot tea, and some bread and butter.

"You and Miss Nellie had better take some too; the
Doctor will not be in till nearly six, ma'am."

"Drink yours, Nellie," said Christina; "I shall take this to

Ada first. I can stay a day or two if you like."

"Like!" answered Nellie. "It would be the greatest

comfort I could have."

"Dear Nellie, I am so glad to hear you say so. I shall be

so glad to help you. I told my aunt not to expect me back
yet."

"How did you hear?"

"Your father telegraphed to me this morning."

She took the tea and went up into the darkened room,
passing on her way that other room, so still and quiet,
where she longed to enter, but must not yet.

Ada lay much in the same position as she had done in

the morning, and Christina closed the door and placed her
little tray on the table, in a firm audible way which
somewhat aroused her. She listened, without moving; and
Christina, when her preparations were completed, came to
her side, and said kindly, "Here is your tea, Ada."

Ada looked up surprised. "I thought it was Nellie," she

said without emotion; "but yet I knew it was not her step."

Christina began to raise her, and Ada so far helped

herself as to sit upright and draw the tea towards her.
 Christina busied herself in straightening this room as
she had done the other, and Ada drank the tea and ate
some bread and butter, watching Christina moving about as
if in a dream. When it was done, her misery came over her
again, and pushing away her plate almost pettishly, she
turned round and threw herself over her bed once more,
with a bitter cry.

Christina ceased to put the room tidy; kneeling down by

the bed, she threw her arm round Ada, and whispered
softly, "I know what it is to lose a mother."

"Oh, if she'd only wished me good-bye!" said Ada,

sobbing.

"Ah, dear, we always wish some things had been

different; but perhaps she could not."

"She did kiss me; but then she fell asleep. Oh,
Christina, Christina, she can never know how I loved her,
and all I meant to do to be a comfort to her!"

"She will know some day, dear child."

"I was often tiresome," said Ada, heart-brokenly, "often

grieved and worried her, and I can never, never show her
that I loved her all the same."

"She knew that, dear. I never heard her say one word
but of love to you."

"No; oh, no! But, Christina, it is too dreadful; I could

not believe it could be so."

Christina could only whisper that she had passed

through the same anguish herself, and knew how it felt;
and then she reminded her, too, of Him who sorrowed and
wept with bereaved ones, and of what a tender heart He
had.

"And think, dear Ada, of her joy now. Think of her

redeemed spirit among the multitude whom no man can
number. No more pain or anxiety or weariness, but with her
Lord, rejoicing in serving Him with perfect service for ever."

Ada listened at last, till the soft voice and the

comforting words soothed her, and ere long the eyes which
had been raised, trying to follow Christina's thoughts right
into heaven, gradually closed, and sleep ended to her that
mournful day which had found her motherless.

Then Christina drew some bed-clothes over her, and

putting her head in a more comfortable position left her,
glad that for a time she would have some relief from her
sorrow.

On the next landing she paused, and was just entering

the chamber of death when Nellie joined her.

They went in, and stood hand-in-hand by the bed.

Nellie uncovered the sweet, peaceful face.

"She looks just the same," said Christina, very low.

"Yes, only so still. She was always doing something for

others before," whispered Nellie in return.

"What a lovely smile she has. Nellie, I can't cry; I can

only feel just now that she has gone in to see the King."

"It brings heaven very near," said Nellie. "I have felt as
if I had gone almost to the gate."
 "And dear little Tom?" asked Christina softly.

"He seems half in heaven," she answered.

CHAPTER XVII.
IN THE NIGHT.

"COULD you not save yourself, dear papa?" asked

Nellie, looking up in Dr. Arundel's grave face as he prepared
to visit his patients the next morning.

"No, my child; it would be no 'save' to me to know they

were neglected or troubled. I would rather go and see
them."

Nellie still looked at him; but hardly liked to say more.

 "I shall do very well, dear," he added; "do not be
anxious about me."

On that second day Ada was ill. She begged to be

allowed to stay in bed, and her father had told her it would
be the best place for her.

She lay hour after hour in hopeless grief. She wept till
she seemed to have no more tears, and her aching,
throbbing head warned her that she could bear no more
thinking. And yet thought after thought came over her, and
again and again she wept, till her heart seemed broken.

Tom had asked to be in the drawing room all day, so as

to be near Nellie and Christina. He lay perfectly quiet, not
crying or making any complaint, his anxiety seeming to be
to comfort the others, and be as little trouble as possible.

Netta and Isabel brought their work downstairs, and sat

in a corner busying themselves over it, wonderfully
comforted to think they could be doing what would have
pleased their mamma.

At times they would be overpowered by fits of weeping;

but, as a rule, there was more a subdued sense of loss and
sorrow, than any outward show of it.

Arthur only broke down once, and that was when he

had first gone in to see his mother's face. After that he was
silent and thoughtful, and only desirous of helping Nellie.

How sad were the details of the mourning and the

funeral preparations. Nellie was thankful that Ada was out
of it all. She continued very unwell, and would accept no
one to wait on her but Christina, who went in and out with
the greatest kindness and consideration.
 One morning after she had put Ada comfortable, and
made all neat, she stood by the bedside looking earnestly at
her.

"What is it?" asked Ada, thinking she wished to say

something.

"Ada, dear, I do not like to keep you in ignorance; but

would you not like to see your dear mamma before—"

Ada started, and sat up in bed, raising her heavy eyes

in surprise.

"There is no hurry, dear," answered Christina tenderly;

"but I knew what I should have liked."

"Yes, yes; I have been so ill. I forgot she could not be

here always."

Christina gave her her clothes, and Ada dressed,

trembling in every limb.

"Can you, dear?" asked Christina, with tears in her

eyes.

"Oh, yes, please. I shall come back to bed after."

"Shall I come with you, or wait here?"

"If you would help me to the door? Oh, I'm so giddy."

Christina put her strong arm round her, and they went
slowly downstairs. When they came to the room, Ada kissed
her, and, steadying herself with a strong effort, entered and
shut the door.

Long Christina waited, but not a sound came from

within. At last with beating heart she ventured to go in.
 In the centre of the room was all that remained of the
one they so loved, and on the floor by it lay Ada, in a
death-like swoon.

Christina was glad to hear Dr. Arundel's step behind her,

and together they lifted the poor child back to her bed,
where she lay again hour after hour, till grief should have
time to spend its bitter force.

On the morning after the funeral, Dr. Arundel told them

it would be wise to take up their usual avocations.

"Let us remember your dear mother always, and speak

of her to each other whenever we like; but we will also do
as she would wish, and that will be to remember we have
One higher than even her to please; that we must go about
our Father's business."

He kissed them all gravely and lovingly, and then,

taking Nellie's hand, led her into his study.

"My dear child," he said, "you must thank all in the

house for their consideration and love to me this week; I
cannot. And to you, my dear, I must now look to be my
housekeeper, and comfort, and friend. You have always
been so, my child, next to dear mamma; and now I have
only you."

He was too overcome to say more; and perhaps that

day in which they turned over a new leaf was the most
hopelessly sad one they had passed.

In the evening, just as Nellie was coming down from

their mamma's duty of saying the little ones' prayers, she
heard some one being shown into the drawing room.
 She waited till the door was shut, and then descending,
met Simmons, who told her Mr. Elliot had called, and had
asked for her.

"Is papa in?" she said.

"He went out five minutes ago."

So Nellie went in, trying to be calm.

Wilmot Elliot came forward, and took her hand, and

looked in her face; and Nellie looked up in his in silence.

"I am so dreadfully sorry," he said, speaking in a

hushed and altered tone from any she had ever heard from
him.

She was going to answer, but her lips quivered, and she
hastily turned away, and sat down with her face averted.

"I longed to come, if I might be permitted to try to

comfort you or help you; but yet I feared to intrude on you,
and so I stayed away, Miss Arundel."

"Oh, no!" said Nellie.

"But you will believe I have not forgotten you?"

"I am sure of that, thank you," she answered low.

"Could you bear to tell me a little about it?"

"Would you care to hear?"

"Indeed I should; I have felt so sorrowful for you all this

time."
 Nellie glanced up gratefully, but found it very difficult to
open afresh the wounds which were so slightly healed. She
could not begin yet.

"I hear that Hope is getting better," she said.

"At last; she was very ill after you left."

"I feared she was going to be, and I was so sorry I

could not stay to nurse her."

"She was very sorry to lose you; but our grief at all you
came home to, seemed to put poor Hope's illness quite in
the background."

"What was the matter? I have never heard."

"It was almost rheumatic fever; but I am glad to say

she escaped it. But she is so happy, Miss Arundel."

"I am very glad to hear that," answered Nellie.

"And so am I. If for nothing else, I shall ever thank you

for that."

He sat on talking for about half-an-hour; and Nellie told

him the sad story, feeling comforted by his sympathy. The
time flew by quickly, and just as he was rising to go, Dr.
Arundel came in.

Wilmot did not, however, remain long, as he feared to

intrude on the bereaved family; and soon Nellie was free to
go upstairs to see how Ada was, and find Christina.

"Why did you not come down, dear?" she asked,

bending over her future sister, and giving her a kiss.
 "I did not know he was here till just now, and anyhow I
should not have come."

"Why not?"

"He would not know me, and I think you would get on
better alone this first visit."

"Yes, it was very sad; so different from the pleasant call

I thought he would make when he would come to see dear
mamma."

Nellie's eyes filled with tears at the thought, and

Christina stood by her with a hand on her shoulder, looking
out into the twilight of the square. That night, when Nellie
knew Christina and Ada were fast asleep, she tried to set
her thoughts in order. How was it that things looked less
unutterably sad than they had done two or three hours ago?
What difference was there?

A few kind words, a little sympathy, a short friendly call.

Nellie's head was buried closer into her pillow. "He was
so kind and gentle," she thought; "he seemed to
understand what I feel, and to sympathize so much. It did
me good, I suppose."

Then she began to review all the day, and her father's
words came back to her—

"Now I have only you."

"Only me," said Nellie to herself; "and I must be all he

wants, and stay with him always, and be his comfort. Dear,
dear papa."
 Long, long hours passed by before she slept. The
streets got quieter and quieter; only an occasional carriage
or cab broke the stillness, till the sound of the wheels died
away in the distance, and all was again silent.

Through her open window, she could hear Big Ben send
forth its thrilling sound as the hours went by, and when
each one struck, it came almost as a knell to her.

"I have left the best of life behind," she thought sadly;
and then once more she looked forward, not into the little
life which was close to her, but beyond that, above the
mists of the valley at her feet, up to the everlasting hills.

"In Thy presence is fulness of joy: at Thy right hand

there are pleasures for evermore."

CHAPTER XVIII.
 "THAT WHICH WAS LOST."

ADA gradually recovered; but still seemed not to wish to

get up.

Christina remained with them during this time, and they

hardly knew how much her presence lightened the gloom of
their fresh sorrow.

She took the little girls, Nellie, and often Arthur, for a
daily walk, and sometimes would persuade them into a
short omnibus ride, and a walk in the park with her. Though
they felt as if they could not bear to go, they yet came back
refreshed and strengthened; and Christina was well repaid
for her trouble, if trouble it could be called, when she saw
the little girls looking happy, and Nellie's pale face relapse
into a smile.

One day she told them she was going to make a call,
and asked them if they would wait for her an hour; and if
she did not come back to them in that time, would they
return home.

"It is some one I do not know, Nellie; but when I come

back, I will tell you about it; not just now. It is to the
mother of that little boy I have, little Charlie Wood."

"Are you going to see her?" asked Arthur.

"Yes; I cannot bear not to try."

So, when they arrived at Hyde Park, Christina left them

to enjoy themselves while she went to pay her visit in one
of the streets near.
 At the end of nearly an hour she came towards them,
and asking Arthur to call a cab, she put them in it,
promising to be home about six o'clock.

She returned at the appointed time, very tired; but

after tea as they all sat together, Dr. Arundel having gone to
his study, Christina said she would begin her story.

"I rang at the bell," she said, "and was asked up by an

untidy servant to a shabby drawing room. I should however
say that I knew she was in town on business, from her
husband, who writes to me occasionally.

"Here's a lady, 'm,' said the untidy servant, announcing

me without any warning.

"A person rose to greet me who had once been very

pretty, and bowing, asked if I wished to speak to her on
business.

"I said that I had come to see her, if she would allow
me.

"She looked extremely astonished; and when I sat down

by her and took her hand, and told her I knew she was in
trouble, she was rather angry.

"But I would not be put off; and I explained to her that

I knew of her sad history, and felt sure she desired to lead a
new and better life.

"After a while she broke down utterly, and confessed

that she was the most miserable woman living.

"'I did love my husband, and I do love my children; but

I can't do without what I take, and it's of no use. I have
tried, and tried, and tried; but it's of no use.'
 "Oh, how hopelessly she wept! And I wept with her.

"'Why do you come to me?' she asked at last, looking

suspiciously at me.

"'Because I grieve for you; because I have heard of

your little baby without a mother's love, and I want you to
begin a new life.'

"She looked at me wonderingly. 'Heard of my baby?

Where is he?'

"'Where you will never see him again if you do not give
up drink.'

"She bowed her head down on the table. 'I can't give it
up,' she breathed, despair written on her face.

"'Are you willing to give it up?'

"'I am to-day—now, while you are talking to me; but to-

morrow, or presently, when you are gone, the thirst will
come on, and I shall go to it again; yes, I know I shall.'

"'Have you ever heard of the Son of God?' I asked.

"'Of course I have.'

"'Do you suppose that the devil is stronger than the Son
of God?'

"'Sometimes I think he is.'

"'Then I am sure he is not. Christ is able to save to the

uttermost them that come unto God by Him. He can save
you even from desiring it again, if you ask Him.'
 "She looked at me earnestly, and faltered out, 'Say
those words again. Able—'

"'Able to save to the uttermost.'

"Then she sank on her knees by my side, and buried

her face in her hands, and sobbed out words of prayer and
entreaty.

"You may be sure I prayed too; and when she grew

silent, and lifted her eyes to mine, I raised her up to sit by
me once more.

"'Now,' I said, 'will you go for a visit to a nice house I

know of at the sea-side, where they will help you to keep
your resolve?'

"'But my business?'

"'You are not of much use in your business now, are

you?'

"'No,' she said humbly.

"'Then leave all that. Your husband will see to it. He will
only be too glad. I will come in an hour's time, and put you
into the train, and telegraph to them to meet you, and with
God's blessing, in three months' time you will be a different
woman.'

"'And my children?'

"'Ask God to bless them, and make you fit to come back
to them.'

"'I will,' she answered.

"Then I rose to go, promising to come back in an hour.

 "'May I kiss you?' she said, holding my hand and gazing
at me.

"So I stooped and kissed her.

"'Oh! I am not worthy,' she said, sinking beside me and

burying her head in my lap.

"And then I felt, dears, that I was not worthy.

"But as I bent over her bowed head, I remembered He

had come to seek and to save that which was lost—me, as
well as her; blessed Jesus."

CHAPTER XIX.
ADA'S STORY.

DR. ARUNDEL used to go in each morning and evening

to see Ada, but the visits were not very satisfactory to
either of them.

She was afraid of adding to his grief by showing her

own, and generally answered his questions as briefly as
possible, keeping a composed face, which was truly much
more painful to him than tears would have been.

On his part, he felt obliged to keep up the same self-

control. Such grief as his could only be borne by putting it
in the background, and living a life apart from it.

One evening, however, before Ada had yet ventured

from her room, her father tapped at the door and entered.

She was still in bed, but looking better. Just now her
eyes were red with weeping, and Dr. Arundel sat down by
her side, and took her hand in silence.

Ada tried to rouse herself; and quietly wiped away her

tears. Her father bent down and kissed her, saying kindly,
"Is anything special troubling you, my dear?"

"It is, papa, that she can never know—"

"I think she will, dear; and I have been thinking, too, of
the best way of being sure she will know."

"How?" asked Ada, checking her sobs.

"By helping to bring all whom she loved to be stars in

her Saviour's crown."

"Oh, papa, I feel as if I should never do anything again!

Life is worth nothing without her."

Dr. Arundel paused; he seemed unable to answer, and a

heavy, heavy sigh escaped him; but after a minute he said:

"Ada, I am afraid that is a very rebellious thought. Shall

we receive good at the hand of the Lord, and shall we not
receive evil?"
 "Oh, I can't, papa," said Ada bitterly.

"You cannot make yourself, my dear; but He can make

you. Ada, submit. Oh, my child, it must come sooner or
later! You are His, and He sends it in love. It is so, though
we cannot see it now; but I can, and do trust Him that it is
so."

Ada lay silently looking in the face which seemed almost

grey with sorrow. She pressed his hand earnestly, with
filling eyes.

"Think of it, my dear, and ask Him to enable you to do

His will, not only by submitting, but by acquiescing."

"Papa!"

"I mean it, dear. Being willing to carry out His perfect
will."

Ada felt, as he wished her good-night, that this would

never, never, be possible.

But before she fell asleep that night, she had taken the
first and the most difficult step in the ladder of self-
surrender.

She had prayed to be made willing.

The next morning, when Nellie came in to take away

her breakfast tray, she was already half dressed.

"Ada!" exclaimed her sister, astonished. "Can you, dear?

Are you well enough?"

"Yes, Nellie, I am getting on very well; you go about

your business. When I am dressed, I shall go into the
nursery. I'm afraid I've been very selfish."

"They will be so glad to have you," said Nellie, thankful

that her sister was so far recovering.

They were very glad to have her, though greatly

surprised to see her sitting there when they returned from
their early walk.

How tall and slim she looked in her new black dress.
Ada had felt at first as if she could not put it on; but when
she had taken down her usual frock, she put it hastily back
again, glad that Nellie's thoughtfulness had placed the other
where she could get it without asking.

The little ones gathered round her, and many were the
inquiries as to whether she was "better." They looked with
somewhat of awe in their faces; for Ada had not been able
to bear their presence in her room, and she felt quite a
stranger to them.

They showed her their work, and she was ready to

admire and praise, while Dolly's performance was duly
inspected.

No one touched on the tender ground of their sorrow.

The little girls felt instinctively that Ada would not be able to
bear it, and they therefore did their best to comfort her in
their own loving little way.

Ada shrank from going down to the empty rooms, and

told Mary she should have meals with them till she felt
stronger. Mary gladly consented, and was herself cheered
with the prospect of someone to talk to.

"Where is Tom?" asked Ada.

 "He is downstairs now," said Isabel, slightly grumbling;
"he is always there; we do miss him so."

"Yes," said Netta; "but Nellie said she thought he would

miss —;" she paused, and then remembering her father's
words, she went on gently, "miss dear mamma less if he
were downstairs with her, and Christina, and Arthur."

"It is quite right," said Ada; "dear little Tom."

"We don't have any school," Isabel observed rather

dolefully. "Nellie has been so busy, and all; but the days do
seem so long."

Ada looked up. "Perhaps to-morrow I might give you

lessons, till I go back to school myself."

"We've nothing to do to-day," said Netta.

"I suppose you're not well enough to tell us a tiny story,

are you, Ada?" asked Dolly, looking coaxingly up in her face.

"I will try," answered Ada, "because I've been thinking

of one, and perhaps it will do you good, like it did me. Get
your work, children."

Mary took out her basket, and sat down to listen; the
little girls ran to the cupboard to fetch theirs, and soon they
settled down to quietness.

"Come, Cecil," said Ada to the baby, patting her knee

invitingly, "you must be my boy now."

She took him up on her lap, and made him lean against
her. Something in her eyes must have won him, for he did
not generally condescend to notice her; but to-day, whether
from a certain unexplainable void in his own little heart, or
because his sister looked so very lovingly at him, she could
not tell, but he nestled his little head against her so
confidingly, that Ada felt it very difficult to go on with her
story. At last she said, looking up at them:

"But you will think I am never going to begin.

"It is a story I once read. I do not know that I can tell it

you in the same words, nor do I know where I read it, but I
am sure you will like it," she said.

"There was once a large vessel. She was making a long

voyage, and there were a great many people on board.

"She had been driven out of her course by long stress of

bad weather, and, unable to reach the port where they
usually took in water, the people began to be very short of
it.

"At first they were reduced to a very small quantity, but

as the days passed on they had less and less, and at last
every drop of fresh water was gone.

"Oh, how desolate and sad were those poor forlorn

people! Oh, the terrible pangs which that thirst gave them!

"They stood on deck straining their eyes for the sight of

land, or for a friendly ship which might give them a little, till
they should reach the port.

"Worse and worse got that thirst, and as the sun's rays
poured down upon them, they would many of them have
exchanged their misery for death, if they could have
chosen.

"At last a sail came in sight. They made signals of

distress, and then waited in agony to see if they would be
attended to.

"Yes; the steamer altered her course and came slowly—

oh, so slowly it seemed!—towards them.

"The captain had signalled that they were short of

water; but as the steamer came nearer, what was the
dismay of the despairing crew to find that she was not
intending to stop, or put down a boat!

"The captain of the distressed ship took a trumpet and

shouted, 'Water! Water! We are dying. For mercy's sake
give us some water!'

"And the captain of the steamer answered back through

his trumpet the mocking reply, 'Dip down your buckets and
drink!'

"On sped the steamer on her way, with her flag gaily
flying, while the dying, thirsting people bemoaned
themselves in bitter wailing.

"Yet thoughts of the captain of the steamer having

taken the trouble to come miles out of his way to tell them
what he did, came over them. No one would have taken so
much trouble were he ever so cruel, they thought.

"'Let us try his advice,' said one.

"When they drew up the bucket of sparkling water, it

was found to be fresh, and clear, and life-giving!

"Then the captain knew that they were sailing calmly in

the mouth of the great river Amazon, and that while they
had been almost dying for want of fresh water, it had
surrounded them all the time!"
 * * * * * *

"And I have been thinking, children, that this is just like

Jesus.

"We, like those sailors, sorrowful, and helpless, and

thirsty, are longing to get to port, that we may be satisfied
and lose our misery; and there, all the while, is the life-
giving water close to us, only waiting for us to let down the
bucket of our faith, and drink, and find that He satisfies
every need."

CHAPTER XX.
CROSSES.
 AFTER this Christina went home, and Ada gradually
resumed her old ways, so far as they could be taken up
without the one who had been the moving spring of the
house. There were times when she felt the misery of being
motherless was almost more than she could bear; times
when everything went wrong; when the children were cross,
and there was no one to settle the quarrels; when Nellie
wished things done which she considered unreasonable;
when Arthur was wretched, and she could offer no comfort;
when Tom was suffering, and there was no one to appeal to
about him.

At these times Ada would chafe bitterly against the

cruel blow which had ruined her happiness, and she would
add to the general discomfort by going about with a cloud
on her face, and irritation in her whole manner.

Such days as these were hard to bear. Nellie at times

well-nigh fainted under their difficulty. But when things
seemed at the worst, words would come back to her, and
the blessed Spirit would remind her: "When my heart was
overwhelmed within me, then Thou knewest my path;" and
the thought that God knew, and was ready to help, was
sure consolation, and Nellie took courage again.

"There cannot be a need greater than He can supply,

dear," her father said to her one day, when he found her
mending a great heap of stockings, and looking lonely and
desolate.

She could only kiss his kind face, and go on with her
work, blinded with tears. Her dear, unselfish father!

But where Nellie could submit, and rest on the

assurance of everlasting love, Ada had to fight a hard
battle, inch by inch; and in the struggle, she grew older
fast, and felt as if the days were years.

About a fortnight after Wilmot Elliot's first call, he came

again.

This time it was in the evening, when they were all

sitting together in the drawing room. Arthur was reading
aloud, and Dr. Arundel rested in his armchair listening, or
perhaps not listening, with his eyes closed.

When Wilmot was asked in, Dr. Arundel roused himself,

and entered into a pleasant conversation with the young
fellow, and Nellie learnt more about his avocations and
doings in that hour's talk, than she had done all the while
she was at Shellford. More in that way, but not so much in
other ways, she thought.

Presently the conversation turned on the narrow escape

they had all had at the picnic, and Wilmot explained to
them many things which Nellie had never yet had the heart
to tell them. Arthur was delighted to hear it all, and the
circle gathered round their visitor, eagerly asking questions.

"Hope told me," said Wilmot, "that it would have been

but a sorry affair without Miss Arundel."

He looked across to where Nellie sat, so quiet and

gentle, stitching away at her work.

"That's always what our Nellie is!" exclaimed Arthur.

"The best little woman in the world."

"Hush, Arthur," said Nellie, looking pained; "you know

you always think too well of me."
 "Do I, though?" he answered. "What should we do
without you, I should like to know?"

"You have not need to think," said Nellie, "as here I

am."

Wilmot soon after this said he thought he had better go.

He was pressed to stay to supper, but replied that, if they
would allow him, he would come another day and do that.

"It is most delightful to be admitted into a family circle,"

he added, as he shook hands with Dr. Arundel; "I have so
few friends in London."

"We shall be pleased to see you whenever you can look

in," answered Dr. Arundel, feeling what a pleasant change it
had been from his sad thoughts, and thinking also what a
nice friend Wilmot would be for Arthur.

On the following Sunday, as Nellie, Arthur, and the little

girls returned from church in the evening, Wilmot joined
them, having been to their service.

"I wanted to see you for a few minutes," he said,

coming close to Nellie's side and speaking in a slightly
lowered tone.

"Did you?" asked Nellie.

"Yes; I was a little disappointed the other night when I

came. You said you would be glad to see me."

"I said—," answered Nellie, hesitating.

"I remember every word. But you implied you would be

glad to see me."