Download the Full Version of textbook for Fast Typing at textbookfull.

com

Deep Learning with JavaScript: Neural networks in

TensorFlow.js 1st Edition Shanqing Cai

https://textbookfull.com/product/deep-learning-with-
javascript-neural-networks-in-tensorflow-js-1st-edition-
shanqing-cai/

OR CLICK BUTTON

DOWNLOAD NOW

Download More textbook Instantly Today - Get Yours Now at textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Deep Learning with JavaScript Neural networks in

TensorFlow js 1st Edition Shanqing Cai Stanley Bileschi
Eric D Nielsen
https://textbookfull.com/product/deep-learning-with-javascript-neural-
networks-in-tensorflow-js-1st-edition-shanqing-cai-stanley-bileschi-
eric-d-nielsen/
textboxfull.com

MATLAB Deep Learning: With Machine Learning, Neural

Networks and Artificial Intelligence 1st Edition Phil Kim

https://textbookfull.com/product/matlab-deep-learning-with-machine-
learning-neural-networks-and-artificial-intelligence-1st-edition-phil-
kim/
textboxfull.com

Applied Neural Networks with TensorFlow 2: API Oriented

Deep Learning with Python Orhan Gazi Yalç■n

https://textbookfull.com/product/applied-neural-networks-with-
tensorflow-2-api-oriented-deep-learning-with-python-orhan-gazi-yalcin/

textboxfull.com

Neural Networks and Deep Learning A Textbook Charu C.

Aggarwal

https://textbookfull.com/product/neural-networks-and-deep-learning-a-
textbook-charu-c-aggarwal/

textboxfull.com
Applied Neural Networks with TensorFlow 2 API Oriented
Deep Learning with Python 1st Edition Orhan Gazi Yalc■n
Yalç■n Orhan
https://textbookfull.com/product/applied-neural-networks-with-
tensorflow-2-api-oriented-deep-learning-with-python-1st-edition-orhan-
gazi-yalcin-yalcin-orhan/
textboxfull.com

Learn Keras for Deep Neural Networks: A Fast-Track

Approach to Modern Deep Learning with Python 1st Edition
Jojo John Moolayil
https://textbookfull.com/product/learn-keras-for-deep-neural-networks-
a-fast-track-approach-to-modern-deep-learning-with-python-1st-edition-
jojo-john-moolayil/
textboxfull.com

Practical TensorFlow.js: Deep Learning in Web App

Development Juan De Dios Santos Rivera

https://textbookfull.com/product/practical-tensorflow-js-deep-
learning-in-web-app-development-juan-de-dios-santos-rivera/

textboxfull.com

Python Deep Learning: Understand how deep neural networks

work and apply them to real-world tasks 3rd Edition
Vasilev
https://textbookfull.com/product/python-deep-learning-understand-how-
deep-neural-networks-work-and-apply-them-to-real-world-tasks-3rd-
edition-vasilev/
textboxfull.com

Artificial Neural Networks and Machine Learning ICANN 2019

Deep Learning 28th International Conference on Artificial
Neural Networks Munich Germany September 17 19 2019
Proceedings Part II Igor V. Tetko
https://textbookfull.com/product/artificial-neural-networks-and-
machine-learning-icann-2019-deep-learning-28th-international-
conference-on-artificial-neural-networks-munich-germany-
september-17-19-2019-proceedings-part-ii-igor-v-tet/
textboxfull.com
Neural networks in TensorFlow.js

Shanqing Cai
Stanley Bileschi
Eric D. Nielsen
François Chollet
Foreword by
Nikhil Thorat and Daniel Smilkov

MANNING
 Working with data

Ingest data Clean data Augment data

Data Sect. 6.1, 6.2, 6.3 Sect. 6.4 Sect. 6.5

Visualize data
Sect. 7.1

Model building 1: Choosing key layer types based on your data

Input data type Recommended layer API Reference

Numerical data Dense Chapters 2 and 3

(without sequential order)

Images or data that can be represented 2D convolution and pooling Chapters 4 and 5
as images (e.g., audio, game board)

Sequential data, including text • RNN (LSTM, GRU) • Sect. 9.1.2

• Embedding • Sect. 9.2.3
• 1D convolution • Sect. 9.2.4
• Attentional • Sect. 9.3

Model building 2: Choosing last-layer activation, loss, and metric functions

Task type Last-layer

(What are you predicting?) activation Loss function Metric(s) Reference

Regression Linear meanSquaredError (Same as loss) Chapter 2

(predicting a real number) meanAbsoluteError Sect. 9.1

Binary classification Sigmoid binaryCrossentropy Accuracy, precision, Sect. 3.1, 3.2, 9.2
(making a binary decision) recall, sensitivity, TPR,
FPR, ROC, AUC
Multi-class classification Softmax categoricalCrossentropy Accuracy, confusion Sect. 3.3, 9.3
(deciding among multiple matrix
classes)
A mix of the above (for example, (Multiple) Custom loss function (multiple) Sect. 5.2
numbers plus classes)

Advanced and miscellaneous task types Reference

Transfer learning Chapter 5

(applying a pretrained model to new data)

Generative learning Chapter 10

(generating new examples based on training data)

Reinforcement learning Chapter 11

(training an agent to interact with the environment)

Continues inside back cover

Deep Learning with JavaScript
 Deep Learning
with JavaScript
NEURAL NETWORKS IN TENSORFLOW.JS

SHANQING CAI
STANLEY BILESCHI
ERIC D. NIELSEN
WITH FRANÇOIS CHOLLET

FOREWORD BY NIKHIL THORAT

DANIEL SMILKOV

MANNING
SHELTER ISLAND
For online information and ordering of this and other Manning books, please visit
www.manning.com. The publisher offers discounts on this book when ordered in quantity.
For more information, please contact
Special Sales Department
Manning Publications Co.
20 Baldwin Road
PO Box 761
Shelter Island, NY 11964
Email: [email protected]

© 2020 by Manning Publications Co. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in

any form or by means electronic, mechanical, photocopying, or otherwise, without prior written
permission of the publisher.

Many of the designations used by manufacturers and sellers to distinguish their products are
claimed as trademarks. Where those designations appear in the book, and Manning
Publications was aware of a trademark claim, the designations have been printed in initial caps
or all caps.

Recognizing the importance of preserving what has been written, it is Manning’s policy to have
the books we publish printed on acid-free paper, and we exert our best efforts to that end.
Recognizing also our responsibility to conserve the resources of our planet, Manning books
are printed on paper that is at least 15 percent recycled and processed without the use of
elemental chlorine.

Manning Publications Co. Development editor: Jenny Stout

20 Baldwin Road Technical development editor: Marc-Phillipe Huget
PO Box 761 Review editor: Ivan Martinovič
Shelter Island, NY 11964 Project editor: Lori Weidert
Copy editor: Rebecca Deuel-Gallegos
Proofreader: Jason Everett
Technical proofreader: Karsten Strøbæck
Typesetter: Dottie Marsico
Cover designer: Marija Tudor

ISBN 9781617296178
Printed in the United States of America
 brief contents
PART 1 MOTIVATION AND BASIC CONCEPTS . .................................1
1 ■ Deep learning and JavaScript 3

PART 2 A GENTLE INTRODUCTION TO TENSORFLOW.JS . ............. 35

2 ■ Getting started: Simple linear regression in TensorFlow.js 37
3 ■ Adding nonlinearity: Beyond weighted sums 79
4 ■ Recognizing images and sounds using convnets 117
5 ■ Transfer learning: Reusing pretrained neural networks 152

PART 3 ADVANCED DEEP LEARNING WITH TENSORFLOW.JS. ....... 199

6 ■ Working with data 201
7 ■ Visualizing data and models 246
8 ■ Underfitting, overfitting, and the universal workflow
of machine learning 273
9 ■ Deep learning for sequences and text 292
10 ■ Generative deep learning 334
11 ■ Basics of deep reinforcement learning 371

PART 4 SUMMARY AND CLOSING WORDS . .................................. 415

12 ■ Testing, optimizing, and deploying models 417
13 ■ Summary, conclusions, and beyond 453

v
 contents
foreword xiii
preface xv
acknowledgments xvii
about this book xix
about the authors xxii
about the cover illustration xxiii

PART 1 MOTIVATION AND BASIC CONCEPTS ....................1

1 Deep learning and JavaScript

1.1
3
Artificial intelligence, machine learning, neural networks,
and deep learning 6
Artificial intelligence 6 Machine learning: How it differs from
■

traditional programming 7 Neural networks and deep

■

learning 12 Why deep learning? Why now? 16

■

1.2 Why combine JavaScript and machine learning? 18

Deep learning with Node.js 24 ■
The JavaScript ecosystem 25
1.3 Why TensorFlow.js? 27
A brief history of TensorFlow, Keras, and TensorFlow.js 27 Why ■

TensorFlow.js: A brief comparison with similar libraries 31 How ■

is TensorFlow.js being used by the world? 31 What this book will

■

and will not teach you about TensorFlow.js 32

vii
viii CONTENTS

PART 2 A GENTLE INTRODUCTION TO

TENSORFLOW.JS ..............................................35

2 Getting started: Simple linear regression in TensorFlow.js

2.1 Example 1: Predicting the duration of a download using
37

TensorFlow.js 38
Project overview: Duration prediction 38 A note on code listings
■

and console interactions 39 Creating and formatting the

■

data 40 Defining a simple model 43 Fitting the model

■ ■

to the training data 46 Using our trained model to make

■

predictions 48 Summary of our first example 49

■

2.2 Inside Model.fit(): Dissecting gradient descent

from example 1 50
The intuitions behind gradient-descent optimization 50
Backpropagation: Inside gradient descent 56
2.3 Linear regression with multiple input features 59
The Boston Housing Prices dataset 60 Getting and running the
■

Boston-housing project from GitHub 61 Accessing the Boston-

■

housing data 63 Precisely defining the Boston-housing

■

problem 65 A slight diversion into data normalization 66

■

Linear regression on the Boston-housing data 70

2.4 How to interpret your model 74
Extracting meaning from learned weights 74 Extracting internal ■

weights from the model 75 Caveats on interpretability 77

■

3 Adding nonlinearity: Beyond weighted sums

3.1 Nonlinearity: What it is and what it is good for 80
79

Building the intuition for nonlinearity in neural networks 82

Hyperparameters and hyperparameter optimization 89
3.2 Nonlinearity at output: Models for classification 92
What is binary classification? 92 Measuring the quality of
■

binary classifiers: Precision, recall, accuracy, and ROC curves 96

The ROC curve: Showing trade-offs in binary classification 99
Binary cross entropy: The loss function for binary classification 103
3.3 Multiclass classification 106
One-hot encoding of categorical data 107 Softmax ■

activation 109 Categorical cross entropy: The loss function

■

for multiclass classification 111 Confusion matrix: Fine-grained

■

analysis of multiclass classification 113

 CONTENTS ix

4 Recognizing images and sounds using convnets 117

4.1 From vectors to tensors: Representing images 118
The MNIST dataset 119
4.2 Your first convnet 120
conv2d layer 122 maxPooling2d layer 126 Repeating
■ ■

motifs of convolution and pooling 127 Flatten and dense

■

layers 128 Training the convnet 130 Using a convnet to

■ ■

make predictions 134

4.3 Beyond browsers: Training models faster using
Node.js 137
Dependencies and imports for using tfjs-node 137 Saving the
■

model from Node.js and loading it in the browser 142

4.4 Spoken-word recognition: Applying convnets on
audio data 144
Spectrograms: Representing sounds as images 145

5 Transfer learning: Reusing pretrained neural networks 152

5.1 Introduction to transfer learning: Reusing pretrained
models 153
Transfer learning based on compatible output shapes: Freezing
layers 155 Transfer learning on incompatible output shapes:
■

Creating a new model using outputs from the base model 161
Getting the most out of transfer learning through fine-tuning: An
audio example 174
5.2 Object detection through transfer learning on a
convnet 185
A simple object-detection problem based on synthesized scenes 186
Deep dive into simple object detection 187

PART 3 ADVANCED DEEP LEARNING WITH

TENSORFLOW.JS . ...........................................199

6 Working with data 201

6.1 Using tf.data to manage data 202
The tf.data.Dataset object 203 Creating a tf.data.Dataset 203
■

Accessing the data in your dataset 209 Manipulating tfjs-data

■

datasets 210
6.2 Training models with model.fitDataset 214
x CONTENTS

6.3 Common patterns for accessing data 220

Working with CSV format data 220 Accessing video data using
■

tf.data.webcam() 225 Accessing audio data using

■

tf.data.microphone() 228
6.4 Your data is likely flawed: Dealing with problems
in your data 230
Theory of data 231 ■
Detecting and cleaning problems with
data 235
6.5 Data augmentation 242

7 Visualizing data and models 246

7.1 Data visualization 247
Visualizing data using tfjs-vis 247 ■
An integrative case study:
Visualizing weather data with tfjs-vis 255
7.2 Visualizing models after training 260
Visualizing the internal activations of a convnet 262
Visualizing what convolutional layers are sensitive to: Maximally
activating images 265 Visual interpretation of a convnet’s
■

classification result 269

8 Underfitting, overfitting, and the universal workflow

of machine learning 273
8.1 Formulation of the temperature-prediction problem 274
8.2 Underfitting, overfitting, and countermeasures 278
Underfitting 278 Overfitting 280 Reducing overfitting
■ ■

with weight regularization and visualizing it working 282

8.3 The universal workflow of machine learning 287

9 Deep learning for sequences and text 292

9.1 Second attempt at weather prediction:
Introducing RNNs 294
Why dense layers fail to model sequential order 294 ■
How RNNs
model sequential order 296
9.2 Building deep-learning models for text 305
How text is represented in machine learning: One-hot and multi-hot
encoding 306 First attempt at the sentiment-analysis
■

problem 308 A more efficient representation of text: Word

■

embeddings 310 1D convnets 312

■
 CONTENTS xi

9.3 Sequence-to-sequence tasks with attention

mechanism 321
Formulation of the sequence-to-sequence task 321 The encoder-
■

decoder architecture and the attention mechanism 324 Deep dive ■

into the attention-based encoder-decoder model 327

10 Generative deep learning 334

10.1 Generating text with LSTM 335
Next-character predictor: A simple way to generate text 335
The LSTM-text-generation example 337 Temperature:■

Adjustable randomness in the generated text 342

10.2 Variational autoencoders: Finding an efficient and
structured vector representation of images 345
Classical autoencoder and VAE: Basic ideas 345 A detailed ■

example of VAE: The Fashion-MNIST example 349

10.3 Image generation with GANs 356
The basic idea behind GANs 357 The building blocks of
■

ACGAN 360 Diving deeper into the training of ACGAN 363

■

Seeing the MNIST ACGAN training and generation 366

11 Basics of deep reinforcement learning 371

11.1 The formulation of reinforcement-learning
problems 373
11.2 Policy networks and policy gradients: The cart-pole
example 376
Cart-pole as a reinforcement-learning problem 376 Policy ■

network 378 Training the policy network: The REINFORCE

■

algorithm 381
11.3 Value networks and Q-learning: The snake game
example 389
Snake as a reinforcement-learning problem 389 Markov decision
■

process and Q-values 392 Deep Q-network 396 Training

■ ■

the deep Q-network 399

PART 4 SUMMARY AND CLOSING WORDS .....................415

12 Testing, optimizing, and deploying models

12.1 Testing TensorFlow.js models 418
417

Traditional unit testing 419 Testing with golden values

■
422
Considerations around continuous training 424
xii CONTENTS

12.2 Model optimization 425

Model-size optimization through post-training weight
quantization 426 Inference-speed optimization using
■

GraphModel conversion 434

12.3 Deploying TensorFlow.js models on various platforms
and environments 439
Additional considerations when deploying to the web 439
Deployment to cloud serving 440 Deploying to a browser
■

extension, like Chrome Extension 441 Deploying TensorFlow.js

■

models in JavaScript-based mobile applications 443 Deploying

■

TensorFlow.js models in JavaScript-based cross-platform desktop

applications 445 Deploying TensorFlow.js models on WeChat
■

and other JavaScript-based mobile app plugin systems 447

Deploying TensorFlow.js models on single-board computers 448
Summary of deployments 450

13 Summary, conclusions, and beyond 453

13.1 Key concepts in review 454
Various approaches to AI 454 What makes deep learning stand
■

out among the subfields of machine learning 455 How to think

■

about deep learning at a high level 455 Key enabling

■

technologies of deep learning 456 Applications and

■

opportunities unlocked by deep learning in JavaScript 457

13.2 Quick overview of the deep-learning workflow
and algorithms in TensorFlow.js 458
The universal workflow of supervised deep learning 458
Reviewing model and layer types in TensorFlow.js: A quick
reference 460 Using pretrained models from TensorFlow.js
■
465
The space of possibilities 468 Limitations of deep learning
■
470
13.3 Trends in deep learning 473
13.4 Pointers for further exploration 474
Practice real-world machine-learning problems on Kaggle 474
Read about the latest developments on arXiv 475 Explore the
■

TensorFlow.js Ecosystem 475

appendix A Installing tfjs-node-gpu and its dependencies 477

appendix B A quick tutorial of tensors and operations in TensorFlow.js 482
glossary 507
index 519
 foreword
When we started TensorFlow.js (TF.js), formerly called deeplearn.js, machine learning
(ML) was done mostly in Python. As both JavaScript developers and ML practitioners
on the Google Brain team, we quickly realized that there was an opportunity to bridge
the two worlds. Today, TF.js has empowered a new set of developers from the extensive
JavaScript community to build and deploy ML models and enabled new classes of on-
device computation.
TF.js would not exist in its form today without Shanqing, Stan, and Eric. Their con-
tributions to TensorFlow Python, including the TensorFlow Debugger, eager execu-
tion, and build and test infrastructure, uniquely positioned them to tie the Python
and JavaScript worlds together. Early on in the development, their team realized the
need for a library on top of deeplearn.js that would provide high-level building blocks
to develop ML models. Shanqing, Stan, and Eric, among others, built TF.js Layers,
allowing conversion of Keras models to JavaScript, which dramatically increased the
wealth of available models in the TF.js ecosystem. When TF.js Layers was ready, we
released TF.js to the world.
To investigate the motivations, hurdles, and desires of software developers, Carrie
Cai and Philip Guo deployed a survey to the TF.js website. This book is in direct
response to the study’s summary: “Our analysis found that developers’ desires for ML
frameworks extended beyond simply wanting help with APIs: more fundamentally,
they desired guidance on understanding and applying the conceptual underpinnings
of ML itself.”1
Deep Learning with JavaScript contains a mix of deep learning theory as well as real-
world examples in JavaScript with TF.js. It is a great resource for JavaScript developers

1
C. Cai and P. Guo, (2019) “Software Developers Learning Machine Learning: Motivations, Hurdles, and
Desires,” IEEE Symposium on Visual Languages and Human-Centric Computing, 2019.

xiii
xiv FOREWORD

with no ML experience or formal math background, as well as ML practitioners who

would like to extend their work into the JavaScript ecosystem. This book follows the
template of Deep Learning with Python, one of the most popular applied-ML texts, writ-
ten by the Keras creator, François Chollet. Expanding on Chollet’s work, Deep Learning
with JavaScript does an amazing job building on the unique things that JavaScript has
to offer: interactivity, portability, and on-device computation. It covers core ML con-
cepts, but does not shy away from state-of-the-art ML topics, such as text translation,
generative models, and reinforcement learning. It even gives pragmatic advice on
deploying ML models into real-world applications written by practitioners who have
extensive experience deploying ML to the real world. The examples in this book are
backed by interactive demos that demonstrate the unique advantages of the JavaScript
ecosystem. All the code is open-sourced, so you can interact with it and fork it online.
This book should serve as the authoritative source for readers who want to learn
ML and use JavaScript as their main language. Sitting at the forefront of ML and
JavaScript, we hope you find the concepts in this book useful and the journey in Java-
Script ML a fruitful and exciting one.
—NIKHIL THORAT AND DANIEL SMILKOV,
inventors of deeplearn.js
and technical leads of TensorFlow.js
 preface
The most significant event in the recent history of technology is perhaps the explo-
sion in the power of neural networks since 2012. This was when the growth in labeled
datasets, increases in computation power, and innovations in algorithms came
together and reached a critical mass. Since then, deep neural networks have made
previously unachievable tasks achievable and boosted the accuracies in other tasks,
pushing them beyond academic research and into practical applications in domains
such as speech recognition, image labeling, generative models, and recommendation
systems, just to name a few.
It was against this backdrop that our team at Google Brain started developing
TensorFlow.js. When the project started, many regarded “deep learning in JavaScript”
as a novelty, perhaps a gimmick, fun for certain use cases, but not to be pursued with
seriousness. While Python already had several well-established and powerful frame-
works for deep learning, the JavaScript machine-learning landscape remained splin-
tered and incomplete. Of the handful of JavaScript libraries available back then, most
only supported deploying models pretrained in other languages (usually in Python).
For the few that supported building and training models from scratch, the scope of
supported model types was limited. Considering JavaScript’s popular status and its
ubiquity that straddles client and server sides, this was a strange situation.
TensorFlow.js is the first full-fledged industry-quality library for doing neural net-
works in JavaScript. The range of capabilities it provides spans multiple dimensions.
First, it supports a wide range of neural-networks layers, suitable for various data types
ranging from numeric to text, from audio to images. Second, it provides APIs for load-
ing pretrained models for inference, fine-tuning pretrained models, and building and
training models from scratch. Third, it provides both a high-level, Keras-like API for
practitioners who opt to use well-established layer types, and a low-level, TensorFlow-
like API for those who wish to implement more novel algorithms. Finally, it is designed

xv
xvi PREFACE

to be runnable in a wide selection of environments and hardware types, including the

web browser, server side (Node.js), mobile (e.g., React Native and WeChat), and desk-
top (electron). Adding to the multidimensional capability of TensorFlow.js is its status
as a first-class integrated part of the larger TensorFlow/Keras ecosystem, specifically its
API consistency and two-way model-format compatibility with the Python libraries.
The book you have in your hands will guide your grand tour through this multi-
dimensional space of capabilities. We’ve chosen a path that primarily cuts through the
first dimension (modeling tasks), enriched by excursions along the remaining dimen-
sions. We start from the relatively simpler task of predicting numbers from numbers
(regression) to the more complex ones such as predicting classes from images and
sequences, ending our trip on the fascinating topics of using neural networks to gen-
erate new images and training agents to make decisions (reinforcement learning).
We wrote the book not just as a recipe for how to write code in TensorFlow.js, but
as an introductory course in the foundations of machine learning in the native lan-
guage of JavaScript and web developers. The field of deep learning is a fast-evolving
one. It is our belief that a firm understanding of machine learning is possible without
formal mathematical treatment, and this understanding will enable you to keep your-
self up-to-date in future evolution of the techniques.
With this book you’ve made the first step in becoming a member of the growing
community of JavaScript machine-learning practitioners, who’ve already brought
about many impactful applications at the intersection between JavaScript and deep
learning. It is our sincere hope that this book will kindle your own creativity and inge-
nuity in this space.
SHANQING CAI, STAN BILESCHI, AND ERIC NIELSEN
September 2019
Cambridge, MA
 acknowledgments
This book owes Deep Learning with Python by François Chollet for its overall structure.
Despite the fact that the code was rewritten in a different language and much new
content was added for the JavaScript ecosystem and to reflect new developments in
the field, neither this book nor the entire high-level API of TensorFlow.js would have
been a reality without pioneer work on Keras led by François.
Our journey to the completion of this book and all the related code was made
pleasant and fulfilling thanks to the incredible support from our colleagues on Goo-
gle’s TensorFlow.js Team. The seminal and foundational work by Daniel Smilkov and
Nikhil Thorat on the low-level WebGL kernels and backpropagation forms a rock-
solid foundation for model building and training. The work by Nick Kreeger on the
Node.js binding to TensorFlow’s C library is the main reason why we can run neural
networks in the browser and Node.js with the same code. The TensorFlow.js data API
by David Soergel and Kangyi Zhang makes chapter 6 of the book possible, while chap-
ter 7 was enabled by the visualization work by Yannick Assogba. The performance
optimization techniques described in chapter 11 wouldn’t be possible without Ping
Yu’s work on op-level interface with TensorFlow. The speed of our examples wouldn’t
be nearly as fast as it is today without the focused performance optimization work by
Ann Yuan. The leadership of Sarah Sirajuddin, Sandeep Gupta, and Brijesh Krishnas-
wami is critical to the overall long-term success of the TensorFlow.js project.
We would have fallen off the track without the support and encouragement of
D. Sculley, who carefully reviewed all the chapters of the book. We’re also immensely
grateful for all the encouragement we received from Fernanda Viegas, Martin Watten-
berg, Hal Abelson, and many other colleagues of ours at Google. Our writing and
content were greatly improved as a result of the detailed review by François Chollet,

xvii
xviii ACKNOWLEDGMENTS

Nikhil Thorat, Daniel Smilkov, Jamie Smith, Brian K. Lee, and Augustus Odena, as
well as by in-depth discussion with Suharsh Sivakumar.
One of the unique pleasures of working on a project such as TensorFlow.js is the
opportunity to work alongside and interact with the worldwide open-source software
community. TensorFlow.js was fortunate to have a group of talented and driven con-
tributors including Manraj Singh, Kai Sasaki, Josh Gartman, Sasha Illarionov, David
Sanders, syt123450@, and many many others, whose tireless work on the library
expanded its capability and improved its quality. Manraj Singh also contributed the
phishing-detection example used in chapter 3 of the book.
We are grateful to our editorial team at Manning Publications. The dedicated and
tireless work by Brian Sawyer, Jennifer Stout, Rebecca Rinehart, and Mehmed Pasic,
and many others made it possible for we authors to focus on writing the content.
Marc-Philip Huget provided extensive and incisive technical review throughout the
development process. Special thanks go to our reviewers, Alain Lompo, Andreas Refs-
gaard, Buu Nguyen, David DiMaria, Edin Kapic, Edwin Kwok, Eoghan O’Donnell,
Evan Wallace, George thomas, Giuliano Bertoti, Jason Hales, Marcio Nicolau, Michael
Wall, Paulo Nuin, Pietro Maffi, Polina Keselman, Prabhuti Prakash, Ryan Burrows,
Satej Sahu, Suresh Rangarajulu, Ursin Stauss, and Vaijanath Rao, whose suggestions
helped make this a better book.
We thank our MEAP readers for catching and pointing out quite a few typographi-
cal and technical errors.
Finally, none of this would be possible without the tremendous understanding and
sacrifice on the part of our families. Shanqing Cai would like to express the deepest
gratitude to his wife, Wei, as well as his parents and parents-in-law for their help and
support during this book’s year-long writing process. Stan Bileschi would like to thank
his mother and father, as well as his step-mother and step-father, for providing a foun-
dation and direction to build a successful career in science and engineering. He
would also like to thank his wife, Constance, for her love and support. Eric Nielsen
would like to say to his friends and family, thank you.
 about this book
Who should read this book
This book is written for programmers who have a working knowledge of JavaScript,
from prior experience with either web frontend development or Node.js-based back-
end development, and wish to venture into the world of deep learning. It aims to sat-
isfy the learning needs of the following two subgroups of readers:
 JavaScript programmers who aspire to go from little-to-no experience with
machine learning or its mathematical background, to a decent knowledge of
how deep learning works and a practical understanding of the deep-learning
workflow that is sufficient for solving common data-science problems such as
classification and regression
 Web or Node.js developers who are tasked with deploying pre-trained models in
their web app or backend stack as new features
For the first group of readers, this book develops the basic concepts of machine learn-
ing and deep learning in a ground-up fashion, using JavaScript code examples that
are fun and ready for fiddling and hacking. We use diagrams, pseudo-code, and con-
crete examples in lieu of formal mathematics to help you form an intuitive, yet firm,
grasp of the foundations of how deep learning works.
For the second group of readers, we cover the key steps of converting existing
models (e.g., from Python training libraries) into a web- and/or Node-compatible for-
mat suitable for deployment in the frontend or the Node stack. We emphasize practi-
cal aspects such as optimizing model size and performance, as well as considerations
for various deployment environments ranging from a server to browser extensions
and mobile apps.

xix
xx ABOUT THIS BOOK

This book provides in-depth coverage of the TensorFlow.js API for ingesting and
formatting data, for building and loading models, and for running inference, evalua-
tion, and training for all readers.
Finally, technically minded people who don’t code regularly in JavaScript or any
other language will also find this book useful as an introductory text for both basic
and advanced neural networks.

How this book is organized: A roadmap

This book is organized into four parts. The first part, consisting of chapter 1 only,
introduces you to the landscape of artificial intelligence, machine learning, and deep
learning, and why it makes sense to practice deep learning in JavaScript.
The second part forms a gentle introduction to the most foundational and
frequently encountered concepts in deep learning. In particular:
 Chapters 2 and 3 are your gentle on-ramp to machine learning. Chapter 2
works through a simple problem of predicting a single number from another
number by fitting a straight line (linear regression) and uses it to illustrate how
backpropagation (the engine of deep learning) works. Chapter 3 builds on
chapter 2 by introducing nonlinearity, multi-layered networks, and classification
tasks. From this chapter you will gain an understanding of what nonlinearity is,
how it works, and why it gives deep neural networks their expressive power.
 Chapter 4 deals with image data and the neural-network architecture dedicated
to solving image-related machine-learning problems: convolutional networks
(convnets). We will also show you why convolution is a generic method that has
uses beyond images by using audio inputs as an example.
 Chapter 5 continues the focus on convnets and image-like inputs, but shifts into
the topic of transfer learning: how to train new models based on existing ones,
instead of starting from scratch.
Part 3 of the book systematically covers more advanced topics in deep learning for
users who wish to build an understanding of more cutting-edge techniques, with a
focus on specific challenging areas of ML systems, and the TensorFlow.js tools to work
with them:
 Chapter 6 discusses techniques for dealing with data in the context of deep
learning.
 Chapter 7 shows the techniques for visualizing data and the models that process
them, an important and indispensable step for any deep-learning workflow.
 Chapter 8 focuses on the important topics of underfitting and overfitting in
deep learning, and techniques for analyzing and mitigating them. Through this
discussion, we condense what we’ve learned in this book so far into a recipe
referred to as “the universal workflow of machine learning.” This chapter pre-
pares you for the advanced neural-network architectures and problems in chap-
ters 9–11.
 ABOUT THIS BOOK xxi

 Chapter 9 is dedicated to deep neural networks that process sequential data

and text inputs.
 Chapters 10 and 11 cover the advanced deep-learning areas of generative mod-
els (including generative adversarial networks) and reinforcement learning,
respectively.
In the fourth and final part of the book, we cover techniques for testing, optimizing
and deploying models trained or converted with TensorFlow.js (chapter 12) and wrap
up the whole book by recapitulating the most important concepts and workflows
(chapter 13).
Each chapter finishes with exercises to help you gauge your level of understanding
and hone your deep-learning skills in TensorFlow.js in a hands-on fashion.

About the code

This book contains many examples of source code both in numbered listings and in
line with normal text. In both cases, source code is formatted in a fixed-width font
like this to separate it from ordinary text. Sometimes code is also in bold to high-
light code that has changed from previous steps in the chapter, such as when a new
feature adds to an existing line of code.
In many cases, the original source code has been reformatted; we’ve added line
breaks and reworked indentation to accommodate the available page space in the
book. In rare cases, even this was not enough, and listings include line-continuation
markers (➥). Additionally, comments in the source code have often been removed
from the listings when the code is described in the text. Code annotations accom-
pany many of the listings, highlighting important concepts. The code for the exam-
ples in this book is available for download from GitHub at https:/ /github.com/
tensorflow/tfjs-examples.

liveBook discussion forum

Purchase of Deep Learning with JavaScript includes free access to a private web forum
run by Manning Publications where you can make comments about the book, ask
technical questions, and receive help from the author and from other users. To
access the forum, go to https://livebook.manning.com/#!/book/deep-learning-with-
javascript/discussion. You can also learn more about Manning’s forums and the rules
of conduct at https://livebook.manning.com/#!/discussion.
Manning’s commitment to our readers is to provide a venue where a meaningful
dialogue between individual readers and between readers and the author can take
place. It is not a commitment to any specific amount of participation on the part of
the author, whose contribution to the forum remains voluntary (and unpaid). We sug-
gest you try asking the authors some challenging questions lest their interest stray!
The forum and the archives of previous discussions will be accessible from the pub-
lisher’s website as long as the book is in print.
 about the authors
SHANQING CAI, STANLEY BILESCHI, AND ERIC NIELSEN are software engineers on the
Google Brain team. They were the primary developers of the high-level API of Tensor-
Flow.js, including the examples, the documentation, and the related tooling. They
have applied TensorFlow.js-based deep learning to real-world problems such as alter-
native communication for people with disabilities. They each have advanced degrees
from MIT.

xxii
 about the cover illustration
The figure on the cover of Deep Learning with JavaScript is captioned “Finne Katschin,”
or a girl from the Katschin tribe. The illustration is taken from a collection of dress
costumes from various countries by Jacques Grasset de Saint-Sauveur (1757-1810),
titled Costumes de Différents Pays, published in France in 1797. Each illustration is finely
drawn and colored by hand. The rich variety of Grasset de Saint-Sauveur’s collection
reminds us vividly of how culturally apart the world’s towns and regions were just 200
years ago. Isolated from each other, people spoke different dialects and languages. In
the streets or in the countryside, it was easy to identify where they lived and what their
trade or station in life was just by their dress.
The way we dress has changed since then and the diversity by region, so rich at the
time, has faded away. It is now hard to tell apart the inhabitants of different conti-
nents, let alone different towns, regions, or countries. Perhaps we have traded cultural
diversity for a more varied personal life—certainly for a more varied and fast-paced
technological life.
At a time when it is hard to tell one computer book from another, Manning cele-
brates the inventiveness and initiative of the computer business with book covers
based on the rich diversity of regional life of two centuries ago, brought back to life by
Grasset de Saint-Sauveur’s pictures.

xxiii
 Part 1
Motivation
and basic concepts

P art 1 consists of a single chapter that orients you to the basic concepts that
will form the backdrop for the rest of the book. These include artificial intelli-
gence, machine learning, and deep learning and the relations between them.
Chapter 1 also addresses the value and potential of practicing deep learning in
JavaScript.
 Deep learning
and JavaScript

This chapter covers

 What deep learning is and how it is related to artificial
intelligence (AI) and machine learning
 What makes deep learning stand out among various
machine-learning techniques, and the factors that led to the
current “deep-learning revolution”
 The reasons for doing deep learning in JavaScript using
TensorFlow.js
 The overall organization of this book

All the buzz around artificial intelligence (AI) is happening for a good reason: the
deep-learning revolution, as it is sometimes called, has indeed happened. Deep-
learning revolution refers to the rapid progress made in the speed and techniques of
deep neural networks that started around 2012 and is still ongoing. Since then,
deep neural networks have been applied to an increasingly wide range of prob-
lems, enabling machines to solve previously unsolvable problems in some cases and
dramatically improving solution accuracy in others (see table 1.1 for examples). To
experts in AI, many of these breakthroughs in neural networks were stunning.

3
Other documents randomly have
different content
we have bully good times of nights. You’re as good as a show—
better. Tune up your Julius Cæsar!”
“Your attitude—if you will overlook the involuntary rhyme,” said
George, “is one of base ingratitude. I endeavor to instruct and uplift
you. You might be absorbing sweetness and light at every pore,
acquiring a love for the true, the good and the beautiful—and you are
merely amused! It is disheartening. As for this golden volume, this
masterpiece of William Shakspere’s genius—‘which, pardon me, I do
not mean to read’——”
“Oh, go on! Of course you’re going to read it. We’ve got almost
through it. You left off just beyond ‘the-will-give-us-the-will, we-will-
have-the-will.’”
“Why, you lazy pup, why didn’t you read it yourself? You have
nothing to do. I have to work.”
“I did read it through to-day. And began at the first again. But,” said
Jeff admiringly, “I like to hear you read it. You have such a lovely
voice, Mr. Crow.”
Aughinbaugh bowed. “Thank you, Mr. Bransford, thank you! But I am
proof against even such subtle and insidious flattery as yours.
Hereafter, sir, I shall read no book through to you. I shall select
works suited to your parts and your station in life and read barely
enough to stimulate your sluggish mind. Then you can shell corn or
be buried alive. To-night, for instance, I shall read some salient
extracts from Carlyle’s ‘French Revolution.’ You will not in the least
understand it, but your interest and curiosity will be aroused. You will
then finish it, with such collateral reading as I shall direct.”
“Sure you got all those ‘shalls’ and ‘wills’ just right?” suggested Jeff.
“It’s mighty easy to get ’em tangled up.”
“That is the only proper way to study history,” George went on,
wisely ignoring the interruption. “Read history lightly, about some
period, then read the best works of poetry or fiction dealing with the
same events. Then come back to history again. The characters will
be real people to you and not mere names. You will eagerly extend
your researches to details about these familiar acquaintances and
friends, and learn particulars that you would else have shirked as
dull and laborious.” He took a book from the shelf. “I will now read to
you—after you replenish the fire—a few chapters here and there,
especially there, dealing with the taking of the Bastille.”
Without, a wild March wind shrilled and moaned at the trembling
casements; within, firelight’s cozy cheer, Aughinbaugh’s slim youth lit
by the glowing circle of the shaded lamp, the dusky corners beyond.
The flexible voice sank with pity or swelled with hot indignation. And
Bransford, as he listened to that stupendous, chaotic drama of
incoherent clangorous World Bedlam, saw, in the glowing coals,
tumultuous, dim-confused figures come and go, passionate, terrible
and grim; the young, the gay, the beautiful, the brave, the brave in
vain; fire-hearted, vehement, proud, swallowed up by delirium.
Newer shapes, wild, portentous, spluttering, flashing, whirling,
leaping in wild dervish dance. In the black shadows, in the eddying
thick smoke, lurked crowding shapes more terrible still, abominable,
malignant, demoniacal, imbecile—Proteus shapes that changed,
dwindled, leaped and roared to an indistinguishable sulphurous
whirlpool, sport of all the winds. Brief flashes of clearer light there
were, as the smoke billowed aside; faces gleamed a moment
distinct, resolute, indomitable, bright-sparkling; blazed high—and fell,
trampled down by fresh legion-changing apparitions. Sad visions,
some monstrous, some heroic, all pitiful; thronging innumerable,
consuming and consumed.

“Likewise ashlar stones of the Bastille continue thundering through

the dusk; its paper archives shall fly white. Old secrets come to view;
and long buried despair finds voice. Read this portion of an old letter:
‘If for my consolation monseigneur would grant me, for the sake of
God and the most blessed Trinity, that I should have news of my
dear wife; were it only her name on a card to show that she is alive!
It were the greatest consolation I could receive; and I should forever
bless the greatness of monseigneur.’ Poor prisoner, who signest
thyself Quéret-Démery, and hast no other history, she is dead, that
dear wife of thine, and thou art dead! ’Tis fifty years since thy
breaking heart put this question; to be heard now first, and long
heard, in the hearts of men.”

A long silence. The fire was low. One dim, blurred form was there—
an old man, writing, in a stone cell.
Aughinbaugh closed the book. His eyes were moist. “One of the
greatest novels ever written, ‘The Tale of Two Cities,’ is based
entirely upon and turns upon this last paragraph. Read that to-
morrow and then come back to the ‘French Revolution.’ You’ll be
around to-morrow night?”
Jeff rose, laughing. “You remind me of my roommate at school.”
“Your—what? Where?” said George in astonishment.
“Oh, yes, I’ve been to school, but not very long. When the boys used
to stay too late he’d yawn and say to me: ‘Jeff, perhaps we’d better
go to bed. These people may want to go home!’”
“Oh, well, it’s nearly twelve o’clock,” said George, unabashed. “And I
have to work if you don’t. Bless you, my children, bless you! Be
happy and you will be good! Buenas noches!”
“Buenas noches!”

A trolley car whirred by, with scintillation of blue-crackling sparks. Jeff

elected to walk, companied by his storied ghosts—their footsteps
sounded through the rustling leaves. The wind was dead; the night
was overcast, dark and chill. Aughinbaugh’s lodgings were in the
outskirts of the residence section; the streets at this hour were
deserted. Jeff had walked briskly for ten minutes when, as he neared
a corner in a quiet neighborhood, he saw a tall man in gray come
from the farther side of the intersecting street just ahead. The gray
man paused under the electric light to let a recklessly-driven cab
overtake and pass him, and then turned diagonally over toward Jeff,
whistling as he came. He was half-way across, and Jeff was within a
yard of the corner, when another man, short and squat, hurried from
the street to the left, brushing by so close that Jeff might have
touched him. So unexpected was his appearance—for his footsteps
had been drowned by the clattering cab—that Jeff was startled. He
paused, midstep, for the merest fraction of a second. The town clock
boomed midnight.
Thereafter, events moved with all the breathless unreality of dream.
The second man turned across to meet the first. A revolver leaped
up, shining in the light; he fired point-blank. The gray man staggered
back. Yet, taken all unaware, so deadly swift he was that both men
fired now together.
Nor was Jeff imprudently idle. He was in the line of fire, directly
behind the short man. To the left, across the sidewalk, the hole of a
tree was just visible beyond the house corner. Jeff leaped for this
friendly shelter—and butted headlong into human ribs.
A one-hundred-and-sixty-pound projectile deals no light blow, and
Jeff’s initial velocity was the highest he could command at such
slight notice. The owner of the ribs reeled out into the street, beyond
the shadows. A huge man, breathless, gasping, with a revolver
drawn; his thumb was on the hammer. So much Jeff knew and
closed on him, his left hand clutched the gun, the hammer was
through his finger. They wrenched and tore at the gun; and had the
bigger man grappled now he might have crushed Jeff at once,
broken him by main strength. But he was a man of one idea—and he
had a second gun. A violent jerk threw Jeff to his knee, but he kept
his desperate grip. The second gun flashed in the giant’s left hand,
rising and falling with the frontier firing motion; but Jeff’s own gun
was out, he struck up the falling death, the bullet sang above him.
He was on his feet, in trampling, unreal struggle; again he struck the
gun aside as it belched fire. Turning, whirling, straining, Jeff was
dizzily conscious that the men beneath the light were down, both still
shooting; the cab had stopped, men were running toward him
shouting. The giant’s dreadful strength was undirected, heaving and
thrusting purposeless; time for order and response would be time for
crashing death to find him; his one frantic thought was to shoot first,
to shoot fast. Shaken, tossed and thrown, Jeff kept his feet, kept his
head, kept close in; as the great man’s gun rose and fell he parried
with his own. Three shots, four—the others fired no longer; five—one
more—Six! It was warded, Jeff drew back, fired his first shot from his
hip; the giant dragged at him, heaved forward, and struck out
mightily, hammerwise. Jeff saw the blow gleaming down as he fired
again. Glint of myriad lights streamed sparkwise across an infinite
blackness; he knew no more.
The clock was still striking.
 Chapter III

“Please go ’way and let me sleep,

I would rather sleep than eat!”

—The Sluggard.

“HE’S coming round. That man’s suhtenly got a cast-iron skull. Such
a blow with a .45 would ’a’ killed most fellers. What you goin’ to do
with him, Judge?”
“I don’t know. It strikes me that he would be a valuable man for us.
That was the nerviest performance I ever saw. Had I been told that
any one could mix it that way with Oily Broderick and two guns, and
get off with it scot-free except for this little love tap, I should never
have believed it.” The voice was rich, clear, slow, well-modulated.
“Perhaps he may be induced to join us. If not——”
The words reached Jeff from immeasurable distances. He was
floating on a particularly soft and billowy cloud at the time: a cloud
with a buoyant and undulant motion, very soothing. Jeff noted it with
approval. Underneath and a little ahead, a high and exceedingly
steep mountain rose abruptly from the sea. It was built entirely of
piled, roundish boulders. The contour seemed familiar. Madagascar,
of course! How clever of him to remember! Jeff turned the cloud. It
sank in slow and graceful spirals to the peak. Doubtless the voices
came from there. The words seemed to have an unexplained
connection with some circumstance that he could not quite recall. He
felt the elusive memory slipping away. However, it made no
difference. He drifted into a delicious vagueness.
Something hard was forced between his teeth; a fiery liquid trickled
down his throat. He gasped and struggled; his eyes fluttered open.
To his intense disappointment the cloud was gone. An arm was
propping him up. Mysterious blankets appeared before him from
somewhere or other. On them lay an arm and a bandaged hand. The
hand was hurting some one very much. Jeff wondered whose it was.
He looked at the hand fixedly for a long time and, on further
examination, found it to be his own. Here was a pretty state of
affairs!
A pillow was thrust behind him and the supporting arm withdrawn. At
once he felt a throbbing pain in his head. He put his hand up and lo!
his head was also heavily bandaged! He regretted Madagascar more
than ever. He settled back for reflection. Looking up, after a little, he
saw a chair with the back turned toward him; astride the chair, a
middle-aged man, large, clean-shaven, rosy, well-dressed, and, as it
seemed to Jeff, unnecessarily cheerful. His eyes twinkled; his hands,
which were white and plump and well kept, played a little ditty on the
chair-back. There was a ruby on one finger. Beyond him sat a gross,
fat man with a stubbly beard, a coarse, flat nose and little, piggish,
red eyes. His legs were crossed and he smoked a villainous pipe.
There were other men behind these two. Jeff was just turning to look
at them when his attention was recalled by a voice from the man
astride the chair.
“And how are we now, my young friend? A trifle dazed, I fancy?
Something of a headache?” He showed his white teeth in a friendly
smile; his voice was soft and playful. “Are we well enough to eat
something? What with our recent disagreeable shock and our long
abstinence from food, we must find ourself rather feeble.”
Jeff stared at the man while he digested this communication. “A little
coffee,” he said at last. “I can’t eat anything now. I am dizzy and
most everlasting sick at my stomach. Put out that damned pipe!”
The soft-voiced man chuckled delightedly, as if he found this
peremptory command exquisitely humorous. “You hear, Borrowman?
Evidently Mr. Bransford is of those who want what they want when
they want it. Bring a little soup, too. He’ll feel better after he drinks
his coffee.”
The man addressed as Borrowman disappeared with a shuffling gait.
Jeff lay back and considered. His half-shut eyes wandered around.
Whitewashed stone walls, a heavily-ironed door, no window—that
was queer, too!—floor and ceiling of rough boards, a small fireplace,
two chairs, a pine table, a lighted lamp. That was all. His gaze came
back to the man in the chair, to find that gentleman’s large blue eyes
watching him with a quizzical and humorous look—a look highly
suggestive of a cat enjoying a little casual entertainment with a
mouse. In his weakened condition Jeff found this feline regard
disconcerting.
The coffee came, and the soup. After Jeff’s refreshment the man in
the chair rose. “We will leave you to the care of our good
Borrowman,” he said, baring his white, even teeth. “I will be back this
evening and, if you are stronger, we will then discuss some rather
momentous affairs. Go to sleep now.”
The caressing advice seemed good. Jeff was just dropping off when
a disturbing thought intruded itself.
This evening? Then it must be day now. Why did they burn a lamp in
daytime? The problem was too much for Jeff. Still pondering it, he
dozed off.
When he woke the lamp was yet burning; the objectionable fat man
sat by the fire. When he turned his head, presently, Jeff was startled
to observe that this man had got hold of an entirely new set of
features. Here was an extraordinary thing! Hard features, and
unprepossessing still, but clean at least. How very curious!
After a while a simple solution presented itself. It was not the same
man at all! Jeff wondered why he had not hit upon that at first. It
seemed that he had now become a body entirely surrounded by fat
men—no—that wasn’t right. “Let me—let me name the Supreme
Court of a nation and I care not who makes the laws.” No, that was
John Wesley Pringle’s gag. Good old Wes’! Wonder where he is? He
wasn’t fat. How did that go? Oh, yes! “Let me have men about me
that are fat!”—Something snapped—and Jeff remembered.
Not all at once. He lay silent, with closed eyes, and pieced together
scraps of recollection, here and there, bit by bit. It was like a picture
puzzle; so much so that Jeff quite identified each random memory
with some definite shape, eagerly fitting them together in a frame;
and, when he had adjusted them satisfactorily to a perfect square,
fell peacefully asleep.
 Chapter IV

“Good fellow, thy shooting is good,

An’ if thy heart be as good as thy hand,
Thou art better than Robin Hood.”

—Guy of Gisborne.

WHEN he woke the soft-voiced, white-handed man again sat beside

the bed, again in the same equestrian attitude, clasping the back of
the chair, beaming with good humor.
“And how is our young friend now? Much better, I trust. We have had
a long and refreshing sleep. Is our brain quite clear?”
Here the fat man—the less ill-favored one—rose silently from beside
the fire and left them.
“Our young friend is extremely hungry,” said Jeff. “Our young friend’s
brain is clear, but our young friend’s head is rather sore. Where am
I? In jail?” He sat up and pushed back the bandage for clearer vision.
The jovial gentleman laughed—a merry and mellow peal. “What a
spirited fellow you are! And what an extremely durable headpiece
you have! A jail? Well, not exactly, my dear fellow, not exactly. Let us
say, in a cache, in a retreat, sometimes used by gentlemen wishing
temporary retirement from society. You are also, though I grieve to
say it, in a jackpot—to use a phrase the precise meaning and origin
of which I do not comprehend, but which seems to be, in the
vernacular, a synonym for the more common word predicament.” He
shook his head sorrowfully. “A very sad predicament, indeed! Quite
unintentionally, and in obedience to a chivalrous impulse—which
does you great credit, I assure you—you have had the misfortune to
mar a very-well-laid plan of mine. Had I not been a quick thinker,
marvelously fertile in expedients, your officiousness would have
placed me in an awkward quandary. However, in the very brief time
at my disposal I was able to hit upon a device equally satisfactory—I
may say even more satisfactory than the original.”
“Hold on!” said Jeff. “I don’t quite keep up. You planned a midnight
assassination which did not go off smoothly. I’ve got that. You were
one of the men in the cab. There was a fight——”
“There was, indeed!” interrupted the genial gentleman. His eyes lit
up with enthusiasm; his shapely fingers tapped the chair-back. “Such
a fight! It was magnificent! Believe me, my dear Bransford, it inspired
me with an almost affectionate admiration for you! And your
opponent was a most redoubtable person, with a sensitive trigger
finger——”
“Excuse the interruption,” said Jeff. “But you seem to have the
advantage of me in the matter of names.”
“So I have, so I have! As you will infer, I looked through your
pockets. Thorpe is my name—S. S. Thorpe. Stay—here is my card.
You will see that I am entitled to the prefix ‘Hon.,’ having been
sometime State Senator. Call me Judge. I have never occupied that
exalted position, but all the boys call me Judge. To go back—we
were speaking of your opponent. Perhaps you knew him? No? Mr.
Broderick, Mr. Oily Broderick, once of San Antonio, a man of some
renown. We shall miss him, Mr. Bransford, we shall miss him! A very
useful fellow! But your eyes ask the question—Dead? Dear me, yes!
Dead and buried these many hours. He never knew what ailed him.
Both of your bullets found a vital spot. A sad loss! But I interrupt. I
am much interested to see how nearly accurate your analysis of the
situation will be.”
“The short man—was he killed, too?” asked Jeff.
“The worthy Krouse was killed as well,” said Judge Thorpe, sighing
with comfortable resignation. “But Krouse was a negligible quantity.
Amiable, but a bungler. Go on!”
“Your intended victim seems to have escaped——”
“Survived,” corrected Judge Thorpe gently, with complacent
inspection of his shapely hands. “Survived is the better word, believe
me. Captain Charles Tillotson, Captain of the Rangers. An estimable
gentleman, with whom, I grieve to say, I was not on the best of
terms. To our political enmity, of long standing—and you perhaps
know that Southwestern politics are extremely bitter—has been
added of late a certain social rivalry. But I digress. You were saying
——”
“But you are prompting me,” said Jeff testily. “It is hardly necessary.
Your enemy not being killed outright, you choose to assassinate his
good name, juggling appearances to make it seem that he was the
murderer—and to that end you have spirited me away.”
“Exactly! You are a man after my own heart—a man of acumen and
discernment,” said Judge Thorpe, beaming, “although I did, as you
suggest, prompt you at some points—knowing that you were not
familiar with all the premises. Really, Mr. Bransford, though I would
not unduly exalt myself, I cannot help but think my little device
showed more than mere talent. It was, considering the agitating
circumstances, considering that both conception and execution had
to be instantaneous, little less than Napoleonic! I feel sure that when
I tell you the details you will share my enthusiasm.”
Jeff was doing some quick thinking. He recalled what he had heard
of Thorpe. He was best known as a powerful and wealthy politician
of El Paso, who in his younger days had been a dangerous
gunfighter. Of late years, however, he had become respected and
reputable, his youthful foibles forgotten.
The appalling frankness of this avowal could bode no good to Jeff.
Evidently he was helplessly in this man’s power, and his life had
been spared for some sinister and shameful purpose.
“Before you favor me with any more details, Judge,” said Jeff, “can’t
you give me an old boot to chew on?”
“What wonderful spirits, what splendid nerves! I compliment you!”
said the Judge. “Our good Mac went, when you first awoke, to
prepare steak, eggs and coffee for you. You will pardon us if we do
not have your meals brought in from a restaurant. It would not do.
We are quiet here, we do not court observation. For the same
reason we have been forced to abstain from medical attendance for
you, otherwise so desirable. I, myself, have filled that office to the
best of my ability. Now as to the replenishing of the inner man. Mac
is an excellent cook.”
“Cleaner than Borrowman,” said Jeff.
“And is, as you observe, much cleaner than Borrowman. He will
prepare whatever the market affords. You have only to ask. And,
while we are waiting, I will return to my story.”
“I was, as you so readily surmised, in the cab, together with my good
friend, colleague and lieutenant, Mr. Sam Patterson. We had
telephoned ahead to Krouse and Broderick that Tillotson was on his
way. We were to be witnesses that Krouse acted purely in self-
defense, you know—as, indeed, were also the cab driver and
Broderick. Broderick was to hold himself in reserve and not to assist,
except in case of mishap. We supposed that Krouse would kill
Tillotson without difficulty. Krouse bungled. He inflicted three
wounds, painful but not dangerous; including one which creased the
scalp and produced unconsciousness.”
The man took such shameless delight in parading his wickedness
that Jeff began to wonder if, after all, it would not have saved himself
much difficulty if Broderick had killed him. But he set his mind like a
flint to thwart this smiling monster at any cost.
The Judge went on: “Such was the distressing situation when I came
up. Some men would have finished Tillotson on the spot. But I kept
my presence of mind; I exercised admirable self-restraint. It would be
but an instant before the aroused neighborhood would be on the
street. We bundled you and your gun into the cab and the driver
hurried you away to a certain rendezvous of ours. To have done with
the driver, I will say at this time that he came in and gave his
testimony the next day very effectively, fully confirming ours;
accounting for his conduct by the very natural excuse that he was
scared and so ran away lest he should be shot.
“The gun in Broderick’s right hand, you may remember, had not been
fired. His stiffening fingers still held it. I picked up his other gun,
unbuckled his belt, buckled it around Tillotson, and dropped
Broderick’s empty gun by him. No more was needed. The populace
found me caring for Captain Tillotson like a brother, pouring whisky
down him—and thereby heaping coals of fire on his head.
“Now, as to our evidence. As you may readily guess, we were driving
by when the trouble began. We saw Captain Tillotson when he fired
the first shot, killing Broderick with it. He continued to shoot after
Broderick dropped; Krouse, defending his friend, was killed also,
wounding Tillotson, who kept on shooting blindly after he fell. The
circumstantial evidence, too, was damning, and bore us out in every
respect. Broderick, a man of deadly quickness, had been killed
before he could shoot. Tillotson had emptied one gun and fired four
shots from the other; his carrying two guns pointed toward
deliberate, fore-planned murder. The marks on the houses, made by
a number of his wild bullets, were in a line directly beyond
Broderick’s body from where Tillotson lay. Broderick was between
you and the others, you know,” explained the Judge parenthetically.
“But as nothing is known of you, the marks of Broderick’s bullets are
supposed to be made by Tillotson’s—incontrovertible evidence that
he began the fighting.”
Nothing could have been more hateful, more revolting, than this
bland, smiling complacency: Jeff’s fingers itched to be at his throat. It
became clear to him that either this man would be his death, or,
which was highly improbable, the other way about. His resolution
hardened; he began to have visions of this smiling face above a
noose.
“When Tillotson regained consciousness he told a most amazing
story, obviously conflicting with the facts. He had carried but one
gun; Krouse had made a wanton attack upon him, without warning;
he had returned the fire. Simultaneously Broderick had been killed
by some fourth man, a stranger, whom Tillotson did not know, and
who had mysteriously disappeared when the people of the
neighborhood arrived. It looks very black for Captain Tillotson,”
purred the Judge, shaking his hands and head sorrowfully. “Even
those who uphold him do not credit this wildly-improbable tale. It is
universally thought that his wealth and position will not save him
from the noose. El Paso is reforming; El Paso is weary of two-gun
men.
“And now, my dear Bransford, comes the crucial point, a matter so
delicate that I hesitate to touch upon it. All of my ingenious little
impromptu was built and founded on the natural hypothesis of your
demise, which, in my haste, I did not stop to verify. It did not occur to
me as among the possibilities that any man—even myself—could
weather six shots, at hand-grips, from Oily Broderick. Imagine, then,
my surprise and chagrin when I learned that you were not even
seriously hurt! It was a shock, I assure you! But here comes Mac
with the tray. I will bathe your hands, Mr. Bransford. Then I beg that
you will fall to at once. We will discourse while you break your fast.”
“Oh, I can get up,” said Jeff. “I’m not hurt. Put it on the table.”
 Chapter V

“Quoth Robin, ‘I dwell by dale and down

By thee I set right naught.’”

—Guy of Gisborne.

“I PERCEIVE,” said the Judge, surveying the tempting viands, “that

Mac has thoughtfully cut your meat for you. You are provided with
many spoons, but neither knife nor fork. A wise and wholesome
precaution, I may remark. After your recent exploit we stand quite in
awe of you. Pray be seated. I will take a cup of coffee with you—if
you will allow me?
“It will not have escaped a man of your penetration that an obvious
course was open to me. But your gallantry had quite won my heart,
and I refrained from that obvious course, though strongly urged to it.
Mac, tell Mr. Bransford what your advice was.”
“I said: ‘Dead men tell no tales!’” replied Mac sturdily. “And I say it
again. Yon is a fearsome man.”
“You are a dangerous man yourself, Mac. Yet I trust you. And why?
Because,” said the Judge cooingly, “I am more dangerous still—
leader by right of the strongest. I admire you, Mr. Bransford; I
needed such a man as you seem to be. Moreover, singular as it may
seem, I boggled at cutting you off in cold blood. I have as good a
heart as can be made out of brains. You had not intentionally
harmed me; I bore you no grudge; it seemed a pity. I decided to give
you a chance. I refused this advice. If you but knew it, Mr. Bransford,
you owe me a heavy debt of gratitude. So we brought you across
quite unostentatiously. That brings us up to date.
“You see the logic of the situation, my dear fellow? Your silence must
be insured. Either you must throw in your lot with us, commit yourself
entirely and irrevocably to us, or suffer the consequences of—shall
we say, your indiscretion?”
The Judge sipped his coffee daintily. “It is distressing even to
mention the alternative; it is needless to lay undue emphasis upon it;
circumstances have already done that. You see for yourself that it
must be thus, and not otherwise.”
Jeff took a toothpick, pushed his chair back and crossed his legs
comfortably. “I must have time to consider the matter and look at it
from all sides,” he said meditatively. “But I can tell you now how it
strikes me at first blush. Do you believe in presentiments, Judge?”
The Judge shook his head. “I am singularly free from all
superstition.”
“Now, I do,” said Jeff steadily, his face wearing as engaging an
expression as its damaged condition would permit. “And I have a
very strong presentiment that I shall see you hung, or perhaps I
should say, hanged.”
The Judge went off in another peal of laughter. Even the saturnine
Mac relaxed to a grim smile. The Judge pounded on the table. “But
what a droll dog it is!” he cried. “Positively, I like you better every
moment. Such high spirits! Such hardihood! Really, we need you, we
must have you. I cannot imagine any one better fitted to fill the place
of the departed brother whom you—as the instrument of an
inscrutable and all-wise Providence—have removed from our midst.”
At this disloyalty to the dead, Jeff’s gorge rose at the man;
treacherous, heartless, revolting. But he kept a tranquil, untroubled
face. The Judge went on: “Your resolution may change. You will
suffer from ennui. I may mention that, should you join us, the
pecuniary reward will be great. I am wealthy and powerful, and our
little organization—informal, but very select—shares my fortunes.
They push me up from below and I pull them up from above. I will
add that we seldom find it necessary to resort to such extreme
measures as we did in the Tillotson case. He was a very
troublesome man; he has been a thorn in my side for years.
“On the contrary, we conduct many open and perfectly-legitimate
enterprises, political, legal, financial. We are interested in mining
propositions; we have cattle ranches in Texas and Old Mexico; we
handle real estate. As side lines, we do a miscellaneous business—
smuggle a vast amount of opium and a few Chinamen, keep
sanctuary for unhappy fugitives, jump good mines and sell poor
ones, furnish or remove witnesses—Oh, many things! But, perhaps,
our greatest activity is simply to exert moral pressure in aid of our
strictly-legitimate enterprises.
“Tut, tut! I have been so charmed that I have overstayed my time.
Think this matter over carefully, my dear fellow. There is much to
gain or to lose. You shall have ample time for consideration. Mac
and Borrowman will get you anything you want, within the bounds of
reason—clothes, books, tobacco, such knickknacks. And, by the
way, here are yesterday’s papers. You may care to read the Tillotson
case. The editorials, both those that condemn him and those that
defend, are particularly amusing.”
“Mac and Borrowman are to be my jailers?” said Jeff.
The Judge raised his hands in expostulation. “Jailers?” he repeated.
“What a harsh term! Let us say, companions. You might break out of
jail,” said the Judge, tapping Jeff’s breast with his strong fingers, “but
you will not get away from me. They will tell you their instructions. I
will attend to your hurts, now, and then I must go.”
“I would like clean clothes,” said Jeff, while the Judge dressed his
wounds skilfully. “A safety razor—they can keep it when I’m not
using it—the daily papers, cigars, tobacco—let me see, what else?
Oh, yes—I was trying to learn the typewriter. I’d like to try it again
when my finger gets better. For books, send in Shakspere’s works
and Carlyle’s ‘French Revolution,’ for the present.”
“You’re quite sure that’s all?” said the Judge, entertained and
delighted. “You must intend to take your time about making up your
mind.”
“My mind is entirely made up now. I would insure you against a
watery death,” said Jeff with utmost calmness, “for a dime!”
“We shall see, we shall see!” said the Judge skeptically. “Time works
many wonders. You will be ennuied! I prophesy it. Besides, I count
upon your gratitude. Good-night!”
“Good-night!”

So you “brought me unostentatiously across,” did you? You made a

slip that time. You talk well, Judge, but you talk too much. Across?
Across the Rio Grande. I am in Juarez. I had already guessed it, for I
hear the sounds of many whistling engines from far off, and but few
from near at hand. My prison is underground, since those whistles
are the only sounds that reach me, and they muffled and indistinct;
coming by the fireplace. That chimney goes through a house above,
since they keep up a fire. What to do?
Through the long hours he lay on his bed, sleepless. When he
opened his eyes, at intervals, it was always to find the guard’s face
toward him, watching him intently. They were taking no chances.
His vigorous brain was busy with the possibilities; contriving,
hopeless as the situation might seem, more than one scheme,
feasible only to desperation, and with terrible odds against success.
These he put by to be used only as a last recourse, and fell to his
Sisyphean task again with such concentration of all his powers upon
the work in hand as few men have ever dreadful need to attain—
such focused concentration that, had his mind been an actual
searchlight, capable, in its turning, to throw a shining circle upon
actual, living, moving men, in all places, far or near, in time past,
present or to come—where it paused, the places, men and events
could not have been more real, more clear, more brightly illumined.
When this inner light wearied and grew faint he turned it back till it
pierced the thick walls to another prison, dwelt on another prisoner
there: a tall, gray figure, whose face was turned away; ringed round
with hate, with ignominy, shame despair and death; not friendless.
And the light rose again, strong and unwavering, ranging the earth
for what help was there; so fell at last upon a plan, not after to be
altered. A rough plan only—the details to be worked out—to-morrow
and to-morrow. So thinking, utter exhaustion came upon him and he
fell asleep.
 Chapter VI

“The bosun’s mate was very sedate, but fond of amusement

too,
So he played hop-scotch with the larboard watch, while the
Captain tickled the crew.”

—Ballad of The Walloping Window Blind.

“AND what are these famous instructions of yours, Mae?”

“They are verra precise, Mr. Bransford. One of us will be always in
the room. That one will keep close and constant watch upon you,
even when you are asleep. Your wound will be dressed only when
we are both here. Coal and water, your meals, the things you send
for, will be brought in only when we are both here. And on any
slightest eendication of an attempted rescue or escape we are to kill
you without hesitation!”
It was plain that Mac was following the manner as well as the matter
of his instructions. He gave this information slowly, with dour
satisfaction, checking each item by forcibly doubling down, with his
right hand, the fingers of his left. Having now doubled them all down,
he undoubled them and began again.
“If you attempt to give any alarm, if you attempt to make any attack,
if on any pretext you try to get near enough for a possible attack, we
will kill you without hesitation.” He rolled the phrase under his tongue
with great relish.
“Your precautions are most flattering, I’m sure,” said Jeff idly. “I must
be very careful. The room is large, but I might inadvertently break
your last rule at any time. If I understand you correctly, should I so
much as drop my pencil and, picking it up, forgetfully come too close
——”
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com