100% found this document useful (5 votes)
71 views

Download full Applied Neural Networks with TensorFlow 2: API Oriented Deep Learning with Python Orhan Gazi Yalçın ebook all chapters

Deep

Uploaded by

frumanayel84
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
100% found this document useful (5 votes)
71 views

Download full Applied Neural Networks with TensorFlow 2: API Oriented Deep Learning with Python Orhan Gazi Yalçın ebook all chapters

Deep

Uploaded by

frumanayel84
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 55

Download the Full Version of textbook for Fast Typing at textbookfull.

com

Applied Neural Networks with TensorFlow 2: API


Oriented Deep Learning with Python Orhan Gazi
Yalçın

https://textbookfull.com/product/applied-neural-networks-
with-tensorflow-2-api-oriented-deep-learning-with-python-
orhan-gazi-yalcin/

OR CLICK BUTTON

DOWNLOAD NOW

Download More textbook Instantly Today - Get Yours Now at textbookfull.com


Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Applied Neural Networks with TensorFlow 2 API Oriented


Deep Learning with Python 1st Edition Orhan Gazi Yalc■n
Yalç■n Orhan
https://textbookfull.com/product/applied-neural-networks-with-
tensorflow-2-api-oriented-deep-learning-with-python-1st-edition-orhan-
gazi-yalcin-yalcin-orhan/
textboxfull.com

Deep Learning Projects Using TensorFlow 2: Neural Network


Development with Python and Keras 1st Edition Vinita
Silaparasetty
https://textbookfull.com/product/deep-learning-projects-using-
tensorflow-2-neural-network-development-with-python-and-keras-1st-
edition-vinita-silaparasetty/
textboxfull.com

Artificial Neural Networks with TensorFlow 2: ANN


Architecture Machine Learning Projects Poornachandra
Sarang
https://textbookfull.com/product/artificial-neural-networks-with-
tensorflow-2-ann-architecture-machine-learning-projects-poornachandra-
sarang/
textboxfull.com

Applied Reinforcement Learning with Python: With OpenAI


Gym, Tensorflow, and Keras Beysolow Ii

https://textbookfull.com/product/applied-reinforcement-learning-with-
python-with-openai-gym-tensorflow-and-keras-beysolow-ii/

textboxfull.com
Deep Learning with JavaScript Neural networks in
TensorFlow js 1st Edition Shanqing Cai Stanley Bileschi
Eric D Nielsen
https://textbookfull.com/product/deep-learning-with-javascript-neural-
networks-in-tensorflow-js-1st-edition-shanqing-cai-stanley-bileschi-
eric-d-nielsen/
textboxfull.com

Deep Learning with Python Develop Deep Learning Models on


Theano and TensorFLow Using Keras Jason Brownlee

https://textbookfull.com/product/deep-learning-with-python-develop-
deep-learning-models-on-theano-and-tensorflow-using-keras-jason-
brownlee/
textboxfull.com

Deep Learning with JavaScript: Neural networks in


TensorFlow.js 1st Edition Shanqing Cai

https://textbookfull.com/product/deep-learning-with-javascript-neural-
networks-in-tensorflow-js-1st-edition-shanqing-cai/

textboxfull.com

Learn Keras for Deep Neural Networks: A Fast-Track


Approach to Modern Deep Learning with Python 1st Edition
Jojo John Moolayil
https://textbookfull.com/product/learn-keras-for-deep-neural-networks-
a-fast-track-approach-to-modern-deep-learning-with-python-1st-edition-
jojo-john-moolayil/
textboxfull.com

MATLAB Deep Learning: With Machine Learning, Neural


Networks and Artificial Intelligence 1st Edition Phil Kim

https://textbookfull.com/product/matlab-deep-learning-with-machine-
learning-neural-networks-and-artificial-intelligence-1st-edition-phil-
kim/
textboxfull.com
Applied Neural
Networks with
TensorFlow 2
API Oriented Deep Learning
with Python

Orhan Gazi Yalçın
Applied Neural
Networks with
TensorFlow 2
API Oriented Deep Learning
with Python

Orhan Gazi Yalçın


Applied Neural Networks with TensorFlow 2: API Oriented Deep
Learning with Python

Orhan Gazi Yalçın


Istanbul, Turkey

ISBN-13 (pbk): 978-1-4842-6512-3 ISBN-13 (electronic): 978-1-4842-6513-0


https://doi.org/10.1007/978-1-4842-6513-0

Copyright © 2021 by Orhan Gazi Yalçın


This work is subject to copyright. All rights are reserved by the Publisher, whether the whole
or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical
way, and transmission or information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodology now known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a
trademark symbol with every occurrence of a trademarked name, logo, or image we use the
names, logos, and images only in an editorial fashion and to the benefit of the trademark
owner, with no intention of infringement of the trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms,
even if they are not identified as such, is not to be taken as an expression of opinion as to
whether or not they are subject to proprietary rights.
While the advice and information in this book are believed to be true and accurate at the
date of publication, neither the authors nor the editors nor the publisher can accept any
legal responsibility for any errors or omissions that may be made. The publisher makes no
warranty, express or implied, with respect to the material contained herein.
Managing Director, Apress Media LLC: Welmoed Spahr
Acquisitions Editor: Aaron Black
Development Editor: James Markham
Coordinating Editor: Jessica Vakili
Distributed to the book trade worldwide by Springer Science+Business Media New York,1
NY Plazar, New York, NY 10014. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail
[email protected], or visit www.springeronline.com. Apress Media, LLC is a
California LLC and the sole member (owner) is Springer Science + Business Media Finance
Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.
For information on translations, please e-mail [email protected]; for
reprint, paperback, or audio rights, please e-mail [email protected].
Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook
versions and licenses are also available for most titles. For more information, reference our
Print and eBook Bulk Sales web page at http://www.apress.com/bulk-sales.
Any source code or other supplementary material referenced by the author in this book is
available to readers on GitHub via the book’s product page, located at www.apress.com/
978-1-4842-6512-3. For more detailed information, please visit http://www.apress.com/
source-code.
Printed on acid-free paper
I dedicate this book to my overcurious dad, Lutfi – who kept
sneaking into the study room to see how far I was into the
book – and to my mom, Ayşe, for always supporting and
encouraging me.

I would also like to thank my friend, Enes, for encouraging


me to write this book in the first place.

Finally, I would like to thank my sister and brother, Merve


and Kürşat, and all my friends who supported me
throughout the whole process – all the way – till the last word.
Table of Contents
About the Author��������������������������������������������������������������������������������xv

About the Technical Reviewer����������������������������������������������������������xvii


Acknowledgments�����������������������������������������������������������������������������xix

Chapter 1: Introduction������������������������������������������������������������������������1
Python as Programming Language�����������������������������������������������������������������������3
Timeline of Python�������������������������������������������������������������������������������������������3
Python 2 vs. Python 3��������������������������������������������������������������������������������������4
Why Python?����������������������������������������������������������������������������������������������������5
TensorFlow As Deep Learning Framework������������������������������������������������������������7
Timeline of TensorFlow������������������������������������������������������������������������������������8
Why TensorFlow?�������������������������������������������������������������������������������������������10
What’s New in TensorFlow 2.x�����������������������������������������������������������������������10
TensorFlow Competitors��������������������������������������������������������������������������������15
Installation and Environment Setup��������������������������������������������������������������������20
Interactive Programming Environments: IPython, Jupyter Notebook,
and Google Colab�������������������������������������������������������������������������������������������21
IPython�����������������������������������������������������������������������������������������������������������22
Jupyter Notebook������������������������������������������������������������������������������������������23
Google Colab��������������������������������������������������������������������������������������������������30
Hardware Options and Requirements�����������������������������������������������������������������32

v
Table of Contents

Chapter 2: Introduction to Machine Learning�������������������������������������33


What Is Machine Learning?��������������������������������������������������������������������������������33
Scope of Machine Learning and Its Relation to Adjacent Fields�������������������������37
Artificial Intelligence��������������������������������������������������������������������������������������37
Deep Learning�����������������������������������������������������������������������������������������������38
Data Science�������������������������������������������������������������������������������������������������39
Big Data���������������������������������������������������������������������������������������������������������39
The Taxonomy Diagram���������������������������������������������������������������������������������39
Machine Learning Approaches and Models��������������������������������������������������������40
Supervised Learning��������������������������������������������������������������������������������������41
Unsupervised Learning����������������������������������������������������������������������������������44
Semi-supervised Learning����������������������������������������������������������������������������46
Reinforcement Learning��������������������������������������������������������������������������������47
Evaluation of Different Approaches���������������������������������������������������������������48
Steps of Machine Learning���������������������������������������������������������������������������������49
Gathering Data�����������������������������������������������������������������������������������������������50
Preparing Data�����������������������������������������������������������������������������������������������50
Model Selection���������������������������������������������������������������������������������������������51
Training����������������������������������������������������������������������������������������������������������51
Evaluation������������������������������������������������������������������������������������������������������52
Hyperparameter Tuning���������������������������������������������������������������������������������54
Prediction������������������������������������������������������������������������������������������������������55
Final Evaluations�������������������������������������������������������������������������������������������������55

Chapter 3: Deep Learning and Neural Networks Overview�����������������57


Timeline of Neural Networks and Deep Learning Studies����������������������������������59
Structure of Artificial Neural Networks���������������������������������������������������������������63
McCulloch-Pitts Neuron���������������������������������������������������������������������������������63
Linear Threshold Unit (LTU)����������������������������������������������������������������������������64

vi
Table of Contents

Perceptron�����������������������������������������������������������������������������������������������������64
A Modern Deep Neural Network��������������������������������������������������������������������65
Activation Functions��������������������������������������������������������������������������������������������66
Loss (Cost or Error) Functions�����������������������������������������������������������������������������69
Optimization in Deep Learning����������������������������������������������������������������������������70
Backpropagation�������������������������������������������������������������������������������������������71
Optimization Algorithms��������������������������������������������������������������������������������72
Optimization Challenges��������������������������������������������������������������������������������75
Overfitting and Regularization����������������������������������������������������������������������������77
Overfitting������������������������������������������������������������������������������������������������������77
Regularization�����������������������������������������������������������������������������������������������78
Feature Scaling���������������������������������������������������������������������������������������������������79
Final Evaluations�������������������������������������������������������������������������������������������������80

Chapter 4: Complementary Libraries to TensorFlow 2.x���������������������81


Installation with Pip��������������������������������������������������������������������������������������������82
NumPy – Array Processing����������������������������������������������������������������������������������85
SciPy – Scientific Computing������������������������������������������������������������������������������86
Pandas – Array Processing and Data Analysis����������������������������������������������������88
Matplotlib and Seaborn – Data Visualization������������������������������������������������������89
Scikit-learn – Machine Learning�������������������������������������������������������������������������91
Flask – Deployment��������������������������������������������������������������������������������������������93
Final Evaluations�������������������������������������������������������������������������������������������������94

Chapter 5: A Guide to TensorFlow 2.0 and Deep Learning Pipeline����95


TensorFlow Basics����������������������������������������������������������������������������������������������96
Eager Execution���������������������������������������������������������������������������������������������96
Tensor������������������������������������������������������������������������������������������������������������97
Variable����������������������������������������������������������������������������������������������������������99

vii
Table of Contents

TensorFlow Deep Learning Pipeline������������������������������������������������������������������100


Data Loading and Preparation���������������������������������������������������������������������������101
Dataset Object (tf.data.Dataset)������������������������������������������������������������������101
TensorFlow Datasets Catalog����������������������������������������������������������������������102
NumPy Array������������������������������������������������������������������������������������������������106
Pandas DataFrame��������������������������������������������������������������������������������������106
Other Objects�����������������������������������������������������������������������������������������������107
Model Building��������������������������������������������������������������������������������������������������107
Keras API�����������������������������������������������������������������������������������������������������108
Estimator API�����������������������������������������������������������������������������������������������112
Compiling, Training, and Evaluating the Model and Making Predictions�����������113
The Standard Method����������������������������������������������������������������������������������113
Custom Training�������������������������������������������������������������������������������������������116
Saving and Loading the Model��������������������������������������������������������������������������118
Saving the Model�����������������������������������������������������������������������������������������119
Loading the Model���������������������������������������������������������������������������������������120
Conclusion��������������������������������������������������������������������������������������������������������120

Chapter 6: Feedforward Neural Networks����������������������������������������121


Deep and Shallow Feedforward Neural Networks��������������������������������������������122
Shallow Feedforward Neural Network���������������������������������������������������������122
Deep Feedforward Neural Network�������������������������������������������������������������123
Feedforward Neural Network Architecture�������������������������������������������������������125
Layers in a Feedforward Neural Network����������������������������������������������������125
Input Layer���������������������������������������������������������������������������������������������������125
Output Layer������������������������������������������������������������������������������������������������126
Hidden Layer�����������������������������������������������������������������������������������������������126

viii
Table of Contents

Case Study | Fuel Economics with Auto MPG����������������������������������������������������127


Initial Installs and Imports���������������������������������������������������������������������������127
Downloading the Auto MPG Data�����������������������������������������������������������������128
Data Preparation������������������������������������������������������������������������������������������129
Model Building and Training������������������������������������������������������������������������134
Evaluating the Results���������������������������������������������������������������������������������138
Making Predictions with a New Observation�����������������������������������������������141
Conclusion��������������������������������������������������������������������������������������������������������143

Chapter 7: Convolutional Neural Networks���������������������������������������145


Why Convolutional Neural Networks?���������������������������������������������������������������146
CNN Architecture�����������������������������������������������������������������������������������������������147
Layers in a CNN�������������������������������������������������������������������������������������������147
A Full CNN Model�����������������������������������������������������������������������������������������151
Case Study | Image Classification with MNIST�������������������������������������������������152
Downloading the MNIST Data����������������������������������������������������������������������152
Reshaping and Normalizing the Images������������������������������������������������������154
Building the Convolutional Neural Network�������������������������������������������������155
Compiling and Fitting the Model������������������������������������������������������������������156
Evaluating the Model�����������������������������������������������������������������������������������157
Saving the Trained Model����������������������������������������������������������������������������159
Conclusion��������������������������������������������������������������������������������������������������������160

Chapter 8: Recurrent Neural Networks���������������������������������������������161


Sequence Data and Time-Series Data��������������������������������������������������������������161
RNNs and Sequential Data��������������������������������������������������������������������������������163
The Basics of RNNs�������������������������������������������������������������������������������������������164
The History of RNNs������������������������������������������������������������������������������������164
Applications of RNNs�����������������������������������������������������������������������������������165
Mechanism of RNNs������������������������������������������������������������������������������������166
ix
Table of Contents

RNN Types���������������������������������������������������������������������������������������������������������167
Simple RNNs������������������������������������������������������������������������������������������������168
Long Short-Term Memory (LSTM)���������������������������������������������������������������169
Gated Recurrent Units (GRUs)����������������������������������������������������������������������170
Case Study | Sentiment Analysis with IMDB Reviews���������������������������������������171
Preparing Our Colab for GPU Accelerated Training��������������������������������������172
IMDB Reviews���������������������������������������������������������������������������������������������173
Preparing the Dataset����������������������������������������������������������������������������������175
Building the Recurrent Neural Network�������������������������������������������������������176
Compiling and Fitting the Model������������������������������������������������������������������178
Evaluating the Model�����������������������������������������������������������������������������������179
Making New Predictions������������������������������������������������������������������������������181
Saving and Loading the Model��������������������������������������������������������������������182
Conclusion��������������������������������������������������������������������������������������������������������185

Chapter 9: Natural Language Processing�����������������������������������������187


History of NLP���������������������������������������������������������������������������������������������������187
Early Ideas���������������������������������������������������������������������������������������������������188
Rule-Based NLP�������������������������������������������������������������������������������������������188
Statistical NLP and Supervised Learning����������������������������������������������������189
Unsupervised and Semi-supervised NLP����������������������������������������������������190
Real-World Applications of NLP������������������������������������������������������������������������190
Major Evaluations, Techniques, Methods, and Tasks����������������������������������������191
Morphosyntax����������������������������������������������������������������������������������������������192
Semantics����������������������������������������������������������������������������������������������������193
Discourse�����������������������������������������������������������������������������������������������������195
Speech���������������������������������������������������������������������������������������������������������195
Dialogue������������������������������������������������������������������������������������������������������196
Cognition�����������������������������������������������������������������������������������������������������196

x
Table of Contents

Natural Language Toolkit (NLTK)�����������������������������������������������������������������������196


Case Study | Text Generation with Deep NLP����������������������������������������������������198
The Goal of the Case Study�������������������������������������������������������������������������198
Shakespeare Corpus�����������������������������������������������������������������������������������199
Initial Imports����������������������������������������������������������������������������������������������200
Loading the Corpus�������������������������������������������������������������������������������������201
Vectorize the Text����������������������������������������������������������������������������������������202
Creating the Dataset������������������������������������������������������������������������������������203
Building the Model��������������������������������������������������������������������������������������205
Compiling and Training the Model���������������������������������������������������������������207
Generating Text with the Trained Model������������������������������������������������������209
Conclusion��������������������������������������������������������������������������������������������������������213

Chapter 10: Recommender Systems������������������������������������������������215


Popular Approaches������������������������������������������������������������������������������������������216
Collaborative Filtering���������������������������������������������������������������������������������216
Data Collection��������������������������������������������������������������������������������������������217
Content-Based Filtering (Personality-Based Approach)������������������������������219
Other Recommender System Approaches���������������������������������������������������220
Case Study | Deep Collaborative Filtering with MovieLens Dataset������������������221
MovieLens Dataset��������������������������������������������������������������������������������������222
Initial Imports����������������������������������������������������������������������������������������������222
Loading the Data�����������������������������������������������������������������������������������������223
Processing the Data������������������������������������������������������������������������������������225
Splitting the Dataset������������������������������������������������������������������������������������227
Building the Model��������������������������������������������������������������������������������������228
Compile and Train the Model�����������������������������������������������������������������������231
Make Recommendations�����������������������������������������������������������������������������232
Conclusion��������������������������������������������������������������������������������������������������������236

xi
Table of Contents

Chapter 11: Autoencoders����������������������������������������������������������������237


Advantages and Disadvantages of Autoencoders���������������������������������������������238
Autoencoder Architecture���������������������������������������������������������������������������������239
Layers Used in an Autoencoder�������������������������������������������������������������������240
Advantages of Depth�����������������������������������������������������������������������������������241
Variations of Autoencoders�������������������������������������������������������������������������������241
Undercomplete Autoencoders���������������������������������������������������������������������242
Regularized Autoencoders���������������������������������������������������������������������������242
Variational Autoencoder (VAE)���������������������������������������������������������������������244
Use Cases of Autoencoders������������������������������������������������������������������������������245
Case Study | Image Denoising with Fashion MNIST������������������������������������������246
Fashion MNIST Dataset�������������������������������������������������������������������������������������247
Initial Imports����������������������������������������������������������������������������������������������247
Loading and Processing the Data����������������������������������������������������������������248
Adding Noise to Images�������������������������������������������������������������������������������251
Building the Model��������������������������������������������������������������������������������������253
Denoising Noisy Images������������������������������������������������������������������������������255
Conclusion��������������������������������������������������������������������������������������������������������257

Chapter 12: Generative Adversarial Network�����������������������������������259


Method��������������������������������������������������������������������������������������������������������������259
Architecture������������������������������������������������������������������������������������������������������260
GAN Components�����������������������������������������������������������������������������������������261
A Known Issue: Mode Collapse��������������������������������������������������������������������262
Final Notes on Architecture�������������������������������������������������������������������������262
Applications of GANs�����������������������������������������������������������������������������������������263
Art and Fashion�������������������������������������������������������������������������������������������263
Manufacturing, Research, and R&D�������������������������������������������������������������263

xii
Table of Contents

Video Games������������������������������������������������������������������������������������������������264
Malicious Applications and Deep Fake��������������������������������������������������������264
Miscellaneous Applications�������������������������������������������������������������������������264
Case Study | Digit Generation with MNIST��������������������������������������������������������265
Initial Imports����������������������������������������������������������������������������������������������265
Load and Process the MNIST Dataset���������������������������������������������������������266
Build the GAN Model������������������������������������������������������������������������������������267
Train the GAN Model������������������������������������������������������������������������������������275
Animate Generated Digits During the Training��������������������������������������������281
Conclusion��������������������������������������������������������������������������������������������������������284

Index�������������������������������������������������������������������������������������������������285

xiii
About the Author
Orhan Gazi Yalçın is a joint PhD candidate at
the University of Bologna and the Polytechnic
University of Madrid. After completing
his double major in business and law, he
began his career in Istanbul, working for a
city law firm, Allen & Overy, and a global
entrepreneurship network, Endeavor. During
his academic and professional career, he
taught himself programming and excelled
in machine learning. He currently conducts
research on hotly debated law and AI topics
such as explainable artificial intelligence
and the right to explanation by combining his technical and legal skills.
In his spare time, he enjoys free diving, swimming, exercising, as well as
discovering new countries, cultures, and cuisines.

• You can visit Orhan’s personal web page at

www.orhangaziyalcin.com

• Also feel free to connect with Orhan on Linkedin at

www.linkedin.com/in/orhangaziyalcin

xv
About the Technical Reviewer
Vishwesh Ravi Shrimali graduated from BITS Pilani in 2018, where
he studied mechanical engineering. Since then, he has worked with
BigVision LLC on deep learning and computer vision and was involved in
creating official OpenCV AI courses. Currently, he is working at Mercedes
Benz Research and Development India Pvt. Ltd. He has a keen interest
in programming and AI and has applied that interest in mechanical
engineering projects. He has also written multiple blogs on OpenCV and
deep learning on LearnOpenCV, a leading blog on computer vision. He has
also coauthored Machine Learning for OpenCV4 (second edition) by Packt.
When he is not writing blogs or working on projects, he likes to go on long
walks or play his acoustic guitar.

xvii
Acknowledgments
This book was written during a global lockdown due to the Covid-19
pandemic, which created a new normal that I have never experienced
before. Writing a book in the middle of a global crisis was a very intense
experience, and I was uncertain about taking this responsibility for a
long time. Thanks to my family and friends, I was able to complete the
book even earlier than scheduled. Now I am glad that I accepted Aaron’s
invitation, who guided me throughout the whole process. Thank you very
much for reaching out to me in the first place and making it possible to
have this book written.
I would like to thank Jessica Vakili for coordinating the entire project
and for being there whenever I needed. I would also like to thank Vishwesh
Ravi Shrimali for reviewing every single line of the book and providing me
with all the valuable comments, which helped to improve the quality of the
book tremendously.
Being surrounded with people who all have a positive attitude made
this experience very fruitful, and I am looking forward to working with
them in the future. Thank you all very much!

xix
CHAPTER 1

Introduction
In this book, we dive into the realms of deep learning (DL) and cover
several deep learning concepts along with several case studies. These case
studies range from image recognition to recommender systems, from art
generation to object clustering. Deep learning is part of a broader family
of machine learning (ML) methods based on artificial neural networks
(ANNs) with representation learning. These neural networks mimic the
human brain cells, or neurons, for algorithmic learning, and their learning
speed is much faster than human learning speed. Several deep learning
methods offer solutions to different types of machine learning problems:
(i) supervised learning, (ii) unsupervised learning, (iii) semi-supervised
learning, and (iv) reinforcement learning.
This book is structured in a way to also include an introduction to
the discipline of machine learning so that the reader may be acquainted
with the general rules and concepts of machine learning. Then, a detailed
introduction to deep learning is provided to familiarize the reader with the
sub-discipline of deep learning.
After covering the fundamentals of deep learning, the book covers
different types of artificial neural networks with their potential real-life
applications (i.e., case studies). Therefore, at each chapter, this book (i)
introduces the concept of a particular neural network architecture with
details on its components and then (ii) provides a tutorial on how to
apply this network structure to solve a particular artificial intelligence
(AI) problem.

© Orhan Gazi Yalçın 2021 1


O. G. Yalçın, Applied Neural Networks with TensorFlow 2,
https://doi.org/10.1007/978-1-4842-6513-0_1
Chapter 1 Introduction

Since the goal of this book is to provide case studies for deep learning
applications, the competency in several technologies and libraries is
sought for a satisfactory learning experience.
Before diving into machine learning and deep learning, we start with
the introduction to the technologies used in this book. This introduction
includes the latest developments and the reasoning as to why these
technologies are selected. Finally, this chapter also covers how to install
these technologies and prepare your environment with a minimum
amount of hassle. The technologies that are in the center of this book are
as follows:

• Our Selected Programming Language: Python 3.x

• Our Selected Deep Learning Framework: TensorFlow 2.x

• Our Development Environment: Google Colab (with


Jupyter Notebook alternative)

Note A TensorFlow Pipeline Guide showing how to use TensorFlow


can be found in Chapter 5, whereas the relevant libraries used with
TensorFlow are covered in Chapter 4.

Please note that this book assumes that you use Google Colab, which
requires almost no environment setup. The chapter also includes a local
Jupyter Notebook installation guide if you prefer a local environment. You
may skip the Jupyter Notebook installation section if you decide to use
Google Colab.

Note When learning a new programming discipline or technology,


one of the most demoralizing tasks is the environment setup process.
Therefore, it is important to simplify this process as much as possible.
Therefore, this chapter is designed with this principle in mind.

2
Chapter 1 Introduction

Python as Programming Language


Python is a programming language created by Guido van Rossum as a side
project and was initially released in 1991. Python supports object-oriented
programming (OOP), a paradigm based on the concept of objects, which
can contain data, in the form of fields. Python prioritizes the programmer’s
experience. Therefore, programmers can write clear and logical code
for both small and large projects. It also contains support for functional
programming. Python is dynamically typed and garbage collected.
Python is also considered as an interpreted language because it goes
through an interpreter, which turns code you write into the language
understood by your computer’s processor. An interpreter executes
the statements of code “one by one.” On the other hand, in compiled
languages, a compiler executes the code entirely and lists all possible
errors at a time. The compiled code is more efficient than the interpreted
code in terms of speed and performance. However, scripted languages
such as Python show only one error message even though your code has
multiple errors. This feature helps the programmer to clear errors quickly,
and it increases the development speed.

Timeline of Python
Let’s take a look at the timeline of Python:

• In the late 1980s, Python was conceived as a successor


to the ABC language.

• In December 1989, Guido van Rossum started Python’s


implementation.

• In January 1994, Python version 1.0 was released.


The major new features included were the functional
programming tools lambda, map, filter, and reduce.

3
Chapter 1 Introduction

• October 2000, Python 2.0 was released with major new


features, including a cycle-detecting garbage collector
and support for Unicode.

• Python 3.0 was released on December 3, 2008. It was


a major revision of the language that is only partially
backward compatible. Many of its major features were
backported to Python 2.6.x and 2.7.x version series.
Releases of Python 3 include the 2 to 3 utility, which
automates (at least partially) the translation of Python 2
code to Python 3.

• As of January 1, 2020, no new bug reports, fixes, or


changes are made to Python 2, and Python 2 is no
longer supported.

Python 2 vs. Python 3


One of the common questions a new deep learning programmer might
have is whether to use Python 2.x or Python 3.x since there are many
outdated blog posts and web articles comparing two major versions. As of
2020, it is safe to claim that these comparisons are not relevant. As you may
see in the preceding timeline, the delayed deprecation of Python 2.x finally
took place as of January 1, 2020. Therefore, programmers may not find
official support for Python 2.x versions anymore.
One of the essential skills for a programmer is to be up to date with
the latest technology, and therefore, this book only utilizes the use of
Python 3.x versions. For the readers who are only familiar with Python 2.x
versions, this preference should not pose a problem since the differences
between the syntax used in this book for Python 2.x and Python 3.x are
not significant. Therefore, Python 2.x programmers may immediately
familiarize themselves with the source code in this book.

4
Chapter 1 Introduction

W
 hy Python?
Compared to other programming languages, there are several reasons
for Python’s popularity among data scientists and machine learning
engineers. 2019 Kaggle Machine Learning and Data Science Survey
revealed that Python is by far the most popular programming language for
data science and machine learning; see Figure 1-1.

Figure 1-1. 2019 Kaggle Machine Learning and Data Science Survey

There are several reasons for Python’s popularity compared to


other languages. A non-exhaustive list of benefits of Python may be the
following.

5
Chapter 1 Introduction

Ease of Learning
One of the main reasons for newcomers to choose Python as their primary
programming language is its ease of learning. When compared to other
programming languages, Python offers a shorter learning curve so that
programmers can achieve a good level of competency in a short amount
of time. Python’s syntax is easier to learn, and the code is more readable
compared to other popular programming languages. A common example
to show this is the amount of code required by different programming
languages to print out “Hello, World!”. For instance, to be able to print out
Hello, World! in Java, you need the following code:

Hello, World! In Java

public class Main {


   public static void main(String[] args) {
       System.out.println("Hello, World!");
   }
}

The same result may be achieved with a single line of code in Python:

Hello, World! in Python


print("Hello, World!")

A Variety of Available Data Science Libraries


Another powerful characteristic of Python compared to other
programming languages is its wide variety of data science libraries.
The data science libraries such as Pandas, NumPy, SciPy, and scikit-­
learn reduce the time to prepare the data for model training with their
standardized functions and modules for logical and mathematical
operations. Furthermore, thanks to the vibrant community of Python
developers, as soon as the developers detect a common problem, a new
library is immediately designed and released to address this problem.

6
Chapter 1 Introduction

Community Support
The powerful community support is another advantage of Python over
other programming languages. More and more volunteers are releasing
Python libraries, and this practice made Python the language with modern
and powerful libraries. Besides, a high number of seasoned Python
programmers are always ready to help other programmers with their
problems on online community channels such as Stack Overflow.

Visualization Options
Data visualization is an important discipline to extract insights from raw
data, and Python offers several useful visualization options. The good old
Matplotlib is always there with the most customizable options. In addition,
Seaborn and Pandas Plot API are powerful libraries that streamline the
most common visualization tasks used by data scientists. Additionally,
libraries like Plotly and Dash allow users to create interactive plots and
sophisticated dashboards to be served on the Web. With these libraries,
data scientists may easily create charts, draw graphical plots, and facilitate
feature extraction.
Now that we covered why favorite language of data scientists is Python,
we can move on to why we use TensorFlow as our machine learning
framework.

TensorFlow As Deep Learning Framework


TensorFlow is an open source machine
learning platform with a particular focus on neural networks, developed by
the Google Brain team. Despite initially being used for internal purposes,

7
Chapter 1 Introduction

Google released the library under the Apache License 2.0 in November
2015, which made it an open source library.1 Although the use cases of
TensorFlow are not limited to machine learning applications, machine
learning is the field where we see TensorFlow’s strength.
The two programming languages with stable and official
TensorFlow APIs are Python and C. Also, C++, Java, JavaScript, Go, and
Swift are other programming languages where developers may find
limited-to-extensive TensorFlow compatibility. Finally, there are third-
party TensorFlow APIs for C#, Haskell, Julia, MATLAB, R, Scala, Rust,
OCaml, and Crystal.

T imeline of TensorFlow
Although this book focuses on TensorFlow 2.x with Python API, there
are several complementary TensorFlow libraries released by Google.
Understanding the development of the TensorFlow platform is essential to
see the full picture. The timeline of the milestones achieved by Google as
part of the TensorFlow project may be summarized as follows:

• In 2011, Google Brain built a machine learning system


called DistBelief using deep learning neural networks.

• November 2015, Google released the TensorFlow


library under the Apache License 2.0 and made it
open source to accelerate the advancements in
artificial intelligence.

1
Google Just Open Sourced TensorFlow, Its Artificial Intelligence Engine
| WIRED, www.wired.com/2015/11/google-open-sources-its-artificial-
intelligence-engine/ (last visited Jun 5, 2020)

8
Chapter 1 Introduction

• In May 2016, Google announced an application-­


specific integrated circuit (an ASIC) built for machine
learning and tailored for TensorFlow, called Tensor
Processing Unit (TPU).
• In February 2017, Google released TensorFlow 1.0.0.

• In May 2017, Google announced TensorFlow Lite, a


library for machine learning development in mobile
devices.

• In December 2017, Google introduced Kubeflow,


which allows operation and deployment of TensorFlow
on Kubernetes.

• In March 2018, Google announced TensorFlow.js


version 1.0 for machine learning with JavaScript.

• In July 2018, Google announced the Edge TPU. Edge


TPU is Google’s purpose-built ASIC chip designed to
run TensorFlow Lite machine learning (ML) models on
smartphones.

• In January 2019, Google announced TensorFlow 2.0 to


be officially available in September 2019.

• In May 2019, Google announced TensorFlow Graphics


for deep learning in computer graphics.

• In September 2019, TensorFlow Team released


TensorFlow 2.0, a new major version of the library.

This timeline shows that the TensorFlow platform is maturing.


Especially with the release of TensorFlow 2.0, Google has improved the
user-friendliness of TensorFlow APIs significantly. Besides, the TensorFlow
team announced that they don’t intend to introduce any other significant
changes. Therefore, it is safe to assume that the methods and syntax
included in this book are to keep their relevance for a long time.

9
Chapter 1 Introduction

Why TensorFlow?
There are more than two dozens of deep learning libraries developed by
tech giants, tech foundations, and academic institutions that are available
to the public. While each framework has its advantage in a particular sub-­
discipline of deep learning, this book focuses on TensorFlow with Keras
API. The main reason for choosing TensorFlow over other deep learning
frameworks is its popularity. On the other hand, this statement does not
indicate that the other frameworks are better – yet, less popular – than
TensorFlow. Especially with the introduction of version 2.0, TensorFlow
strengthened its power by addressing the issues raised by the deep
learning community. Today, TensorFlow may be seen as the most popular
deep learning framework, which is very powerful and easy to use and has
excellent community support.

What’s New in TensorFlow 2.x


Since its introduction in 2015, TensorFlow has grown into one of the
most advanced machine learning platforms in the market. Researchers,
developers, and companies widely adopted the technologies introduced by
the TensorFlow team. Around its 4th birthday, TensorFlow 2.0 was released
in September 2019. The TensorFlow team put a lot of effort into simplifying
the APIs by cleaning up deprecated APIs and reducing duplication. The
TensorFlow team introduced several updates to achieve simplicity and ease
of use in TensorFlow 2.0. These updates may be listed as follows:

1. Easy model building with Keras and eager execution

2. Robust model deployment in production level on


any platform

3. Robust experimentation for research

4. Simplified API thanks to cleanups and duplication


reduction

10
Chapter 1 Introduction

Easy Model Building with Keras and Eager Execution


The TensorFlow team further streamlined the model building experience
to respond to expectations with the new or improved modules such as
tf.data, tf.keras, and tf.estimators and the Distribution Strategy
API.

Load Your Data Using tf.data


In TensorFlow 2.0, training data is read using input pipelines created with
the tf.data module. tf.feature_column module is used to define feature
characteristics. What is useful for newcomers is the new DataSets module.
TensorFlow 2.0 offers a separate DataSets module which offers a range of
popular datasets and allows developers to experiment with these datasets.

Build, Train, and Validate Your Model with tf.keras, or Use


Premade Estimators
In TensorFlow 1.x, developers could use the previous versions of tf.
contrib, tf.layers, tf.keras, and tf.estimators to build models.
Offering four different options to the same problem confused the
newcomers and drove some of them away, especially to PyTorch.
TensorFlow 2.0 simplified the model building by limiting the options
to two improved modules: tf.keras (TensorFlow Keras API) and tf.
estimators (Estimator API). TensorFlow Keras API offers a high-level
interface that makes model building easy, which is especially useful
for proof of concepts (POC). On the other hand, Estimator API is better
suited for production-level models that require scaled serving and
increased customization capability.

11
Chapter 1 Introduction

Run and Debug with Eager Execution, Then Use AutoGraph API
for the Benefits of Graphs
TensorFlow 1.x versions were prioritizing TensorFlow graphs, which is not
friendly to newcomers. Even though this complicated methodology was
kept in TensorFlow 2.0, eager execution – the contrast concept – was made
default. Google explained the initial reasoning for this change with the
following statement:

Eager execution is an imperative, define-by-run interface where


operations are executed immediately as they are called from
Python. This makes it easier to get started with TensorFlow, and
can make research and development more intuitive.2

Eager execution makes the model building easier. It offers fast


debugging capability with immediate runtime errors and integration
with Python tools, which makes TensorFlow more beginner friendly. On
the other hand, graph execution has advantages for distributed training,
performance optimizations, and production deployment. To fill this gap,
TensorFlow introduced AutoGraph API called via tf.function decorator.
This book prioritizes eager execution over graph execution to achieve a
steep learning curve for the reader.

Use Distribution Strategies for Distributed Training


Model training with large datasets necessitates distributed training with
multiple processors such as CPU, GPU, or TPU. Even though TensorFlow
1.x has support for distributed training, Distribution Strategy API optimizes
and streamlines the distributed training across multiple GPUs, multiple

2
Google AI Blog: Eager Execution: An imperative, define-by-run interface
to TensorFlow, https://ai.googleblog.com/2017/10/eager-execution-
imperative-define-by.html (last visited Jun 8, 2020)

12
Chapter 1 Introduction

machines, or TPUs. TensorFlow also provides templates to deploy training


on Kubernetes clusters in on-prem or cloud environments, which makes
the training more cost-effective.

Export to SavedModel
After training a model, developers may export to SavedModel. tf.saved_
model API may be used to build a complete TensorFlow program with
weights and computations. This standardized SavedModel can be used
interchangeably across different TensorFlow deployment libraries such as
(i) TensorFlow Serving, (ii) TensorFlow Lite, (iii) TensorFlow.js, and (iv)
TensorFlow Hub.

 obust Model Deployment in Production on Any


R
Platform
TensorFlow has always made efforts to provide a direct path to production
on different devices. There are already several libraries which may be used
to serve the trained models on dedicated environments.

TensorFlow Serving
TensorFlow Serving is a flexible and high-performance TensorFlow library
that allows models to be served over HTTP/REST or gRPC/Protocol
Buffers. This platform is platform and language-neutral as you may make
an HTTP call using any programming language.

TensorFlow Lite
TensorFlow Lite is a lightweight deep learning framework to deploy models
to mobile devices (iOS and Android) or embedded devices (Raspberry Pi
or Edge TPUs). Developers may pick a trained model, convert the model
into a compressed fat buffer, and deploy to a mobile or embedded device
with TensorFlow Lite.

13
Another Random Scribd Document
with Unrelated Content
far from intending to give even a sketch of all past geological
speculations, I must notice some of the forms such speculations
have at different times assumed.

Sect. 2.—Of Fanciful Geological Opinions.

Real and permanent geological knowledge, like all other physical


knowledge, can be obtained only by inductions of classification and
law from many clearly seen phenomena. The labor of the most
active, the talent of the most intelligent, are requisite for such a
purpose. But far less than this is sufficient to put in busy operation
the inventive and capricious fancy. A few appearances hastily seen,
and arbitrarily interpreted, are enough to give rise to a wondrous tale
of the past, full of strange events and supernatural agencies. The
mythology and early poetry of nations afford sufficient evidence of
man’s love of the wonderful, and of his inventive powers, in early
stages of intellectual development. The scientific faculty, on the other
hand, 581 and especially that part of it which is requisite for the
induction of laws from facts, emerges slowly and with difficulty from
the crowd of adverse influences, even under the most favorable
circumstances. We have seen that in the ancient world, the Greeks
alone showed themselves to possess this talent; and what they thus
attained to, amounted only to a few sound doctrines in astronomy,
and one or two extremely imperfect truths in mechanics, optics, and
music, which their successors were unable to retain. No other nation,
till we come to the dawn of a better day in modern Europe, made any
positive step at all in sound physical speculation. Empty dreams or
useless exhibitions of ingenuity, formed the whole of their essays at
such knowledge.
It must, therefore, independently of positive evidence, be
considered as extremely improbable, that any of these nations
should, at an early period, have arrived, by observation and
induction, at wide general truths, such as the philosophers of modern
times have only satisfied themselves of by long and patient labor and
thought. If resemblances should be discovered between the
assertions of ancient writers and the discoveries of modern science,
the probability in all cases, the certainty in most, is that these are
accidental coincidences;—that the ancient opinion is no anticipation
of the modern discovery, but is one guess among many, not a whit
the more valuable because its expression agrees with a truth. The
author of the guess could not intend the truth, because his mind was
not prepared to comprehend it. Those of the ancients who spoke of
the harmony which binds all things together, could not mean the
Newtonian gravitation, because they had never been led to conceive
an attractive force, governed by definite mathematical laws in its
quantity and operation.

In agreement with these views, we must, I conceive, estimate the


opinions which we find among the ancients, respecting the changes
which the earth’s surface has undergone. These opinions, when they
are at all of a general kind, are arbitrary fictions of the fancy, showing
man’s love of generality indeed, but indulging it without that expense
of labor and thought which alone can render it legitimate.

We might, therefore, pass by all the traditions and speculations of


Oriental, Egyptian, and Greek cosmogony, as extraneous to our
subject. But since these have recently been spoken of, as
conclusions collected, however vaguely, from observed facts, 95 we
may make a remark or two upon them.
95 Lyell, B. i. c. ii. p. 8. (4th ed.)

582 The notion of a series of creations and destructions of worlds,


which appears in the sacred volume of the Hindoos, which formed
part of the traditionary lore of Egypt, and which was afterwards
adopted into the poetry and philosophy of Greece, must be
considered as a mythological, not a physical, doctrine. When this
doctrine was dwelt upon, men’s thoughts were directed, not to the
terrestrial facts which it seemed to explain, but to the attributes of the
deities which it illustrated. The conception of a Supreme power,
impelling and guiding the progress of events, which is permanent
among all perpetual change, and regular among all seeming chance,
was readily entertained by contemplative and enthusiastic minds;
and when natural phenomena were referred to this doctrine, it was
rather for the purpose of fastening its impressiveness upon the
senses, than in the way of giving to it authority and support. Hence
we perceive that in the exposition of this doctrine, an attempt was
always made to fill and elevate the mind with the notions of
marvellous events, and of infinite times, in which vast cycles of order
recurred. The “great year,” in which all celestial phenomena come
round, offered itself as capable of being calculated; and a similar
great year was readily assumed for terrestrial and human events.
Hence there were to be brought round by great cycles, not only
deluges and conflagrations which were to destroy and renovate the
earth, but also the series of historical occurrences. Not only the sea
and land were to recommence their alternations, but there was to be
another Argo, which should carry warriors on the first sea-foray, 96
and another succession of heroic wars. Looking at the passages of
ancient authors which refer to terrestrial changes in this view, we
shall see that they are addressed almost entirely to the love of the
marvellous and the infinite, and cannot with propriety be taken as
indications of a spirit of physical philosophy. For example, if we turn
to the celebrated passage in Ovid, 97 where Pythagoras is
represented as asserting that land becomes sea, and sea land, and
many other changes which geologists have verified, we find that
these observations are associated with many fables, as being matter
of exactly the same kind;—the fountain of Ammon which was cold by
day and warm by night; 98 —the waters of Salmacis which effeminate
men;—the Clitorian spring which makes them loathe wine;—the
Simplegades islands which were once moveable;—the Tritonian lake
which covered men’s bodies with feathers;—and many similar
marvels. And the general purport of 583 the whole is, to countenance
the doctrine of the metempsychosis, and the Pythagorean injunction
of not eating animal food. It is clear, I think, that facts so introduced
must be considered as having been contemplated rather in the spirit
of poetry than of science.
96 Virg. Eclog. 4.

97 Met. Lib. xv.

98 V. 309, &c.

We must estimate in the same manner, the very remarkable


passage brought to light by M. Elie de Beaumont, 99 from the Arabian
writer, Kazwiri; in which we have a representation of the same spot
of ground, as being, at successive intervals of five hundred years, a
city, a sea, a desert, and again a city. This invention is adduced, I
conceive, rather to feed the appetite of wonder, than to fix it upon
any reality: as the title of his book, The Marvels of Nature obviously
intimates.
99 Ann. des Sc. Nat. xxv. 380.
The speculations of Aristotle, concerning the exchanges of land
and sea which take place in long periods, are not formed in exactly
the same spirit, but they are hardly more substantial; and seem to be
quite as arbitrary, since they are not confirmed by any examples and
proofs. After stating, 100 that the same spots of the earth are not
always land and always water, he gives the reason. “The principle
and cause of this is,” he says, “that the inner parts of the earth, like
the bodies of plants and animals, have their ages of vigor and of
decline; but in plants and animals all the parts are in vigor, and all
grow old, at once: in the earth different parts arrive at maturity at
different times by the operation of cold and heat: they grow and
decay on account of the sun and the revolution of the stars, and thus
the parts of the earth acquire different power, so that for a certain
time they remain moist, and then become dry and old: and then
other places are revivified, and become partially watery.” We are, I
conceive, doing no injustice to such speculations by classing them
among fanciful geological opinions.
100 Meteorol. i. 14.

We must also, I conceive, range in the same division another class


of writers of much more modern times;—I mean those who have
trained their geology by interpretations of Scripture. I have already
endeavored to show that such an attempt is a perversion of the
purpose of a divine communication, and cannot lead to any physical
truth. I do not here speak of geological speculations in which the
Mosaic account of the deluge has been referred to; for whatever
errors may have been committed on that subject, it would be as
absurd to disregard the most ancient historical record, in attempting
to trace back the history of the earth, as it would be, gratuitously to
reject any other 584 source of information. But the interpretations of
the account of the creation have gone further beyond the limits of
sound philosophy: and when we look at the arbitrary and fantastical
inventions by which a few phrases of the writings of Moses have
been moulded into complete systems, we cannot doubt that these
interpretations belong to the present Section.

I shall not attempt to criticize, nor even to enumerate, these


Scriptural Geologies,—Sacred Theories of the Earth, as Burnet
termed his. Ray, Woodward, Whiston, and many other persons to
whom science has considerable obligations, were involved, by the
speculative habits of their times, in these essays; and they have
been resumed by persons of considerable talent and some
knowledge, on various occasions up to the present day; but the more
geology has been studied on its own proper evidence, the more
have geologists seen the unprofitable character of such labors.

I proceed now to the next step in the progress of Theoretical


Geology.

Sect. 3.—Of Premature Geological Theories.

While we were giving our account of Descriptive Geology, the


attentive reader would perceive that we did, in fact, state several
steps in the advance towards general knowledge; but when, in those
cases, the theoretical aspect of such discoveries softened into an
appearance of mere classification, the occurrence was assigned to
the history of Descriptive rather than of Theoretical Geology. Of such
a kind was the establishment, by a long and vehement controversy,
of the fact, that the impressions in rocks are really the traces of
ancient living things; such, again, were the division of rocks into
Primitive, Secondary, Tertiary; the ascertainment of the orderly
succession of organic remains: the consequent fixation of a standard
series of formations and strata; the establishment of the igneous
nature of trap rocks; and the like. These are geological truths which
are assumed and implied in the very language which geology uses;
thus showing how in this, as in all other sciences, the succeeding
steps involve the preceding. But in the history of geological theory,
we have to consider the wider attempts to combine the facts, and to
assign them to their causes.

The close of the last century produced two antagonist theories of


this kind, which long maintained a fierce and doubtful struggle;—that
of Werner and that of Hutton: the one termed Neptunian, from its 585
ascribing the phenomena of the earth’s surface mainly to aqueous
agency; the other Plutonian or Vulcanian, because it employed the
force of subterraneous fire as its principal machinery. The
circumstance which is most worthy of notice in these remarkable
essays is, the endeavor to give, by means of such materials as the
authors possessed, a complete and simple account of all the facts of
the earth’s history. The Saxon professor, proceeding on the
examination of a small district in Germany, maintained the existence
of a chaotic fluid, from which a series of universal formations had
been precipitated, the position of the strata being broken up by the
falling in of subterraneous cavities, in the intervals between these
depositions. The Scotch philosopher, who had observed in England
and Scotland, thought himself justified in declaring that the existing
causes were sufficient to spread new strata on the bottom of the
ocean, and that they are consolidated, elevated, and fractured by
volcanic heat, so as to give rise to new continents.

It will hardly be now denied that all that is to remain as permanent


science in each of these systems must be proved by the examination
of many cases and limited by many conditions and circumstances.
Theories so wide and simple, were consistent only with a
comparatively scanty collection of facts, and belong to the early
stage of geological knowledge. In the progress of the science, the
“theory” of each part of the earth must come out of the examination
of that part, combined with all that is well established, concerning all
the rest; and a general theory must result from the comparison of all
such partial theoretical views. Any attempt to snatch it before its time
must fail; and therefore we may venture at present to designate
general theories, like those of Hutton and Werner, as premature.

This, indeed, is the sentiment of most of the good geologists of the


present day. The time for such general systems, and for the fierce
wars to which the opposition of such generalities gives rise, is
probably now past for ever; and geology will not again witness such
a controversy as that of the Wernerian and Huttonian schools.

. . . . . . As when two black clouds


With heaven’s artillery fraught, come rattling on
Over the Caspian: then stand front to front,
Hovering a space, till winds the signal blow
To join their dark encounter in mid-air.
So frowned the mighty combatants, that hell
Grew darker at their frown; so matched they stood:
For never but once more was either like
To meet so great a foe.

586 The main points really affecting the progress of sound theoretical
geology, will find a place in one of the two next Sections.

[2nd Ed.] [I think I do no injustice to Dr. Hutton in describing his


theory of the earth as premature. Prof. Playfair’s elegant work,
Illustrations of the Huttonian Theory (1802,) so justly admired,
contains many doctrines which the more mature geology of modern
times rejects; such as the igneous origin of chalk-flints, siliceous
pudding stone, and the like; the universal formation of river-beds by
the rivers themselves; and other points. With regard to this last-
mentioned question, I think all who have read Deluc’s Geologie
(1810) will deem his refutation of Playfair complete.

But though Hutton’s theory was premature, as well as Werner’s,


the former had a far greater value as an important step on the road
to truth. Many of its boldest hypotheses and generalizations have
become a part of the general creed of geologists; and its publication
is perhaps the greatest event which has yet occurred in the progress
of Physical Geology.]
CHAPTER VIII.

The Two Antagonist Doctrines of Geology.

Sect. 1.—Of the Doctrine of Geological Catastrophes.

T HAT great changes, of a kind and intensity quite different from


the common course of events, and which may therefore properly
be called catastrophes, have taken place upon the earth’s surface,
was an opinion which appeared to be forced upon men by obvious
facts. Rejecting, as a mere play of fancy, the notions of the
destruction of the earth by cataclysms or conflagrations, of which we
have already spoken, we find that the first really scientific
examination of the materials of the earth, that of the Sub-Apennine
hills, led men to draw this inference. Leonardo da Vinci, whom we
have already noticed for his early and strenuous assertion of the real
marine origin of fossil impressions of shells, also maintained that the
bottom of the sea had become the top of the mountain; yet his mode
of explaining this may perhaps be claimed by the modern advocates
of uniform causes as more allied to their 587 opinion, than to the
doctrine of catastrophes. 101 But Steno, in 1669, approached nearer
to this doctrine; for he asserted that Tuscany must have changed its
face at intervals, so as to acquire six different configurations, by the
successive breaking down of the older strata into inclined positions,
and the horizontal deposit of new ones upon them. Strabo, indeed,
at an earlier period had recourse to earthquakes, to explain the
occurrence of shells in mountains; and Hooke published the same
opinion later. But the Italian geologists prosecuted their researches
under the advantage of having, close at hand, large collections of
conspicuous and consistent phenomena. Lazzaro Moro, in 1740,
attempted to apply the theory of earthquakes to the Italian strata; but
both he and his expositor, Cirillo Generelli, inclined rather to reduce
the violence of these operations within the ordinary course of
nature, 102 and thus leant to the doctrine of uniformity, of which we
have afterwards to speak. Moro was encouraged in this line of
speculation by the extraordinary occurrence, as it was deemed by
most persons, of the rise of a new volcanic island from a deep part of
the Mediterranean, near Santorino, in 1707. 103 But in other
countries, as the geological facts were studied, the doctrine of
catastrophes appeared to gain ground. Thus in England, where,
through a large part of the country, the coal-measures are extremely
inclined and contorted, and covered over by more horizontal
fragmentary beds, the opinion that some violent catastrophe had
occurred to dislocate them, before the superincumbent strata were
deposited, was strongly held. It was conceived that a period of
violent and destructive action must have succeeded to one of
repose; and that, for a time, some unusual and paroxysmal forces
must have been employed in elevating and breaking the pre-existing
strata, and wearing their fragments into smooth pebbles, before
nature subsided into a new age of tranquillity and vitality. In like
manner Cuvier, from the alternations of fresh-water and salt-water
species in the strata of Paris, collected the opinion of a series of
great revolutions, in which “the thread of induction was broken.”
Deluc and others, to whom we owe the first steps in geological
dynamics, attempted carefully to distinguish between causes now in
action, and those which have ceased to act; in which latter class they
reckoned the causes which have 588 elevated the existing
continents. This distinction was assented to by many succeeding
geologists. The forces which have raised into the clouds the vast
chains of the Pyrenees, the Alps, the Andes, must have been, it was
deemed, something very different from any agencies now operating.
101“Here is a part of the earth which has become more light, and
which rises, while the opposite part approaches nearer to the
centre, and what was the bottom of the sea is become the top of
the mountain.”—Venturi’s Léonardo da Vinci.

102 Lyell, i. 3. p. 64. (4th ed.)

103 Ib. p. 60.

This opinion was further confirmed by the appearance of a


complete change in the forms of animal and vegetable life, in
passing from one formation to another. The species of which the
remains occurred, were entirely different, it was said, in two
successive epochs: a new creation appears to have intervened; and
it was readily believed that a transition, so entirely out of the
common course of the world, might be accompanied by paroxysms
of mechanical energy. Such views prevail extensively among
geologists up to the present time: for instance, in the comprehensive
theoretical generalizations of Elie de Beaumont and others,
respecting mountain-chains, it is supposed that, at certain vast
intervals, systems of mountains, which may be recognized by the
parallelism of course of their inclined beds, have been disturbed and
elevated, lifting up with them the aqueous strata which had been
deposited among them in the intervening periods of tranquillity, and
which are recognized and identified by means of their organic
remains: and according to the adherents of this hypothesis, these
sudden elevations of mountain-chains have been followed, again
and again, by mighty waves, desolating whole regions of the earth.
The peculiar bearing of such opinions upon the progress of
physical geology will be better understood by attending to the
doctrine of uniformity, which is opposed to them, and with the
consideration of which we shall close our survey of this science, the
last branch of our present task.

Sect. 2.—Of the Doctrine of Geological Uniformity.

The opinion that the history of the earth had involved a serious of
catastrophes, confirmed by the two great classes of facts, the
symptoms of mechanical violence on a very large scale, and of
complete changes in the living things by which the earth had been
tenanted, took strong hold of the geologists of England, France, and
Germany. Hutton, though he denied that there was evidence of a
beginning of the present state of things, and referred many
processes in the formation of strata to existing causes, did not assert
that the elevatory forces which raise continents from the bottom of
the ocean, were of the same order, 589 as well as of the same kind,
with the volcanoes and earthquakes which now shake the surface.
His doctrine of uniformity was founded rather on the supposed
analogy of other lines of speculation, than on the examination of the
amount of changes now going on. “The Author of nature,” it was
said, “has not permitted in His works any symptom of infancy or of
old age, or any sign by which we may estimate either their future or
their past duration:” and the example of the planetary system was
referred to in illustration of this. 104 And a general persuasion that the
champions of this theory were not disposed to accept the usual
opinions on the subject of creation, was allowed, perhaps very
unjustly, to weigh strongly against them in the public opinion.
104 Lyell, i. 4, p. 94.
While the rest of Europe had a decided bias towards the doctrine
of geological catastrophes, the phenomena of Italy, which, as we
have seen, had already tended to soften the rigor of that doctrine, in
the progress of speculation from Steno to Generelli, were destined to
mitigate it still more, by converting to the belief of uniformity
transalpine geologists who had been bred up in the catastrophist
creed. This effect was, indeed, gradual. For a time the distinction of
the recent and the tertiary period was held to be marked and strong.
Brocchi asserted that a large portion of the Sub-Apennine fossil
shells belonged to a living species of the Mediterranean Sea: but the
geologists of the rest of Europe turned an incredulous ear to this
Italian tenet; and the persuasion of the distinction of the tertiary and
the recent period was deeply impressed on most geologists by the
memorable labors of Cuvier and Brongniart on the Paris basin. Still,
as other tertiary deposits were examined, it was found that they
could by no means be considered as contemporaneous, but that
they formed a chain of posts, advancing nearer and nearer to the
recent period. Above the strata of the basins of London and Paris, 105
lie the newer strata of Touraine, of Bourdeaux, of the valley of the
Bormida and the Superga near Turin, and of the basin of Vienna,
explored by M. Constant Prevost. Newer and higher still than these,
are found the Sub-Apennine formations of Northern Italy, and
probably of the same period, the English “crag” of Norfolk and
Suffolk. And most of these marine formations are associated with
volcanic products and fresh-water deposits, so as to imply
apparently a long train of alternations of corresponding processes. It
may easily be supposed that, when the subject had assumed this
form, the boundary of the present and past condition of the earth 590
was in some measure obscured. But it was not long before a very
able attempt was made to obliterate it altogether. In 1828, Mr. Lyell
set out on a geological tour through France and Italy. 106 He had
already conceived the idea of classing the tertiary groups by
reference to the number of recent species which were found in a
fossil state. But as he passed from the north to the south of Italy, he
found, by communication with the best fossil conchologists, Borelli at
Turin, Guidotti at Parma, Costa at Naples, that the number of extinct
species decreased; so that the last-mentioned naturalist, from an
examination of the fossil shells of Otranto and Calabria, and of the
neighboring seas, was of opinion that few of the tertiary shells were
of extinct species. To complete the series of proof, Mr. Lyell himself
explored the strata of Ischia, and found, 2000 feet above the level of
the sea, shells, which were all pronounced to be of species now
inhabiting the Mediterranean; and soon after, he made collections of
a similar description on the flanks of Etna, in the Val di Noto, and in
other places.
105 Lyell, 1st ed. vol. iii. p. 61.

106 1st ed. vol. iii. Pref.

The impression produced by these researches is described by


himself. 107 “In the course of my tour I had been frequently led to
reflect on the precept of Descartes, that a philosopher should once in
his life doubt everything he had been taught; but I still retained so
much faith in my early geological creed as to feel the most lively
surprize on visiting Sortino, Pentalica, Syracuse, and other parts of
the Val di Noto, at beholding a limestone of enormous thickness,
filled with recent shells, or sometimes with mere casts of shells,
resting on marl in which shells of Mediterranean species were
imbedded in a high state of preservation. All idea of [necessarily]
attaching a high antiquity to a regularly-stratified limestone, in which
the casts and impressions of shells alone were visible, vanished at
once from my mind. At the same time, I was struck with the identity
of the associated igneous rocks of the Val di Noto with well-known
varieties of ‘trap’ in Scotland and other parts of Europe; varieties
which I had also seen entering largely into the structure of Etna.
107 Lyell, 1st ed. Pref. x.

“I occasionally amused myself,” Mr. Lyell adds, “with speculating


on the different rate of progress which geology might have made,
had it been first cultivated with success at Catania, where the
phenomena above alluded to, and the great elevation of the modern
tertiary beds in the Val di Noto, and the changes produced in the
historical era by the Calabrian earthquakes, would have been
familiarly known.” 591

Before Mr. Lyell entered upon his journey, he had put into the
hands of the printer the first volume of his “Principles of Geology,
being an attempt to explain the former Changes of the Earth’s
Surface by reference to the Causes now in Operation.” And after
viewing such phenomena as we have spoken of, he, no doubt,
judged that the doctrine of catastrophes of a kind entirely different
from the existing course of events, would never have been generally
received, if geologists had at first formed their opinions upon the
Sicilian strata. The boundary separating the present from the anterior
state of things crumbled away; the difference of fossil and recent
species had disappeared, and, at the same time, the changes of
position which marine strata had undergone, although not inferior to
those of earlier geological periods, might be ascribed, it was thought,
to the same kind of earthquakes as those which still agitate that
region. Both the supposed proofs of catastrophic transition, the
organical and the mechanical changes, failed at the same time; the
one by the removal of the fact, the other by the exhibition of the
cause. The powers of earthquakes, even such as they now exist,
were, it was supposed, if allowed to operate for an illimitable time,
adequate to produce all the mechanical effects which the strata of all
ages display. And it was declared that all evidence of a beginning of
the present state of the earth, or of any material alteration in the
energy of the forces by which it has been modified at various
epochs, was entirely wanting.

Other circumstances in the progress of geology tended the same


way. Thus, in cases where there had appeared in one country a
sudden and violent transition from one stratum to the next, it was
found, that by tracing the formations into other countries, the chasm
between them was filled up by intermediate strata; so that the
passage became as gradual and gentle as any other step in the
series. For example, though the conglomerates, which in some parts
of England overlie the coal-measures, appear to have been
produced by a complete discontinuity in the series of changes; yet in
the coal-fields of Yorkshire, Durham, and Cumberland, the transition
is smoothed down in such a way that the two formations pass into
each other. A similar passage is observed in Central-Germany, and
in Thuringia is so complete, that the coal-measures have sometimes
been considered as subordinate to the todtliegendes. 108
108 De la Beche, p. 414, Manual.

Upon such evidence and such arguments, the doctrine of 592


catastrophes was rejected with some contempt and ridicule; and it
was maintained, that the operation of the causes of geological
change may properly and philosophically be held to have been
uniform through all ages and periods. On this opinion, and the
grounds on which it he been urged, we shall make a few concluding
remarks.

It must be granted at once, to the advocates of this geological


uniformity, that we are not arbitrarily to assume the existence of
catastrophes. The degree of uniformity and continuity with which
terremotive forces have acted, must be collected, not from any
gratuitous hypothesis, but from the facts of the case. We must
suppose the causes which have produced geological phenomena, to
have been as similar to existing causes, and as dissimilar, as the
effects teach us. We are to avoid all bias in favor of powers deviating
in kind and degree from those which act at present; a bias which, Mr.
Lyell asserts, has extensively prevailed among geologists.

But when Mr. Lyell goes further, and considers it a merit in a


course of geological speculation that it rejects any difference
between the intensity of existing and of past causes, we conceive
that he errs no less than those whom he censures. “An earnest and
patient endeavor to reconcile the former indication of change,” 109
with any restricted class of causes,—a habit which he enjoins,—is
not, we may suggest, the temper in which science ought to be
pursued. The effects must themselves teach us the nature and
intensity of the causes which have operated; and we are in danger of
error, if we seek for slow and shun violent agencies further than the
facts naturally direct us, no less than if we were parsimonious of time
and prodigal of violence. Time, inexhaustible and ever accumulating
his efficacy, can undoubtedly do much for the theorist in geology; but
Force, whose limits we cannot measure, and whose nature we
cannot fathom, is also a power never to be slighted: and to call in the
one to protect us from the other, is equally presumptuous, to
whichever of the two our superstition leans. To invoke Time, with ten
thousand earthquakes, to overturn and set on edge a mountain-
chain, should the phenomena indicate the change to have been
sudden and not successive, would be ill excused by pleading the
obligation of first appealing to known causes. 110
109 Lyell, B. iv. c. i. p. 328, 4th ed.

110 [2nd Ed.] [I have, in the text, quoted the fourth edition of Mr.
Lyell’s Principles, in which he recommends “an earnest and
patient endeavor to reconcile the former indications of change
with the evidence of gradual mutation now in progress.” In the
sixth edition, in that which is, I presume, the corresponding
passage, although it is transferred from the fourth to the first Book
(B. i. c. xiii. p. 325) he recommends, instead, “an earnest and
patient inquiry how far geological appearances are reconcileable
with the effect of changes now in progress.” But while Mr. Lyell
has thus softened the advocate’s character in his language in this
passage, the transposition which I have noticed appears to me to
have an opposite tendency. For in the former edition, the causes
now in action were first described in the second and third Books,
and the great problem of Geology, stated in the first Book, was
attempted to be solved in the fourth. But by incorporating this
fourth Book with the first, and thus prefixing to the study of
existing causes arguments against the belief of their geological
insufficiency, there is an appearance as if the author wished his
reader to be prepared by a previous pleading against the doctrine
of catastrophes, before he went to the study of existing causes.
The Doctrines of Catastrophes and of Uniformity, and the other
leading questions of the Palætiological Sciences, are further
discussed in the Philosophy of the Inductive Sciences, Book x.]

593 In truth, we know causes only by their effects; and in order to


learn the nature of the causes which modify the earth, we must study
them through all ages of their action, and not select arbitrarily the
period in which we live as the standard for all other epochs. The
forces which have produced the Alps and Andes are known to us by
experience, no less than the forces which have raised Etna to its
present height; for we learn their amount in both cases by their
results. Why, then, do we make a merit of using the latter case as a
measure for the former? Or how can we know the true scale of such
force, except by comprehending in our view all the facts which we
can bring together?

In reality when we speak of the uniformity of nature, are we not


obliged to use the term in a very large sense, in order to make the
doctrine at all tenable? It includes catastrophes and convulsions of a
very extensive and intense kind; what is the limit to the violence
which we must allow to these changes? In order to enable ourselves
to represent geological causes as operating with uniform energy
through all time, we must measure our time by long cycles, in which
repose and violence alternate; how long may we extend this cycle of
change, the repetition of which we express by the word uniformity?

And why must we suppose that all our experience, geological as


well as historical, includes more than one such cycle? Why must we
insist upon it, that man has been long enough an observer to obtain
the average of forces which are changing through immeasurable
time? 594

The analogy of other sciences has been referred to, as


sanctioning this attempt to refer the whole train of facts to known
causes. To have done this, it has been said, is the glory of
Astronomy: she seeks no hidden virtues, but explains all by the force
of gravitation, which we witness operating at every moment. But let
us ask, whether it would really have been a merit in the founders of
Physical Astronomy, to assume that the celestial revolutions resulted
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and


personal growth!

textbookfull.com

You might also like