Download the Full Ebook and Access More Features - ebooknice.

com

(Ebook) Machine Learning for Knowledge Discovery

with R: Methodologies for Modeling, Inference and
Prediction by Kao-Tai Tsai ISBN 9781032065366,
1032065362
https://ebooknice.com/product/machine-learning-for-
knowledge-discovery-with-r-methodologies-for-modeling-
inference-and-prediction-33691448

OR CLICK HERE

DOWLOAD EBOOK

Download more ebook instantly today at https://ebooknice.com

 Instant digital products (PDF, ePub, MOBI) ready for you
Download now and discover formats that fit your needs...

Start reading on any device today!

(Ebook) Biota Grow 2C gather 2C cook by Loucas, Jason;

Viles, James ISBN 9781459699816, 9781743365571,
9781925268492, 1459699815, 1743365578, 1925268497
https://ebooknice.com/product/biota-grow-2c-gather-2c-cook-6661374

ebooknice.com

(Ebook) Introduction To Conformal Prediction With Python :

A Short Guide For Quantifying Uncertainty Of Machine
Learning Models by Christoph Molnar
https://ebooknice.com/product/introduction-to-conformal-prediction-
with-python-a-short-guide-for-quantifying-uncertainty-of-machine-
learning-models-49150962
ebooknice.com

(Ebook) Machine Learning with R: Expert Techniques for

Predictive Modeling to Solve All Your Data Analysis
Problems by Brett Lantz ISBN 9781784393908, 1784393908
https://ebooknice.com/product/machine-learning-with-r-expert-
techniques-for-predictive-modeling-to-solve-all-your-data-analysis-
problems-6852218
ebooknice.com

(Ebook) Machine Learning for Financial Risk Management

with Python: Algorithms for Modeling Risk by Karasan,
Abdullah ISBN 9781492085256, 1492085251, B09N9V3X36
https://ebooknice.com/product/machine-learning-for-financial-risk-
management-with-python-algorithms-for-modeling-risk-38313580

ebooknice.com
(Ebook) Prediction and Analysis for Knowledge
Representation and Machine Learning by Avadhesh Kumar
(editor), Shrddha Sagar (editor), T Ganesh Kumar (editor),
K Sampath Kumar (editor) ISBN 9780367649104, 0367649101
https://ebooknice.com/product/prediction-and-analysis-for-knowledge-
representation-and-machine-learning-38303144

ebooknice.com

(Ebook) Machine Learning for Data Science Handbook: Data

Mining and Knowledge Discovery Handbook, 3rd by Lior
Rokach, Oded Maimon, Erez Shmueli, (eds.) ISBN
9783031246272, 3031246276
https://ebooknice.com/product/machine-learning-for-data-science-
handbook-data-mining-and-knowledge-discovery-handbook-3rd-51680834

ebooknice.com

(Ebook) Knowledge-Guided Machine Learning: Accelerating

Discovery Using Scientific Knowledge and Data (Chapman &
Hall/CRC Data Mining and Knowledge Discovery Series) by
Anuj Karpatne, Ramakrishnan Kannan, Vipin Kumar ISBN
https://ebooknice.com/product/knowledge-guided-machine-learning-
9780367693411, 0367693410
accelerating-discovery-using-scientific-knowledge-and-data-chapman-
hall-crc-data-mining-and-knowledge-discovery-series-44169202
ebooknice.com

(Ebook) Machine Learning for Financial Risk Management

with Python: Algorithms for Modeling Risk (Early Release)
by Abdullah Karasan ISBN 9781492085256, 1492085251
https://ebooknice.com/product/machine-learning-for-financial-risk-
management-with-python-algorithms-for-modeling-risk-early-
release-34605294
ebooknice.com

(Ebook) Statistical Methods for Machine Learning: Discover

How to Transform Data into Knowledge with Python by Jason
Brownlee
https://ebooknice.com/product/statistical-methods-for-machine-
learning-discover-how-to-transform-data-into-knowledge-with-
python-10440848
ebooknice.com
Machine Learning for
Knowledge Discovery
with R
Machine Learning for
Knowledge Discovery
with R
Methodologies for Modeling,
Inference, and Prediction

Kao-Tai Tsai
Frontier Informatics Services
Bristol Myers Squibb
Adjunct Professor, Jiann-Ping Hsu College of
Public Health, Georgia Southern University
First edition published 2022

by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742

and by CRC Press

2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN

© 2022 Kao-Tai Tsai

CRC Press is an imprint of Taylor & Francis Group, LLC

The right of Kao-Tai Tsai to be identified as author of this work has been asserted by him in accor-
dance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988.

Reasonable efforts have been made to publish reliable data and information, but the author and pub-
lisher cannot assume responsibility for the validity of all materials or the consequences of their use.
The authors and publishers have attempted to trace the copyright holders of all material reproduced
in this publication and apologize to copyright holders if permission to publish in this form has not
been obtained. If any copyright material has not been acknowledged please write and let us know so
we may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, access www.copyright.
com or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA
01923, 978-750-8400. For works that are not available on CCC please contact mpkbookspermis-
[email protected]

Trademark notice: Product or corporate names may be trademarks or registered trademarks and are
used only for identification and explanation without intent to infringe.

ISBN: 978-1-032-06536-6 (hbk)

ISBN: 978-1-032-07159-6 (pbk)
ISBN: 978-1-003-20568-5 (ebk)

Typeset in CMR10
by KnowledgeWorks Global Ltd.

DOI: 10.1201/9781003205685
Dedicated this book to:
My Parents
My Family
My Teachers
and My Brothers
Contents

Preface xiii

1 Data Analysis 1
1.1 Perspectives of Data Analysis . . . . . . . . . . . . . . . . . 1
1.2 Strategies and Stages of Data Analysis . . . . . . . . . . . . 3
1.3 Data Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3.1 Heterogeneity in Data Sources . . . . . . . . . . . . . 5
1.3.1.1 Heterogeneity in Study Subject Populations 5
1.3.1.2 Heterogeneity in Data due to Timing of Gen-
erations . . . . . . . . . . . . . . . . . . . . . 5
1.3.2 Noise Accumulation . . . . . . . . . . . . . . . . . . . 6
1.3.3 Spurious Correlation . . . . . . . . . . . . . . . . . . . 6
1.3.4 Missing Data . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 Data Sets Analyzed in This Book . . . . . . . . . . . . . . . 7
1.4.1 NCI-60 . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4.2 Riboflavin Production with Bacillus Subtilis . . . . . . 7
1.4.3 TCGA . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4.4 The Boston Housing Data Set . . . . . . . . . . . . . . 8

2 Examining Data Distribution 9

2.1 One Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.1 Histogram, Stem-and-Leaf, Density Plot . . . . . . . . 9
2.1.2 Box Plot . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.1.3 Quantile-Quantile (Q-Q) Plot , Normal Plot, Probability-
Probability (P-P) Plot . . . . . . . . . . . . . . . . . . 11
2.2 Two Dimension . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.1 Scatter Plot . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.2 Ellipse – Visualization of Covariance and Correlation . 13
2.2.3 Multivariate Normality Test . . . . . . . . . . . . . . . 17
2.3 More Than Two Dimension . . . . . . . . . . . . . . . . . . . 19
2.3.1 Scatter Plot Matrix . . . . . . . . . . . . . . . . . . . 19
2.3.2 Andrews0 s Plot . . . . . . . . . . . . . . . . . . . . . . 20
2.3.3 Conditional Plot . . . . . . . . . . . . . . . . . . . . . 23
2.4 Visualization of Categorical Data . . . . . . . . . . . . . . . 25
2.4.1 Mosaic Plot . . . . . . . . . . . . . . . . . . . . . . . . 26
2.4.2 Association Plot . . . . . . . . . . . . . . . . . . . . . 27

vii
viii Contents

3 Regressions 29
3.1 Ridge Regression . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.2 Lasso . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.2.1 Example: Lasso on Continuous Data . . . . . . . . . . 31
3.2.2 Example: Lasso on Binary Data . . . . . . . . . . . . 32
3.2.3 Example: Lasso on Survival Data . . . . . . . . . . . . 33
3.3 Group Lasso . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3.1 Example: Group Lasso on Gene Signatures . . . . . . 35
3.4 Sparse Group Lasso . . . . . . . . . . . . . . . . . . . . . . . 37
3.4.1 Example: Lasso, Group Lasso, Sparse Group Lasso on
Simulated Continuous Data . . . . . . . . . . . . . . . 38
3.4.2 Example: Lasso, Group Lasso, Sparse Group Lasso on
Gene Signatures Continuous Data . . . . . . . . . . . 41
3.5 Adaptive Lasso . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.5.1 Example: Adaptive Lasso on Continuous Data . . . . 46
3.5.2 Example: Adaptive Lasso on Binary Data . . . . . . . 47
3.6 Elastic Net . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.6.1 Example: Elastic Net on Continuous Data . . . . . . . 51
3.6.2 Example: Elastic Net on Binary Data . . . . . . . . . 52
3.7 The Sure Screening Method . . . . . . . . . . . . . . . . . . 53
3.7.1 The Sure Screening Method . . . . . . . . . . . . . . . 54
3.7.2 Sure Independence Screening on Model Selection . . . 55
3.7.3 Example: SIS on Continuous Data . . . . . . . . . . . 56
3.7.4 Example: SIS on Survival Data . . . . . . . . . . . . . 56
3.8 Identify Minimal Class of Models . . . . . . . . . . . . . . . 57
3.8.1 Analysis Using Minimal Models . . . . . . . . . . . . . 58

4 Recursive Partitioning Modeling 59

4.1 Recursive Partitioning Modeling via Trees . . . . . . . . . . 59
4.1.1 Elements of Growing a Tree . . . . . . . . . . . . . . . 59
4.1.1.1 Grow a Tree . . . . . . . . . . . . . . . . . . 60
4.1.2 The Impurity Function . . . . . . . . . . . . . . . . . . 60
4.1.2.1 Definition of Impurity Function . . . . . . . 61
4.1.2.2 Measure of Node Impurity – the Gini Index . 61
4.1.3 Misclassification Cost . . . . . . . . . . . . . . . . . . 61
4.1.4 Size of Trees . . . . . . . . . . . . . . . . . . . . . . . 62
4.1.5 Example of Recursive Partitioning . . . . . . . . . . . 63
4.1.5.1 Recursive Partitioning with Binary Outcomes 63
4.1.5.2 Recursive Partitioning with Continuous Out-
comes . . . . . . . . . . . . . . . . . . . . . . 65
4.1.5.3 Recursive Partitioning for Survival Outcomes 67
4.2 Random Forest . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.2.1 Mechanism of Action of Random Forests . . . . . . . . 72
4.2.2 Variable Importance . . . . . . . . . . . . . . . . . . . 72
4.2.3 Random Forests for Regression . . . . . . . . . . . . . 73
Contents ix

4.2.4 Example of Random Forest Data Analysis . . . . . . . 73

4.2.4.1 randomForest for Binary Data . . . . . . . . 73
4.2.4.2 randomForest for Continuous Data . . . . . . 76
4.3 Random Survival Forest . . . . . . . . . . . . . . . . . . . . . 77
4.3.1 Algorithm to Construct RSF . . . . . . . . . . . . . . 78
4.3.2 Individual and Ensemble Estimate at Terminal Nodes 79
4.3.3 VIMP . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.3.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.4 XGBoost: A Tree Boosting System . . . . . . . . . . . . . . . 81
4.4.1 Example Using xgboost for Data Analysis . . . . . . . 83
4.4.1.1 xgboost for Binary Data . . . . . . . . . . . 83
4.4.1.2 xgboost for Continuous Data . . . . . . . . . 84
4.4.2 Example – xgboost for Cox Regression . . . . . . . . . 87
4.5 Model-based Recursive Partitioning . . . . . . . . . . . . . . 88
4.5.1 The Recursive Partitioning Algorithm . . . . . . . . . 89
4.5.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.6 Recursive Partition for Longitudinal Data . . . . . . . . . . . 91
4.6.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . 91
4.6.2 Recursive Partition for Longitudinal Data Based on
Baseline Covariates . . . . . . . . . . . . . . . . . . . . 92
4.6.2.1 Methodology . . . . . . . . . . . . . . . . . . 92
4.6.3 LongCART Algorithm . . . . . . . . . . . . . . . . . . 93
4.6.4 Example of Recursive Partitioning of Longitudinal Data 93
4.7 Analysis of Ordinal Data . . . . . . . . . . . . . . . . . . . . 95
4.8 Examples – Analysis of Ordinal Data . . . . . . . . . . . . . 96
4.8.1 Analysis of Cleveland Clinic Heart Data (Ordinal) . . 96
4.8.2 Analysis of Cleveland Clinic Heart Data (Twoing) . . 97
4.9 Advantages and Disadvantages of Trees . . . . . . . . . . . . 99

5 Support Vector Machine 101

5.1 General Theory of Classification and Regression in Hyperplane 101
5.1.1 Separable Case . . . . . . . . . . . . . . . . . . . . . . 102
5.1.2 Non-separable Case . . . . . . . . . . . . . . . . . . . 102
5.1.2.1 Method of Stochastic Approximation . . . . 103
5.1.2.2 Method of Sigmoid Approximations . . . . . 103
5.1.2.3 Method of Radial Basis Functions . . . . . . 104
5.2 SVM for Indicator Functions . . . . . . . . . . . . . . . . . . 104
5.2.1 Optimal Hyperplane for Separable Data Sets . . . . . 104
5.2.1.1 Constructing the Optimal Hyperplane . . . . 105
5.2.2 Optimal Hyperplane for Non-Separable Sets . . . . . . 106
5.2.2.1 Generalization of the Optimal Hyperplane . 106
5.2.3 Support Vector Machine . . . . . . . . . . . . . . . . . 108
5.2.4 Constructing SVM . . . . . . . . . . . . . . . . . . . . 109
5.2.4.1 Polynomial Kernel Functions . . . . . . . . . 110
5.2.4.2 Radial Basis Kernel Functions . . . . . . . . 110
x Contents

5.2.5 Example: Analysis of Binary Classification Using SVM 110

5.2.6 Example: Effect of Kernel Selection . . . . . . . . . . 112
5.3 SVM for Continuous Data . . . . . . . . . . . . . . . . . . . 112
5.3.1 Minimizing the Risk with -insensitive Loss Functions 113
5.3.2 Example: Regression Analysis Using SVM . . . . . . . 115
5.4 SVM for Survival Data Analysis . . . . . . . . . . . . . . . . 117
5.4.1 Example: Analysis of Survival Data Using SVM . . . . 118
5.5 Feature Elimination for SVM . . . . . . . . . . . . . . . . . . 119
5.5.1 Example: Gene Selection via SVM with Feature Elimi-
nation . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
5.6 Spare Bayesian Learning with Relevance Vector Machine
(RVM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.6.1 Example: Regression Analysis Using RVM . . . . . . . 125
5.6.2 Example: Curve Fitting for SVM and RVM . . . . . . 125
5.7 SV Machines for Function Estimation . . . . . . . . . . . . . 127

6 Cluster Analysis 129

6.1 Measure of Distance/Dissimilarity . . . . . . . . . . . . . . . 129
6.1.1 Continuous Variables . . . . . . . . . . . . . . . . . . . 130
6.1.2 Binary and Categorical Variables . . . . . . . . . . . . 130
6.1.3 Mixed Data Types . . . . . . . . . . . . . . . . . . . . 130
6.1.4 Other Measure of Dissimilarity . . . . . . . . . . . . . 131
6.2 Hierarchical Clustering . . . . . . . . . . . . . . . . . . . . . 131
6.2.1 Options of Linkage . . . . . . . . . . . . . . . . . . . . 132
6.2.2 Example of Hierarchical Clustering . . . . . . . . . . . 133
6.3 K-means Cluster . . . . . . . . . . . . . . . . . . . . . . . . . 135
6.3.1 General Description of K-means Clustering . . . . . . 135
6.3.2 Estimating the Number of Clusters . . . . . . . . . . . 137
6.4 The PAM Clustering Algorithm . . . . . . . . . . . . . . . . 139
6.4.1 Example of K-means with PAM Clustering Algorithm 141
6.5 Bagged Clustering . . . . . . . . . . . . . . . . . . . . . . . . 141
6.5.1 Example of Bagged Clustering . . . . . . . . . . . . . 142
6.6 RandomForest for Clustering . . . . . . . . . . . . . . . . . . 144
6.6.1 Example: Random Forest for Clustering . . . . . . . . 144
6.7 Mixture Models/Model-based Cluster Analysis . . . . . . . . 145
6.8 Stability of Clusters . . . . . . . . . . . . . . . . . . . . . . . 147
6.9 Consensus Clustering . . . . . . . . . . . . . . . . . . . . . . 147
6.9.1 Determination of Clusters . . . . . . . . . . . . . . . . 148
6.9.2 Example of Consensus Clustering on RNA Sequence
Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
6.10 The Integrative Clustering Framework . . . . . . . . . . . . . 151
6.10.1 Example: Integrative Clustering . . . . . . . . . . . . . 152
Contents xi

7 Neural Network 155

7.1 General Theory of Neural Network . . . . . . . . . . . . . . . 155
7.2 Elemental Aspects and Structure of Artificial Neural Networks 156
7.3 Multilayer Perceptrons . . . . . . . . . . . . . . . . . . . . . 157
7.3.1 The Simple (Single Unit) Perceptron . . . . . . . . . . 157
7.3.2 Training Perceptron Learning . . . . . . . . . . . . . . 157
7.4 Multilayer Perceptrons (MLP) . . . . . . . . . . . . . . . . . 158
7.4.1 Architectures of MLP . . . . . . . . . . . . . . . . . . 158
7.4.2 Training MLP . . . . . . . . . . . . . . . . . . . . . . 159
7.5 Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . 159
7.5.1 Model Parameterization . . . . . . . . . . . . . . . . . 160
7.6 Few Pros and Cons of Neural Networks . . . . . . . . . . . . 161
7.7 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

8 Causal Inference and Matching 173

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
8.2 Three Layer Causal Hierarchy . . . . . . . . . . . . . . . . . 173
8.3 Seven Tools of Causal Inference . . . . . . . . . . . . . . . . 174
8.4 Statistical Framework of Causal Inferences . . . . . . . . . . 176
8.5 Propensity Score . . . . . . . . . . . . . . . . . . . . . . . . . 177
8.6 Methodologies of Matching . . . . . . . . . . . . . . . . . . . 178
8.6.1 Nearest Neighbor (or greedy) Matching . . . . . . . . 178
8.6.1.1 Example Using Nearest Neighbor Matching . 178
8.6.2 Exact Matching . . . . . . . . . . . . . . . . . . . . . . 180
8.6.2.1 Example . . . . . . . . . . . . . . . . . . . . 180
8.6.3 Mahalanobis Distance Matching . . . . . . . . . . . . 181
8.6.3.1 Example . . . . . . . . . . . . . . . . . . . . 181
8.6.4 Genetic Matching . . . . . . . . . . . . . . . . . . . . 182
8.6.4.1 Example . . . . . . . . . . . . . . . . . . . . 183
8.7 Optimal Matching . . . . . . . . . . . . . . . . . . . . . . . . 184
8.7.0.1 Example . . . . . . . . . . . . . . . . . . . . 185
8.8 Full Matching . . . . . . . . . . . . . . . . . . . . . . . . . . 186
8.8.0.1 Example . . . . . . . . . . . . . . . . . . . . 187
8.8.1 Analysis of Data After Matching . . . . . . . . . . . . 188
8.8.1.1 Example . . . . . . . . . . . . . . . . . . . . 189
8.9 Cluster Matching . . . . . . . . . . . . . . . . . . . . . . . . 191
8.9.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . 192

9 Business 197
9.1 Case Study One: Marketing Campaigns of a Portuguese Bank-
ing Institution . . . . . . . . . . . . . . . . . . . . . . . . . . 197
9.1.1 Description of Data . . . . . . . . . . . . . . . . . . . 197
9.1.2 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . 198
9.1.2.1 Analysis via Lasso . . . . . . . . . . . . . . . 198
9.1.2.2 Analysis via Elastic Net . . . . . . . . . . . . 198
xii Contents

9.1.2.3 Analysis via SIS . . . . . . . . . . . . . . . . 199

9.1.2.4 Analysis via rpart . . . . . . . . . . . . . . . 200
9.1.2.5 Analysis via randomForest . . . . . . . . . . 200
9.1.2.6 Analysis via xgboost . . . . . . . . . . . . . . 202
9.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
9.3 Case Study Two: Polish Companies Bankruptcy Data . . . . 204
9.3.1 Description of Data . . . . . . . . . . . . . . . . . . . 204
9.3.2 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . 206
9.3.2.1 Analysis of Year-1 Data (univariate analysis) 207
9.3.2.2 Analysis of Year-3 Data (univariate analysis) 209
9.3.2.3 Analysis of Year-5 Data (univariate analysis) 210
9.3.2.4 Analysis of Year-1 Data (composite analysis) 212
9.3.2.5 Analysis of Year-3 Data (composite analysis) 214
9.3.2.6 Analysis of Year-5 Data (composite analysis) 216
9.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

10 Analysis of Response Profiles 221

10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
10.2 Data Example . . . . . . . . . . . . . . . . . . . . . . . . . . 221
10.3 Transition of Response States . . . . . . . . . . . . . . . . . 224
10.4 Classification of Response Profiles . . . . . . . . . . . . . . . 225
10.4.1 Dissimilarities Between Response Profiles . . . . . . . 225
10.4.2 Visualizing Clusters via Multidimensional Scaling . . . 226
10.4.3 Response Profile Differences among Clusters . . . . . . 227
10.4.4 Significant Clinical Variables for Each Cluster . . . . . 228
10.5 Modeling of Response Profiles via GEE . . . . . . . . . . . . 230
10.5.1 Marginal Models . . . . . . . . . . . . . . . . . . . . . 230
10.5.2 Estimation of Marginal Regression Parameters . . . . 231
10.5.3 Local Odds Ratio . . . . . . . . . . . . . . . . . . . . . 231
10.5.4 Results of Modeling . . . . . . . . . . . . . . . . . . . 231
10.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Bibliography 235

Index 243
Preface

Data analysis has a long history. Actually, the Kingdom of Sumer (Babylonia
c. 4500-1900 BC) is a famous example of where initial records of written data
analysis have been found. Through human history, data analysis had become
the foundation for scientific discoveries, advances of medicine, and government
policies.
Due to the advances of sciences, business, and computing technologies,
etc., data of medium to huge sizes have become ubiquitous. In addition, with
the emphasis of “everything evidence-based,” how to extract valid, useful,
and actionable information is more critical than ever and heavily depends on
well-done data analysis. In a broader scope of data analysis, analyst needs
to consider the relevance of population the data was from and whether it
fits the objectives of the analysis (ref. Mallows). In the ICSA presentation
(2010), I also emphasized the importance of experimental design, rigorous
data collection plan to ensure the data quality, the plan to implement the
analytical findings, and potential revision of the original questions. This cycle
can often repeat itself a few times.
Data analysis should not be just a pure mechanical exercise following math-
ematical models or canned programs, instead, data analysis very often involves
tedious and careful effort to examine the nature of data, including its distribu-
tion, homogeneity, correlation, outliers, and other complex internal structure,
etc. before any effort of estimation or modeling, especially in large data sets
(ref. Chapter 1).
Analysis of large real-world data sets is often very complex and requires a
team with multi-discipline. Therefore, data analysts must look to a heavy em-
phasis on judgment about the subject-matter experience of the field the data
being considered, and the broad experience of the analytical techniques have
worked out in a variety fields of application and its properties (ref. Tukey).
In addition, rather than spend great effort to pursue the “optimal/best” solu-
tions, the time can be better used in real research (ref. Kimball). As professor
Box once said, “all models are wrong, but some are useful.” Similarly echoed
by Tukey, “Far better an approximate answer to the right question, which is
often vague, than an exact answer to the wrong question, which can always
be made precise.”
The contents of the book very much reflect the principles described above.
I emphasize the importance of understanding the data before any attempt of
analysis, which I learned from Mallows (Chapters 1 and 2). Since regression
forms the foundation of many modeling, I include it herein (Chapter 3) with

xiii
xiv Preface

a lot more varieties which can’t ever be inclusive. Even though parsimonious
model derived from shrinkage is important and convenient, one needs to be
careful about its purpose. For example, in genomics, gene-groups with collab-
orated functions often are highly correlated, to select only one or two of them
can mask their importance.
Recursive partition of data (Chapter 4) has becoming ever more important
in the presence of large data with heterogeneity of subject sub-populations
therein. It is rarely that one can have one overall model/estimate to describe
the data well, however, due to the large sample size, any hypothesis test would
become significant even though the effect could be minuscule. Many analysts
use this method to identify important variables and sub-populations, however,
it is important to “confirm/validate” the findings with other methods for
similar purposes (ref. Tukey).
Chapter 5 is about Support Vector Machine which I also had learned when
I was at AT&T Bell Labs, however, it was not easy to use it for real large
data in those days due to the limitation of computing power. It deals with
data from a different angle by trying to identify the important support vectors
instead of variables. With the current capabilities of computing and feature se-
lection as well as the incorporation of kernels, it has become quite flexible and
provide another approach to discriminate or sub-setting the data population.
Graphical presentation of the results is still quite challenging.
Cluster analysis (Chapter 6) has been commonly used in genomics research
even though it is not as commonly used to analyze clinical trial data. Bioin-
formaticians had extended this into more clustering varieties, however, the
final clusters still very much depend on the metrics and linkage methods. It is
prudent to conduct analysis using various linkage methods and discuss with
the subject-matter experts for the proper selection and interpretation.
Neural network analysis (Chapter 7) is quite useful or controversial de-
pending on the problems and the environments where the results are intended
to be used. It can construct a network for high degree of prediction, but it
can also be very difficult to interpretate except for the simple networks. It
goes against the principle of parsimonious model and, therefore, how to use
the model for a new data set can be quite challenging.
As many are aware that most of the data analysis, especially in regressions,
the relationship between outcomes and covariates are mostly for association,
and rarely can be interpreted as a causal-effect relationship. This is especially
challenging for the observational studies. Professors Rubin and colleagues of
Statistics, and Pearl and colleagues of Computer Science had proposed var-
ious approaches to address this topic. Chapter 8 is devoted for this topic
by describing various methods including matching proposed by researchers.
This has become one of the important tools in data analysis when the control
group of the experiment is unobtainable due to either ethic or other reasons.
Therefore, how to construct a proper well-matched control group and proper
estimation of treatment effect remains as an active area of research.
Preface xv

I included in Chapters 9 and 10 two case studies. Chapter 9 analyzes the

bankruptcies of financial institutes based on many financial performance met-
rics. It has a “longitudinal data flavor” even though no institution identities
were available (due to privacy concern), however, with the large number of in-
stitutes included, the factors affecting bankruptcies should be consistent and
informative. Chapter 10 analyzes patient treatment profile based on various
treatment cycles and its relationship to the final outcomes. It is a true lon-
gitudinal data set with each subject having its own observational series. It
is interesting to see how the profiles related to the final outcome, which is
usually not attainable by analyze the outcome from each individual cycle.
The examples shown in the book were all programed using R language. The
important codes are all included in the book; therefore, it should be quite easy
to follow. I can also be reached through my email ([email protected])
for the questions.
This book would be impossible without the help of many of my teachers,
friends, and colleagues of the various organizations, including UC San Diego,
AT&T Bell Labs, and various pharmaceutical companies I had worked. I would
send all of them my sincere appreciation. In addition, without the support of
my family, I would not have time to do my research and finish this writing,
similarly, I would like to send them my deepest gratitude.
To err is human, I am sure one can find many errors in the book and I
am fully responsible for that. I will try to make my best effort to correct any
errors which anyone would kindly let me know.

Kao-Tai Tsai
Berkeley Heights, New Jersey, USA
March 2021
1
Statistical Data Analysis

Statistical data analysis has a long history and, over the years, many great
statisticians had discussed how it should be properly conducted. The topic
has become more important nowadays due to the overwhelming speed of data
generation, and how to properly extract meaningful and correct information
from massive data has become ever more complicated. In the following, I
briefly described the discussions from Fisher, Tukey, Huber, and Mallows of
their views of this topic. It is obvious that it is impossible to be exclusive,
however, these discussions will at least give us some important perspectives

1.1 Perspectives of Data Analysis

In his paper “On the Mathematical Foundations of Theoretical Statistics,”
Fisher [21] stated, “... the object of statistical methods is the reduction of
data.... [This] is accomplished by constructing a hypothetical infinite popula-
tion of which the actual data are regarded as constituting a random sample.”
He then identified three problems, which arise in the processes of data reduc-
tion:
1. Problems of Specification. These arise in the choice of the mathe-
matical form of the population.
2. Problems of Estimation. These involve the choice of methods of
calculating statistics, which are designed to estimate the values of
the parameters of the hypothetical population.
3. Problems of Distribution. These include discussions of the distribu-
tion of statistics derived from samples.
He assumed that one can capture the essence of the real-world problem in
a specification of model(s), which is assumed to be known except for some
parameters to be estimated from data. His thinking had substantial influence
on the development of statistics, especially mathematical statistics.
Tukey, in his monumental paper “The Future of Data Analysis” [90],
stated: “For a long time I have thought I was a statistician, interested
in inferences from the particular to the general. But as I have watched

DOI: 10.1201/9781003205685-1 1
2 Statistical Data Analysis

mathematical statistics evolve, I have had cause to wonder and to doubt. And
when I have pondered about why such techniques as the spectrum analysis of
time series have proved so useful, it has become clear that their “dealing with
fluctuations” aspects are, in many circumstances, of lesser importance than
the aspects that would already have been required to deal effectively with the
simpler case of very extensive data, where fluctuations would no longer be
a problem. All in all, I have come to feel that my central interest is in data
analysis, which I take to include, among other things: procedures for analyz-
ing data, techniques for interpreting the results of such procedures, ways of
planning the gathering of data to make its analysis easier, more precise or
more accurate, and all the machinery and results of (mathematical) statistics
which apply to analyzing data.”
Tukey further stated, “Large parts of data analysis are inferential in the
sample-to-population sense, . . . Large parts of data analysis are incisive, laying
bare indications which we could not perceive by simple and direct examination
of the raw data, . . . Some parts of data analysis, . . . , are allocation, in the
sense that they guide us in the distribution of effort and other valuable consid-
erations in observation, experimentation, or analysis. Data analysis is a larger
and more varied field than inference, or incisive procedures, or allocation.”
He further stated that data analysis must look to a very heavy emphasis on
judgment. He considered at least three different sorts or sources of judgment
that are likely to be involved in almost every instance: (1) judgment based
upon the experience of the particular field of subject matter from which the
data come, (2) judgment based upon a broad experience with how particular
techniques of data analysis have worked out in a variety of fields of applica-
tion, and (3) judgment based upon abstract results about the properties of
particular techniques, whether obtained by mathematical proofs or empirical
sampling. And it is “Far better an approximate answer to the right question,
which is often vague, than an exact answer to the wrong question, which can
always be made precise.”
In his Fisher memorial lecture, Mallows [51], being a distinguished statisti-
cian at the AT&T Bell laboratories, who worked with scientists and engineers
from various disciplines, had emphasized that there are problems that precede
Fisher’s, namely deciding what the relevant population is, what the relevant
data are, and just how these relate to the purpose of the statistical study, in
addition to choosing what problem to study.
This kind of thinking has been a required practice in many scientific in-
dustries, such as in epidemiology, clinical trials for drug discovery, etc. One
often analyzes the observational data, which could be heavily confounded by
unobserved factors. Then further validates the findings by conducting trials
with rigorous experimental designs, which clearly specify the population of
interest and the objectives of study, as well as the specifications of procedures
for trial conducts.
Mallows hence integrated the thinking from Fisher and Tukey and sum-
marized the steps of data analysis into five problems by defining the Zeroth
Strategies and Stages of Data Analysis 3

Problem: “Considering the relevance of the observed data, and other data that
might be observed, to the substantive problem” before Fisher’s specification
of models and the Fourth Problem regarding the presentation of conclusions
from data analysis. He also stated that “statistical thinking concerns the re-
lation of quantitative data to a real-world problem, often in the presence of
variability and uncertainty. It attempts to make precise and explicit what the
data have to say about the problem of interest.”

1.2 Strategies and Stages of Data Analysis

As emphasized by professor Peter Huber, the need for strategic thinking in
data analysis is imposed on us by the nature of ever larger data sets. Data an-
alysts need to consider not just the size, but also the facts that large databases
are less homogeneous and have more complex internal structure. Data which is
heterogeneous in precision, in variation and inevitable incomplete, all impose
great challenges to analysts. Data analysis is an iterative and tedious work;
therefore, it is important to focus on the important tasks and be flexible in
conduct to avoid data dredging to waste time and effort
There is a broad range in the conduct of data analysis. It begins with the
identification of the problem of interest, specification of the analysis protocol,
and plan including the potential statistical models of interest; henceforward
to plan the data collection with good data quality control.
For any extensive and complicated projects, a team of multidisciplinary
subject-matter experts is very important so that the various aspects of the
project can be explored to avoid the potential difficulties in execution and to
produce valid analytical results.
For the planning of data collection, data analysts need to get involved
if possible because this will most likely impact the data quality and make
later analysis more efficient and valid. Interim data check during the collec-
tion processes is important to make sure the collection processes are followed
correctly, the intended data are collected, and mistakes are corrected before a
great amount of effort and resources are wasted.
For the scenario that data are already existent, it is critical for data ana-
lysts to carefully examine the meta-data if available, which usually describes
the contents of the data in detail, so that the database can be better under-
stood before planning the data analysis.
Most of the real-world projects have various assumptions for the unknown
factors; therefore, it is important to conduct sensitivity analysis (or the so-
called what-if analysis) after initial statistical models are fitted so that the
impacts of the assumptions can be tested. It is also important to compare the
results with other related findings from other research to check the validity of
4 Statistical Data Analysis

the findings. As professor George Box once said that no model is correct, but
some are useful; therefore, flexibility in data modeling is always recommended.
To present the final results, the well thought through graphs and tables is
definitely preferred to lines and lines of text. Be careful not to overly crowd the
contents of the graphs and tables to the point the messages are lost, and the
readers are confused. The steps described above can be tactically summarized
as in Figure 1.1.

FIGURE 1.1
Data analysis strategies.

1.3 Data Quality

Data with good quality are the ingredient for analysis. There are many defi-
nitions of data quality, however, in engineering, the general scope includes
• Degree of excellence exhibited by the data in relation to the portrayal of the
actual scenario.

• The state of completeness, standards based, validity, consistency, timeliness,

and accuracy that make data appropriate for a specific use.
Data Quality 5

• The totality of features and characteristics of data that bear on its ability
to satisfy a given purpose: the sum of the degrees of excellence for factors
related to data.
• The processes and technologies involved in ensuring the conformance of data
values to business requirements and acceptance criteria.
In essence, data quality is the degree to which it is complete, valid, accu-
rate, consistent, and timely. In statistics, it adds several other complications,
especially in big data.
Big data promises new levels of scientific discovery and economic value.
The general expectation of big sample size is that it may give better oppor-
tunities to explore the hidden structures of each subpopulation of the data,
which is traditionally not as easy when the sample size is small, and to iden-
tify important common or uncommon features across many sub populations.
However, these expectations are yet to be realized in some greater degrees,
primarily due to the data quality and capacity of the analysts. In the follow-
ing, we describe some common challenges usually facing the analysts during
the course of data analysis.

1.3.1 Heterogeneity in Data Sources

Unlike the well-designed experiments that collect well-controlled data, most
of the big data are observational and coming from aggregation of several data
sets. For example, some data may come from well-controlled studies, some
from observational studies, and some could be from past studies, etc. This
difference may reflect certain degrees of data quality heterogeneity.

1.3.1.1 Heterogeneity in Study Subject Populations

Some heterogeneity may include differences in subject populations, or regional
and cultural differences, the time of data generations. For example, in clinical
trials, study subjects can include various demographics such as age, gender,
medical histories, genotypes, and phenotypes, etc. Even though the pharma-
ceutical companies try to have a more homogeneous subject population by
imposing restrictive inclusion and exclusion criteria, the population homo-
geneity still cannot be guaranteed.

1.3.1.2 Heterogeneity in Data due to Timing of Generations

For example, due to the advance of medical research and increased biological
understanding of diseases in the past decades, the medical treatment practices
during later more advanced days may be very different from the treatments
years ago for the same disease. These are commonly seen in the treatment
of cancers, heart disease, immune system disorder, etc.; therefore, the data
collected from different periods may post substantial amounts of differences
6 Statistical Data Analysis

in various aspects. Whether and how these kinds of data should be aggregated
for analysis of treatment effects is a challenging question.

1.3.2 Noise Accumulation

Every data sample is associated with its own distribution and random error.
The precision of the estimation of the parameters is affected by these random
errors. These estimation errors accumulate when a model or prediction rule
depends on a large number of such parameters. Researchers had suggested to
alleviate the noise issue and to increase the precision involving less variables
by various model assumptions and variable selections. However, variable se-
lection in high dimensions is challenging due to spurious correlation, which is
described in the next section, heterogeneity, and measurement errors, etc.

1.3.3 Spurious Correlation

It is well known that many uncorrelated random variables may have acciden-
tal high sample correlations in high dimensional data. The correlations may
be real or incidental; however, given the large number of dimensions, it is very
difficult to examine them sufficiently. The spurious correlation has significant
impact on variable selection and may lead to false scientific discoveries. Be-
sides variable selection, spurious correlation may also lead to wrong statistical
inference. To cope with the noise accumulation issue, when the dimension d is
comparable to or larger than the sample size n, it is popular to assume that
only a small number of variables contribute to the response, i.e. the set of
unknown parameters is a sparse vector.

1.3.4 Missing Data

Missing data are part of data collections and the problems can be more acute if
data are collected over a long period of time. Even for big data, there is still the
missing data problem. When p >> n with moderate or small n, any missing
data of the p variables from the n subjects will create the same problem as
that in the moderate or small sample data. Missing data imputation may still
be performed; however, it may not be as straightforward as the practices for
data with smaller p due to the correlation among the variables and the issues
can arise if the correlations are spurious. In addition, the large dimension and
invertibility of the covariance matrix of the variables may post some additional
challenges for imputations.
Data Sets Analyzed in This Book 7

1.4 Data Sets Analyzed in This Book

1.4.1 NCI-60
The NCI-60 cancer cell line panel is a group of 60 human cancer cell lines
used by the National Cancer Institute (NCI) for the screening of compounds
to detect potential anticancer activity. The screening procedure is called the
NCI-60 Human Tumor Cell Lines Screen, and it is one of the Discovery and
Development Services of NCI’s Developmental Therapeutics Program(DTP).
Due to the diversity of the cell lines, it is possible to compare tested com-
pounds by their effect patterns, high correlation potentially corresponding to
similar effect mechanisms. The same panel is used in the Molecular Target
Program for the characterization of molecular targets. Measurements include
protein levels, RNA measurements, mutation status, and enzyme activity lev-
els. The panel holds cell lines representing leukemia, melanoma, non-small-cell
lung carcinoma, and cancers of the brain, ovary, breast, colon, kidney, and
prostate. The NCI60 screen supported the development of several anticancer
drugs, such as paclitaxel and bortezomib, which were approved by the US
Food and Drug Administration (FDA) for cancer treatment (26, 27). Data
mining on the NCI60 screen result also led to many findings. For example,
association analysis between gene mutation status and drug efficacy on the
NCI60 panel discovered that the BRAF mutation is a predictor of MEK in-
hibitor sensitivity.

1.4.2 Riboflavin Production with Bacillus Subtilis

This is a data set about riboflavin (vitamin B2) production with B. subtilis
provided by DSM (Kaiseraugst, Switzerland) and can be found at the following
website: http://www.annualreviews.org. There is a single real-valued response
variable, which is the logarithm of the riboflavin production rate with p = 4088
covariates measure the logarithm of the expression level of 4088 genes. These
gene expressions were normalized. It is a rather homogeneous data set for n =
71 samples that were hybridized repeatedly during a fed-batch fermentation
process in which different engineered strains and strains grown under different
fermentation conditions were analyzed. This data set is denoted as riboflavin.

1.4.3 TCGA
The Cancer Genome Atlas (TCGA), a landmark cancer genomics program,
https://www.cancer.gov/about-nci/organization/ccg/research/structural-ge-
nomics/tcga, molecularly characterized over 20,000 primary cancer and
matched normal samples spanning 33 cancer types. This joint effort between
the National Cancer Institute and the National Human Genome Research In-
stitute began in 2006, bringing together researchers from diverse disciplines
8 Statistical Data Analysis

and multiple institutions. Over the next dozen years, TCGA generated over
2.5 petabytes of genomic, epigenomic, transcriptomic, and proteomic data.
The data, which have already lead to improvements in our ability to diag-
nose, treat, and prevent cancer, will remain publicly available for anyone in
the research community to use.

1.4.4 The Boston Housing Data Set

A data set derived from information collected by the U. Census Service con-
cerning housing in the area of Boston, Mass. It was obtained from the StatLib
archive (http://lib.stat.cmu.edu/datasets/boston), and has been used exten-
sively throughout the literature to benchmark algorithms. The data set is
small in size with only 506 cases. The data were originally published by Har-
rison, D. and Rubinfeld, D.L. “Hedonic prices and the demand for clean air,”
J. Environ. Economics & Management, vol.5, 81–102, 1978. There are 14 at-
tributes in each case of the data set. They are CRIM (per capita crime rate
by town), ZN (proportion of residential land zoned for lots over 25,000 sq.ft.),
INDUS (proportion of non-retail business acres per town), CHAS (Charles
River dummy variable (1 if tract bounds river; 0 otherwise)), NOX (nitric ox-
ides concentration (parts per 10 million)), RM (average number of rooms per
dwelling), AGE (proportion of owner-occupied units built prior to 1940), DIS
(weighted distances to five Boston employment centers), RAD (index of acces-
sibility to radial highways), TAX (full-value property-tax rate per $10,000),
PTRATIO (pupil-teacher ratio by town), B (1000(Bk − 0.63)2 , where Bk is
the proportion of blacks by town), LSTAT (% lower status of the population),
and MEDV (median value of owner-occupied homes in $1000s).
In the following chapters, we will present various methods which were
proposed by researchers with the effort to cope with various data issues and
also to produce meaningful and statistical valid results for the interpretations.
2
Examining Data Distribution

As professor Tukey once said that every data set is unique and one should
understand the characteristics of the data before attempting to produce sum-
maries or conduct statistical modeling, and the best ways to do that are via
graphical methods. Professor Huber echoed this philosophy and indicated that
graphical methods for exploratory data analysis are among the natural evo-
lution of statistical data analysis (see “The evolving spiral path of statistics”
in Huber (1997) [38])
In this chapter, we describe some basic tools to examine the data distri-
butions for various dimensions. It is obvious that the following descriptions
cannot be exhaustive and there are many other useful tools for special pur-
poses that do exist in various monographs and literature. Nevertheless, this
is a starting point and the graphs can all be easily produced using the many
packages in R.

2.1 One Dimension

2.1.1 Histogram, Stem-and-Leaf, Density Plot
Histogram, Stem-and-Leaf, and density plot are among the most commonly
used methods to examine the univariate distribution of any continuous vari-
able. This is the first step of any data analysis, especially for statistical mod-
eling, which assumes the data would follow a certain kind of distribution.
In histogram and density plot, one has to decide the number of categories
to divide the data. Too many or too few categories will make the histogram
unable to show the characteristic of the distribution. Proposals exist in the
literature for the optimal bandwidth and most of the software packages had
these functions built-in already, but analysts may want to define the band-
width for special purposes of their analysis.
Stem-and-Leaf is another version of histogram with the actual data values
being displayed in the plot. One advantage of Stem-and-Leaf plot is the ease
of reading data values from the data. In addition, one does not need to decide
the number of categories to display as in the regular histogram.

DOI: 10.1201/9781003205685-2 9
10 Examining Data Distribution

FIGURE 2.1
Box plot with variable width, notch, and w/o outliers identity.

2.1.2 Box Plot

Box plot is among one of the most used graphical methods to summarize
the distribution of the univariate data. It provides five important summary
statistics of the data distribution: median, lower quartile (25th percentile),
upper quartile (75th percentile), and lower and upper hinges (or whiskers).
One can specify the range in the plot to determine how far the plot whiskers
extend out from the box. In general, the whiskers extend to the most extreme
data point, which is no more than 1.5 times the interquartile range from the
box. The data outside the range of whiskers can be considered as outliers.
Options exist to make the box plot more informative. For example, for mul-
tiple box plotted together in the same graph, if the option varwidth=TRUE, the
boxes are drawn with widths proportional to the square roots of the number
of observations in the groups. Another useful option is notch. If notch=TRUE,
a notch is drawn in each side of the boxes. If the notches of two plots do not
overlap, which is “strong evidence” that the two medians differ as indicated
by Chambers et al. [12]. The examples are shown in Figure 2.1. One can also
produce a horizontal box plot by specifying horizontal=TRUE. For example,
the following simple codes produce the box plots described above.
library(MASS)
library(car)
Pim<-Pima.tr2
Boxplot(glu~type, data=Pima.tr2, xlab="", ylab="")
title(xlab="Diabetes Status", ylab="Glucose Levels") #left panel
One Dimension 11

Boxplot(glu~type, data=Pima.tr2, xlab="", ylab="",

notch=TRUE, varwidth=TRUE, id.n=10)
title(xlab="Diabetes Status", ylab="Glucose Levels") #right panel

2.1.3 Quantile-Quantile (Q-Q) Plot , Normal Plot,

Probability-Probability (P-P) Plot
In statistical inference to compare samples, it is important to know whether
the samples follow similar distributions or even have the same distributions.
Q-Q plot is a useful tool that provides insight of the distributional difference
by examining the whole spectrum of the distributions. In contrast, most of
the statistical tests for the equality of distributions, such as the commonly
used normality test or Kolmogorov-Smirnov two-sample test, only utilize the
first few moments.
Q-Q plot does not require equal sample sizes between samples. In addition,
many distributional aspects can be simultaneously tested. For example, shifts
in location, shifts in scale, changes in symmetry, and the presence of outliers
can all be detected from this plot. For example, the two data sets come from
populations whose distributions differ only by a shift in location, the points
should lie along a straight line that is displaced either up or down from the 45-
degree reference line. If the samples are different in variances, the plot should
be tilted away from the 45-degree line. The difference in tail distributions can
also be detected.
More specifically, let F (x) and G(y) be the distributions of these two vari-
ables, and 0 < p < 1 be the probability, then Q-Q plot is constructed by
plotting
{(F −1 (p), G−1 (p))}, 0 < p < 1. (2.1)
Normal plot is a special case of Q-Q plot when one is interested to see
whether the distribution of a data sample is normal. The normal plot can be
constructed by potting

{(Φ−1 (p), G−1 (p))}, 0 < p < 1, (2.2)

where Φ is the normal distribution function.

The P-P plot is another graph method which plots the percentile of one
sample distribution versus the percentile of the other sample distribution.
Specifically, the P-P plot can be constructed by

{(p, F (G−1 (p)))}, 0 < p < 1. (2.3)

Similarly, a normal probability plot is a special case of a P-P plot when

one is interested to see whether the distribution of a variable is normal. The
normal probability plot can be constructed by potting

{(p, F (Φ−1 (p)))}, 0 < p < 1. (2.4)

Even though P-P plot is less commonly used by the practitioners; however,
12 Examining Data Distribution

FIGURE 2.2
Q-Q plot of glucose levels for patients w/o diabetes.

the P-P plot can avoid some inferential difficulties posed by the Q-Q plot. More
details can be found in Holmgren [17]. The following codes create (Figure 2.2).

library(MASS)
Pim<-Pima.tr2
x<-Pim[Pim$type=="Yes", 2]
y<-Pim[Pim$type=="No", 2]
qqplot(x,y, xlab="", ylab="")
abline(0,1, col="red")
title(xlab="Glucose levels of Patients with Diabetes",
ylab="Glucose levels of Patients without Diabetes")

2.2 Two Dimension

2.2.1 Scatter Plot
The scatter diagram is one of the very useful tools of any data analysis. Scat-
ter plots can show how much one variable is affected by another. A scatter
Two Dimension 13

plot can often be employed to identify potential associations between two

variables, where one may be considered to be an explanatory variable and an-
other may be considered a response variable. A positive association between
the variables would be indicated on a scatter plot by a upward trend (positive
slope), similarly, a negative association would be indicated by the opposite ef-
fect (negative slope), or, there might not be any notable association, in which
case a scatter plot would not indicate any trends whatsoever.
An equation for the correlation between the variables can be estimated by
some best fit procedures. For example, the linear correlation can be demon-
strated by using linear regression. One of the most powerful aspects of a scatter
plot, however, is its ability to show nonlinear relationships between variables.
The lowess function (a locally weighted regression) in R is a very useful tool
to identify the relationship. Furthermore, if the data are represented by a mix-
ture model of simple relationships, these relationships will be visually evident
as superimposed patterns. Even though a scatter plot can be easily produced
using the command plot(x, y) for two variables x and y; however, the pack-
age car provides a collection of more comprehensive plots, which can add the
smooth line and the confidence band, as well as the box plot for each variable,
as shown in Figures 2.3 and 2.4.
library(MASS)
library(car)
scatterplot(medv ~ rm, data=Boston,
smooth=FALSE, ellipse=FALSE, reg.line=FALSE, boxplots=FALSE,
xlab="number of rooms", ylab="value")
title(main="Scatterplot of Boston Housing Data")

scatterplot(medv ~ rm, data=Boston,

smooth=TRUE, ellipse=FALSE, reg.line=FALSE, boxplots=FALSE,
xlab="number of rooms", ylab="value")

2.2.2 Ellipse – Visualization of Covariance and Correlation

In a linear model, a confidence region can be expressed as
(β − β̂)0 X 0 X(β − β̂) ≤ s2 pF (p, N − p, α), (2.5)
where s is the empirical variance estimated from the data and F (p, N − p, α)
is the critical value of the F distribution with (p, N − p) degrees of freedom.
In the multivariate-dimensional case, one can approximate the inference
region for the parameters by
(θ − θ̂)0 V̂ 0 V̂ (θ − θ̂) ≤ s2 pF (p, N − p, α). (2.6)
For the bivariate case, the inference region can be calculated and drew ac-
cording to the following relationship:
(x, y) = (cos(θ + d/2), cos(θ − d/2)), (2.7)
14 Examining Data Distribution

FIGURE 2.3 FIGURE 2.4

Simple 2-d scatter plot of two vari- with smooth line and confidence
ables. band.

with cos(d) = ρ, the correlation of x and y.

Using the Cars data mtcars of the MASS package in R, Figure 2.5 shows
the original data plots with the ellipses of the estimated parameters about the
data center and the corresponding 95% confidence regions using the estimated
covariance matrix. The covariance matrix can be estimated either using the
least squares method or robust methods to down-weight the possible outliers.
The shape and direction of the ellipse indicates the relative magnitude of
the variances between the variables, and the positive (north-east direction) or
negative (north-west direction) correlation of the estimated parameters.
Figure 2.5 shows the data ellipses, with least squares and robust estimate
of covariance matrix, respectively, of the cars data that is created by the
following codes. From the graph, one can easily see the positive correlation
between these two variables and the magnitude of the variances.

library(MASS)
library(car)
data(mtcars)
dataEllipse(mtcars[,3], mtcars[,4], levels=c(0.5, 0.95), center.pch=19,
center.cex=1.5, xlab="Displacement", ylab="Horse Power")

dataEllipse(mtcars[,3], mtcars[,4], levels=c(0.5, 0.95), center.pch=19,

center.cex=1.5, robust=TRUE, xlab="Displacement", ylab="Horse Power")

One can also fit the variable mpg by the variables disp, cyl, and the
covariance matrix of the estimated parameters for these variables can also
Two Dimension 15

FIGURE 2.5
Least squares (left) and robust (right) estimates of data ellipse of cars data.

be graphed as shown in Figure 2.6. Similarly, one can easily see the nega-
tive correlation and the relative magnitude of the variance between these two
variables. The graph is produced by the following codes.

fit <- lm(mpg ~ disp + cyl , mtcars)

confidenceEllipse(fit, which.coef= c(’disp’, ’cyl’), levels=0.95,
Scheffe=FALSE, center.pch=19, center.cex=1.5, segments=51,
col=palette()[2], lwd=2, fill=FALSE, fill.alpha=0.3, draw=TRUE,
xlab="Displacement", ylab="Horse Power")

print("Variance-Covariance Matrix of the Estimated Parameters")

print(vcov(fit))
[1] "Variance-Covariance Matrix of the Estimated Parameters"
(Intercept) disp cyl
(Intercept) 6.48722874 0.0164777387 -1.615718386
disp 0.01647774 0.0001052158 -0.006586396
cyl -1.61571839 -0.0065863960 0.506722267

When the dependent variable is discrete, one can perform similar analytical
steps to get the data ellipse and the ellipse for the estimated parameters
via generalized linear modeling. We use the Pima Indian Diabetes data to
illustrate the procedures. The following sample codes produce the data ellipse
as shown in Figures 2.7.

library(MASS)
Pim<-Pima.tr2
16 Examining Data Distribution

FIGURE 2.6
Covariance ellipse of the estimated parameters of cars data.

dataEllipse(Pim$glu, Pim$age, levels=c(0.5, 0.95), center.pch=19,

center.cex=1.5, xlab="Glucose Level", ylab="Age")

dataEllipse(Pim$glu, Pim$age, levels=c(0.5, 0.95), center.pch=19,

center.cex=1.5, robust=TRUE, xlab="Glucose Level", ylab="Age")

fit<- glm(type ~ glu + age, data=Pim, family=binomial())

confidenceEllipse(fit, which.coef= c(’glu’, ’age’), levels=0.95,
Scheffe=FALSE, center.pch=19, center.cex=1.5, segments=51,
col=palette()[2], lwd=2, fill=FALSE, fill.alpha=0.3, draw=TRUE,
xlab="Glucose Level", ylab="Age")

print("Variance-Covariance Matrix of the Estimated Parameters")

print(vcov(fit))
[1] "Variance-Covariance Matrix of the Estimated Parameters"
(Intercept) glu age
(Intercept) 0.608524815 -3.715419e-03 -3.200757e-03
glu -0.003715419 3.206397e-05 -1.245096e-05
age -0.003200757 -1.245096e-05 1.408382e-04
Two Dimension 17

FIGURE 2.7
Least squares (left) and robust (right) estimates of data ellipse of Pima Indian
diabetes data.

2.2.3 Multivariate Normality Test

Most of the joint normality tests in the literature are based on numerical
calculations and most of them suffer low power to reject the null hypotheses
of multivariate normality.
For graphical tests of normality, one can start with the normal plot of
each marginal distribution and examine any deviation from a straight line.
For multivariate responses, Andrews et al. [3] have suggested a graphical
method that utilizes a radius-and-angels representation of multivariate data.
For a set of multivariate observations {yi , i = 1, · · · , n}, the squared radii
of the observations are defined as

ri2 = (yi − ȳ)0 S−1 (yi − ȳ). (2.8)

If the observations are from a p-variate normal distribution, ri2 should follow
a χ2 distribution with p-degrees of freedom. For example, one can use the
following codes to test multivariate normality and the graphical exhibition as
shown in Figure 2.8.

library(MASS)

Sigma <- matrix(c(10,3,3,2),2,2)

mvdata<-mvrnorm(n=1000, rep(0, 2), Sigma)
zvar<-var(mvdata)
mvmean<-apply(mvdata, 2, mean)
18 Examining Data Distribution

FIGURE 2.8
Bivariate and trivariate normality test.

radii<-diag((mvdata-mvmean)%*%solve(zvar)%*%t(mvdata-mvmean))
qchi<-qchisq(c(1:length(radii))/(length(radii)+0.05), nrow(Sigma),
ncp=0, lower.tail = TRUE, log.p = FALSE)
plot(qchi, sort(radii), xlab="", ylab="")
abline(0,1)
title(xlab="Quantiles of Chi-square dist with 2 df",
ylab="Squared Radii")

Sigma <- matrix(c(10,3,2,3,5,2,2,2,3),3,3)

mvdata<-mvrnorm(n=1000, rep(0, 3), Sigma)
zvar<-var(mvdata)
mvmean<-apply(mvdata, 2, mean)
radii<-diag((mvdata-mvmean)%*%solve(zvar)%*%t(mvdata-mvmean))
qchi<-qchisq(c(1:length(radii))/(length(radii)+0.05), nrow(Sigma),
ncp=0, lower.tail = TRUE, log.p = FALSE)
plot(qchi, sort(radii), xlab="", ylab="")
abline(0,1)
title(xlab="Quantiles of Chi-square dist with 3 df",
ylab="Squared Radii")
More Than Two Dimension 19

2.3 More Than Two Dimension

2.3.1 Scatter Plot Matrix
In the practice of data analysis, one usually has many variables representing
the data of interest. Before any modeling or inferences, it is important to have
a good understanding of the inter-relationship and its intensity between the
variables, and the identification of possible outliers. Pairwise plot of each pair
of variables becomes a handy tool for this purpose. A more efficient way to
examine the pairwise relationship is the scatter plot matrix, which takes a few
variables together to produce the plot.
Many software packages have easy ways to produce the pairwise scat-
ter plot matrix. For example, R has the function pairs which produces the
matrix. It also has several variations that can add more information at the
discretion of the analysts. The left panel of Figure 2.9 is a simple example us-
ing Anderson’s iris data, and the right panel of Figure 2.9 superimposes the
trend line produced by the function lowess at the lower-left part of the plot,
the histogram of each variable at the diagonal, and the pairwise correlation
at the upper-right part of the plot. The R codes to produce these plots are
shown below. Following codes produce examples of Scatter Plot Matrix Using
pairs.
library(MASS)

pairs(iris[1:4], main = "Anderson’s Iris Data of 3 species",

pch = 21, bg = c("red", "green", "blue")[unclass(iris$Species)])

panel.hist <- function(x, ...){

usr <- par("usr"); on.exit(par(usr))
par(usr = c(usr[1:2], 0, 1.5) )
h <- hist(x, plot = FALSE)
breaks <- h$breaks; nB <- length(breaks)
y <- h$counts; y <- y/max(y)
rect(breaks[-nB], 0, breaks[-1], y, col="cyan", ...)
}

panel.cor <- function(x, y, digits=2, prefix="", cex.cor, ...){

usr <- par("usr"); on.exit(par(usr))
par(usr = c(0, 1, 0, 1))
r <- abs(cor(x, y))
txt <- format(c(r, 0.123456789), digits=digits)[1]
txt <- paste(prefix, txt, sep="")
if(missing(cex.cor)) cex.cor <- 0.8/strwidth(txt)
text(0.5, 0.5, txt, cex = 2)
}

pairs(iris[1:4], main = "Anderson’s Iris Data -- 3 species",

lower.panel=panel.smooth, diag.panel=panel.hist, upper.panel=panel.cor,
20 Examining Data Distribution

pch = 21, bg = c("red", "green", "blue")[unclass(iris$Species)])

FIGURE 2.9
Pairwise scatter plot matrix.

The scatterplotMatrix of the cars package in R has several additional

features, which produces other added information to the plot. By properly se-
lecting the values of the options diagonal, ellipse, transform, smooth,
reg.line, one can add or remove the additional information in the plot
and even perform the power transformation of the variables. The R pro-
gram for these plots is shown below and the corresponding plots are shown in
Figures 2.10 and 2.11. The following is an example of Scatter Plot Matrix
using scatterplotMatrix.

library(car)
scatterplotMatrix(iris[1:4], diagonal="density", ellipse=TRUE,
transform=TRUE, smooth=TRUE, reg.line=FALSE, data=iris,
main="Pairwise Plot of Iris Data")

2.3.2 Andrews0 s Plot

Andrews0 s plot (Andrews, 1972) is a graphical technique to display high-
dimensional data. The essential idea in Andrews0 s plot is to map each multi-
dimensional observation into a function, f (t), which is a linear combination
of ortho-normal functions in t with the coefficients in the linear combination
being the observed responses.
More Than Two Dimension 21

FIGURE 2.10
Pairwise scatter plot matrix.

Specifically, let

yi = (yi1 , yi2 , · · · , yip )0 ,

at = (a1 (t), a2 (t), · · · )0
√
= (1/ 2, sin(t), cos(t), sin(2t), cos(2t), · · · )0 ,

for 1 ≤ i ≤ n and t ∈ (−π, π).

The n functions {f1 (t), f2 t, · · · , fn (t)} can then be plotted versus t, namely,

{(t, fi (t)) | t ∈ (−π, π), i = 1, 2, · · · , n}. (2.9)

Thus, the initial multi-dimensional observations will now appear as n

curves in a two-dimensional display whose ordinate corresponds to the func-
tion values f (t) and the abscissa is the values of t. If one changes the parameter
t in the ortho-normal functions by 2πt, then the plot can be created with t
ranging in (−1, 1).
The choice of the ortho-normal function is not unique. Tukey had suggested
using
√ √ √
at = (cos(t), cos( 2t), cos( 3t), cos( 5t), · · · )0 t ∈ (0, kπ). (2.10)

The following codes produce Andrew’s plot at two different angels as shown
in Figure 2.12:
22 Examining Data Distribution

FIGURE 2.11
Pairwise scatter plot matrix.

library(andrews)
library(MASS)

iris2<-iris[c(1:10, 101:110),]
numstep<-100
steps<-2*pi/numstep
proj<-matrix(rep(0, numstep*nrow(iris2)), nrow(iris2), numstep)
for(j in 1:numstep){
i<-j-1
stepi<- -pi+i*steps
fa<-c(1/sqrt(2), sin(stepi), cos(stepi), sin(2*(stepi)),
cos(2*(stepi)), sin(3*(stepi)), cos(3*(stepi)))
for(kk in 1:nrow(iris2)){
proj[kk,j]<- iris2[kk, 1]*fa[1] + iris2[kk, 2]*fa[2]
+ iris2[kk, 3]*fa[3] + iris2[kk, 4]*fa[4]
}}
plot(1:numstep, proj[1,], type="n", ylim=c(min(proj)-1, max(proj)+1),
xlab="", ylab="")
title(xlab=expression(paste("Range:(", -pi, ", ", pi, ")")),
ylab="Values of Projections")
for(i in 1:10){
lines(1:numstep, proj[i,])
}
for(i in 11:nrow(proj)){
lines(1:numstep, proj[i,], col="red")
}
More Than Two Dimension 23

FIGURE 2.12
Andrew’s plot of two different projects.

legend("topleft", c("red: virginia, black:setosa"))

proj<-matrix(rep(0, numstep*nrow(iris2)), nrow(iris2), numstep)

for(j in 1:numstep){
i<-j-1
stepi<- -pi+i*steps
fa<-c(1/sqrt(2), sin(stepi), cos(stepi), sin(2*(stepi)),
cos(2*(stepi)), sin(3*(stepi)), cos(3*(stepi)))
for(kk in 1:nrow(iris2)){
proj[kk,j]<- iris2[kk, 1]*fa[2] + iris2[kk, 2]*fa[3]
+ iris2[kk, 3]*fa[4] + iris2[kk, 4]*fa[1]
}}
plot(1:numstep, proj[1,], type="n", ylim=c(min(proj)-1, max(proj)+1),
xlab="", ylab="")
title(xlab=expression(paste("Range:(", -pi, ", ", pi, ")")),
ylab="Values of Projections")

for(i in 1:10){lines(1:numstep, proj[i,])}

for(i in 11:nrow(proj)){lines(1:numstep, proj[i,], col="red")}

2.3.3 Conditional Plot

A conditional plot, also known as a coplot or subset plot, is a plot of two
variables conditional on the value of a third variable (called the conditioning
variable). The conditioning variable may be either a variable that takes on
only a few discrete values or a continuous variable that is divided into a
24 Examining Data Distribution

FIGURE 2.13
Conditional plots.

limited number of subsets. One limitation of the scatter plot matrix is that it
cannot show interaction effects with another variable. This is the strength of
the conditioning plot. It is also useful for displaying scatter plots for groups
in the data. Although these groups can also be plotted on a single plot with
different plot symbols, it can often be visually easier to distinguish the groups
using the conditional plot.
Although the basic concept of the conditioning plot matrix is simple, there
are numerous alternatives in the details of the plots. It can be helpful to overlay
some type of fitted curve on the scatter plot. Although a linear or quadratic fit
can be used, the most common alternative is to overlay a lowess curve. Given
the variables X, Y , and Z, the condition plot is formed by dividing the values
of Z into k groups. There are several ways that these groups may be formed.
There may be a natural grouping of the data, the data may be divided into
several equal sized groups, the grouping may be determined by clusters in the
data, and so on. The page will be divided into n rows and c columns, where
n × c ≥ k. Each row and column defines a single scatter plot. For example,
the following codes create the plot (Figure 2.13) of Tonga trench earthquakes
conditioning on depth of the ocean.
## Conditioning on 1 variable - ocean depth:
coplot(lat ~ long | depth, data = quakes)
given.depth <- co.intervals(quakes$depth, number=4, overlap=.1)
coplot(lat ~ long | depth, data = quakes, given.v=given.depth, rows=1)

One can also create the conditional plot by conditioning on more than
Visualization of Categorical Data 25

one variable. The following codes produce a conditioning plot (Figure 2.14) of
wool strength by conditioning on types of wool and tension strength:

## Conditioning on 2 factors - wool and tension:

Index <- seq(length=nrow(warpbreaks)) # to get nicer default labels
coplot(breaks ~ Index | wool * tension, data = warpbreaks,
show.given = 0:1)
coplot(breaks ~ Index | wool * tension, data = warpbreaks,
col = "red", bg = "pink", pch = 21,
bar.bg = c(fac = "light blue"))

FIGURE 2.14
Example of conditional plot.

2.4 Visualization of Categorical Data

In order to explain multi-dimensional categorical data, statisticians typically
look for (conditional) independence structures. Whether the task is purely
exploratory or model-based, techniques such as mosaic and association plots
offer good support for people to better understand the data distribution. Both
26 Examining Data Distribution

aspects of (possibly higher-dimensional) contingency tables, with several ex-

tensions introduced over the last two decades, and implementations available
in many statistical environments.

2.4.1 Mosaic Plot

Hartigan and Kleiner [35] proposed a mosaic plot, which is basically an area-
proportional display of the observed frequencies, composed of tiles for each
corresponding cells created by recursive vertical and horizontal splits of a
rectangle. Thus, the area of each tile is proportional to the corresponding
cell entry given the dimensions of previous splits. For example, the follow-
ing simple codes produce the mosaic plot of hair x eye x sex, as shown in
Figure 2.15:

library(MASS)
library(vcd)
data("HairEyeColor")
mosaic(HairEyeColor, shade = TRUE)

FIGURE 2.15
Mosaic plot of of hair x eye x sex.
Visualization of Categorical Data 27

2.4.2 Association Plot

For a contingency table, the signed contribution to Pearson’s χ2 for cell
{ij · · · k} is
Oij···k − Eij···k
dij···k = p ,
Eij···k
where Oij···k and Eij···k are the observed and expected cell counts ,respectively,
under the independence model.
Cohen [13] proposed an association plot, which displays the standardized
deviations of observed frequencies from those expected under a certain inde-
pendence hypothesis. In the association plot, each cell is represented by a rect-
angle
p that has (signed) height proportional to dij···k and width proportional
to Eij···k , so that the area of the box is proportional to the difference in
observed and expected frequencies. The rectangles in each row are positioned
relative to a baseline indicating independence dij···k = 0. If the observed fre-
quency of a cell is greater than the expected one, the box rises above the
baseline and falls below otherwise.
Friendly [25] extended the association plot and provided a means for visu-
alizing the residuals of an independence model for a contingency table. assoc
in R package vcd is a function to produce (extended) association plots. The
following codes produce two different ways to visualize multi-way table of
cross-classification, as shown in Figure 2.16:

library(MASS)
library(vcd)
data("HairEyeColor")
(x <- margin.table(HairEyeColor, c(1, 2)))
assoc(x, main = "Relation between hair and eye color", shade = TRUE)

assoc(aperm(HairEyeColor), expected = ~ (Hair + Eye) * Sex,

labeling_args = list(just_labels = c(Eye = "left"),
offset_labels = c(right = -0.5), offset_varnames = c(right = 1.2),
rot_labels = c(right = 0), tl_varnames = c(Eye = TRUE)))
28 Examining Data Distribution

FIGURE 2.16
Two different ways to visualize multi-way table of cross-classification.
Discovering Diverse Content Through
Random Scribd Documents
having died on the day he left her, and as having been repeatedly
buried in the various dull country houses by whose proprietors he
was hospitably received. He thus proceeds:—
‘But whilst my body, madam, was thus disposed of, my spirit (as
when alive) was still hovering, though invisible, round your Majesty,
anxious for your welfare, and watching to do you any little service
that lay within my power.
‘On Monday, whilst you walked, my shade still turned on the side
of the sun to guard you from its beams.
‘On Tuesday morning, at breakfast, I brushed away a fly that
had escaped Teed’s observation’ (Teed was one of the Queen’s
attendants) ‘and was just going to be the taster of your chocolate.
‘On Wednesday, in the afternoon, I took off the chillness of some
strawberry-water your Majesty was going to drink as you came in
hot from walking; and at night I hunted a bat out of your
bedchamber, and shut a sash just as you fell asleep, which your
Majesty had a little indiscreetly ordered Mrs. Purcel to leave open.
‘On Thursday, in the drawing-room, I took the forms and voices
of several of my acquaintances, made strange faces, put myself into
awkward postures, and talked a good deal of nonsense, whilst your
Majesty entertained me very gravely, recommended me very
graciously, and laughed at me internally very heartily.
‘On Friday, being post-day, I proposed to get the best pen in the
other world for your Majesty’s use, and slip it invisibly into your
standish just as Mr. Shaw was bringing it into your gallery for you to
write; and accordingly I went to Voiture, and desired him to hand
me his pen; but when I told him for whom it was designed, he only
laughed at me for a blockhead, and asked me if I had been at court
for four years to so little purpose as not to know that your Majesty
had a much better of your own.
‘On Saturday I went on the shaft of your Majesty’s chaise to
Richmond; as you walked there I went before you, and with an
invisible wand I brushed the dew and the worms out of your path all
the way, and several times uncrumpled your Majesty’s stocking.
‘Sunday.—This very day, at chapel, I did your Majesty some
service, by tearing six leaves out of the parson’s sermon and
shortening his discourse six minutes.’
While these imaginary services were being rendered by the
visionary Lord Hervey to the Queen, realities more serious and not
less amusing were claiming the attention of Caroline and her
consort.
In return for the information communicated by the King to the
Queen on the subject of Madame Walmoden and her charms,
Caroline had to inform her husband of the marriage we have spoken
of between Lady Suffolk and Mr. George Berkeley. The royal ex-lover
noticed the communication in his reply in a coarse way, and
expressed his entire satisfaction at being rid of the lady, and at the
lady’s disposal of herself.
When Caroline informed her vice-chamberlain, Lord Hervey, of
the report of this marriage, his alleged disbelief of the report made
her peevish with him, and induced her to call him an ‘obstinate
devil,’ who would not believe merely improbable facts to be truths.
Caroline then railed at Lady Suffolk in good set terms as a sayer and
doer of silly things, entirely unworthy of the reputation she had with
some people of being the sayer and doer of wise ones.
It was on this occasion that Caroline herself described to Lord
Hervey the farewell interview she had had with Lady Suffolk. The ex-
mistress took a sentimental view of her position, and lamented to
the wife that she, the mistress, was no longer so kindly treated as
formerly by the husband. ‘I told her,’ said the Queen, ‘in reply, that
she and I were not of an age to think of these sort of things in such
a romantic way, and said, “My good Lady Suffolk, you are the best
servant in the world; and, as I should be most extremely sorry to
lose you, pray take a week to consider of this business, and give me
your word not to read any romances in that time, and then I dare
say you will lay aside all thoughts of doing what, believe me, you will
20
repent, and what I am very sure I shall be very sorry for.”’ It was
at one of these conversations with Lord Hervey that the Queen told
him that Lady Suffolk ‘had had 2,000l. a year constantly from the
King whilst he was prince, and 3,200l. ever since he was King;
besides several little dabs of money both before and since he came
to the crown.’
A letter of Lady Pomfret’s will serve to show us not only a picture
of the Queen at this time, but an illustration of feeling in a fine lady.
Lady Pomfret, writing to Lady Sundon, in 1735, says: ‘All I can
say of Kensington is, that it is just the same as it was, only pared as
close as the bishop does the sacrament. My Lord Pomfret and I were
the greatest strangers there; no secretary of state, no chamberlain
or vice-chamberlain, but Lord Robert, and he just in the same coat,
the same spot of ground, and the same words in his mouth that he
had when I left there. Mrs. Meadows in the window at work; but,
though half an hour after two, the Queen was not quite dressed, so
that I had the honour of seeing her before she came out of her little
blue room, where I was graciously received, and acquainted her
Majesty, to her great sorrow, how ill you had been; and then, to
alleviate that sorrow, I informed her how much Sundon was altered
for the better, and that it looked like a castle. From thence we
proceeded to a very short drawing-room, where the Queen joked
much with my Lord Pomfret about Barbadoes. The two ladies of the
bedchamber and the governess are yet on so bad a foot, that upon
the latter coming into the room to dine with Lady Bristol, the others
went away, though just going to sit down, and strangers in the
place.’
The writer of this letter soon after lost a son, the Honourable
Thomas Fermor. It was a severely felt loss; so severe that some
weeks elapsed before the disconsolate mother was able, as she
says, ‘to enjoy the kind and obliging concern’ expressed by the
Queen’s bedchamber-woman in her late misfortune. Christianity
itself, as this charming mother averred, would have authorised her in
lamenting such a calamity during the remainder of her life; but then,
oh joy! her maternal lamentation was put an end to and Rachel was
comforted, and all because—‘It was impossible for any behaviour to
be more gracious than that of the Queen on this occasion, who
made it quite fashionable to be concerned’ at the death of Lady
Pomfret’s son.
But there were more bustling scenes at Kensington than such as
those described by this fashionably sorrowing lady and the
sympathising sovereign.
On Sunday, the 26th of October, the Queen and her court had
just left the little chapel in the palace of Kensington, when intimation
was given to her Majesty that the King, who had left Hanover on the
previous Wednesday, was approaching the gate. Caroline, at the
head of her ladies and the gentlemen of her suite, hastened down to
receive him; and, as he alighted from his ponderous coach, she took
his hand and kissed it. This ceremony performed by the regent, a
very unceremonious, hearty, and honest kiss was impressed on his
lips by the wife. The King endured the latter without emotion, and
then, taking the Queen-regent by the fingers, he led her upstairs in
a very stately and formal manner. In the gallery there was a grand
presentation, at which his Majesty exhibited much ill-humour, and
conversed with everybody but the Queen.
His ill-humour arose from various sources. He had heated
himself by rapid and continual travelling, whereby he had brought on
an attack of a complaint to which he was subject, which made him
very ill at ease, and which is irritating enough to break down the
patience of the most patient of people.
On ordinary occasions of his return from Hanover his most
sacred Majesty was generally of as sour disposition as man so little
heroic could well be. He loved the Electorate better than he did his
kingdom, and would not allow that there was anything in the latter
which could not be found in Hanover of a superior quality. There was
no exception to this: men, women, artists, philosophers, actors,
citizens, the virtues, the sciences, and the wits, the country, its
natural beauties and productions, the courage of the men and the
attractions of the women—all of these in England seemed to him
worthless. In Hanover they assumed the guise of perfection.
This time he returned to his ‘old’ wife laden with a fresh sorrow
—the memory of a new favourite. He had left his heart with the
insinuating Walmoden, and he brought to his superb Caroline
nothing but a tribute of ill-humour and spite. He hated more than
ever the change from an Electorate where he was so delightfully
despotic, to a country where he was only chief magistrate, and
where the people, through their representatives, kept a very sharp
watch upon him in the execution of his duties. He was accordingly as
coarse and evil-disposed towards the circle of his court as he was to
her who was the centre of it. He, too, was like one of those
pantomime potentates who are for ever in King Cambyses’ vein, and
who sweep through the scene in a whirlwind of farcically furious
words and of violent acts, or of threats almost as bad as if the
menaces had been actually realised. It was observed that his
behaviour to Caroline had never been so little tinged with outward
respect as now. She bore his ill-humour with admirable patience;
and her quiet endurance only the more provoked the petulance of
the little and worthless King.
He was not only ill-tempered with the mistress of the palace, but
was made, or chose to think himself, especially angry at trifling
improvements which Caroline had carried into effect in the suburban
palace during the temporary absence of its master. The
improvements consisted chiefly in removing some worthless pictures
and indifferent statues and placing master-pieces in their stead. The
King would have all restored to the condition it was in when he had
last left the palace; and he treated Lord Hervey as a fool for
venturing to defend the Queen’s taste and the changes which had
followed the exercise of it. ‘I suppose,’ said the dignified King to the
courteous vice-chamberlain, ‘I suppose you assisted the Queen with
your fine advice when she was pulling my house to pieces, and
spoiling all my furniture. Thank God! at least she has left the walls
standing!’
 Lord Hervey asked if he would not allow the two Vandykes which
the Queen had substituted for ‘two signposts,’ to remain. George
pettishly answered, that he didn’t care whether they were changed
or no; ‘but,’ he added, ‘for the picture with the dirty frame over the
door, and the three nasty little children, I will have them taken away,
and the old ones restored. I will have it done, too, to-morrow
morning, before I go to London, or else I know it will not be done at
all.’
Lord Hervey next enquired if his Majesty would also have ‘his
gigantic fat Venus restored too?’ The King replied that he would, for
he liked his fat Venus better than anything which had been put in its
place. Upon this Lord Hervey says he fell to thinking ‘that if his
Majesty had liked his fat Venus as well as he used to do, there would
have been none of these disputations.’
By a night’s calm repose the ill-humour of the Sovereign was not
dispersed. On the following morning we meet with the insufferable
little man in the gallery, where the Queen and her daughters were
taking chocolate; her son, the Duke of Cumberland, standing by. He
only stayed five minutes, but in that short time the husband and
father contrived to wound the feelings of his wife and children. ‘He
snubbed the Queen, who was drinking chocolate, for being always
stuffing; the Princess Amelia for not hearing him; the Princess
Caroline for being grown fat; the Duke of Cumberland for standing
awkwardly; and then he carried the Queen out to walk, to be re-
21
snubbed in the garden.’
Sir Robert Walpole told his friend Hervey that he had done his
utmost to prepare the Queen for this change in the King’s feelings
and actions towards her. He reminded her that her personal
attractions were not what they had been, and he counselled her to
depend more upon her intellectual superiority than ever. The
virtuous man advised her to secure the good temper of the King by
throwing certain ladies in his way of an evening. Sir Robert
mentioned, among others, Lady Tankerville, ‘a very safe fool, who
would give the King some amusement without giving her Majesty
any trouble.’ Lady Deloraine, the Delia from whose rage Pope bade
his readers dread slander and poison, had already attracted the royal
notice, and the King liked to play cards with her in his daughter’s
apartments. This lady, who had the loosest tongue of the least
modest women about the court, was characterised by Walpole as
likely to exercise a dangerous influence over the King. If Caroline
would retain her power, he insinuated, she must select her
husband’s favourites, through whom she might still reign supreme.
Caroline is said to have taken this advice in good part. There
would be difficulty in believing that it ever was given did we not
know that the Queen herself could joke, not very delicately, in full
court, on her position as a woman not first in her husband’s regard.
Sir Robert would comment on these jokes in the same locality, and
with increase of coarseness. The Queen, however, though she
affected to laugh, was both hurt and displeased—hurt by the joke
and displeased with the joker, of whom Swift has said, that—

By favour and fortune fastidiously blest,

He was loud in his laugh and was coarse in his jest.

In spite of the King’s increased ill-temper towards the Queen,

and in spite of what Sir Robert Walpole thought and said upon that
delicate subject, Lord Hervey maintains that at this very time the
King’s heart, as affected towards the Queen, was not less warm than
his temper. The facts which are detailed by the gentle official
immediately after he has made this assertion go strongly to disprove
the latter. The detail involves a rather long extract; but its interest,
and the elaborate minuteness with which this picture of a royal
interior is painted, will doubtless be considered ample excuse for
reproducing the passages. Lord Hervey was eye and ear-witness of
what he here so well describes:—
‘About nine o’clock every night the King used to return to the
Queen’s apartment from that of his daughter’s, where, from the time
of Lady Suffolk’s disgrace, he used to pass those evenings he did not
go to the opera or play at quadrille, constraining them, tiring himself,
and talking a little indecently to Lady Deloraine, who was always of
the party.
‘At his return to the Queen’s side, the Queen used often to send
for Lord Hervey to entertain them till they retired, which was
generally at eleven. One evening among the rest, as soon as Lord
Hervey came into the room, the Queen, who was knotting, while the
King walked backwards and forwards, began jocosely to attack Lord
Hervey upon an answer just published to a book of his friend Bishop
Hoadly’s on the Sacrament, in which the bishop was very ill-treated;
but before she had uttered half what she had a mind to say, the
King interrupted her, and told her she always loved talking of such
nonsense, and things she knew nothing of; adding, that if it were
not for such foolish people loving to talk of these things when they
were written, the fools who wrote upon them would never think of
publishing their nonsense, and disturbing the government with
impertinent disputes that nobody of any sense ever troubled himself
about. The Queen bowed, and said, “Sir, I only did it to let Lord
Hervey know that his friend’s book had not met with that general
approbation he had pretended.” “A pretty fellow for a friend!” said
the King, turning to Lord Hervey. “Pray what is it that charms you in
him? His pretty limping gait?” And then he acted the bishop’s
lameness, and entered upon some unpleasant defects which it is not
necessary to repeat. The stomachs of the listeners must have been
strong, if they experienced no qualm at the too graphic and nasty
detail. “Or is it,” continued the King, “his great honesty that charms
your lordship? His asking a thing of me for one man, and when he
came to have it in his own power to bestow, refusing the Queen to
give it to the very man for whom he had asked it? Or do you admire
his conscience, that makes him now put out a book that, till he was
Bishop of Winchester, for fear his conscience might hurt his
preferment, he kept locked up in his chest? Is his conscience so
much improved beyond what it was when he was Bishop of Bangor,
or Hereford, or Salisbury—for this book, I fear, was written so long
ago—or is it that he would not risk losing a shilling a year more
whilst there was anything better to be got than what he had? I
cannot help saying, that if the Bishop of Winchester is your friend,
you have a great puppy, and a very dull fellow, and a great rascal,
for your friend. It is a very pretty thing for such scoundrels, when
they are raised by favour above their deserts, to be talking and
writing their stuff, to give trouble to the government which has
showed them that favour; and very modest for a canting,
hypocritical knave to be crying that the kingdom of Christ is not of
this world at the same time that he, as Christ’s ambassador, receives
6,000l. or 7,000l. a year. But he is just the same thing in the Church
that he is in the government, and as ready to receive the best pay
for preaching the Bible, though he does not believe a word of it, as
he is to take favour from the Crown, though, by his republican spirit
and doctrine, he would be glad to abolish its power.”’
There is something melancholily suggestive in thus hearing the
temporal head of a Church accusing of rank infidelity a man whom
he had raised to be an overseer and bishop of souls in that very
Church. If George knew that Hoadly did not believe in Scripture, he
was infinitely worse than the prelate for the simple fact of his having
made him a prelate, or having translated him from one diocese to
another of more importance and more value. But, to resume:—
‘During the whole time the King was speaking, the Queen, by
smiling and nodding in proper places, endeavoured all she could, but
in vain, to make her court, by seeming to approve everything he
said.’ Lord Hervey then attempted to give a pleasant turn to the
conversation by remarking on prelates who were more docile
towards government than Hoadly, and who, for being dull branches
of episcopacy, and ignorant piecers of orthodoxy, were none the less
good and quiet subjects. From the persons of the Church the vice-
chamberlain got to the fabric, and then descanted to the Queen
upon the newly restored bronze gates in Henry VII.’s Chapel. This
excited the King’s ire anew. ‘My lord,’ said he, ‘you are always
putting some of these fine things in the Queen’s head, and then I
am to be plagued with a thousand plans and workmen.’ He grew
sarcastic, too, on the Queen’s grotto in Richmond Gardens, which
was known as Merlin’s Cave, from a statue of the great enchanter
therein; and in which there was a collection of books, over which
Stephen Duck, thresher, poet, and parson, had been constituted
librarian. The Craftsman paper had attacked this plaything of the
Queen, and her husband was delighted at the annoyance caused to
her by such an attack.
The poor Queen probably thought she had succeeded in cleverly
changing the topic of conversation by referring to and expressing
disapproval of the expensive habit of giving vails to the servants of
the house at which a person has been visiting. She remarked that
she had found it no inconsiderable expense during the past summer
to visit her friends even in town. ‘That is your own fault,’ growled the
King; ‘for my father, when he went to people’s houses in town, never
was fool enough to give away his money.’ The Queen pleaded that
she only gave what her chamberlain, Lord Grantham, informed her
was usual; whereupon poor Lord Grantham came in for his full share
of censure. The Queen, said her consort, ‘was always asking some
fool or another what she was to do, and that none but a fool would
ask another fool’s advice.’
The vice-chamberlain gently hinted that liberality would be
expected from a Queen on such occasions as her visits at the houses
of her subjects. ‘Then let her stay at home, as I do,’ said the King.
‘You do not see me running into every puppy’s house to see his new
chairs and stools.’ And then, turning to the Queen, he added: ‘Nor is
it for you to be running your nose everywhere, and to be trotting
about the town, to every fellow that will give you some bread and
butter, like an old girl who loves to go abroad, no matter where, or
whether it be proper or no.’ The Queen coloured, and knotted a
good deal faster during this speech than before; whilst the tears
came into her eyes, but she said not one word.
Such is the description of Lord Hervey, and it shows Caroline in a
favourable light. The vice-chamberlain struck in for her, by observing
that her Majesty could not see private collections of pictures without
going to the owners’ houses, and honouring them by her presence.
‘Supposing,’ said the King, ‘she had a curiosity to see a tavern, would
it be fit for her to satisfy it? and yet the innkeeper would be very
glad to see her.’ The vice-chamberlain did not fail to see that this
was a most illogical remark, and he very well observed, in reply,
that, ‘if the innkeepers were used to be well received by her Majesty
in her palace, he should think that the Queen’s seeing them at their
own houses would give no additional scandal.’ As George found
himself foiled by this observation, he felt only the more displeasure,
and he gave vent to the last by bursting forth into a torrent of
German, which sounded like abuse, and during the outpouring of
which ‘the Queen made not one word of reply, but knotted on till she
tangled her thread, then snuffed the candles that stood on the table
before her, and snuffed one of them out. Upon which the King, in
English, began a new dissertation upon her Majesty, and took her
22
awkwardness for his text.’
Unmoved as Caroline appeared at this degrading scene, she felt
it acutely; but she did not wish that others should be aware of her
feelings under such a visitation. Lord Hervey was aware of this; and
when, on the following morning, she remarked that he had looked at
her the evening before as if he thought she had been going to cry,
the courtier protested that he had neither done the one nor thought
the other, but had expressly directed his eyes on another object, lest
if they met hers, the comicality of the scene should have set both of
them laughing.
And such scenes were of constant occurrence. The King
extracted something unpleasant from his very pleasures, just as
acids may be produced from sugar. Sometimes he fell into a difficulty
during the process. Thus, on one occasion, when the party were
again assembled for their usual delightful evening, the Queen had
mentioned the name of a person whose father, she said, was known
to the King. It was at the time when his Majesty was most bitterly
incensed against his eldest son. Caroline was on better terms with
Frederick; but, as she remarked, they each knew the other too well
to love or trust one another. Well, the King hearing father and son
alluded to, observed, that ‘one very often sees fathers and sons very
little alike; a wise father has very often a fool for his son. One sees a
father a very brave man, and his son a scoundrel; a father very
honest, and his son a great knave; a father a man of truth, and his
son a great liar; in short, a father that has all sorts of good qualities,
23
and a son who is good for nothing.’ The Queen and all present
betrayed, by their countenances, that they comprehended the
historical parallel; whereupon the King attempted, as he thought, to
make it less flagrantly applicable, by running the comparison in
another sense. ‘Sometimes,’ he said, ‘the case was just the reverse,
and that very disagreeable fathers had very agreeable men for their
sons.’ In this case, the King, as Lord Hervey suggests, was thinking
of his own father, as in the former one he had been thinking of his
son.
But how he drew what was sour from the sweetest of his
pleasures is shown from his remarks after having been to the
theatre to see Shakspeare’s ‘Henry IV.’ He was tolerably well pleased
with all the actors, save the ‘Prince of Wales.’ He had never seen, he
said, so awkward a fellow and so mean a looking scoundrel in his
life. Everybody, says Lord Hervey, who hated the actual Prince of
Wales thought of him as the King here expressed himself of the
player; ‘but all very properly pretended to understand his Majesty
literally, joined in the censure, and abused the theatrical Prince of
Wales for a quarter of an hour together.’
It may be here noticed that Shakspeare owed some of his
reputation, at this time, to the dissensions which existed between
the King and his son. Had it, at least, not been for this circumstance,
it is not likely that the play of ‘Henry IV.’ would have been so often
represented as it was at the three theatres—Lincoln’s-Inn-Fields,
Covent Garden, and Drury Lane. Every auditor knew how to make
special application of the complainings and sorrowings of a royal sire
over a somewhat profligate son; or of the unfilial speeches and
hypocritical assurance of a princely heir, flung at his Sovereign and
impatient sire. The house in Lincoln’s-Inn-Fields had the reputation
of being the Tory house; and the Prince of Wales there was probably
represented as a proper gentleman; not out of love to him, but
rather out of contempt to the father. It was not a house which
received the favour of either Caroline or her consort. The new pieces
there ran too strongly against the despotic rule of kings—the only
sort of rule for which George at all cared, and the lack of which
made him constantly abusive of England, her institutions,
parliament, and public men. It is difficult to say what the real
opinion of Caroline was upon this matter, for at divers times we find
her uttering opposite sentiments. She could be as abusive against
free institutions and civil and religious rights as ever her husband
was. She has been heard to declare that sovereignty was worth little
where it was merely nominal, and that to be king or queen in a
country where people governed through their parliament was to
wear a crown and to exercise none of the prerogatives which are
ordinarily attached to it. At other times she would declare that the
real glory of England was the result of her free institutions; the
people were industrious and enterprising because they were free,
and knew that their property was secure from any attack on the part
of prince or government. They consequently regarded their
sovereign with more affection than a despotic monarch could be
regarded by a slavish people; and she added, that she would not
have cared to share a throne in England, if the people by whom it
was surrounded had been slaves without a will of their own, or
without a heart that throbbed at the name of liberty. The King never
had but one opinion on the subject, and therefore the theatre at
Lincoln’s-Inn-Fields was for ever resounding with clap-traps against
despotism, and that in presence of an audience of whom Frederick,
Prince of Wales, was chief, and Bolingbroke led the applause.
But even Drury Lane could be as democratic as Lincoln’s Inn.
Thus, in the very year of which we are treating, Lillo brought out his
‘Christian Hero’ at Drury Lane, and the audience had as little
difficulty to apply the parts to living potentates as they had
reluctance to applaud to the echo passages like the following against
despotic rulers:—
 Despotic power, that root of bitterness,
That tree of death that spreads its baleful arms
Almost from pole to pole, beneath whose cursed shade
No good thing thrives, and every ill finds shelter,
Had found no time for its detested growth
But for the follies and the crimes of men.

But ‘Drury’ did not often offend in this guise, and even George
and Caroline might have gone to see ‘Junius Brutus,’ and have been
amused. The Queen, who well knew the corruption of the senate,
might have smiled as Mills, in Brutus, with gravity declared that the
senators—

Have heaped no wealth, though hoary grown in honours,

and George might have silently assented to the reply of Cibber, Jun.,
in ‘Messala,’ that—

On crowns they trample with superior pride;

They haughtily affect the pomp of princes.

The Queen’s vice-chamberlain asserts that the King’s heart still

beat for Caroline as warmly as his temper did against her. This
assertion is not proved, but the contrary, by the facts. These facts
were of so painful a nature to the Queen that she did not like to
speak of them, even to Sir Robert Walpole. One of them is a
precious instance of the conjugal warmth of heart pledged for by
Lord Hervey.
The night before the King had last left Hanover for England he
supped gaily, in company with Madame Walmoden and her friends,
who were not so nice as to think that the woman who had deserted
her husband for a King who betrayed his consort had at all lost caste
by such conduct. Towards the close of the banquet, the frail lady, all
wreathed in mingled tears and smiles, rose, and gave as a toast, or
sentiment, the ‘next 29th of May.’ On that day the old libertine had
promised to be again at the feet of his new concubine; and as this
was known to the select and delicate company, they drank the ‘toast’
amid shouts of loyalty and congratulations.
The knowledge of this fact gave more pain to Caroline than all
the royal fits of ill-humour together. The pain was increased by the
King’s conduct at home. It had been his custom of a morning, at St.
James’s, to tarry in the Queen’s rooms until after he had, from
behind the blinds, seen the guard relieved in the court-yard below:
this took place about eleven o’clock. This year he ceased to visit the
Queen or to watch the soldiers; but by nine o’clock in the morning
he was seated at his desk, writing lengthy epistles to Madame
Walmoden, in reply to the equally long letters from the lady, who
received and despatched a missive every post.
‘He wants to go to Hanover, does he?’ asked Sir Robert Walpole
of Lord Hervey; ‘and to be there by the 29th of May. Well, he shan’t
go for all that.’
Domestic griefs could not depress the Queen’s wit. An illustration
of this is afforded by her remark on the Triple Alliance. ‘It always put
her in mind,’ she said, ‘of the South Sea scheme, which the parties
concerned entered into, not without knowing the cheat, but hoping
to make advantage of it, everybody designing, when he had made
his own fortune, to be the first in scrambling out of it, and each
thinking himself wise enough to be able to leave his fellow-
adventurers in the lurch.’
It has been well observed that the King’s good humour was now
as insulting to her Majesty as his bad. When he was in the former
rare vein, he exhibited it by entertaining the Queen with accounts of
her rival, and the many pleasures which he and that lady had
enjoyed together. He appears at Hanover to have been as
extravagant in the entertainments which he gave as his grandfather,
Ernest Augustus. Some of these court revels he caused to be painted
on canvas; the ladies represented therein were all portraits of the
actual revellers. Several of such pictures were brought over to
England, and five of them were hung up in the Queen’s dressing-
room. Occasionally, of an evening, the King would take a candle
from the Queen’s table, and go from picture to picture, with Lord
Hervey, telling him its history, explaining the joyous incidents,
naming the persons represented, and detailing all that had been said
or done on the particular occasion before them. ‘During which
lecture,’ says the vice-chamberlain himself, ‘Lord Hervey, while
peeping over his Majesty’s shoulders at those pictures, was
shrugging up his own, and now and then stealing a look, to make
faces at the Queen, who, a little angry, a little peevish, and a little
tired at her husband’s absurdity, and a little entertained with his
lordship’s grimaces, used to sit and knot in a corner of the room,
sometimes yawning, and sometimes smiling, and equally afraid of
betraying those signs, either of her lassitude or mirth.’
In the course of the year which we have now reached, Queen
Caroline communicated to Lord Hervey a fact, which is not so much
evidence of her Majesty’s common-sense, as of the presumption and
immorality of those who gave Caroline little credit for having even
the sense which is so qualified. Lord Bolingbroke had married the
Marchioness de Villette, niece of Madame de Maintenon, about the
year 1716. The union, however, was not only kept secret for many
years, but when Bolingbroke was under attainder, and a sum of
52,000l. belonging to his wife was in the hands of Decker, the
banker, Lady Bolingbroke swore that she was not married to him,
and so obtained possession of a sum which, being hers, was her
husband’s, and which being her husband’s, who was attainted as a
traitor, was forfeit to the Crown. However, as some of it went
through the hands of poor Sophia Dorothea’s rival, the easy Duchess
of Kendal, and her rapacious niece, Lady Walsingham, the matter
was not enquired into. Subsequently Lady Bolingbroke attempted to
excuse her husband’s alleged dealings with the Pretender, by
asserting that he entered into them solely for the purpose of serving
the Court of London. ‘That was, in short,’ said Caroline to Lord
Hervey, ‘to betray the Pretender; for though Madame de Villette
softened the word, she did not soften the thing, which I own,’
continued the Queen, ‘was a speech which had so much impudence
and villainy mixed up in it, that I could never bear him or her from
that hour, and could hardly hinder myself from saying to her—“And
pray, madam, what security can the King have that my Lord
Bolingbroke does not desire to come here with the same honest
desire that he went to Rome? or that he swears that he is no longer
a Jacobite, with any more truth than you have sworn you are not his
wife?”’ The only wonder is, considering Caroline’s vivacious
character, that she restrained herself from giving expression to her
thoughts. She was eminently fond of ‘speaking daggers’ to those
who merited such a gladiatorial visitation.
 CHAPTER V.
THE MARRIAGE OF FREDERICK, PRINCE OF
WALES.

The Queen’s cleverness—Princess Augusta of Saxe Gotha, the selected bride

of Prince Frederick—Spirited conduct of Miss Vane, the Prince’s mistress—
The King anxious for a matrimonial alliance with the Court of Prussia—
Prussian intrigue to prevent this—The Prussian mandats for entrapping
recruits—Quarrel, and challenge to duel, between King George and the
Prussian monarch—The silly duel prevented—Arrival of the bride—The
royal lovers—Disgraceful squabbles of the Princes and Princesses—The
marriage—Brilliant assemblage in the bridal chamber—Lady Diana
Spencer proposed as a match for the Prince—Débût of Mr. Pitt, afterwards
Lord Chatham, in the House of Commons—Riot of the footmen at Drury
Lane Theatre—Ill-humour exhibited by the Prince towards the Queen.

The Queen never exhibited her cleverness in a clearer light than

when, in 1735, she got over the expected difficulty arising from a
threatened parliamentary address to the throne for the marriage and
settlement of the Prince of Wales. She ‘crushed’ it, to use the term
employed by Lord Hervey, by gaining the King’s consent—no difficult
matter—to tell the prince that it was his royal sire’s intention to
marry him forthwith. The King had no princess in view for him; but
was ready to sanction any choice he might think proper to make,
and the sooner the better. As if the thing were already settled, the
Queen, on her side, talked publicly of the coming marriage of the
heir-apparent; but not a word was breathed as to the person of the
bride. Caroline, moreover, to give the matter a greater air of reality,
purchased clothes for the wedding of her son with the yet ‘invisible
lady,’ and sent perpetually to jewellers to get presents for the ideal
future Princess of Wales.
 The lady, however, was not a merely visionary bride. It was
during the absence of the King in Hanover that it was delicately
contrived for him to see a marriageable princess—Augusta of Saxe
Gotha. He approved of what he saw, and wrote home to the Queen,
bidding her to prepare her son for the bridal.
Caroline communicated the order to Frederick, who received it
with due resignation. His mother, who had great respect for outward
observances, counselled him to begin his preparations for marriage
by sending away his ostentatiously maintained favourite, Miss Vane.
Frederick pleased his mother by dismissing Miss Vane, and then
pleased himself by raising to the vacant bad eminence Lady
Archibald Hamilton, a woman of thirty-five years of age and the
mother of ten children. The prince visited her at her husband’s
house, where he was as well received by the master as by the
mistress. He saw her constantly at her sister’s, rode out with her,
walked with her daily for hours in St. James’s Park, ‘and, whenever
she was at the drawing-room (which was pretty frequently), his
behaviour was so remarkable that his nose and her ear were
inseparable, whilst, without discontinuing, he would talk to her as if
he had rather been relating than conversing, from the time he came
into the room to the moment he left it, and then seemed to be
24
rather interrupted than to have finished.’
The first request made by Lady Archibald to her royal lover was,
that he would not be satisfied with putting away Miss Vane; but that
he would send her out of the country. The prince did not hesitate a
moment; he sent a royal message, wherein he was guilty of an act
of which no man would be guilty to the woman whom he had loved.
The message was taken by Lord Baltimore, who bore proposals,
offering an annuity of 1,600l. a year to the lady, on condition that
she would proceed to the continent, and give up the little son which
owed to her the disgrace of his birth, but to whom both she and the
prince were most affectionately attached. The alternative was
starvation in England.
 Miss Vane had an old admirer, to whom she sent in the hour of
adversity, and who was the more happy to aid her in her extremity
as, by so doing, he would not only have some claim on her
gratitude, but that he could, to the utmost of his heart’s desire,
annoy the prince, whom he intensely despised.
Lord Hervey sat down, and imagining himself for the nonce in
the place of Miss Vane, he wrote a letter in that lady’s name. The
supposed writer softly reproved the fickle prince, reminded him of
the fond old times ere love yet had expired, resigned herself to the
necessity of sacrificing her own interests to that of England, and
then running over the sacrifices which a foolish woman must ever
make—of character, friends, family, and peace of mind—for the fool
or knave whom she loves with more irregularity than wisdom, she
burst forth into a tone of indignation at the mingled meanness and
cruelty of which she was now made the object, and finally refused to
leave either England or her child, spurning the money offered by the
father, and preferring any fate which might come, provided she were
not banished from the presence and the love of her boy.
Frederick was simple enough to exhibit this letter to his mother,
sisters, and friends, observing at the same time that it was far too
clever a production to come from the hand of Miss Vane, and that he
would not give her a farthing until she had revealed the name of the
‘rascal’ who had written it. The author was popularly set down as
being Mr. Pulteney.
On the other hand, Miss Vane published the prince’s offer to her,
and therewith her own letter in reply. The world was unanimous in
condemning him as mean and cruel. Not a soul ever thought of
finding fault with him as immoral. At length a compromise was
effected. The prince explained away the cruel terms of his own
epistle, and Miss Vane withdrew what was painful to him in hers.
The pension of 1,600l. a year was settled on her, with which she
retired to a mansion in Grosvenor Street, her little son accompanying
her. But the anxiety she had undergone had so seriously affected her
health that she was very soon after compelled to proceed to Bath.
The waters were not healing waters for her. She died in that city, on
the 11th of March 1736, having had one felicity reserved for her in
her decline, the inexpressible one of seeing her little son die before
her. ‘The Queen and the Princess Caroline,’ says Lord Hervey,
‘thought the prince more afflicted for the loss of this child than they
had ever seen him on any occasion, or thought him capable of
being.’
One of the most cherished projects of George the Second was
the union by marriage of two of his own children with two of the
children of the King of Prussia. Such an alliance would have bound
more intimately the descendants of Sophia Dorothea through her
son and daughter. The double marriage was proposed to the King of
Prussia, in the name of the King of England, by Sir Charles Hotham,
minister-plenipotentiary. George proposed that his eldest son,
Frederick, should marry the eldest daughter of the King of Prussia,
and that his second daughter should marry the same King’s eldest
son. To these terms the Prussian monarch would not agree,
objecting that if he gave his eldest daughter to the Prince of Wales,
he must have the eldest, and not the second, daughter of George
and Caroline for the Prince of Prussia. Caroline would have agreed to
these terms; but George would not yield: the proposed
intermarriages were broken off, and the two courts were estranged
for years.
The Prussian princess, Frederica Wilhelmina, has published the
memoirs of her life and times; and Ranke, quoting them in his
‘History of the House of Brandenburgh,’ enters largely into the
matrimonial question, which was involved in mazes of diplomacy.
Into the latter it is not necessary to enter; but to those who would
know the actual causes of the failure of these proposed royal
marriages the following passage from Ranke’s work will not be
without interest:—
‘Whatever be their exaggerations and errors, the memoirs of the
Princess Frederica Wilhelmina must always be considered as one of
the most remarkable records of the state of the Prussian court of
that period. From these it is evident that neither she herself, nor the
Queen, had the least idea of the grounds which made the King
reluctant to give an immediate consent to the proposals. They saw
in him a domestic tyrant, severe only towards his family, and weak
to indifferent persons. The hearts on both sides became filled with
bitterness and aversion. The Crown Prince, too, who was still of an
age when young men are obnoxious to the influence of a clever
elder sister, was infected with these sentiments. With a view to
promote her marriage, he suffered himself to be induced to draw up
in secret a formal declaration that he would give his hand to no
other than an English princess. On the other hand, it is inconceivable
to what measures the other party had recourse, in order to keep the
King steady to his resolution. Seckendorf had entirely won over
General Grumbkoo, the King’s daily and confidential companion, to
his side; both of them kept up a correspondence of a revolting
nature with Reichenbach, the Prussian resident in London. This
Reichenbach, who boasts somewhere of his indifference to outward
honours, and who was, at all events, chiefly deficient in an inward
sense of honour, not only kept up a direct correspondence with
Seckendorf, in which he informed him of all that was passing in
England in relation to the marriage, and assured the Austrian agent
that he might reckon on him as on himself; but, what is far worse,
he allowed Grumbkoo to dictate to him what he was to write to the
King, and composed his despatches according to his directions. It is
hardly conceivable that these letters should not have been
destroyed; they were, however, found among Grumbkoo’s papers at
his death. Reichenbach, who played a subordinate part, but who
regarded himself as the third party to this conspiracy, furnished on
his side facts and arguments which were to be urged orally to the
King, in support of his statements. Their system was to represent to
the King that the only purpose of England was to reduce Prussia to
the condition of a province, and to turn a party around him that
might fetter and control all his actions; representations to which
Frederick William was already disposed to lend an ear. He wished to
avoid having an English daughter-in-law because he feared he
should be no longer master in his own house; perhaps she would
think herself of more importance than he; he should die, inch by
inch, of vexation. On comparing these intrigues, carried on on either
side of the King, we must admit that the former—those in his own
family—were the more excusable, since their sole object was the
accomplishment of those marriages, upon the mere suspicion of
which the King broke out into acts of violence which terrified his
family and his kingdom and astonished Europe. The designs of the
other party were far more serious; their purpose was to bind Prussia
in every point to the existing system, and to keep her aloof from
England. Of this the King had no idea; he received without suspicion
whatever Reichenbach wrote or Grumbkoo reported to him.’
The mutual friends, whose interest it was to keep Prussia and
England wide apart, laboured with a zeal worthy of a better cause,
and not only broke the proposed marriages, but made enemies of
the two Kings. A dispute was built up between them touching
Mecklenburgh; and Prussian press-gangs and recruiting parties
crossed into the Hanoverian territory, and carried off or inveigled the
King of England’s Electoral subjects into the military service of
Prussia. This was the most outrageous insult that could have been
devised against the English monarch, and it was the most cruel that
could be inflicted upon the inhabitants of the Electorate.
The King of Prussia was not nice of his means for entrapping
men, nor careful on whose territory he seized them, provided only
they were obtained. The districts touching on the Prussian frontier
were kept in a constant state of alarm, and border frays were as
frequent and as fatal as they were on England and Scotland’s neutral
ground, which derived its name from an oblique application of
etymology, and was so called because neither country’s faction
hesitated to commit murder or robbery upon it. I have seen in the
inns near these frontiers some strange memorials of these old times.
Those I allude to are in the shape of mandats, or directions, issued
by the authorities, and they are kept framed and glazed, old
curiosities, like the ancient way-bill at the Swan at York, which
announces a new fast coach travelling to London, God willing, in a
week. These mandats, which were very common in Hanover when
Frederick, after refusing the English alliance, took to sending his
Werbers, or recruiters, to lay hold of such of the people as were
likely to make good tall soldiers, were to this effect: they enjoined all
the dwellers near the frontiers to be provided with arms and
ammunition; the militia to hold themselves ready against any
surprise; the arms to be examined every Sunday by the proper
authorities; watch and ward to be maintained day and night; patrols
to be active; and it was ordered, that, the instant any strange
soldiers were seen approaching, the alarm-bells should be sounded
and preparations be made for repelling force by force. The Prussian
Werbers, as they were called, were wont sometimes to do their
spiriting in shape so questionable that the most anti-belligerent
travellers and the most unwarlike and well-intentioned bodies were
liable to be fired upon if their characters were not at once explained
and understood. These were times when Hanoverians, who stood in
fear of Prussia, never lay down in bed but with arms at their side;
times when young peasants who, influenced by soft attractions, stole
by night from one village to another to pay their devoirs to bright
eyes waking to receive them, walked through perils, love in their
hearts, and a musket on their shoulders. The enrollers of Frederick,
and indeed those of his great son after him, cast a chill shadow of
fear over every age, sex, and station of life.
In the meantime the two Kings reviled each other as coarsely as
any two dragoons in their respective services. The quarrel was
nursed until it was proposed to be settled, not by diplomacy, but by
a duel. When this was first suggested, the place, but not the time, of
meeting, was immediately agreed upon. The territory of Hildesheim
was to be the spot whereon were to meet in deadly combat two
monarchs—two fathers, who could not quietly arrange a marriage
between their sons and daughters. It really seemed as if the blood
of Sophia Dorothea of Zell was ever to be fatal to peace and averse
from connubial felicity.
The son of Sophia Dorothea selected Brigadier-General Sutton
for his second. Her son-in-law (it will be remembered that he had
married that unhappy lady’s daughter) conferred a similar honour on
Colonel Derschein. His English Majesty was to proceed to the
designated arena from Hanover; Frederick was to make his way
thither from Saltzdhal, near Brunswick. The two Kings of Brentford
could not have looked more ridiculous than these two. They would,
undoubtedly, have crossed weapons, had it not been for the strong
common sense of a Prussian diplomatist, named Borck. ‘It is quite
right and exceedingly dignified,’ said Borck one day, to his master,
when the latter was foaming with rage against George the Second,
and expressing an eager desire for fixing a near day whereon to
settle their quarrel—‘it is most fitting and seemly, since your Majesty
will not marry with England, to cut the throat, if possible, of the
English monarch; but your faithful servant would still advise your
Majesty not to be over-hasty in fixing the day: ill-luck might come of
it.’ On being urged to show how this might be, he remarked—‘Your
gracious Majesty has lately been ill, is now far from well, and might,
by naming an early day for voidance of this quarrel, be unable to
keep the appointment.’ ‘We would name another,’ said the King. ‘And
in the meantime,’ observed Borck, ‘all Europe generally, and George
of England in particular, would be smiling, laughing, commenting on,
and ridiculing the King who failed to appear where he had promised
to be present with his sword. Your Majesty must not expose your
sacred person and character to such a catastrophe as this: settle
nothing till there is certainty that the pledge will be kept; and, in the
meantime, defer naming the day of battle for a fortnight.’
The advice of Borck was followed, and of course the fight never
‘came off.’ The ministers of both governments exerted themselves to
save their respective masters from rendering themselves supremely,
and perhaps sanguinarily, ridiculous—for the blood of both would not
have washed out the absurdity of the thing. Choler abated,
common-sense came up to the surface, assumed the supremacy,
and saved a couple of foolish kings from slaying or mangling each
other. George, however, was resolved, and that for more reasons
than it is necessary to specify, that a wife must be found for his heir-
apparent; and it was Caroline who directed him to look at the
princesses in the small and despotic court of Saxe Gotha. Walpole
was the more anxious that the Prince of Wales should be fittingly
matched, as a report had reached him that Frederick had accepted
an offer from the Duchess of Marlborough of a hundred thousand
pounds and the hand of her favourite grand-daughter, Lady Diana
Spencer. The marriage, it was said, was to come off privately, at the
duchess’s lodge in Richmond Park.
Lord Delawar, who was sent to demand the hand of the Princess
Augusta from her brother, the Duke of Saxe Gotha, was long, lank,
awkward, and unpolished. There was no fear here of the catastrophe
which followed on the introduction to Francesca da Rimini of the
handsome envoy whom she mistook for her bridegroom, and with
whom she fell in love as soon as she beheld him.
Walpole, writing from King’s College on the 2nd of May 1736,
says: ‘I believe the princess will have more beauties bestowed upon
her by the occasional poets than even a painter would afford her.
They will cook up a new Pandora, and in the bottom of the box
enclose Hope—that all they have said is true. A great many, out of
excess of good breeding, who have heard that it was rude to talk
Latin before women, proposed complimenting her in English; which
she will be much the better for. I doubt most of them, instead of
fearing their compositions should not be understood, should fear
they should; they wish they don’t know what to be read by they
don’t know who.’
When the King despatched some half dozen lords of his council
to propose to the prince that he should espouse the youthful
Princess Augusta, he replied, with a tone of mingled duty and
indifference, something like Captain Absolute in the play, that
‘whoever his Majesty thought a proper match for his son would be
agreeable to him.’
The match was straightway resolved upon; and as the young
lady knew little of French and less of English, it was suggested to
her mother that a few lessons in both languages would not be
thrown away. The Duchess of Saxe Gotha, however, was wiser in her
own conceit than her officious counsellors; and remembering that
the Hanoverian family had been a score of years, and more, upon
the throne of England, she very naturally concluded that the people
all spoke or understood German, and that it would really be
needlessly troubling the child to make her learn two languages, to
acquire a knowledge of which would not be worth the pains spent
upon the labour.
When princesses then espoused heirs to thrones they were
treated but with very scanty ceremony. Their own feelings were
allowed to exercise very little influence in the matter; there was no
pleasant wooing time; the bridegroom did not even give himself the
trouble to seek the bride—he does not always do so, even now; and
when the bride married the deputy who was despatched to espouse
her by proxy, she knew as little of the principal as she did of his
representative. But the blooming young Princess of Saxe Gotha
submitted joyfully to custom and the chance of becoming Queen of
England. She was willing to come and win what the Prince of Wales,
had not dignity made him ungallant, should have gone and laid at
her feet and besought her to accept. Accordingly, the royal yacht,
William and Mary, destined to carry many a less noble freight before
its career was completed, bore the bride to our shores. When Lord
Delawar handed the bride ashore at Greenwich, on the 25th of April
1736, she excited general admiration by her fresh air, good humour,
and tasteful dress. It was St. George’s day; no inauspicious day
whereon landing should be made in England by the young girl of
seventeen, who was to be the mother of the first king born and bred
in England since the birthday of James II.
The royal bride was conducted to the Queen’s house in the park,
where, as my fair readers, and indeed all readers with equal good
sense and a proper idea of the fitness of things, will naturally
conclude that all the royal family had assembled to welcome, with
more than ordinary warmth, one who came among them under
circumstances of more than ordinary interest. But the truth is that
there was no one to give her welcome but solemn officers of state
and criticising ladies-in-waiting. The people were there of course,
and the princess had no cause to complain of any lack of warmth on
their part. For want of better company, she spent half an hour with
the English commonalty; and as she sat in the balcony overlooking
the park, the gallant mob shouted themselves hoarse in her praise,
and did her all homage until the tardy lover arrived, whose own
peculiar homage he should have been in a little more lover-like haste
to pay. However, Frederick came at last, and he came alone. The
King, Queen, duke, and princesses sent ‘their compliments, and
hoped she was well!’ They could not have sent or said less had she
been Griselda, fresh from her native cottage and about to become
the bride of the prince without their consent and altogether without
their will. But the day was Sunday, and perhaps those distinguished
personages were reluctant to indulge in too much expansion of
feeling on the sacred day.
On the following day, Monday, Greenwich was as much alive as it
used to be on a fine fair-day: for the princess dined in public, and all
the world was there to see her. That is to say, she and the prince
dined together in an apartment the windows of which were thrown
open ‘to oblige the curiosity of the people;’ and it is only to be hoped
that the springs of the period were not such inclement seasons as
those generally known by the name of spring to us. The people
having stared their fill, and the princess having banqueted as
comfortably as she could under such circumstances, the Prince of
Wales took her down to the water, led her into a gaily decorated
barge, and slowly up the river went the lovers—with horns playing,
streamers flying, and under a fusillade from old stocks of old guns,
the modest artillery of colliers and Other craft anxious to render to
the pair the usual noisy honours of the way. They returned to
Greenwich in like manner, similarly honoured, and there, having
supped in public, the prince kissed her hand, took his leave, and
promised to return upon the morrow.
On the Tuesday the already enamoured Frederick thought better
of his engagement, and tarried at home till the princess arrived
there. She had left Greenwich in one of the royal carriages, from
which she alighted at Lambeth, where, taking boat, she crossed to
Whitehall. Here one of Queen Caroline’s state chairs was awaiting
her, and in it she was borne, by two stout carriers, plump as Cupids
but more vigorous, to St. James’s Palace. The reception here was
magnificent and tasteful. On the arrival of the bride, the bridegroom,
already there to receive her, took her by the hand as she stepped
out of the chair, softly checked the motion she made to kneel to him
and kiss his hand, and, drawing her to him, gallantly impressed a
kiss—nay two, for the record is very precise on this matter—upon
her lips. All confusion and happiness, the illustrious couple ascended
the staircase hand in hand. The prince led her into the presence of a
splendid and numerous court, first introducing her to the King, who
would not suffer her to kneel, but, putting his arm around her,
sainted her on each cheek. Queen Caroline greeted as warmly the
bride of her eldest son; and the Duke of Cumberland and the
princesses congratulated her on her arrival in terms of warm
affection.
The King, who had been irritably impatient for the arrival of the
bride, and had declared that the ceremony should take place without
him if it were not speedily concluded, was softened by the behaviour
of the youthful princess on her first appearing in his presence. ‘She
threw herself all along on the floor, first at the King’s and then at the
25
Queen’s feet.’ This prostration was known to be so acceptable a
homage to his Majesty’s pride, that, joined to the propriety of her
whole behaviour on this occasion, it gave the spectators great
prejudice in favour of her understanding.
The poor young princess, who came into England
unaccompanied by a single female friend, behaved with a propriety
and ease which won the admiration of Walpole and the sneers of old
ladies who criticised her. Her self-possession, joined as it was with
modesty, showed that she was ‘well-bred.’ She was not
irreproachable of shape or carriage, but she was fair, youthful, and
sensible—much more sensible than the bridegroom, who quarrelled
with his brothers and sisters, in her very presence, upon the right of
sitting down and being waited on in such presence!
 The squabbles between the brothers and sisters touching
etiquette show the extreme littleness of the minds of those who
engaged in them. The prince would have had them, on the occasion
of their dining with himself and bride the day before the wedding, be
satisfied with stools instead of chairs, and consent to being served
with something less than the measure of respect shown to him and
the bride. To meet this, they refused to enter the dining-room till the
stools were taken away and chairs substituted. They then were
waited upon by their own servants, who had orders to imitate the
servants of the Prince of Wales in every ceremony used at table.
Later in the evening, when coffee was brought round by the prince’s
servants, his visitors declined to take any, out of fear that their
brother’s domestics might have had instructions to inflict ‘some
disgrace (had they accepted of any) in the manner of giving it!’
On the day of the arrival of the bride at St. James’s, after a
dinner of some state, and after some rearrangement of costume, the
ceremony of marriage was performed, under a running salute from
artillery, which told to the metropolis the progress made in the
nuptial solemnity. The bride ‘was in her hair,’ and wore a crown with
one bar, as Princess of Wales, a profusion of diamonds adding lustre
to a youthful bearing that could have done without it. Over her white
robe she wore a mantle of crimson velvet, bordered with row upon
row of ermine. Her train was supported by four ‘maids,’ three of
whom were daughters of dukes. They were Lady Caroline Lennox,
daughter of the Duke of Richmond; Lady Caroline Fitzroy, daughter
of the Duke of Grafton; Lady Caroline Cavendish, daughter of the
Duke of Devonshire,—and with the three bridesmaids who bore the
name of the Queen was one who bore that of her whom the King
had looked upon as really Queen of England—of Sophia, his mother.
This fourth lady was Lady Sophia Fermor, daughter of the Earl of
Pomfret. Excepting the mantle, the ‘maids’ were dressed precisely
similar to the ‘bride’ whom they surrounded and served. They were
all in ‘virgin habits of silver.’ Each bridesmaid wore diamonds of the
value of from twenty to thirty thousand pounds.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebooknice.com