David J.

Olive

Linear
Regression
Linear Regression
David J. Olive

Linear Regression

123
David J. Olive
Department of Mathematics
Southern Illinois University
Carbondale, IL, USA

ISBN 978-3-319-55250-7 ISBN 978-3-319-55252-1 (eBook)

DOI 10.1007/978-3-319-55252-1

Library of Congress Control Number: 2017934111

Mathematics Subject Classification (2010): 62J05

Springer International Publishing AG 2017

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, express or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

Printed on acid-free paper

This Springer imprint is published by Springer Nature

The registered company is Springer International Publishing AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface

Regression is the study of the conditional distribution Y |x of the response

variable Y given the p 1 vector of predictors x. In a linear regression
model, Y = T x + e, and Y is conditionally independent of x given a single
linear combination T x of the predictors, written

Y x| T x.

Multiple linear regression and many experimental design models are spe-
cial cases of the linear regression model, and the models can be presented
compactly by dening the population model in terms of the sucient predic-
tor SP = T x and the estimated model in terms of the estimated sucient
T
predictor ESP = x. In particular, the response plot or estimated su-
cient summary plot of the ESP versus Y is used to visualize the conditional
distribution Y | T x. The residual plot of the ESP versus the residuals is used
to visualize the conditional distribution of the residuals given the ESP.
The literature on multiple linear regression is enormous. See Stigler (1986)
and Harter (1974a,b, 1975a,b,c, 1976) for history. Draper (2002) is a good
source for more recent literature. Some texts that were standard at one time
include Wright (1884), Johnson (1892), Bartlett (1900), Merriman (1907),
Weld (1916), Leland (1921), Ezekiel (1930), Bennett and Franklin (1954),
Ezekiel and Fox (1959), and Brownlee (1965). Recent reprints of several of
these texts are available from www.amazon.com.
Draper and Smith (1966) was a breakthrough because it popularized the
use of residual plots, making the earlier texts obsolete. Excellent texts include
Chatterjee and Hadi (2012), Draper and Smith (1998), Fox (2015), Hamil-
ton (1992), Kutner et al. (2005), Montgomery et al. (2012), Mosteller and
Tukey (1977), Ryan (2009), Sheather (2009), and Weisberg (2014). Cook and
Weisberg (1999a) was a breakthrough because of its use of response plots.

v
vi Preface

Other texts of interest include Abraham and Ledolter (2006), Harrell

(2015), Pardoe (2012), Mickey et al. (2004), Cohen et al. (2003), Kleinbaum
et al. (2014), Mendenhall and Sincich (2011), Vittingho et al. (2012), and
Berk (2003).
This text is an introduction to linear regression models for undergraduates
and beginning graduate students in a mathematics or statistics department.
The text is for graduate students in elds like quantitative psychology. The
prerequisites for this text are linear algebra and a calculus-based course in
statistics at the level of Chihara and Hesterberg (2011), Hogg et al. (2014),
Rice (2006), or Wackerly et al. (2008). The student should be familiar with
vectors, matrices, condence intervals, expectation, variance, normal distri-
bution, and hypothesis testing.
This text will not be easy reading for nonmathematical students. Lindsey
(2004) and Bowerman and OConnell (2000) attempt to present regression
models to students who have not had calculus or linear algebra. Also see
Kachigan (1991, ch. 35) and Allison (1999).
This text does not give much history of regression, but it should be noted
that many of the most important ideas in statistics are due to Fisher, Ney-
man, E.S. Pearson, and K. Pearson. See Lehmann (2011). For example, David
(20062007) says that the following terms were due to Fisher: analysis of
variance, confounding, consistency, covariance, degrees of freedom, eciency,
factorial design, information, information matrix, interaction, level of sig-
nicance, likelihood, location, maximum likelihood, null hypothesis, pivotal
quantity, randomization, randomized blocks, sampling distribution, scale,
statistic, Students t, test of signicance, and variance.
David (20062007) says that terms due to Neyman and E.S. Pearson in-
clude alternative hypothesis, composite hypothesis, likelihood ratio, power,
power function, simple hypothesis, size of critical region, test criterion, test
of hypotheses, and type I and type II errors. Neyman also coined the term
condence interval. David (20062007) says that terms due to K. Pearson
include bivariate normal, goodness of t, multiple regression, nonlinear re-
gression, random sampling, skewness, standard deviation, and weighted least
squares.
This text is dierent from the massive competing literature in several
ways. First, response plots are heavily used in this text. With the response
plot, the presentation for multiple linear regression is about the same as the
presentation for simple linear regression. Hence the text immediately starts
with the multiple linear regression model, rather than spending 100 pages on
simple linear regression and then covering multiple regression.
Second, the assumption of iid normal N (0, 2 ) errors is replaced by the as-
sumption that the iid zero mean errors have constant variance 2 . Then large
sample theory can be used to justify hypothesis tests, condence intervals,
and prediction intervals.
Third, the multivariate linear model y i = B T xi + i for i = 1, . . . , n has
m 2 response variables Y1 , . . . , Ym and p predictor variables x1 , x2 , . . . , xp .
Preface vii

Multivariate linear regression and MANOVA models are special cases. Recent
results from Kakizawa (2009), Su and Cook (2012), Olive et al. (2015), and
Olive (2016b) make the multivariate linear regression model (Chapter 12)
easy to learn after the student has mastered the multiple linear regression
model (Chapters 2 and 3). For the multivariate linear regression model, it is
assumed that the iid zero mean error vectors have fourth moments.
Fourth, recent literature on plots for goodness and lack of t, bootstrapping,
outlier detection, response transformations, prediction intervals, prediction
regions, and variable selection has been incorporated into the text. See Olive
(2004b, 2007, 2013a,b, 2016a,b,c) and Olive and Hawkins (2005).
Chapter 1 reviews the material to be covered in the text and can be
skimmed and then referred to as needed. Chapters 2 and 3 cover multiple lin-
ear regression, Chapter 4 considers generalized least squares, and Chapters 5
through 9 consider experimental design models. Chapters 10 and 11 cover lin-
ear model theory and the multivariate normal distribution. These chapters
are needed for the multivariate linear regression model covered in Chapter 12.
Chapter 13 covers generalized linear models (GLMs) and generalized additive
models (GAMs).
The text also uses recent literature to provide answers to the following
important questions:
How can the conditional distribution Y | T x be visualized?
How can be estimated?
How can variable selection be performed eciently?
How can Y be predicted?
The text emphasizes prediction and visualizing the models. Some of the
applications in this text using this research are listed below.
1) It is shown how to use the response plot to detect outliers and to assess
the adequacy of linear models for multiple linear regression and experimental
design.
2) A graphical method for selecting a response transformation for linear
models is given. Linear models include multiple linear regression and many
experimental design models. This method is also useful for multivariate linear
regression.
3) A graphical method for assessing variable selection for the multiple
linear regression model is described. It is shown that for submodels I with
k predictors, the widely used screen Cp (I) k is too narrow. More good
submodels are considered if the screen Cp (I) min(2k, p) is used. Variable
selection methods originally meant for multiple linear regression can be ex-
tended to GLMs. See Chapter 13. Similar ideas from Olive and Hawkins
(2005) have been incorporated in Agresti (2013). Section 3.4.1 shows how to
bootstrap the variable selection estimator.
4) Asymptotically optimal prediction intervals for a future response Yf
are given for models of the form Y = T x + e where the errors are iid,
viii Preface

unimodal, and independent of x. Asymptotically optimal prediction regions

are developed for multivariate linear regression.
5) Rules of thumb for selecting predictor transformations are given.
6) The DD plot is a graphical diagnostic for whether the predictor dis-
tribution is multivariate normal or from some other elliptically contoured
distribution. The DD plot is also useful for detecting outliers in the predic-
tors and for displaying prediction regions for multivariate linear regression.
7) The multivariate linear regression model has m response variables.
Plots, prediction regions, and tests are developed that make this model nearly
as easy to use as the multiple linear regression model (m = 1), at least for
small m.
Throughout the book, there are goodness of t and lack of t plots for
examining the model. The response plot is especially important.
The website (http://lagrange.math.siu.edu/Olive/lregbk.htm) for
this book provides R programs in the le lregpack.txt and several R data
sets in the le lregdata.txt. Section 14.1 discusses how to get the data sets
and programs into the software, but the following commands will work.
Downloading the books R functions lregpack.txt and data les lreg-
data.txt into R: The commands
source("http://lagrange.math.siu.edu/Olive/lregpack.txt")
source("http://lagrange.math.siu.edu/Olive/lregdata.txt")

can be used to download the R functions and data sets into R. Type ls(). Over
65 R functions from lregpack.txt should appear. In R, enter the command q().
A window asking Save workspace image? will appear. Click on No to remove
the functions from the computer (clicking on Yes saves the functions on R,
but the functions and data are easily obtained with the source commands).
Chapters 27 can be used for a one-semester course in regression and
experimental design. For a course in generalized linear models, replace some of
the design chapters by Chapter 13. Design chapters could also be replaced by
Chapters 12 and 13. A more theoretical course would cover Chapters 1,10, 11,
and 12.
Acknowledgments
This work has been partially supported by NSF grants DMS 0202922 and
DMS 0600933. Collaborations with Douglas M. Hawkins and R. Dennis Cook
were extremely valuable. I am grateful to the developers of useful mathemat-
ical and statistical techniques and to the developers of computer software
and hardware (including R Core Team (2016)). Cook (1998) and Cook and
Weisberg (1999a) inuenced this book. Teaching material from this text has
been invaluable. Some of the material in this text has been used in a Math
583 regression graphics course, a Math 583 experimental design course, and
a Math 583 robust statistics course. In 2009 and 2016, Chapters 2 to 7 were
used in Math 484, a course on multiple linear regression and experimental
design. Chapters 11 and 12 were used in a 2014 Math 583 theory of linear
Preface ix

models course. Chapter 12 was also used in a 2012 Math 583 multivariate
analysis course. Chapter 13 was used for a categorical data analysis course.
Thanks also goes to Springer, to Springers associate editor Donna
Chernyk, and to several reviewers.

Carbondale, IL, USA David J. Olive

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Some Regression Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Multiple Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Variable Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4 Other Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.5 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2 Multiple Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.1 The MLR Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 Checking Goodness of Fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3 Checking Lack of Fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.1 Residual Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.2 Other Model Violations . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.4 The ANOVA F Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.5 Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.6 The Partial F Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.7 The Wald t Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.8 The OLS Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.9 Two Important Special Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.9.1 The Location Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.9.2 Simple Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . 57
2.10 The No Intercept MLR Model . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.12 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2.12.1 Lack of Fit Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
2.13 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3 Building an MLR Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

3.1 Predictor Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.2 Graphical Methods for Response Transformations . . . . . . . . . 92

xi
xii Contents

3.3 Main Eects, Interactions, and Indicators . . . . . . . . . . . . . . . . 97

3.4 Variable Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.4.1 Bootstrapping Variable Selection . . . . . . . . . . . . . . . . . 119
3.5 Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
3.6 Outlier Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
3.8 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
3.9 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

4 WLS and Generalized Least Squares . . . . . . . . . . . . . . . . . . . . . . 163

4.1 Random Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
4.2 GLS, WLS, and FGLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
4.3 Inference for GLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4.4 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
4.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

5 One Way Anova . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
5.2 Fixed Eects One Way Anova . . . . . . . . . . . . . . . . . . . . . . . . . . 177
5.3 Random Eects One Way Anova . . . . . . . . . . . . . . . . . . . . . . . . 189
5.4 Response Transformations for Experimental Design . . . . . . . 191
5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
5.6 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
5.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

6 The K Way Anova Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

6.1 Two Way Anova . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
6.2 K Way Anova Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
6.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
6.4 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
6.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

7 Block Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

7.1 One Way Block Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
7.2 Blocking with the K Way Anova Design . . . . . . . . . . . . . . . . . 233
7.3 Latin Square Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
7.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
7.5 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
7.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

8 Orthogonal Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

8.1 Factorial Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
8.2 Fractional Factorial Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
8.3 Plackett Burman Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
8.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
8.5 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
8.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
Contents xiii

9 More on Experimental Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

9.1 Split Plot Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
9.1.1 Whole Plots Randomly Assigned to A . . . . . . . . . . . . . 284
9.1.2 Whole Plots Assigned to A as in a CRBD . . . . . . . . . 286
9.2 Review of the DOE Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
9.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
9.4 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
9.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

10 Multivariate Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

10.1 The Multivariate Normal Distribution . . . . . . . . . . . . . . . . . . . 300
10.2 Elliptically Contoured Distributions . . . . . . . . . . . . . . . . . . . . . 303
10.3 Sample Mahalanobis Distances . . . . . . . . . . . . . . . . . . . . . . . . . 307
10.4 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
10.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309

11 Theory for Linear Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

11.1 Projection Matrices and the Column Space . . . . . . . . . . . . . . . 313
11.2 Quadratic Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
11.3 Least Squares Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
11.3.1 Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
11.4 Nonfull Rank Linear Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
11.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
11.6 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
11.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340

12 Multivariate Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
12.2 Plots for the Multivariate Linear Regression Model . . . . . . . . 348
12.3 Asymptotically Optimal Prediction Regions . . . . . . . . . . . . . . 350
12.4 Testing Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
12.5 An Example and Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
12.5.1 Simulations for Testing . . . . . . . . . . . . . . . . . . . . . . . . . . 372
12.5.2 Simulations for Prediction Regions . . . . . . . . . . . . . . . . 375
12.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
12.7 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
12.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383

13 GLMs and GAMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389

13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
13.2 Additive Error Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
13.3 Binary, Binomial, and Logistic Regression . . . . . . . . . . . . . . . . 394
13.4 Poisson Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
13.5 Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
13.6 Variable Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
xiv Contents

13.7 Generalized Additive Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 428

13.7.1 Response Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
13.7.2 The EE Plot for Variable Selection . . . . . . . . . . . . . . . 432
13.7.3 An EE Plot for Checking the GLM . . . . . . . . . . . . . . . 433
13.7.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433
13.8 Overdispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
13.9 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
13.10 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442

14 Stu for Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459

14.1 R and Arc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459
14.2 Hints for Selected Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465
14.3 Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489
Chapter 1
Introduction

This chapter provides a preview of the book but is presented in a rather ab-
stract setting and will be easier to follow after reading the rest of the book.
The reader may omit this chapter on rst reading and refer back to it as nec-
essary. Chapters 2 to 9 consider multiple linear regression and experimental
design models t with least squares. Chapter 1 is useful for extending several
techniques, such as response plots and plots for response transformations used
in those chapters, to alternative tting methods and to alternative regression
models. Chapter 13 illustrates some of these extensions for the generalized
linear model (GLM) and the generalized additive model (GAM).
Response variables are the variables of interest, and are predicted with
a p 1 vector of predictor variables x = (x1 , . . . , xp )T where xT is the
transpose of x. A multivariate regression model has m > 1 response variables.
For example, predict Y1 = systolic blood pressure and Y2 = diastolic blood
pressure using a constant x1 , x2 = age, x3 = weight, and x4 = dosage amount
of blood pressure medicine. The multivariate location and dispersion model
of Chapter 10 is a special case of the multivariate linear regression model of
Chapter 12.
A univariate regression model has one response variable Y . Suppose Y
is independent of the predictor variables x given a function h(x), written
Y x|h(x), where h : Rp Rd and the integer d is as small as possible. Then
Y follows a dD regression model, where d p since Y x|x. If Y x, then
Y follows a 0D regression model. Then there are 0D, 1D, . . . , pD regression
models, and all univariate regression models are dD regression models for
some integer 0 d p. Cook (1998, p. 49) and Cook and Weisberg (1999a,
p. 414) use similar notation with h(x) = (xT 1 , . . . , xT d )T .
The remainder of this chapter considers 1D regression models, where h :
Rp R is a real function. The additive error regression model Y = m(x) + e
is an important special case with h(x) = m(x). See Section 13.2. An impor-
tant special case of the additive error model is the linear regression model
Y = xT + e = x1 1 + + xp p + e. Multiple linear regression and many
experimental design models are special cases of the linear regression model.

Springer International Publishing AG 2017 1

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 1
2 1 Introduction

The multiple linear regression model has at least one predictor xi that
takes on many values. Chapter 2 ts this model with least squares and Chap-
ter 3 considers variable selection models such as forward selection. There are
many other methods for tting the multiple linear regression model, includ-
ing lasso, ridge regression, partial least squares (PLS), and principal com-
ponent regression (PCR). See James et al. (2013), Olive (2017), and Pelawa
Watagoda and Olive (2017). Chapters 2 and 3 consider response plots, plots
for response transformations, and prediction intervals for the multiple linear
regression model t by least squares. All of these techniques can be extended
to alternative tting methods.

1.1 Some Regression Models

All models are wrong, but some are useful.

Box (1979)

In data analysis, an investigator is presented with a problem and data from

some population. The population might be the collection of all possible out-
comes from an experiment while the problem might be predicting a future
value of the response variable Y or summarizing the relationship between Y
and the p 1 vector of predictor variables x. A statistical model is used to
provide a useful approximation to some of the important underlying charac-
teristics of the population which generated the data. Many of the most used
models for 1D regression, dened below, are families of conditional distribu-
tions Y |x = xo indexed by x = xo . A 1D regression model is a parametric
model if the conditional distribution is completely specied except for a xed
nite number of parameters, otherwise, the 1D model is a semiparametric
model. GLMs and GAMs, dened below, are covered in Chapter 13.

Denition 1.1. Regression investigates how the response variable Y

changes with the value of a p 1 vector x of predictors. Often this condi-
tional distribution Y |x is described by a 1D regression model, where Y is
conditionally independent of x given the sucient predictor SP = h(x),
written
Y x|SP or Y x|h(x), (1.1)
where the real valued function h : Rp R. The estimated sucient predictor
ESP = h(x). An important special case is a model with a linear predictor
T
h(x) = + T x where ESP = + x. This class of models includes the gen-
eralized linear model (GLM). Another important special case is a generalized
additive model (GAM), where Y is independent of x = (x1 , . . . , xp )T given the
p
additive predictor AP = + j=1 Sj (xj ) for some (usually unknown) func-
p
tions Sj . The estimated additive predictor EAP = ESP = + j=1 Sj (xj ).
1.1 Some Regression Models 3

Notation: In this text, a plot of x versus Y will have x on the horizontal

axis, and Y on the vertical axis.

Plots are extremely important for regression. When p = 1, x is both a

sucient predictor and an estimated sucient predictor. So a plot of x versus
Y is both a sucient summary plot and a response plot. Usually the SP is
unknown, so only the response plot can be made. The response plot will be
extremely useful for checking the goodness of t of the 1D regression model.

Denition 1.2. A sucient summary plot is a plot of the SP versus Y .

An estimated sucient summary plot (ESSP) or response plot is a plot of
the ESP versus Y .

Notation. Often the index i will be suppressed. For example, the linear
regression model
Yi = + T xi + ei (1.2)
for i = 1, . . . , n where is a p 1 unknown vector of parameters, and ei
is a random error. This model could be written Y = + T x + e. More
accurately, Y |x = + T x + e, but the conditioning on x will often be
suppressed. Often the errors e1 , . . . , en are iid (independent and identically
distributed) from a distribution that is known except for a scale parameter.
For example, the ei s might be iid from a normal (Gaussian) distribution
with mean 0 and unknown standard deviation . For this Gaussian model,
estimation of , , and is important for inference and for predicting a new
value of the response variable Yf given a new vector of predictors xf .

The class of 1D regression models is very rich, and many of the most
used statistical models, including GLMs and GAMs, are 1D regression mod-
els. Nonlinear regression, nonparametric regression, and linear regression are
special cases of the additive error regression model

Y = h(x) + e = SP + e. (1.3)

The multiple linear regression model and experimental design model or

ANOVA model are special cases of the linear regression model. Another
important class of parametric or semiparametric 1D regression models has
the form
Y = g( + xT , e) or Y = g(xT , e). (1.4)
Special cases include GLMs and the response transformation model

Z = t1 ( + T x + e) (1.5)

where t1 is a one to one (typically monotone) function. Hence

Y = t(Z) = + T x + e. (1.6)
4 1 Introduction

Sections 3.2 and 5.4 show how to choose the response transformation t(Z)
graphically, and these techniques are easy to extend to the additive error
regression model Y = h(x) + e. Then the response transformation model is
Y = t (Z) = h (x) + e, and the graphical method for selecting the response
transformation is to plot hi (x) versus ti (Z) for several values of i , choosing
the value of = 0 where the plotted points follow the identity line with unit
slope and zero intercept. For the multiple linear regression model, hi (x) =
xT i where i can be found using the desired tting method, e.g. lasso.

In the literature, the response variable is sometimes called the dependent

variable while the predictor variables are sometimes called carriers, covariates,
explanatory variables, or independent variables. The ith case (Yi , xTi ) consists
of the values of the response variable Yi and the predictor variables xTi =
(xi,1 , . . . , xi,p ) where p is the number of predictors and i = 1, . . . , n. The
sample size n is the number of cases.

Box (1979) warns that all models are wrong, but some are useful. For
example, the function g in equation (1.4) or the error distribution could
be misspecied. Diagnostics are used to check whether model assumptions
such as the form of g and the proposed error distribution are reasonable.
Often diagnostics use residuals ri . For example, the additive error regression
model (1.3) uses
ri = Yi h(xi )
where h(x) is an estimate of h(x).

Exploratory data analysis (EDA) can be used to nd useful models when

the form of the regression model is unknown. For example, if the monotone
function t is unknown, and

Z = t1 ( + T x + e), (1.7)

then the transformation

Y = t(Z) = + T x + e (1.8)

follows a linear regression model. EDA can be used to nd response and

predictor transformations to build a model. See Sections 3.1 and 3.2.

After selecting a 1D regression model such as a GLM, the investigator will

often want to check whether the model is useful and to perform inference.
Several things to consider are listed below.
i) Use the response plot (and the sucient summary plot) to explain the
1D regression model to consulting clients, students, or researchers.
ii) Goodness of t: use the response plot to show that the model provides a
simple, useful approximation for the relationship between the response vari-
1.2 Multiple Linear Regression 5

able Y and the predictors x. The response plot is used to visualize the con-
ditional distribution of Y |x, Y |SP , and Y |( + T x) if SP = + T x.
iii) Check for lack of t of the model with a residual plot of the ESP versus
the residuals.
iv) Fit the model and nd h(x). If SP = + T x, estimate and , e.g.,
using maximum likelihood estimators.
v) Estimate the mean function E(Yi |xi ) = (xi ) = di (xi ) or estimate
(xi ) where the di are known constants.
vii) Check for overdispersion with an OD plot. See Section 13.8.
viii) Check whether Y is independent of x, that is, check whether the
nontrivial predictors x are needed in the model. Check whether SP = h(x)
c where the constant c does not depend on the xi . If SP = + T x, check
whether = 0, for example, test Ho : = 0,
ix) Check whether a reduced model can be used instead of the full model.
If SP = + T x = + TR xR + TO xO where the r 1 vector xR consists
of the nontrivial predictors in the reduced model, test Ho : O = 0.
x) Use variable selection to nd a good submodel.
xi) Predict Yi given xi .

The eld of statistics known as regression graphics gives useful results for
examining the 1D regression model (1.1) even when the model is unknown or
misspecied. The following section shows that the sucient summary plot is
useful for explaining the given 1D model while the response plot can often be
used to visualize the conditional distribution of Y |SP . Also see Chapter 13
and Olive (2013b).

1.2 Multiple Linear Regression

Suppose that the response variable Y is quantitative and that at least one pre-
dictor variable xi is quantitative. Then the multiple linear regression (MLR)
model is often a very useful model. For the MLR model,

Yi = +xi,1 1 +xi,2 2 + +xi,p p +ei = +xTi +ei = + T xi +ei (1.9)

for i = 1, . . . , n. Here Yi is the response variable, xi is a p 1 vector of

nontrivial predictors, is an unknown constant, is a p1 vector of unknown
coecients, and ei is a random variable called the error.
The Gaussian or normal MLR model makes the additional assumption that
the errors ei are iid N (0, 2 ) random variables. This model can also be written
as Y = + T x + e where e N (0, 2 ), or Y |x N ( + T x, 2 ), or Y |x
N (SP, 2 ), or Y |SP N (SP, 2 ). The normal MLR model is a parametric
model since, given x, the family of conditional distributions is completely
6 1 Introduction

specied by the parameters , , and 2 . Since Y |SP N (SP, 2 ), the

conditional mean function E(Y |SP ) M (SP ) = (SP ) = SP = + T x.
The MLR model is discussed in detail in Chapters 2, 3, and 4.

5
0
Y
5
10

10 5 0 5
SP

Fig. 1.1 SSP for MLR Data

A sucient summary plot (SSP) of the sucient predictor SP = + T xi

versus the response variable Yi with the mean function added as a visual aid
can be useful for describing the multiple linear regression model. This plot
can not be used for real data since and are unknown. To make Figure 1.1,
the articial data used n = 100 cases with k = 5 nontrivial predictors. The
data used = 1, = (1, 2, 3, 0, 0)T , ei N (0, 1) and x from a multivariate
normal distribution x N5 (0, I).
In Figure 1.1, notice that the identity line with unit slope and zero in-
tercept corresponds to the mean function since the identity line is the line
Y = SP = + T x = (SP ) = E(Y |SP ). The vertical deviation of Yi
from the line is equal to ei = Yi ( + T xi ). For a given value of SP ,
Yi N (SP, 2 ). For the articial data, 2 = 1. Hence if SP = 0 then
Yi N (0, 1), and if SP = 5 then Yi N (5, 1). Imagine superimposing the
N (SP, 2 ) curve at various values of SP . If all of the curves were shown, then
the plot would resemble a road through a tunnel. For the articial data, each
Yi is a sample of size 1 from the normal curve with mean + T xi .
T
The estimated sucient summary plot (ESSP) is a plot of + xi versus
Yi with the identity line added as a visual aid. For MLR, the ESP = +
1.2 Multiple Linear Regression 7

5
0
Y
5
10

10 5 0 5
ESP

Fig. 1.2 ESSP = Response Plot for MLR Data

2
1
RES
0 1
2

10 5 0 5
ESP

Fig. 1.3 Residual Plot for MLR Data

8 1 Introduction

5
0
Y
5
10

1.4 1.2 1.0 0.8 0.6 0.4 0.2

ESP

Fig. 1.4 Response Plot when Y is Independent of the Predictors

T
x and the estimated conditional mean function is (ESP ) = ESP. The
T
estimated or tted value of Yi is equal to Yi = + x. Now the vertical
deviation of Yi from the identity line is equal to the residual ri = Yi ( +
T
xi ). The interpretation of the ESSP is almost the same as that of the
SSP, but now the mean SP is estimated by the estimated sucient predictor
(ESP). This plot is also called the response plot and is used as a goodness
of t diagnostic. The residual plot is a plot of the ESP versus ri and is used
as a lack of t diagnostic. These two plots should be made immediately after
tting the MLR model and before performing inference. Figures 1.2 and 1.3
show the response plot and residual plot for the articial data.

The response
 plot is also a useful visual aid for describing the ANOVA F
test (see 2.4) which tests whether = 0, that is, whether the nontrivial
predictors x are needed in the model. If the predictors are not needed in the
model, then Yi and E(Yi |xi ) should be estimated by the sample mean Y . If
the predictors are needed, then Yi and E(Yi |xi ) should be estimated by the
T
ESP Yi = + xi . If the identity line clearly ts the data better than the
horizontal line Y = Y , then the ANOVA F test should have a small pvalue
and reject the null hypothesis Ho that the predictors x are not needed in the
MLR model. Figure 1.2 shows that the identity line ts the data better than
any horizontal line. Figure 1.4 shows the response plot for the articial data
when only X4 and X5 are used as predictors with the identity line and the
line Y = Y added as visual aids. In this plot the horizontal line ts the data
1.3 Variable Selection 9

about as well as the identity line which was expected since Y is independent
of X4 and X5 .

It is easy to nd data sets where the response plot looks like Figure 1.4,
but the pvalue for the ANOVA F test is very small. In this case, the MLR
model is statistically signicant, but the investigator needs to decide whether
the MLR model is practically signicant.

1.3 Variable Selection

A standard problem in 1D regression is variable selection, also called subset or

model selection. Assume that the 1D regression model uses a linear predictor

Y x|( + T x), (1.10)

that a constant is always included, that x = (x1 , . . . , xp1 )T are the p 1

nontrivial predictors, and that the np matrix X with ith row (1, xTi ) has full
rank p. Then variable selection is a search for a subset of predictor variables
that can be deleted without important loss of information.
To clarify ideas, assume that there exists a subset S of predictor variables
such that if xS is in the 1D model, then none of the other predictors are
needed in the model. Write E for these (extraneous) variables not in S,
partitioning x = (xTS , xTE )T . Then

SP = + T x = + TS xS + TE xE = + TS xS . (1.11)

The extraneous terms that can be eliminated given that the subset S is in
the model have zero coecients: E = 0.
Now suppose that I is a candidate subset of predictors, that S I and
that O is the set of predictors not in I. Then

SP = + T x = + TS xS = + TS xS + T(I/S) xI/S + 0T xO = + TI xI ,

where xI/S denotes the predictors in I that are not in S. Since this is true
regardless of the values of the predictors, O = 0 if S I. Hence for any
subset I that includes all relevant predictors, the population correlation

corr( + T xi , + T
I xI,i ) = 1. (1.12)

This observation, which is true regardless of the explanatory power of the

model, suggests that variable selection for a 1D regression model (1.11) is
simple in principle. For each value of j = 1, 2, . . . , p 1 nontrivial predictors,
keep track of subsets I that provide the largest values of corr(ESP,ESP(I)).
Any such subset for which the correlation is high is worth closer investigation
10 1 Introduction

and consideration. To make this advice more specic, use the rule of thumb
that a candidate subset of predictors I is worth considering if the sample
correlation of ESP and ESP(I) satises
T T T T
corr( + xi , I + I xI,i ) = corr( xi , I xI,i ) 0.95. (1.13)

The diculty with this approach is that tting large numbers of possi-
ble submodels involves substantial computation. Fortunately, (ordinary) least
squares (OLS) frequently gives a useful ESP, and methods originally meant
for multiple linear regression using the Mallows Cp criterion (see Jones 1946
and Mallows 1973) also work for more general 1D regression models. As a rule
of thumb, the OLS ESP is useful if |corr(OLS ESP, ESP)| 0.95 where ESP
T
is the standard ESP (e.g., for generalized linear models, the ESP is + x
where (, ) is the maximum likelihood estimator of (, )), or if the OLS
response plot suggests that the OLS ESP is good. Variable selection will be
discussed in much greater detail in Chapters 3 and 13, but the following
methods are useful for a large class of 1D regression models.

Perhaps the simplest method of variable selection is the t directed search

(see Daniel and Wood 1980, pp. 100101). Let k be the number of pre-
dictors in the model, including the constant. Hence k = p for the full
model. Let X1 , . . . , Xp1 denote the nontrivial predictor variables and let
W1 , W2 , . . . , Wp1 be the predictor variables in decreasing order of impor-
tance. Use theory if possible, but if no theory is available then t the full
model using OLS and let ti denote the t statistic for testing Ho : i = 0. Let
|t|(1) |t|(2) |t|(p1) . Then Wi corresponds to the Xj with |t|(pi)
for i = 1, 2, . . . , p 1. That is, W1 has the largest t statistic, W2 the next
largest, etc. Then use OLS to compute Cp (Ij ) for the p 1 models Ij where
Ij contains W1 , . . . , Wj and a constant for j = 1, . . . , p 1.

Forward selection starts with a constant = W0 .

Step 1) k = 2: compute Cp for all models containing the constant and a
single predictor Xi . Keep the predictor W1 = Xj , say, that corresponds to
the model with the smallest value of Cp .
Step 2) k = 3: Fit all models with k = 3 that contain W0 and W1 . Keep the
predictor W2 that minimizes Cp .
Step j) k = j + 1: Fit all models with k = j + 1 that contains W0 , W1 , . . . , Wj .
Keep the predictor Wj+1 that minimizes Cp .
Step p 1) k = p: Fit the full model.

Backward elimination starts with the full model. All models contain a
constant = U0 . Hence the full model contains U0 , X1 , . . . , Xp1 . We will also
say that the full model contains U0 , U1 , . . . , Up1 where Ui need not equal Xi
for i 1.
Step 1) k = p 1: t each model with p 1 predictors including a constant.
Delete the predictor Up1 , say, that corresponds to the model with the small-
est Cp . Keep U0 , . . . , Up2 .
1.3 Variable Selection 11

Step 2) k = p 2: t each model with p 2 predictors including the con-

stant. Delete the predictor Up2 that corresponds to the smallest Cp . Keep
U0 , U1 , . . . , Up3 .
Step j) k = pj: t each model with pj predictors and a constant. Delete the
predictor Upj that corresponds to the smallest Cp . Keep U0 , U1 , . . . , Upj1 .
Step p 2) k = 2: The current model contains U0 , U1 , and U2 . Fit the model
U0 , U1 and the model U0 , U2 . Assume that model U0 , U1 minimizes Cp . Then
delete U2 and keep U0 and U1 .
(Step p 1) which nds Cp for the model that only contains the constant U0
is often omitted.)

All subsets variable selection examines all subsets and keeps track of
several (up to three, say) subsets with the smallest Cp (I) for each group of
submodels containing k predictors including a constant. This method can be
used for p 30 by using the ecient leaps and bounds algorithms when
OLS and Cp is used (see Furnival and Wilson 1974).

Rule of thumb for variable selection (assuming that the cost of each
predictor is the same): nd the submodel Im with the minimum Cp . If Im
uses km predictors including a constant, do not use any submodel that has
more than km predictors. Since the minimum Cp submodel often has too
many predictors, also look at the submodel Io with the smallest value of k,
say ko , such that Cp 2k. This submodel may have too few predictors.
So look at the predictors in Im but not in Io and see if they can be deleted
or not. (If Im = Io , then it is a good candidate for the best submodel.)

Variable selection with the Cp criterion is closely related to the partial F

test for testing whether a reduced model should be used instead of the full
model. See Section 2.6. The following results are properties of OLS and hold
even if the data does not follow a 1D model. If the candidate model of xI has
k terms (including the constant), then the partial F test for reduced model
I uses test statistic
 
SSE(I) SSE SSE n p SSE(I)
FI = / = 1
(n k) (n p) n p pk SSE

where SSE is the residual sum of squares from the full model and SSE(I) is
the residual sum of squares from the candidate submodel. Then

SSE(I)
Cp (I) = + 2k n = (p k)(FI 1) + k (1.14)
M SE
where MSE is the residual mean square for the full model. Let ESP(I) =
T
I + I x be the ESP for the submodel and let VI = Y ESP (I) so that
T
VI,i = Yi I + I xi . Let ESP and V denote the corresponding quantities
12 1 Introduction

for the full model. Then Olive and Hawkins (2005) show that corr(VI , V ) 1
forces corr(OLS ESP, OLS ESP(I)) 1 and that
  
SSE np np
corr(V, VI ) = = = .
SSE(I) Cp (I) + n 2k (p k)FI + n p

Also Cp (I) 2k corresponds to corr(VI , V ) dn where


p
dn = 1 .
n

Notice that the submodel Ik that minimizes Cp (I) also maximizes corr(V, VI )
among all submodels I with k predictors including a constant. If Cp (I) 2k
and n 10p, then 0.948 corr(V, V (I)), and both corr(V, V (I)) 1.0 and
corr(OLS ESP, OLS ESP(I)) 1.0 as n .

If a 1D model (1.11) holds, a common assumption made for variable se-

lection is that the tted full model ESP is a good estimator of the sucient
predictor, and the usual graphical and numerical checks on this assumption
should be made. Also assume that the OLS ESP is useful. This assump-
tion can be checked by making an OLS response plot or by verifying that
|corr(OLS ESP, ESP)| 0.95. Then we suggest that submodels I are inter-
esting if Cp (I) min(2k, p).

Suppose that the OLS ESP and the standard ESP are highly correlated:
|corr(ESP, OLS ESP)| 0.95. Then often OLS variable selection can be used
for the 1D data, and using the pvalues from OLS output seems to be a useful
benchmark. To see this, suppose that n 5p and rst consider the model
Ii that deletes the predictor Xi . Then the model has k = p 1 predictors
including the constant, and the test statistic is ti where

t2i = FIi .

Using (1.14) and Cp (If ull ) = p, notice that

Cp (Ii ) = (p (p 1))(t2i 1) + (p 1) = t2i 1 + Cp (If ull ) 1,

or
Cp (Ii ) = Cp (If ull ) + (t2i 2).
Using the screen Cp (I) min(2k, p) suggests that the predictor Xi should
not be deleted if
|ti | > 2 1.414.

If |ti | < 2, then the predictor can probably be deleted since Cp decreases.
More generally, for the partial F test, notice that by (1.14), Cp (I) 2k
i (p k)FI p + 2k 2k i (p k)Fi p i
1.4 Other Issues 13

p
FI .
pk
Now k is the number of terms in the model including a constant while p k
is the number of terms set to 0. As k 0, the partial F test will reject Ho
(i.e., say that the full model should be used instead of the submodel I) unless
FI is not much larger than 1. If p is very large and p k is very small, then
the partial F test will tend to suggest that there is a model I that is about
as good as the full model even though model I deletes p k predictors.
The Cp (I) k screen tends to overt. An additive error single index model
is Y = m( + xT ) + e. We simulated multiple linear regression and single
index model data sets with p = 8 and n = 50, 100, 1000, and 10000. The true
model S satised Cp (S) k for about 60% of the simulated data sets, but S
satised Cp (S) 2k for about 97% of the data sets.

1.4 Other Issues

The 1D regression models oer a unifying framework for many of the most
used regression models. By writing the model in terms of the sucient predic-
tor SP = h(x), many important topics valid for all 1D regression models can
be explained compactly. For example, the previous section presented variable
selection, and equation (1.14) can be used to motivate the test for whether
the reduced model can be used instead of the full model. Similarly, the su-
cient predictor can be used to unify the interpretation of coecients and to
explain models that contain interactions and factors.

Interpretation of Coecients
One interpretation of the coecients in a 1D model (1.11) is that i is the
rate of change in the SP associated with a unit increase in xi when all other
predictor variables x1 , . . . , xi1 , xi+1 , . . . , xp are held xed. Denote a model
by SP = + T x = + 1 x1 + + p xp . Then

SP
i = for i = 1, . . . , p.
xi
Of course, holding all other variables xed while changing xi may not be
possible. For example, if x1 = x, x2 = x2 and SP = + 1 x + 2 x2 , then x2
cannot be held xed when x1 increases by one unit, but
d SP
= 1 + 22 x.
dx
The interpretation of i changes with the model in two ways. First,
the interpretation changes as terms are added and deleted from the SP.
Hence the interpretation of 1 diers for models SP = + 1 x1 and
14 1 Introduction

SP = +1 x1 +2 x2 . Secondly, the interpretation changes as the parametric

or semiparametric form of the model changes. For multiple linear regression,
E(Y |SP ) = SP and an increase in one unit of xi increases the conditional
expectation by i . For binary logistic regression,

exp(SP )
E(Y |SP ) = (SP ) = ,
1 + exp(SP )

and the change in the conditional expectation associated with a one unit
increase in xi is more complex.

Factors for Qualitative Variables

The interpretation of the coecients also changes if interactions and fac-
tors are present. Suppose a factor W is a qualitative random variable that
takes on c categories a1 , . . . , ac . Then the 1D model will use c 1 indicator
variables Wi = 1 if W = ai and Wi = 0 otherwise, where one of the levels ai
is omitted, e.g. use i = 1, . . . , c 1.

Interactions
Suppose X1 is quantitative and X2 is qualitative with 2 levels and X2 = 1
for level a2 and X2 = 0 for level a1 . Then a rst order model with interaction
is SP = +1 x1 +2 x2 +3 x1 x2 . This model yields two unrelated lines in the
sucient predictor depending on the value of x2 : SP = + 2 + (1 + 3 )x1
if x2 = 1 and SP = + 1 x1 if x2 = 0. If 3 = 0, then there are two
parallel lines: SP = + 2 + 1 x1 if x2 = 1 and SP = + 1 x1 if x2 = 0.
If 2 = 3 = 0, then the two lines are coincident: SP = + 1 x1 for
both values of x2 . If 2 = 0, then the two lines have the same intercept:
SP = + (1 + 3 )x1 if x2 = 1 and SP = + 1 x1 if x2 = 0. In general,
as factors have more levels and interactions have more terms, e.g. x1 x2 x3 x4 ,
the interpretation of the model rapidly becomes very complex.

1.5 Complements

Cook and Weisberg (1999a, p. 411) dene a sucient summary plot to be a

plot that contains all the sample regression information about the conditional
distribution of the response given the predictors. To help explain the given 1D
model, use the sucient summary plot (SSP) of SP versus Yi with the mean
function added as a visual aid. If p = 1, then Y x|x and the plot of xi versus
Yi is a SSP and has been widely used to explain regression models such as the
simple linear regression (SLR) model and the logistic regression model with
one nontrivial predictor. See Agresti (2002, cover illustration and p. 169)
and Collett (1999, p. 74). Replacing x by SP has two major advantages.
First, the plot can be made for p 1 and secondly, the possible shapes that
the plot can take is greatly reduced. For example, in a plot of xi versus Yi ,
1.6 Problems 15

the plotted points will fall about some line with slope and intercept if
the SLR model holds, but in a plot of SP = + T xi versus Yi , the plotted
points will fall about the identity line with unit slope and zero intercept if the
multiple linear regression model holds. If there are more than two nontrivial
predictors, then we generally cannot nd a sucient summary plot and need
to use an estimated sucient summary plot.
Important theoretical results for the additive error single index model Y =
m(+ T x)+e were given by Brillinger (1977, 1983) and Aldrin et al. (1993).
Li and Duan (1989) extended these results to models of the form

Y = g( + T x, e) (1.15)

where g is a bivariate inverse link function. Olive and Hawkins (2005) discuss
variable selection while Chang (2006) and Chang and Olive (2007, 2010)
discuss (ordinary) least squares (OLS) tests. Severini (1998) discusses when
OLS output is relevant for the Gaussian additive error single index model.

1.6 Problems

1.1. Explain why the model Y = g( + T x, e) can also be written as Y =

g( + xT , e).

R Problem

Use the command source(G:/lregpack.txt) to download the func-

tions and the command source(G:/lregdata.txt) to download the data.
See Preface or Section 14.1. Typing the name of the lregpack function,
e.g. lrplot2, will display the code for the function. Use the args command,
e.g. args(lrplot2), to display the needed arguments for the function. For the
following problem, the R command can be copied and pasted from (http://
lagrange.math.siu.edu/Olive/lreghw.txt) into R.
1.2. The Beaton et al. (1996) TIMMS data has response variable Y = 1 if
there was a statistically signicant gender dierence in the nations 8th grade
TIMSS science test, and Y = 0 otherwise. There were n = 35 countries and 12
predictors, including x1 = male 8th grade score, x2 = female 8th grade score,
x3 = male 7th grade score, x4 = female 7th grade score, and x5 = percent
of 8th graders with educational aids (dictionary, study table, and computer).
Enter (or copy and paste) the R command lrplot2(xtimss,ytimss) for this
problem to make a logistic regression response plot using x1 x5 as predictors.
See Chapter 13. Include the response plot in Word.
Chapter 2
Multiple Linear Regression

This chapter introduces the multiple linear regression model, the response
plot for checking goodness of t, the residual plot for checking lack of t,
the ANOVA F test, the partial F test, the t tests, and least squares. The
problems use software R, SAS, Minitab, and Arc.

2.1 The MLR Model

Denition 2.1. The response variable is the variable that you want to
predict. The predictor variables are the variables used to predict the re-
sponse variable.
Notation. In this text the response variable will usually be denoted by
Y and the p predictor variables will often be denoted by x1 , . . . , xp . The
response variable is also called the dependent variable while the predictor
variables are also called independent variables, explanatory variables, carri-
ers, or covariates. Often the predictor variables will be collected in a vector x.
Then xT is the transpose of x.
Denition 2.2. Regression is the study of the conditional distribu-
tion Y |x of the response variable Y given the vector of predictors x =
(x1 , . . . , xp )T .
Denition 2.3. A quantitative variable takes on numerical values
while a qualitative variable takes on categorical values.
Example 2.1. Archeologists and crime scene investigators sometimes
want to predict the height of a person from partial skeletal remains. A model
for prediction can be built from nearly complete skeletons or from living
humans, depending on the population of interest (e.g., ancient Egyptians
or modern US citizens). The response variable Y is height and the predic-
tor variables might be x1 1, x2 = femur length, and x3 = ulna length.

Springer International Publishing AG 2017 17

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 2
18 2 Multiple Linear Regression

The heights of individuals with x2 = 200mm and x3 = 140mm should be

shorter on average than the heights of individuals with x2 = 500mm and
x3 = 350mm. In this example Y , x2 , and x3 are quantitative variables. If
x4 = gender is a predictor variable, then gender (coded as male = 1 and
female = 0) is qualitative.

Denition 2.4. Suppose that the response variable Y and at least one
predictor variable xi are quantitative. Then the multiple linear regression
(MLR) model is

Yi = xi,1 1 + xi,2 2 + + xi,p p + ei = xTi + ei (2.1)

for i = 1, . . . , n. Here n is the sample size and the random variable ei is the
ith error. Suppressing the subscript i, the model is Y = xT + e.

In matrix notation, these n equations become

Y = X + e, (2.2)

where Y is an n 1 vector of dependent variables, X is an n p matrix

of predictors, is a p 1 vector of unknown coecients, and e is an n 1
vector of unknown errors. Equivalently,

Y1 x1,1 x1,2 . . . x1,p 1 e1
Y2 x2,1 x2,2 . . . x2,p 2 e2

.. = .. .. . . .. .. + .. . (2.3)
. . . . . . .
Yn xn,1 xn,2 . . . xn,p p en

Often the rst column of X is X1 = 1, the n 1 vector of ones. The ith

case (xTi , Yi ) = (xi1 , xi2 , . . . , xip , Yi ) corresponds to the ith row xTi of X
and the ith element of Y (if xi1 1, then xi1 could be omitted). In the
MLR model Y = xT + e, the Y and e are random variables, but we only
have observed values Yi and xi . If the ei are iid (independent and identically
distributed) with zero mean E(ei ) = 0 and variance VAR(ei ) = V (ei ) = 2 ,
then regression is used to estimate the unknown parameters and 2 .

Denition 2.5. The constant variance MLR model uses the

assumption that the errors e1 , . . . , en are iid with mean E(ei ) = 0 and
variance VAR(ei ) = 2 < . Also assume that the errors are independent
of the predictor variables xi . The predictor variables xi are assumed to be
xed and measured without error. The cases (xTi , Yi ) are independent for
i = 1, . . . , n.

If the predictor variables are random variables, then the above MLR model
is conditional on the observed values of the xi . That is, observe the xi and
then act as if the observed xi are xed.
2.1 The MLR Model 19

Denition 2.6. The unimodal MLR model has the same assumptions
as the constant variance MLR model, as well as the assumption that the zero
mean constant variance errors e1 , . . . , en are iid from a unimodal distribution
that is not highly skewed. Note that E(ei ) = 0 and V (ei ) = 2 < .

Denition 2.7. The normal MLR model or Gaussian MLR model has
the same assumptions as the unimodal MLR model but adds the assumption
that the errors e1 , . . . , en are iid N (0, 2 ) random variables. That is, the ei
are iid normal random variables with zero mean and variance 2 .

The unknown coecients for the above 3 models are usually estimated
using (ordinary) least squares (OLS).

Notation. The symbol A B = f (c) means that A and B are equivalent

and equal, and that f (c) is the formula used to compute A and B.

Denition 2.8. Given an estimate b of , the corresponding vector of

predicted values or tted values is Y Y (b) = Xb. Thus the ith tted value

Yi Yi (b) = xTi b = xi,1 b1 + + xi,p bp .

The vector of residuals is r r(b) = Y Y (b). Thus ith residual ri

ri (b) = Yi Yi (b) = Yi xi,1 b1 xi,p bp .

Most regression methods attempt to nd an estimate of which mini-

mizes some criterion function Q(b) of the residuals.

Denition 2.9. The ordinary least squares (OLS) estimator OLS mini-
mizes
n
QOLS (b) = ri2 (b), (2.4)
i=1

and OLS = (X T X)1 X T Y .

The vector of predicted or tted values Y OLS = X OLS = HY where the

hat matrix H = X(X T X)1 X T provided the inverse exists. Typically the
subscript OLS is omitted, and the least squares regression equation is
Y = 1 x1 + 2 x2 + + p xp where x1 1 if the model contains a constant.

There are many statistical models besides the MLR model, and you should
learn how to quickly recognize an MLR model. A regression model has a
response variable Y and the conditional distribution of Y given the predic-
tors x = (x1 , . . . , xp )T is of interest. Regression models are used to predict Y
and to summarize the relationship between Y and x. If a constant xi,1 1
(this notation means that xi,1 = 1 for i = 1, . . . , n) is in the model, then
xi,1 is often called the trivial predictor, and the MLR model is said to have
20 2 Multiple Linear Regression

a constant or intercept. All nonconstant predictors are called nontrivial pre-

dictors. The term multiple is used if the model uses one or more nontrivial
predictors. (Some authors use multivariate instead of multiple, but in this
text a multivariate linear regression model will have m 2 response vari-
ables. See Chapter 12.) The simple linear regression model is a special case
of the MLR model that uses exactly one nontrivial predictor. Suppose the
response variable is Y and data has been collected on additional variables
x1 , . . . , xp .
An MLR model is linear in the unknown coecients . Thus the model
is an MLR model in Y and if we can write Yi = xTi + ei or Yi = wTi + ei
where each wi is a function of x1 , . . . , xp . Symbols other than w or x may be
used. Alternatively, the model is linear in the parameters if Y /i does
not depend on the parameters. If Y = xT + e = x1 1 + + xp p + e, then
Y /i = xi . Similarly, if Y = wT + e, then Y /i = wi .

Example 2.2. a) Suppose that interest is in predicting a function of Z

from functions of w1 , . . . , wk . If Y = t(Z) = xT + e where t is a function
and each xi is some function of w1 , . . . , wk , then there is an MLR model in
Y and . Similarly, Z = t(Y ) = wT + e is an MLR model in Z and .
b) To see that Y = 1 + 2 x + 3 x2 + e is an MLR model in Y and ,
take w1 = 1, w2 = x, and w3 = x2 . Then Y = wT + e.
c) If Y = 1 + 2 exp(3 x) + e, then the model is a nonlinear regression
model that is not an MLR model in Y and . Notice that the model can
not be written in the form Y = wT + e and that Y /2 = exp(3 x) and
Y /3 = 2 x exp(3 x) depend on the parameters.

2.2 Checking Goodness of Fit

It is crucial to realize that an MLR model is not necessarily a

useful model for the data, even if the data set consists of a response
variable and several predictor variables. For example, a nonlinear regression
model or a much more complicated model may be needed. Chapters 1 and 13
describe several alternative models. Let p be the number of predictors and n
the number of cases. Assume that n 5p, then plots can be used to check
whether the MLR model is useful for studying the data. This technique is
known as checking the goodness of t of the MLR model.

Notation. Plots will be used to simplify regression analysis, and in this

text a plot of W versus Z uses W on the horizontal axis and Z on the vertical
axis.

Denition 2.10. A scatterplot of X versus Y is a plot of X versus Y

and is used to visualize the conditional distribution Y |X of Y given X.
2.2 Checking Goodness of Fit 21

Denition 2.11. A response plot is a plot of a variable wi versus Yi .

Typically wi is a linear combination of the predictors: wi = xTi where is
a known p 1 vector. The most commonly used response plot is a plot of the
tted values Yi versus the response Yi .

Proposition 2.1. Suppose that the regression estimator b of is used to

nd the residuals ri ri (b) and the tted values Yi Yi (b) = xTi b. Then
in the response plot of Yi versus Yi , the vertical deviations from the identity
line (that has unit slope and zero intercept) are the residuals ri (b).

Proof. The identity line in the response plot is Y = xT b. Hence the

vertical deviation is Yi xTi b = ri (b). 

Denition 2.12. A residual plot is a plot of a variable wi versus the

residuals ri . The most commonly used residual plot is a plot of Yi versus ri .

Notation: For MLR, the residual plot will often mean the residual
plot of Yi versus ri , and the response plot will often mean the plot of Yi
versus Yi .

If the unimodal MLR model as estimated by least squares is useful, then

in the response plot the plotted points should scatter about the identity line
while in the residual plot of Y versus r the plotted points should scatter
about the r = 0 line (the horizontal axis) with no other pattern. Figures 1.2
and 1.3 show what a response plot and residual plot look like for an articial
MLR data set where the MLR regression relationship is rather strong in that
the sample correlation corr(Y , Y ) is near 1. Figure 1.4 shows a response plot
where the response Y is independent of the nontrivial predictors in the model.
Here corr(Y , Y ) is near 0 but the points still scatter about the identity line.
When the MLR relationship is very weak, the response plot will look like the
residual plot.
The above ideal shapes for the response and residual plots are for when
the unimodal MLR model gives a good approximation for the data. If the
plots have the ideal shapes and n 10p, then expect inference, except for
classical prediction intervals, to be approximately correct for many unimodal
distributions that are close to the normal distribution.
If the response and residual plots suggest an MLR model with iid skewed
errors, then add lowess to both plots. The scatterplot smoother tries to esti-
mate the mean function E(Y |Y ) or E(r|Y ) without using any model. If the
lowess curve is close to the identity line in the response plot and close to the
r = 0 line in the residual plot, then the constant variance MLR model may be
a good approximation to the data, but sample sizes much larger than n = 10p
may be needed before inference is approximately correct. Such skewed data
sets seem rather rare, but see Chen et al. (2009) and see Problem 2.28.

Remark 2.1. For any MLR analysis, always make the response
plot and the residual plot of Yi versus Yi and ri , respectively.
22 2 Multiple Linear Regression

Response Plot

3
1500 1600 1700 1800
Y

63
44

1550 1600 1650 1700 1750 1800

FIT

Residual Plot

3
0 50
RES
50

63
100

44

1550 1600 1650 1700 1750 1800

FIT

Fig. 2.1 Residual and Response Plots for the Tremearne Data

Denition 2.13. An outlier is an observation that lies far away from the
bulk of the data.

Remark 2.2. For MLR, the response plot is important because MLR
is the study of the conditional distribution of Y |xT , and the response
plot is used to visualize the conditional distribution of Y |xT since
Y = xT is a good estimator of xT if is a good estimator of .

If the MLR model is useful, then the plotted points in the response plot
should be linear and scatter about the identity line with no gross outliers.
Suppose the tted values range in value from wL to wH with no outliers. Fix
the t = w in this range and mentally add a narrow vertical strip centered at
w to the response plot. The plotted points in the vertical strip should have a
mean near w since they scatter about the identity line. Hence Y |f it = w is
like a sample from a distribution with mean w. The following example helps
illustrate this remark.

Example 2.3. Tremearne (1911) presents a data set of about 17 mea-

surements on 115 people of Hausa nationality. We deleted 3 cases because
of missing values and used height as the response variable Y . Along with a
2.2 Checking Goodness of Fit 23

constant xi,1 1, the ve additional predictor variables used were height

when sitting, height when kneeling, head length, nasal breadth, and span (per-
haps from left hand to right hand). Figure 2.1 presents the OLS response and
residual plots for this data set. These plots show that an MLR model should
be a useful model for the data since the plotted points in the response plot
are linear and follow the identity line while the plotted points in the residual
plot follow the r = 0 line with no other pattern (except for a possible outlier
marked 44).
To use the response plot to visualize the conditional distribution of Y |xT ,
use the fact that the tted values Y = xT . For example, suppose the height
given t = 1700 is of interest. Mentally examine the plot about a narrow
vertical strip about t = 1700, perhaps from 1685 to 1715. The cases in the
narrow strip have a mean close to 1700 since they fall close to the identity
line. Similarly, when the t = w for w between 1500 and 1850, the cases have
heights near w, on average.
Cases 3, 44, and 63 are highlighted. The 3rd person was very tall while
the 44th person was rather short. Beginners often label too many points as
outliers. Mentally draw a box about the bulk of the data ignoring any outliers.
Double the width of the box (about the identity line for the response plot
and about the horizontal line for the residual plot). Cases outside of this
imaginary doubled box are potential outliers. Alternatively, visually estimate
the standard deviation of the residuals in both plots. In the residual plot look
for residuals that are more than 5 standard deviations from the r = 0 line.
In Figure 2.1, the standard deviation of the residuals appears to be around
10. Hence cases 3 and 44 are certainly worth examining.
The identity line can also pass through or near an outlier or a cluster
of outliers. Then the outliers will be in the upper right or lower left of the
response plot, and there will be a large gap between the cluster of outliers
and the bulk of the data. See Figure 3.11. Figure 2.1 was made with the
following R commands, using lregpack function MLRplot and the major.lsp
data set from the texts webpage.

major <- matrix(scan(),nrow=112,ncol=7,byrow=T)

#copy and paste the data set then press enter
major <- major[,-1]
X<-major[,-6]
Y <- major[,6]
MLRplot(X,Y) #left click the 3 highlighted cases,
#then right click Stop for each of the two plots
24 2 Multiple Linear Regression

2.3 Checking Lack of Fit

The response plot may look good while the residual plot suggests that the
unimodal MLR model can be improved. Examining plots to nd model vio-
lations is called checking for lack of t. Again assume that n 5p.
The unimodal MLR model often provides a useful model for the data, but
the following assumptions do need to be checked.
i) Is the MLR model appropriate?
ii) Are outliers present?
iii) Is the error variance constant or nonconstant? The constant variance
assumption VAR(ei ) 2 is known as homoscedasticity. The nonconstant
variance assumption VAR(ei ) = i2 is known as heteroscedasticity.
iv) Are any important predictors left out of the model?
v) Are the errors e1 , . . . , en iid?
vi) Are the errors ei independent of the predictors xi ?

Make the response plot and the residual plot to check i), ii), and iii). An
MLR model is reasonable if the plots look like Figures 1.2, 1.3, 1.4, and 2.1.
A response plot that looks like Figure 13.7 suggests that the model is not
linear. If the plotted points in the residual plot do not scatter about the
r = 0 line with no other pattern (i.e., if the cloud of points is not ellipsoidal or
rectangular with zero slope), then the unimodal MLR model is not sustained.
The ith residual ri is an estimator of the ith error ei . The constant variance
assumption may have been violated if the variability of the point cloud in the
residual plot depends on the value of Y . Often the variability of the residuals
increases as Y increases, resulting in a right opening megaphone shape. (Fig-
ure 4.1b has this shape.) Often the variability of the residuals decreases as Y
increases, resulting in a left opening megaphone shape. Sometimes the vari-
ability decreases then increases again, and sometimes the variability increases
then decreases again (like a stretched or compressed football).

2.3.1 Residual Plots

Remark 2.3. Residual plots magnify departures from the model while the
response plot emphasizes how well the MLR model ts the data.

Since the residuals ri = ei are estimators of the errors, the residual plot
is used to visualize the conditional distribution e|SP of the errors given the
sucient predictor SP = xT , where SP is estimated by Y = xT . For the
unimodal MLR model, there should not be any pattern in the residual plot:
as a narrow vertical strip is moved from left to right, the behavior of the
residuals within the strip should show little change.
2.3 Checking Lack of Fit 25

Notation. A rule of thumb is a rule that often but not always works well
in practice.
Rule of thumb 2.1. If the residual plot would look good after several
points have been deleted, and if these deleted points were not gross outliers
(points far from the point cloud formed by the bulk of the data), then the
residual plot is probably good. Beginners often nd too many things wrong
with a good model. For practice, use the lregpack function MLRsim to generate
several MLR data sets, and make the response and residual plots for these
data sets: type MLRsim(nruns=10) in R and right click Stop for each plot
(20 times) to generate 10 pairs of response and residual plots. This exercise
will help show that the plots can have considerable variability even when the
MLR model is good. See Problem 2.30.
Rule of thumb 2.2. If the plotted points in the residual plot look like
a left or right opening megaphone, the rst model violation to check is the
assumption of nonconstant variance. (This is a rule of thumb because it is
possible that such a residual plot results from another model violation such
as nonlinearity, but nonconstant variance is much more common.)
The residual plot of Y versus r should always be made. It is also a good idea
to plot each nontrivial predictor xj versus r and to plot potential predictors
wj versus r. If the predictor is quantitative, then the residual plot of xj versus
r should look like the residual plot of Y versus r. If the predictor is qualitative,
e.g. gender, then interpreting the residual plot is much more dicult; however,
if each category contains many observations, then the plotted points for each
category should form a vertical line centered at r = 0 with roughly the same
variability (spread or range).

Rule of thumb 2.3. Suppose that the MLR model uses predictors xj
and that data has been collected on variables wj that are not included in
the MLR model. To check whether important predictors have been left out,
make residual plots of xj and wj versus r. If these plots scatter about the
r = 0 line with no other pattern, then there is no evidence that x2j or wj are
needed in the model. If the plotted points scatter about a parabolic curve,
try adding x2j or wj and wj2 to the MLR model. If the plot of the potential
predictor wj versus r has a linear trend, try adding wj to the MLR model.
The additive error regression model and EE plot in Section 13.7 can also be
used to check whether important predictors have been left out.
Rule of thumb 2.4. To check that the errors are independent of the pre-
dictors, make residual plots of xj versus r. If the plot of xj versus r scatters
about the r = 0 line with no other pattern, then there is no evidence that the
errors depend on xj . If the variability of the residuals changes with the value
of xj , e.g. if the plot resembles a left or right opening megaphone, the errors
may depend on xj . Some remedies for nonconstant variance are considered
in Chapter 4.
26 2 Multiple Linear Regression

To study residual plots, some notation and properties of the least squares
estimator are needed. MLR is the study of the conditional distribution of
Yi |xTi , and the MLR model is Y = X + e where X is an n p matrix
of full rank p. Hence the number of predictors p n. The ith row of X is
xTi = (xi,1 , . . . , xi,p ) where xi,k is the value of the ith observation on the
kth predictor xk . We will denote the jth column of X by Xj v j which
corresponds to the jth variable or predictor xj .
Example 2.4. If Y is brain weight in grams, x1 1, x2 is age, and x3 is
the size of the head in (mm)3 , then for the Gladstone (1905) data

3738 1 39 149.5
4261 1 35 152.5

Y = . , X = . . .. = [v 1 v 2 v 3 ].
.. .. .. .
3306 1 19 141

Hence the rst person had brain weight = 3738, age = 39, and size = 149.5.
After deleting observations with missing values, there were n = 267 cases
(people measured on brain weight, age, and size), and x267 = (1, 19, 141)T .
The second predictor x2 = age corresponds to the 2nd column of X and
is X2 = v 2 = (39, 35, . . . , 19)T . Notice that X1 v 1 = 1 = (1, . . . , 1)T
corresponds to the constant x1 .
The results in the following proposition are properties of least squares
(OLS), not of the underlying MLR model. See Chapter 11 for more linear
model theory. Denitions 2.8 and 2.9 dene the hat matrix H, vector of
tted values Y , and vector of residuals r. Parts f) and g) make residual plots
useful. If the plotted points are linear with roughly constant variance and the
correlation is zero, then the plotted points scatter about the r = 0 line with
no other pattern. If the plotted points in a residual plot of w versus r do show
a pattern such as a curve or a right opening megaphone, zero correlation will
usually force symmetry about either the r = 0 line or the w = median(w)
line. Hence departures from the ideal plot of random scatter about the r = 0
line are often easy to detect.

Warning: If n > p, as is usually the case, X is not square, so (X T X)1 =

X (X T )1 since X 1 does not exist.
1

Proposition 2.2. Suppose that X is an n p matrix of full rank p. Then

a) H is symmetric: H = H T .
b) H is idempotent: HH = H.
c) X T r = 0 so that XjT r = v Tj r = 0.
d) If there is aconstant X1 v 1 = 1 in the model, then the sum of the
n
residuals is zero: i=1 ri = 0.
e) r T Y = 0.
f) If there is a constant in the model, then the sample correlation of the
tted values and the residuals is 0: corr(r, Y ) = 0.
2.3 Checking Lack of Fit 27

g) If there is a constant in the model, then the sample correlation of the

jth predictor with the residuals is 0: corr(r, v j ) = 0 for j = 1, . . . , p.

Proof. a) X T X is symmetric since (X T X)T = X T (X T )T = X T X.

Hence (X T X)1 is symmetric since the inverse of a symmetric matrix is
symmetric. (Recall that if A has an inverse then (AT )1 = (A1 )T .) Thus
using (AT )T = A and (ABC)T = C T B T AT shows that

H T = X T [(X T X)1 ]T (X T )T = H.

b) HH = X(X T X)1 X T X(X T X)1 X T = H since (X T X)1 X T X =

I p , the p p identity matrix.
c) X T r = X T (I p H)Y = [X T X T X(X T X)1 X T ]Y =
[X T X T ]Y = 0. Since v j is the jth column of X, v Tj is the jth row of X T
and v Tj r = 0 for j = 1, . . . , p.
n
d) Since v 1 = 1, v T1 r = i=1 ri = 0 by c).
e) r T Y = [(I n H)Y ]T HY = Y T (I n H)HY = Y T (H H)Y = 0.
f) The sample correlation between W and Z is corr(W, Z) =
n n
i=1 (wi w)(zi z) (wi w)(zi z)
= n i=1 n
(n 1)sw sz i=1 (wi w)
2
i=1 (zi z)
2

where sm is the sample standard deviation of m for m = w, z. So the result

n
follows if A = i=1 (Yi Y )(ri r) = 0. Now r = 0 by d), and thus


n 
n 
n
A= Yi ri Y ri = Yi ri
i=1 i=1 i=1

n
by d) again. But i=1 Yi ri = r T Y = 0 by e).
ng) Following the argument in f), the resultn follows if A =
i=1 (x i,j x j )(ri r) = 0 where
n x j = i=1 xi,j /n is the sample mean of
the jth predictor. Now r = i=1 ri /n = 0 by d), and thus


n 
n 
n
A= xi,j ri xj ri = xi,j ri
i=1 i=1 i=1
n
by d) again. But i=1 xi,j ri = XjT r = v Tj r = 0 by c). 
28 2 Multiple Linear Regression

2.3.2 Other Model Violations

Without loss of generality, E(e) = 0 for the unimodal MLR model with a
constant, in that if E(e) = = 0, then the MLR model can always be written
as Y = xT + e where E(e) = 0 and E(Y ) E(Y |x) = xT . To see this
claim notice that

Y = 1 + x2 2 + + xp p + e = 1 + E(e) + x2 2 + + xp p + e E(e)

= 1 + x 2 2 + + x p p + e
where 1 = 1 + E(e) and e = e E(e). For example, if the errors ei are iid
exponential () with E(ei ) = , use ei = ei .

For least squares, it is crucial that 2 exists. For example, if the ei are iid
Cauchy(0,1), then 2 does not exist and the least squares estimators tend to
perform very poorly.
The performance of least squares is analogous to the performance of Y .
The sample mean Y is a very good estimator of the population mean if the
Yi are iid N (, 2 ), and Y is a good estimator of if the sample size is large
and the Yi are iid with mean and variance 2 . This result follows from
the central limit theorem (CLT), but how large is large depends on the
underlying distribution. The n > 30 rule tends to hold for distributions that
are close to normal in that they take on many values and 2 is not huge. Error
distributions that are highly nonnormal with tiny 2 often need n >> 30.
For example, if Y1 , . . . , Yn are iid Gamma(1/m, 1), then n > 25m may be
needed. Another example is distributions that take on one value with very
high probability, e.g. a Poisson random variable with very small variance.
Bimodal and multimodal distributions and highly skewed distributions with
large variances also need larger n. Chihara and Hesterberg (2011, p. 177)
suggest using n > 5000 for moderately skewed distributions.
There are central limit type theorems for the least squares estimators that
depend on the error distribution of the iid errors ei . See Theorems 2.8, 11.25,
and 12.7. We always assume that the ei are continuous random variables with
a probability density function. Error distributions that are close to normal
may give good results for moderate n if n 10p and np 30 where p is the
number of predictors. Error distributions that need large n for the CLT to
apply for e, will tend to need large n for the limit theorems for least squares
to apply (to give good approximations).

Checking whether the errors are iid is often dicult. The iid assumption is
often reasonable if measurements are taken on dierent objects, e.g. people.
In industry often several measurements are taken on a batch of material.
For example a batch of cement is mixed and then several small cylinders of
concrete are made from the batch. Then the cylinders are tested for strength.
2.4 The ANOVA F Test 29

Experience from such experiments suggests that objects (e.g., cylinders) from
dierent batches are independent, but objects from the same batch are not
independent.
One check on independence can also be made if the time order of the
observations is known. Let r[t] be the residual where [t] is the time order of
the trial. Hence [1] was the 1st and [n] was the last trial. Plot the time order
t versus r[t] if the time order is known. Again, trends and outliers suggest
that the model could be improved. A box shaped plot with no trend suggests
that the MLR model is good. A plot similar to the Durbin Watson test plots
r[t1] versus r[t] for t = 2, . . . , n. Linear trend suggests serial correlation while
random scatter suggests that there is no lag 1 autocorrelation. As a rule of
thumb, if the OLS slope b is computed for the plotted points, b > 0.25 gives
some evidence that there is positive correlation between r[t1] and r[t] . Time
series plots, such as the ACF or PACF of the residuals, may be useful.

If it is assumed that the error distribution is symmetric, make a histogram

of the residuals. Check whether the histogram is roughly symmetric or clearly
skewed. If it is assumed that the errors ei are iid N (0, 2 ) again check
whether the histogram is mound shaped with short tails. A commonly
used alternative is to make a normal probability plot of the residuals. Let
r(1) < r(2) < < r(n) denote the residuals ordered from smallest to largest.
Hence r(1) is the value of the smallest residual. The normal probability plot
plots the e(i) versus r(i) where the e(i) are the expected values of the order
statistics from a sample of size n from an N (0, 1) distribution. (Often the e(i)
are the standard normal percentiles that satisfy P (Z e(i) ) = (i 0.5)/n
where Z N (0, 1).)
Rules of thumb: i) if the plotted points scatter about some straight line
in the normal probability plot, then there is no evidence against the normal
assumption. ii) if the plotted points have an ess shape (concave up then
concave down), then the error distribution is symmetric with lighter tails
than the normal distribution. iii) If the plot resembles a cubic function, then
the error distribution is symmetric with heavier tails than the normal distri-
bution. iv) If the plotted points look concave up (e.g. like x2 where x > 0),
then the error distribution is right skewed.

2.4 The ANOVA F Test

After tting least squares and checking the response and residual plots to see
that an MLR model is reasonable, the next step is to check whether there is
an MLR relationship between Y and the nontrivial predictors x2 , . . . , xp . If
at least one of these predictors is useful, then the OLS tted values Yi should
be used. If none of the nontrivial predictors is useful, then Y will give as good
predictions as Yi . Here the sample mean
30 2 Multiple Linear Regression

1
n
Y = Yi . (2.5)
n i=1

In the denition below, SSE is the sum of squared residuals and a residual
ri = ei = errorhat. In the literature errorhat is often rather misleadingly
abbreviated as error.

Denition 2.14. Assume that a constant is in the MLR model.

a) The total sum of squares


n
SST O = (Yi Y )2 . (2.6)
i=1

b) The regression sum of squares


n
SSR = (Yi Y )2 . (2.7)
i=1

c) The residual sum of squares or error sum of squares is


n 
n
SSE = (Yi Yi )2 = ri2 . (2.8)
i=1 i=1

The result in the following proposition is a property of least squares (OLS),

not of the underlying MLR model. An obvious application is that given any
two of SSTO, SSE, and SSR, the 3rd sum of squares can be found using the
formula SST O = SSE + SSR.

Proposition 2.3. Assume that a constant is in the MLR model. Then

SST O = SSE + SSR.

Proof.

n 
n
SST O = (Yi Yi + Yi Y )2 = SSE + SSR + 2 (Yi Yi )(Yi Y ).
i=1 i=1

Hence the result follows if


n
A ri (Yi Y ) = 0.
i=1

But

n 
n
A= ri Yi Y ri = 0
i=1 i=1

by Proposition 2.2 d) and e). 

2.4 The ANOVA F Test 31

Denition 2.15. Assume that a constant is in the MLR model and that
SSTO = 0. The coecient of multiple determination
SSR SSE
R2 = [corr(Yi , Yi )]2 = =1
SSTO SSTO

where corr(Yi , Yi ) is the sample correlation of Yi and Yi .

Warnings: i) 0 R2 1, but small R2 does not imply that the MLR

model is bad.
ii) If the MLR model contains a constant, then there are several equivalent
formulas for R2 . If the model does not contain a constant, then R2 depends
on the software package.
iii) R2 does not have much meaning unless the response plot and residual
plot both look good.
iv) R2 tends to be too high if n is small.
v) R2 tends to be too high if there are two or more separated clusters of
data in the response plot.
vi) R2 is too high if the number of predictors p is close to n.
vii) In large samples R2 will be large (close to one) if 2 is small compared
to the sample variance SY2 of the response variable Y . R2 is also large if the
sample variance of Y is close to SY2 . Thus R2 is sometimes interpreted as
the proportion of the variability of Y explained by conditioning on x, but
warnings i) - v) suggest that R2 may not have much meaning.

The following 2 propositions suggest that R2 does not behave well when
many predictors that are not needed in the model are included in the model.
Such a variable is sometimes called a noise variable and the MLR model
is tting noise. Proposition 2.5 appears, for example, in Cramer (1946,
pp. 414415), and suggests that R2 should be considerably larger than p/n
if the predictors are useful. Note that if n = 10p and p 2, then under the
conditions of Proposition 2.5, E(R2 ) 0.1.

Proposition 2.4. Assume that a constant is in the MLR model. Adding

a variable to the MLR model does not decrease (and usually increases) R2 .

Proposition 2.5. Assume that a constant 1 is in the MLR model, that

2 = = p = 0 and that the ei are iid N (0, 2 ). Hence the Yi are iid
N (1 , 2 ). Then

a) R2 follows a beta distribution: R2 beta( p1 np

2 , 2 ).
b)
p1
E(R2 ) = .
n1
c)
2(p 1)(n p)
VAR(R2 ) = .
(n 1)2 (n + 1)
32 2 Multiple Linear Regression

Notice that each SS/n estimates the variability of some quantity. SST O/n
SY2 , SSE/n Se2 = 2 , and SSR/n SY2 .

Denition 2.16. Assume that a constant is in the MLR model. Asso-

ciated with each SS in Denition 2.14 is a degrees of freedom (df) and a
mean square = SS/df . For SSTO, df = n 1 and M ST O = SST O/(n 1).
For SSR, df = p 1 and M SR = SSR/(p 1). For SSE, df = n p and
M SE = SSE/(n p).

Seber and Lee (2003, pp. 4447) show that when the MLR model holds,
MSE is often a good estimator of 2 . Under regularity conditions, the MSE
is one of the best unbiased quadratic estimators of 2 . For the normal MLR
model, MSE is the uniformly minimum variance unbiased estimator of 2 .
Seber and Lee also give the following theorem that shows that the MSE is an
unbiased estimator of 2 under very weak assumptions
if the MLR model is
appropriate. From Theorem 12.7 MSE is a n consistent estimator of 2 .
Theorem 2.6. If Y = X +e where X is an np matrix of full rank p, if
the ei are independent with E(ei ) = 0, and VAR(ei ) = 2 , then 2 = M SE
is an unbiased estimator of 2 .
The ANOVA F test tests whether any of the nontrivial predictors
x2 , . . . , xp are needed in the OLS MLR model, that is, whether Yi should
be predicted by the OLS t Yi = 1 + xi,2 2 + + xi,p p or with the
sample mean Y . ANOVA stands for analysis of variance, and the computer
output needed to perform the test is contained in the ANOVA table. Below
is an ANOVA table given in symbols. Sometimes Regression is replaced by
Model and Residual by Error.

Summary Analysis of Variance Table

Source df SS MS F p-value
Regression p-1 SSR MSR Fo=MSR/MSE for Ho:
Residual n-p SSE MSE 2 = = p = 0

Remark 2.4. Recall that for a 4 step test of hypotheses, the pvalue is the
probability of getting a test statistic as extreme as the test statistic actually
observed and that Ho is rejected if the pvalue < . As a benchmark for this
textbook, use = 0.05 if is not given. The 4th step is the nontechnical
conclusion which is crucial for presenting your results to people who are not
familiar with MLR. Replace Y and x2 , . . . , xp by the actual variables used in
the MLR model. Follow Example 2.5.

Notation. The pvalue pvalue given by output tends to only be cor-

rect for the normal MLR model. Hence the output is usually only giving an
estimate of the pvalue, which will often be denoted by pval. So reject H0 if
pval . Often
2.4 The ANOVA F Test 33

P
pval pvalue 0
(converges to 0 in probability, so pval is a consistent estimator of pvalue)
as the sample size n . See Theorem 11.25, Section 11.6, and Chang
and Olive (2010). Then the computer output pval is a good estimator of the
unknown pvalue. We will use Ho H0 and Ha HA H1 .

Be able to perform the 4 step ANOVA F test of hypotheses.

i) State the hypotheses Ho: 2 = = p = 0 Ha: not Ho.
ii) Find the test statistic Fo = M SR/M SE or obtain it from output.
iii) Find the pval from output or use the F table: pval =

P (Fp1,np > Fo ).

iv) State whether you reject Ho or fail to reject Ho. If Ho is rejected, conclude
that there is an MLR relationship between Y and the predictors x2 , . . . , xp . If
you fail to reject Ho, conclude that there is not an MLR relationship between
Y and the predictors x2 , . . . , xp . (Or there is not enough evidence to conclude
that there is an MLR relationship between Y and the predictors.)
Example 2.5. For the Gladstone (1905) data, the response variable Y =
brain weight, x1 1, x2 = size of head, x3 = sex, x4 = breadth of head,
x5 = circumference of head. Assume that the response and residual plots
look good and test whether at least one of the nontrivial predictors is needed
in the model using the output shown below.
Summary Analysis of Variance Table
Source df SS MS F p-value
Regression 4 5396942. 1349235. 196.24 0.0000
Residual 262 1801333. 6875.32
Solution: i) Ho: 2 = = 5 = 0 Ha: not Ho
ii) Fo = 196.24 from output.
iii) The pval = 0.0 from output.
iv) The pval < (= 0.05 since was not given). So reject Ho. Hence there
is an MLR relationship between brain weight and the predictors size, sex,
breadth, and circumference.
Remark 2.5. There is a close relationship between the response plot and
the ANOVA F test. If n 10p and n p 30 and if the plotted points
follow the identity line, typically Ho will be rejected if the identity line ts
the plotted points better than any horizontal line (in particular, the line
Y = Y ). If a horizontal line ts the plotted points about as well as the identity
line, as in Figure 1.4, this graphical diagnostic is inconclusive (sometimes the
ANOVA F test will reject Ho and sometimes fail to reject Ho), but the MLR
relationship is at best weak. In Figures 1.2 and 2.1, the ANOVA F test
should reject Ho since the identity line ts the plotted points better than
any horizontal line. Under the above conditions, a graphical ANOVA F test
34 2 Multiple Linear Regression

rejects Ho if the response plot is not similar to the residual plot. The graphical
test is inconclusive if the response plot looks similar to the residual plot. The
graphical test is also useful for multiple linear regression methods other than
least squares, such as M -estimators and other robust regression estimators.

Denition 2.17. An RR plot is a plot of residuals from 2 dierent mod-

els or tting methods.

Remark 2.6. If the RR plot of the residuals Yi Y versus the OLS resid-
uals ri = Yi Yi shows tight clustering about the identity line, then the MLR
relationship is weak: Y ts the data about as well as the OLS t.

Example 2.6. Cook and Weisberg (1999a, pp. 261, 371) describe a data
set where rats were injected with a dose of a drug approximately proportional
to body weight. The response Y was the fraction of the drug recovered from
the rats liver. The three predictors were the body weight of the rat, the dose
of the drug, and the liver weight. A constant was also used. The experimenter
expected the response to be independent of the predictors, and 19 cases
were used. However, the ANOVA F test suggested that the predictors were
important. The third case was an outlier and easily detected in the response
and residual plots (not shown). After deleting the outlier, the response and
residual plots looked ok and the following output was obtained.

Summary Analysis of Variance Table

Source df SS MS F p-value
Regression 3 0.00184396 0.000614652 0.10 0.9585
Residual 14 0.0857172 0.00612265

The 4 step ANOVA F test is

i) Ho: 2 = = 4 = 0 Ha: not Ho
ii) Fo = 0.10.
iii) pval = 0.9585.
iv) The pval > (= 0.05 since was not given). So fail to reject Ho. Hence
there is not an MLR relationship between fraction of drug recovered and the
predictors body weight, dose, and liver weight. (More accurately, there is not
enough statistical evidence to conclude that there is an MLR relationship:
failing to reject Ho is not the same as accepting Ho; however, it may be a
good idea to keep the nontechnical conclusions nontechnical.)
Figure 2.2 shows the RR plot where the residuals from the full model
are plotted against Yi Y , the residuals from the model using no nontrivial
predictors. This plot reinforces the conclusion that the response Y is inde-
pendent of the nontrivial predictors. The identity line and the OLS line from
regressing ri on Yi Y (that is, use Yi = ri , a constant and xi,2 = Yi Y ,
nd the OLS line and then plot it) are shown as visual aids. If the OLS line
and identity line nearly coincide in that it is dicult to tell that the two lines
intersect at the origin, then the 2 sets of residuals are close.
2.4 The ANOVA F Test 35

RR Plot

0.10
0.05
full$residual
0.0 -0.05
-0.10

-0.10 -0.05 0.0 0.05 0.10

subresidual

Fig. 2.2 RR Plot With Outlier Deleted, Submodel Uses Only the Trivial Predictor with
Y = Y

Some assumptions are needed on the ANOVA F test. Assume that both
the response and residual plots look good. It is crucial that there are no
outliers. Then a rule of thumb is that if n p is large, then the ANOVA
F test pvalue is approximately correct. An analogy can be made with the
central limit theorem, Y is a good estimator for if the Yi are iid N (, 2 )
and also a good estimator for if the data are iid with mean and variance
2 if n is large enough. Also see Theorem 11.25. More on the robustness and
lack of robustness of the ANOVA F test can be found in Wilcox (2012).
If all of the xi are dierent (no replication) and if the number of predictors
p = n, then the OLS t Yi = Yi and R2 = 1. Notice that Ho is rejected if the
statistic Fo is large. More precisely, reject Ho if

Fo > Fp1,np,1

where
P (F Fp1,np,1 ) = 1
when F Fp1,np . Since R2 increases to 1 while (n p)/(p 1) decreases
to 0 as p increases to n, Theorem 2.7a below implies that if p is large then
the Fo statistic may be small even if some of the predictors are very good. It
is a good idea to use n 10p or at least n 5p if possible. Theorem 11.25
can be used to show that pval is a consistent estimator of the pvalue under
reasonable conditions.
36 2 Multiple Linear Regression

Theorem 2.7. Assume that the MLR model has a constant 1 .

a)
M SR R2 np
Fo = = .
M SE 1 R2 p 1
b) If the errors ei are iid N (0, 2 ), and if Ho: 2 = = p = 0 is true,
then Fo has an F distribution with p 1 numerator and n p denominator
degrees of freedom: Fo Fp1,np .
c) If the errors are iid with mean 0 and variance 2 , if the error distribution
is close to normal, and if n p is large enough, and if Ho is true, then
Fo Fp1,np in that the p-value from the software (pval) is approximately
correct.

Remark 2.7. When a constant is not contained in the model (i.e., xi,1 is
not equal to 1 for all i), then the computer output still produces an ANOVA
table with the test statistic and pvalue, and nearly the same 4 step test of
hypotheses can be used. The hypotheses are now Ho: 1 = = p = 0 Ha:
not Ho, and you are testing whether or not there is an MLR relationship
between Y and x1 , . . . , xp . An MLR model without a constant (no intercept)
is sometimes called a regression through the origin. See Section 2.10.

2.5 Prediction

This section gives estimators for predicting a future or new value Yf of

the response variable given the predictors xf , and for estimating the mean
E(Yf ) E(Yf |xf ). This mean is conditional on the values of the predictors
xf , but the conditioning is often suppressed.
Warning: All too often the MLR model seems to t the training data

(Y1 , x1 ), . . . , (Yn , xn )

well, but when new test data is collected, a very dierent MLR model is
needed to t the new data well. In particular, the MLR model seems to t
the data (Yi , xi ) well for i = 1, . . . , n, but when the researcher tries to predict
Yf for a new vector of predictors xf , the prediction is very poor in that Yf is
not close to the Yf actually observed. Wait until after the MLR model
has been shown to make good predictions before claiming that the
model gives good predictions!
There are several reasons why the MLR model may not t new data well.
i) The model building process is usually iterative. Data Z, w1 , . . . , wr is col-
lected. If the model is not linear, then functions of Z are used as a potential
response variable and functions of the wi as potential predictors. After trial
and error, the functions are chosen, resulting in a nal MLR model using Y
and x1 , . . . , xp . Since the same data set was used during the model building
2.5 Prediction 37

process, biases are introduced and the MLR model ts the training data
better than it ts new test data. Suppose that Y , x1 , . . . , xp are specied
before collecting data and that the residual and response plots from the re-
sulting MLR model look good. Then predictions from the prespecied model
will often be better for predicting new data than a model built from an iter-
ative process.
ii) If (Yf , xf ) come from a dierent population than the population of
(Y1 , x1 ), . . . , (Yn , xn ), then prediction for Yf can be arbitrarily bad.
iii) Even a good MLR model may not provide good predictions for an xf
that is far from the xi (extrapolation).
iv) The MLR model may be missing important predictors (undertting).
v) The MLR model may contain unnecessary predictors (overtting).

Three remedies for i) are a) use previously published studies to select

an MLR model before gathering data. Unfortunately, often the population
from a published study is quite dierent from the population of the data
collected by the researcher. Then the MLR model from the published study
is not adequate. b) Do a trial study. Collect some data, build an MLR model
using the iterative process. Then use this model as the prespecied model and
collect data for the main part of the study. Better yet, do a trial study, specify
a model, collect more trial data, improve the specied model, and repeat until
the latest specied model works well. Unfortunately, trial studies are often
too expensive or not possible because the data is dicult to collect. c) If
the data set is large enough, use a training set of a random sample of k
of the n cases to build a model where 10p n/2 k 0.9n. Then use
validation set of the other n k cases to conrm that the model built
with the training set is good. This technique may help reduce biases, but
needs n 20p. See James et al. (2013, pp. 176178). In particular, build the
model with the training set, then check the asymptotically optimal prediction
interval (2.20), derived later in this section, on the validation set.

Denition 2.18. Consider the MLR model Y = X + e and the hat

matrix H = X(X T X)1 X T . Let hi = hii be the ith diagonal element of H
for i = 1, . . . , n. Then hi is called the ith leverage and hi = xTi (X T X)1 xi .
Suppose new data is to be collected with predictor vector xf . Then the
leverage of xf is hf = xTf (X T X)1 xf . Extrapolation occurs if xf is far
from the x1 , . . . , xn .

Rule of thumb 2.5. Predictions based on extrapolation are not reliable.

A rule of thumb is that extrapolation occurs if hf > max(h1 , . . . , hn ). This
rule works best if the predictors are linearly related in that a plot of xi versus
xj should not have any strong nonlinearities. If there are strong nonlinearities
among the predictors, then xf could be far from the xi but still have hf <
max(h1 , . . . , hn ).
38 2 Multiple Linear Regression

Example 2.7. Consider predicting Y = weight from x = height and a

constant from data collected on men between 18 and 24 where the minimum
height was 57 and the maximum height was 79 inches. The OLS equation
was Y = 167 + 4.7x. If x = 70, then Y = 167 + 4.7(70) = 162 pounds.
If x = 1 inch, then Y = 167 + 4.7(1) = 162.3 pounds. It is impossible
to have negative weight, but it is also impossible to nd a 1 inch man. This
MLR model should not be used for x far from the interval (57, 79).

Denition 2.19. Consider the unimodal MLR model Y = xT + e where

E(e) = 0. Then regression function is the hyperplane

E(Y ) E(Y |x) = x1 1 + x2 2 + + xp p = xT . (2.9)

Assume OLS is used to nd . Then the point estimator of Yf given x = xf

is
Yf = xf,1 1 + + xf,p p = xTf . (2.10)

The point estimator of E(Yf ) E(Yf |xf ) given x = xf is also Yf = xTf .

Assume that the MLR model contains a constant 1 so that x1 1. The large
sample 100 (1 )% condence interval (CI) for E(Yf |xf ) = xTf = E(Yf )
is
Yf tnp,1/2 se(Yf ) (2.11)
where P (T tnp, ) = if T has a t distribution with n p degrees of
freedom. Generally se(Yf ) will come from output, but
 
se(Yf ) = M SE hf = M SE xTf (X T X)1 xf .

Recall the interpretation of a 100 (1 )% CI for a parameter is that

if you collect data then form the CI, and repeat for a total of k times where
the k trials are independent from the same population, then the probability
that m of the CIs will contain follows a binomial(k, = 1 ) distribution.
Hence if 100 95% CIs are made, = 0.95 and about 95 of the CIs will contain
while about 5 will not. Any given CI may (good sample) or may not (bad
sample) contain , but the probability of a bad sample is .

The following theorem is analogous to the central limit theorem and the
theory for the tinterval for based on Y and the sample standard deviation
(SD) SY . If the data Y1 , . . . , Yn are iid with mean 0 and variance 2 , then Y
is asymptotically normal and the tinterval will perform well if the sample
size is large enough. The result below suggests that the OLS estimators Yi
and are good if the sample size is large enough. The condition max hi 0
in probability usually holds if the researcher picked the design matrix X or
if the xi are iid random vectors from a well-behaved population. Outliers
D
can cause the condition to fail. Convergence in distribution, Zn Np (0, ),
means the multivariate normal approximation can be used for probability
2.5 Prediction 39

calculations involving Zn . When p = 1, the univariate normal distribution

can be used. See Sen and Singer (1993, p. 280) for the theorem, which implies
that Np (, 2 (X T X)1 ). See Chapter 10 for the multivariate normal
distribution.
Theorem 2.8, LS CLT (Least Squares Central Limit Theorem):
Consider the MLR model Yi = xTi + ei and assume that the zero mean
errors are iid with E(ei ) = 0 and VAR(ei ) = 2 . Also assume that
maxi (h1 , . . . , hn ) 0 in probability as n and

XT X
W 1
n

as n . Then the least squares (OLS) estimator satises

D
n( ) Np (0, 2 W ). (2.12)

Equivalently,
D
(X T X)1/2 ( ) Np (0, 2 I p ). (2.13)

Denition 2.20. A large sample 100(1 )% prediction interval (PI) has

the form [Ln , Un ] where P (Ln Yf Un ) 1 as the sample size
n . For the Gaussian MLR model, assume that the random variable Yf
is independent of Y1 , . . . , Yn . Then the 100 (1 )% PI for Yf is

Yf tnp,1/2 se(pred) (2.14)

where P (T tnp, ) = if T has a t distribution with n p degrees of

freedom. Generally se(pred) will come from output, but

se(pred) = M SE (1 + hf ).

The interpretation of a 100 (1 )% PI for a random variable Yf is similar

to that of a CI. Collect data, then form the PI, and repeat for a total of k
times where k trials are independent from the same population. If Yf i is the
ith random variable and P Ii is the ith PI, then the probability that Yf i P Ii
for m of the PIs follows a binomial(k, = 1 ) distribution. Hence if 100
95% PIs are made, = 0.95 and Yf i P Ii happens about 95 times.
There are two big dierences between CIs and PIs. First, the length of
the CI goes to 0 as the sample size n goes to while the length of the PI
converges to some nonzero number J, say. Secondly, the CI for E(Yf |xf )
given in Denition 2.19 tends to work well for the unimodal MLR model if
the sample size is large while the PI in Denition 2.20 is made under the
assumption that the ei are iid N (0, 2 ) and may not perform well if the
normality assumption is violated.
40 2 Multiple Linear Regression

To see this, consider xf such that the heights Y of women between 18

and 24 is normal with a mean of 66 inches and an SD of 3 inches. A 95%
CI for E(Y |xf ) should be centered at about 66 and the length should go
to zero as n gets large. But a 95% PI needs to contain about 95% of the
heights so the PI should converge to the interval 66 1.96(3). This result
follows because if Y N (66, 9) then P (Y < 66 1.96(3)) = P (Y > 66 +
1.96(3)) 0.025. In other words, the endpoints of the PI estimate the 97.5
and 2.5 percentiles of the normal distribution. However, the percentiles of a
parametric error distribution depend heavily on the parametric distribution
and the parametric formulas are violated if the assumed error distribution is
incorrect.
Assume that the unimodal MLR model is valid so that e is from some
distribution with 0 mean and variance 2 . Olive (2007) shows that if 1 is
the asymptotic coverage of the classical nominal (1 )100% PI (2.14), then
1
1 = P (z1/2 < e < z1/2 ) 1 2 (2.15)
z1/2

where the inequality follows from Chebyshevs inequality. Hence the asymp-
totic coverage of the nominal 95% PI is at least 73.9%. The 95% PI (2.14)
was often quite accurate in that the asymptotic coverage was close to 95% for
a wide variety of error distributions. The 99% and 90% PIs did not perform
as well.

Let be the percentile of the error e, i.e. P (e ) = . Let be

the sample percentile of the residuals. Then the results from Theorem 2.8
suggest that the residuals ri estimate the errors ei , and that the sample
percentiles of the residuals estimate . For many error distributions,
 n   n 
 r2  e2
i 2 i
E(M SE) = E = =E .
i=1
np i=1
n

This result suggests that 

n
ri ei .
np
Using   
15 n
an = 1+ (1 + hf ), (2.16)
n np
a large sample semiparametric 100(1 )% PI for Yf is

[Yf + an /2 , Yf + an 1/2 ]. (2.17)

This PI is very similar to the classical PI except that is used instead of

z to estimate the error percentiles . The large sample coverage 1 of
this nominal 100(1 )% PI is asymptotically correct: 1 = 1 .
2.5 Prediction 41

Example 2.8. For the Buxton (1920) data suppose that the response Y
= height and the predictors were a constant, head length, nasal height, bigo-
nal breadth, and cephalic index. Five outliers were deleted leaving 82 cases.
Figure 2.3 shows a response plot of the tted values versus the response Y
with the identity line added as a visual aid. The plot suggests that the model
is good since the plotted points scatter about the identity line in an evenly
populated band although the relationship is rather weak since the correlation
of the plotted points is not very high. The triangles represent the upper and
lower limits of the semiparametric 95% PI (2.17). For this example, 79 (or
96%) of the Yi fell within their corresponding PI while 3 Yi did not. A plot
using the classical PI (2.14) would be very similar for this data. The plot was
made with the following R commands, using the lregpack function piplot.

x <- buxx[-c(61,62,63,64,65),]
Y <- buxy[-c(61,62,63,64,65)]
piplot(x,Y)
1900
1850
1800
1750
1700
Y
1650
1600
1550

1640 1660 1680 1700 1720 1740 1760

FIT

Fig. 2.3 95% PI Limits for Buxton Data

Given output showing i and given xf , se(pred), and se(Yf ), Example 2.9
shows how to nd Yf , a CI for E(Yf |xf ), and the classical PI (2.14) for Yf .
42 2 Multiple Linear Regression

Below is shown typical output in symbols. Sometimes Label is replaced by

Predictor and Estimate by coef or Coecients.

Label Estimate Std. Error t-value p-value

Constant 1 se(1 ) to,1 for Ho: 1 = 0
x2 2 se(2 ) to,2 = 2 /se(2 ) for Ho: 2 = 0
..
.
xp p se(p ) to,p = p /se(p ) for Ho: p = 0

Example 2.9. The Rounceeld (1995) data povc.lsp are female and
male life expectancies from n = 91 countries where 6 cases with missing GNP
were deleted. Suppose that it is desired to predict female life expectancy Y
from male life expectancy X. Suppose that if Xf = 60, then se(pred) =
2.1285, and se(Yf ) = 0.2241. Below is some output.

Label Estimate Std. Error t-value p-value

Constant -2.93739 1.42523 -2.061 0.0422
mlife 1.12359 0.0229362 48.988 0.0000

a) Find Yf if Xf = 60.
Solution: In this example, xf = (1, Xf )T since a constant is in the output
above. Thus Yf = 1 + 2 Xf = 2.93739 + 1.12359(60) = 64.478.
b) If Xf = 60, nd a 90% condence interval for E(Y ) E(Yf |xf ).
Solution: The CI is Yf tn2,1/2 se(Yf ) = 64.478 1.645(0.2241) =
64.478 0.3686 = [64.1094, 64.8466]. To use the ttable on the last page of
Chapter 14, use the 2nd to last row marked by Z since d = df = n 2 =
89 > 30. In the last row nd CI = 90% and intersect the 90% column and
the Z row to get the value of t89,0.95 z.95 = 1.645.
c) If Xf = 60, nd a 90% prediction interval for Yf .
Solution: The PI is Yf tn2,1/2 se(pred) = 64.478 1.645(2.1285) =
64.478 3.5014 = [60.9766, 67.9794].

Two more PIs will be dened and then the 4 PIs (2.14), (2.17), (2.18),
and (2.20) will be compared via simulation. An asymptotically conservative
(ac) 100(1 )% PI has asymptotic coverage 1 1 . We used the (ac)
100(1 )% PI
 
n
Yf max(|/2 |, |1/2 |) (1 + hf ) (2.18)
np

which has asymptotic coverage

2.5 Prediction 43

1 = P [ max(|/2 |, |1/2 |) < e < max(|/2 |, |1/2 |)]. (2.19)

Notice that 1 1 1 /2 and 1 = 1 if the error distribution

is symmetric.

In the simulations described below, will be the sample percentile for the
PIs (2.17) and (2.18). A PI is asymptotically optimal if it has the shortest
asymptotic length that gives the desired asymptotic coverage. If the error
distribution is unimodal, an asymptotically optimal PI can be created by
applying the shorth(c) estimator to the residuals where c = n(1 ) and
x is the smallest integer x, e.g., 7.7 = 8. That is, let r(1) , . . . , r(n) be the
order statistics of the residuals. Compute r(c) r(1) , r(c+1) r(2) , . . . , r(n)
r(nc+1) . Let [r(d) , r(d+c1) ] = [1 , 12 ] correspond to the interval with the
smallest distance. Then the large sample asymptotically optimal 100 (1)%
PI for Yf is
[Yf + an 1 , Yf + an 12 ] (2.20)
where an is given by (2.16).
Remark 2.8. We recommend using the asymptotically optimal PI (2.20)
instead of the classical PI (2.14). The lregpack function pisim can be used to
recreate the simulation described below. See Problem 2.29.
A small simulation study compares the PI lengths and coverages for sample
sizes n = 50, 100, and 1000 for several error distributions. The value n =
gives the asymptotic coverages and lengths. The MLR model with E(Yi ) =
1 + xi2 + + xi8 was used. The vectors (x2 , . . . , x8 )T were iid N7 (0, I 7 ).
The error distributions were N(0,1), t3 , and exponential(1) 1. Also, a small
sensitivity study to examine the eects of changing (1 + 15/n) to (1 + k/n)
on the 99% PIs (2.17) and (2.20) was performed. For n = 50 and k between
10 and 20, the coverage increased by roughly 0.001 as k increased by 1.

Table 2.1 N(0,1) Errors.

n clen slen alen olen ccov scov acov ocov
0.01 50 5.860 6.172 5.191 6.448 .989 .988 .972 .990
0.01 100 5.470 5.625 5.257 5.412 .990 .988 .985 .985
0.01 1000 5.182 5.181 5.263 5.097 .992 .993 .994 .992
0.01 5.152 5.152 5.152 5.152 .990 .990 .990 .990
0.05 50 4.379 5.167 4.290 5.111 .948 .974 .940 .968
0.05 100 4.136 4.531 4.172 4.359 .956 .970 .956 .958
0.05 1000 3.938 3.977 4.001 3.927 .952 .952 .954 .948
0.05 3.920 3.920 3.920 3.920 .950 .950 .950 .950
0.1 50 3.642 4.445 3.658 4.193 .894 .945 .895 .929
0.1 100 3.455 3.841 3.519 3.690 .900 .930 .905 .913
0.1 1000 3.304 3.343 3.352 3.304 .901 .903 .907 .901
0.1 3.290 3.290 3.290 3.290 .900 .900 .900 .900
44 2 Multiple Linear Regression

Table 2.2 t3 Errors.

n clen slen alen olen ccov scov acov ocov
0.01 50 9.539 12.164 11.398 13.297 .972 .978 .975 .981
0.01 100 9.114 12.202 12.747 10.621 .978 .983 .985 .978
0.01 1000 8.840 11.614 12.411 11.142 .975 .990 .992 .988
0.01 8.924 11.681 11.681 11.681 .979 .990 .990 .990
0.05 50 7.160 8.313 7.210 8.139 .945 .956 .943 .956
0.05 100 6.874 7.326 7.030 6.834 .950 .955 .951 .945
0.05 1000 6.732 6.452 6.599 6.317 .951 .947 .950 .945
0.05 6.790 6.365 6.365 6.365 .957 .950 .950 .950
0.1 50 5.978 6.591 5.532 6.098 .915 .935 .900 .917
0.1 100 5.696 5.756 5.223 5.274 .916 .913 .901 .900
0.1 1000 5.648 4.784 4.842 4.706 .929 .901 .904 .898
0.1 5.698 4.707 4.707 4.707 .935 .900 .900 .900

Table 2.3 Exponential(1) 1 Errors.

n clen slen alen olen ccov scov acov ocov
0.01 50 5.795 6.432 6.821 6.817 .971 .987 .976 .988
0.01 100 5.427 5.907 7.525 5.377 .974 .987 .986 .985
0.01 1000 5.182 5.387 8.432 4.807 .972 .987 .992 .987
0.01 5.152 5.293 8.597 4.605 .972 .990 .995 .990
0.05 50 4.310 5.047 5.036 4.746 .946 .971 .955 .964
0.05 100 4.100 4.381 5.189 3.840 .947 .971 .966 .955
0.05 1000 3.932 3.745 5.354 3.175 .945 .954 .972 .947
0.05 3.920 3.664 5.378 2.996 .948 .950 .975 .950
0.1 50 3.601 4.183 3.960 3.629 .920 .945 .925 .916
0.1 100 3.429 3.557 3.959 3.047 .930 .943 .945 .913
0.1 1000 3.303 3.005 3.989 2.460 .931 .906 .951 .901
0.1 3.290 2.944 3.991 2.303 .929 .900 .950 .900

The simulation compared coverages and lengths of the classical (2.14),

semiparametric (2.17), asymptotically conservative (2.18), and asymptoti-
cally optimal (2.20) PIs. The latter 3 intervals are asymptotically optimal
for symmetric unimodal error distributions in that they have the shortest
asymptotic length that gives the desired asymptotic coverage. The PIs (2.17)
and (2.20) also give the correct asymptotic coverage if the unimodal errors
are not symmetric, while the PI (2.18) gives higher coverage (is conserva-
tive). The simulation used 5000 runs and gave the proportion p of runs
where Yf fell within the nominal 100(1 )% PI. The count mp has a
binomial(m = 5000, p = 1 n ) distribution where 1 n converges to
theasymptotic coverage (1 ). The standard error for the proportion
is p(1 p)/5000 = 0.0014, 0.0031, and 0.0042 for p = 0.01, 0.05 and
0.1, respectively. Hence an observed coverage p (0.986, 0.994) for 99%,
p (0.941, 0.959) for 95%, and p (0.887, 0.913) for 90% PIs suggests that
there is no reason to doubt that the PI has the nominal coverage.
2.6 The Partial F Test 45

Tables 2.12.3 show the results of the simulations for the 3 error distri-
butions. The letters c, s, a, and o refer to intervals (2.14), (2.17), (2.18),
and (2.20), respectively. For the normal errors, the coverages were about
right and the semiparametric interval tended to be rather long for n = 50
and 100. The classical PI asymptotic coverage 1 tended to be fairly close
to the nominal coverage 1 for all 3 distributions and = 0.01, 0.05, and
0.1. The asymptotically optimal PI tended to have short length and simulated
coverage close to the nominal coverage.

2.6 The Partial F Test

Suppose that there is data on variables Z, w1 , . . . , wr and that a useful MLR

model has been made using Y = t(Z), x1 1, x2 , . . . , xp where each xi is
some function of w1 , . . . , wr . This useful model will be called the full model. It
is important to realize that the full model does not need to use every variable
wj that was collected. For example, variables with outliers or missing values
may not be used. Forming a useful full model is often very dicult, and it is
often not reasonable to assume that the candidate full model is good based
on a single data set, especially if the model is to be used for prediction.
Even if the full model is useful, the investigator will often be interested in
checking whether a model that uses fewer predictors will work just as well.
For example, perhaps xp is a very expensive predictor but is not needed given
that x1 , . . . , xp1 are in the model. Also a model with fewer predictors tends
to be easier to understand.

Denition 2.21. Let the full model use Y , x1 1, x2 , . . . , xp and let

the reduced model use Y , x1 , xi2 , . . . , xiq where {i2 , . . . , iq } {2, . . . , p}.

The partial F test is used to test whether the reduced model is good in
that it can be used instead of the full model. It is crucial that the reduced
model be selected before looking at the data. If the reduced model is selected
after looking at output and discarding the worst variables, then the pvalue
for the partial F test will be too high. For (ordinary) least squares, usually
a constant is used, and we are assuming that both the full model and the
reduced model contain a constant. The partial F test has null hypothesis
Ho : iq+1 = = ip = 0, and alternative hypothesis HA : at least one of the
ij = 0 for j > q. The null hypothesis is equivalent to Ho: the reduced model
is good. Since only the full model and reduced model are being compared,
the alternative hypothesis is equivalent to HA : the reduced model is not as
good as the full model, so use the full model, or more simply, HA : use the
full model.

To perform the partial F test, t the full model and the reduced model
and obtain the ANOVA table for each model. The quantities dfF , SSE(F)
46 2 Multiple Linear Regression

and MSE(F) are for the full model and the corresponding quantities from
the reduced model use an R instead of an F . Hence SSE(F) and SSE(R) are
the residual sums of squares for the full and reduced models, respectively.
Shown below is output only using symbols.

Full model

Source df SS MS Fo and p-value

Regression p 1 SSR MSR Fo=MSR/MSE
Residual dfF = n p SSE(F) MSE(F) for Ho:2 = = p = 0

Reduced model

Source df SS MS Fo and p-value

Regression q 1 SSR MSR Fo=MSR/MSE
Residual dfR = n q SSE(R) MSE(R) for Ho: 2 = = q = 0

Be able to perform the 4 step partial F test of hypotheses. i) State

the hypotheses. Ho: the reduced model is good Ha: use the full model
ii) Find the test statistic. FR =
 
SSE(R) SSE(F )
/M SE(F )
dfR dfF

iii) Find the pval = P(FdfR dfF ,dfF > FR ). (On exams typically an F table is
used. Here dfR dfF = pq = number of parameters set to 0, and dfF = np,
while pval is the estimated pvalue.)
iv) State whether you reject Ho or fail to reject Ho. Reject Ho if the pval
and conclude that the full model should be used. Otherwise, fail to reject Ho
and conclude that the reduced model is good.

Sometimes software has a shortcut. In particular, the R software uses the

anova command. See Problem 2.27. As an example, assume that the full
model uses x2 and x3 while the reduced model uses x2 . Both models contain
a constant. Then the following commands will perform the partial F test.
(On the computer screen the second command looks more like
red < lm(yx2).)
full <- lm(y~x2+x3)
red <- lm(y~x2)
anova(red,full)
For an n 1 vector a, let

a = a21 + + a2n = aT a
2.6 The Partial F Test 47

be the Euclidean norm of a. If r and r R are the vector of residuals from

the full and reduced models, respectively, notice that SSE(F ) = r2 and
SSE(R) = r R 2 .

The following proposition suggests that Ho is rejected in the partial F test

if the change in residual sum of squares SSE(R)SSE(F ) is large compared
to SSE(F ). If the change is small, then FR is small and the test suggests
that the reduced model can be used.

Proposition 2.6. Let R2 and RR 2

be the multiple coecients of determi-
nation for the full and reduced models, respectively. Let Y and Y R be the
vectors of tted values for the full and reduced models, respectively. Then
the test statistic in the partial F test is
 
SSE(R) SSE(F )
FR = /M SE(F ) =
dfR dfF
 
Y 2 Y R 2
/M SE(F ) =
dfR dfF

SSE(R) SSE(F ) n p R 2 RR2

np
= .
SSE(F ) pq 1R 2 pq
Denition 2.22. An FF plot is a plot of tted values from 2 dierent
models or tting methods.

Six plots are useful diagnostics for the partial F test: the RR plot with
the full model residuals on the vertical axis and the reduced model residuals
on the horizontal axis, the FF plot with the full model tted values on the
vertical axis, and always make the response and residual plots for the full
and reduced models. Suppose that the full model is a useful MLR model. If
the reduced model is good, then the response plots from the full and reduced
models should be very similar, visually. Similarly, the residual plots (of the
tted values versus the residuals) from the full and reduced models should be
very similar, visually. Finally, the correlation of the plotted points in the RR
and FF plots should be high, 0.95, say, and the plotted points in the RR
and FF plots should cluster tightly about the identity line. Add the identity
line to both the RR and FF plots as a visual aid. Also add the OLS line from
regressing r on r R to the RR plot (the OLS line is the identity line in the FF
plot). If the reduced model is good, then the OLS line should nearly coincide
with the identity line in that it should be dicult to see that the two lines
intersect at the origin, as in Figure 2.2. If the FF plot looks good but the
RR plot does not, the reduced model may be good if the main goal of the
analysis is to predict Y.
48 2 Multiple Linear Regression

In Chapter 3, Example 3.8 describes the Gladstone (1905) data. Let the
reduced model use a constant, (size)1/3 , sex, and age. Then Figure 3.7 shows
the response and residual plots for the full and reduced models, and Figure 3.9
shows the RR and FF plots.

Example 2.10. For the Buxton (1920) data, n = 76 after 5 outliers and
6 cases with missing values are removed. Assume that the response variable
Y is height, and the explanatory variables are x2 = bigonal breadth, x3 =
cephalic index, x4 = nger to ground, x5 = head length, x6 = nasal height,
and x7 = sternal height. Suppose that the full model uses all 6 predictors plus
a constant (x1 ) while the reduced model uses the constant, cephalic index,
and nger to ground. Test whether the reduced model can be used instead of
the full model using the output below.

Summary Analysis of Variance Table for the Full Model

Source df SS MS F p-value
Regression 6 260467. 43411.1 87.41 0.0000
Residual 69 34267.4 496.629

Summary Analysis of Variance Table for Reduced Model

Source df SS MS F p-value
Regression 2 94110.5 47055.3 17.12 0.0000
Residual 73 200623. 2748.27

Solution: The 4 step partial F test is shown below.

i) Ho: the reduced model is good Ha: use the full model
ii)
   
SSE(R) SSE(F ) 200623.0 34267.4
FR = /M SE(F ) = /496.629
dfR dfF 73 69

= 41588.9/496.629 = 83.742.
iii) pval = P (F4,69 > 83.742) = 0.00.
iv) The pval < (= 0.05, since was not given), so reject Ho. The full model
should be used instead of the reduced model. (Bigonal breadth, head length,
nasal height, and sternal height are needed in the MLR for height given that
cephalic index and nger to ground are in the model.)
Using a computer to get the pval makes sense, but for exams you may need
to use a table. In ARC, you can use the Calculate probability option from the
ARC menu, enter 83.742 as the value of the statistic, 4 and 69 as the degrees
of freedom, and select the F distribution. To use the table near the end of
Chapter 14, use the bottom row since the denominator degrees of freedom 69
> 30. Intersect with the column corresponding to k = 4 numerator degrees of
freedom. The cuto value is 2.37. If the FR statistic was 2.37, then the pval
would be 0.05. Since 83.472 > 2.37, the pval < 0.05, and since 83.472 >> 2.37,
we can say that the pval 0.0.
2.7 The Wald t Test 49

Example 2.11. Now assume that the reduced model uses the constant,
sternal height, nger to ground, and head length. Using the output below, test
whether the reduced model is good.
Summary Analysis of Variance Table for Reduced Model
Source df SS MS F p-value
Regression 3 259704. 86568. 177.93 0.0000
Residual 72 35030.1 486.528
Solution: The 4 step partial F test follows.
i) Ho: the reduced model is good Ha: use the full model
ii)
   
SSE(R) SSE(F ) 35030.1.0 34267.4
FR = /M SE(F ) = /496.629
dfR dfF 72 69
= 254.2333/496.629 = 0.512.
iii) The pval = P (F3,69 > 0.512) = 0.675.
iv) The pval > , so reject fail to reject Ho. The reduced model is good.
To use the F table near the end of Chapter 14, use the bottom row since
the denominator degrees of freedom 69 > 30. Intersect with the column cor-
responding to k = 3 numerator degrees of freedom. The cuto value is 2.61.
Since 0.512 < 2.61, pval > 0.05, and this is enough information to fail to
reject Ho.
Some R commands and output to do the above problem are shown below.
cyp <- matrix(scan(),nrow=76,ncol=8,byrow=T,dimnames=
list( c(), c("indx", "ht", "sternal", "finger",
"hdlen","nasal","bigonal", "cephalic")))
#copy and paste the data set cyp.lsp then press enter
cyp <- cyp[,-1]; cyp <- as.data.frame(cyp)
full <- lm(ht~.,data=cyp)
red <- lm(ht~sternal+finger+hdlen,data=cyp)
anova(red,full)
Model 1: ht ~ sternal + finger + hdlen
Model 2: ht ~ sternal + finger + hdlen + nasal
+ bigonal + cephalic
Res.Df RSS Df Sum of Sq F Pr(>F)
1 72 35030
2 69 34267 3 762.67 0.5119 0.6754

2.7 The Wald t Test

Often investigators hope to examine k in order to determine the importance

of the predictor xk in the model; however, k is the coecient for xk given
that the other predictors are in the model. Hence k depends strongly on
the other predictors in the model. Suppose that the model has an intercept:
50 2 Multiple Linear Regression

x1 1. The predictor xk is highly correlated with the other predictors if the

OLS regression of xk on x1 , . . . , xk1 , xk+1 , . . . , xp has a high coecient of
determination Rk2 . If this is the case, then often xk is not needed in the model
given that the other predictors are in the model. If at least one Rk2 is high
for k 2, then there is multicollinearity among the predictors.
As an example, suppose that Y = height, x1 1, x2 = left leg length, and
x3 = right leg length. Then x2 should not be needed given x3 is in the model
and 2 = 0 is reasonable. Similarly 3 = 0 is reasonable. On the other hand,
if the model only contains x1 and x2 , then x2 is extremely important with 2
near 2. If the model contains x1 , x2 , x3 , x4 = height at shoulder, x5 = right
arm length, x6 = head length, and x7 = length of back, then Ri2 may be high
for each i 2. Hence xi is not needed in the MLR model for Y given that
the other predictors are in the model.
Denition 2.23. The 100 (1 ) % CI for k is k tnp,1/2 se(k ). If
the degrees of freedom d = n p 30, the N(0,1) cuto z1/2 may be used.
Know how to do the 4 step Wald ttest of hypotheses.
i) State the hypotheses Ho: k = 0 Ha: k = 0.
ii) Find the test statistic to,k = k /se(k ) or obtain it from output.
iii) Find pval from output or use the ttable: pval =

2P (tnp < |to,k |) = 2P (tnp > |to,k |).

Use the normal table or the d = Z line in the ttable if the degrees of freedom
d = n p 30. Again pval is the estimated pvalue.
iv) State whether you reject Ho or fail to reject Ho and give a nontechnical
sentence restating your conclusion in terms of the story problem.

Recall that Ho is rejected if the pval . As a benchmark for this textbook,

use = 0.05 if is not given. If Ho is rejected, then conclude that xk is needed
in the MLR model for Y given that the other predictors are in the model.
If you fail to reject Ho, then conclude that xk is not needed in the MLR
model for Y given that the other predictors are in the model. (Or there is
not enough evidence to conclude that xk is needed in the MLR model given
that the other predictors are in the model.) Note that xk could be a very
useful individual predictor, but may not be needed if other predictors are
added to the model. It is better to use the output to get the test statistic
and pval than to use formulas and the ttable, but exams may not give the
relevant output.
Denition 2.24. Assume that there is a constant x1 1 in the model,
and let x(k) = (x1 , . . . , xk1 , xk+1 , . . . , xp )T be the vector of predictors with
the kth predictor xk deleted. Let r (k) be the residuals from regressing Y on
x(k) , that is, on all of the predictor variables except xk . Let r(xk |x(k) ) denote
the residuals from regressing xk on x(k) . Then an added variable plot for
xk is a plot of r(xk |x(k) ) versus r (k) for k = 2, . . . , p.
2.7 The Wald t Test 51

The added variable plot (also called a partial regression plot) is used to
give information about the test Ho : k = 0. The points in the plot cluster
about a line through the origin with slope = k . An interesting fact is that the
residuals from this line, i.e. the residuals from regressing r (k) on r(xk |x(k) ),
are exactly the same as the usual residuals from regressing Y on x. The range
of the horizontal axis gives information about the collinearity of xk with the
other predictors. Small range implies that xk is well explained by the other
predictors. The r(xk |x(k) ) represent the part of xk that is not explained by
the remaining variables while the r (k) represent the part of Y that is not
explained by the remaining variables.
An added variable plot with a clearly nonzero slope and tight clustering
about a line implies that xk is needed in the MLR for Y given that the other
predictors x2 , . . . , xk1 , xk+1 , . . . , xp are in the model. Slope near zero in the
added variable plot implies that xk may not be needed in the MLR for Y
given that all other predictors x2 , . . . , xk1 , xk+1 , . . . , xp are in the model.
If the zero line with 0 slope and 0 intercept and the OLS line are added to
the added variable plot, the variable is probably needed if it is clear that the
two lines intersect at the origin. Then the point cloud should be tilted away
from the zero line. The variable is probably not needed if the two lines nearly
coincide near the origin in that you cannot clearly tell that they intersect at
the origin.
Shown below is output only using symbols and the following example shows
how to use output to perform the Wald ttest.

Response = Y
Coecient Estimates

Label Estimate Std. Error t-value p-value

Constant 1 se(1 ) to,1 for Ho: 1 = 0
x2 2 se(2 ) to,2 = 2 /se(2 ) for Ho: 2 = 0
..
.
xp p se(p ) to,p = p /se(p ) for Ho: p = 0
Output for Ex. 2.12
Label Estimate Std. Error t-value p-value
Constant -7736.26 2660.36 -2.908 0.0079
x2 0.180225 0.00503871 35.768 0.0000
x3 -1.89411 2.65789 -0.713 0.4832

R Squared: 0.988, Sigma hat: 4756.08, n = 26

Summary Analysis of Variance Table

Source df SS MS F p-value
Regression 2 41380950140. 20690475070. 914.69 0.00
Residual 23 520265969. 22620260.
52 2 Multiple Linear Regression

Example 2.12. The output above was collected from 26 districts in Prus-
sia in 1843. See Hebbler (1847). The goal is to study the relationship between
Y = the number of women married to civilians in the district with the predic-
tors x2 = the population of the district, and x3 = military women = number
of women married to husbands in the military.
a) Find a 95% condence interval for 2 corresponding to population.
The CI is k tnp,1/2 se(k ). Since n = 26, df = n p = 26 3 = 23.
From the ttable at the end of Chapter 14, intersect the df = 23 row with
the column that is labelled by 95% in the CI row near the bottom of the
table. Then tnp,1/2 = 2.069. Using the output shows that the 95% CI is
0.180225 2.069(0.00503871) = [0.16980, 0.19065].
b) Perform a 4 step test for Ho: 2 = 0 corresponding to population.
i) Ho: 2 = 0 HA : 2 = 0
ii) to2 = 35.768
iii) pval = 0.0
iv) Reject Ho, the population is needed in the MLR model for the number
of women married to civilians if the number of military women is in the
model.
c) Perform a 4 step test for Ho: 3 = 0 corresponding to military women.
i) Ho: 3 = 0 HA : 3 = 0
ii) to3 = 0.713
iii) pval = 0.4883
iv) Fail to reject Ho, the number of military women is not needed in the
MLR model for the number of women married to civilians if population is in
the model.

Figure 2.4, made with the commands shown below, shows the added vari-
able plots for x2 and x3 . The plot for x2 strongly suggests that x2 is needed
in the MLR model while the plot for x3 indicates that x3 does not seem to
be very important. The slope of the OLS line in a) is 0.1802 while the slope
of the line in b) is 1.894.

source("G:/lregdata.txt")
x2 <- marry[,1]
x3 <- marry[,5]
y <- marry[,3]
#par(mfrow=c(1,2),pty="s")
#square plots look nice but have too much white space
par(mfrow=c(1,2))
resy2 <- residuals(lm(y~x3))
resx2 <- residuals(lm(x2~x3))
plot(resx2,resy2)
abline(lsfit(resx2,resy2)$coef)
title("a) Added Variable Plot for x2")
resy3 <- residuals(lm(y~x2))
2.8 The OLS Criterion 53

a) Added Variable Plot for x2 b) Added Variable Plot for x3

50000

10000
5000
0
resy2

resy3
0
50000

5000
10000

4e+05 0e+00 500 0 500 1000

resx2 resx3

Fig. 2.4 Added Variable Plots for x2 and x3

resx3 <- residuals(lm(x3~x2))

plot(resx3,resy3)
abline(lsfit(resx3,resy3)$coef)
title("b) Added Variable Plot for x3")
par(mfrow=c(1,1))
If the predictor xk is categorical, e.g. gender, the added variable plot may
look like two spheres, but if the OLS line is added to the plot, it will have
slope equal to k .

2.8 The OLS Criterion

The OLS estimator minimizes the OLS criterion


n
QOLS () = ri2 ()
i=1
54 2 Multiple Linear Regression

a) OLS Minimizes Sum of Squared Vertical Deviations

1600
1300
Y
1000

1000 1100 1200 1300 1400 1500

OLSESP

b) This ESP Has a Much Larger Sum

1600
1300
Y
1000

1100 1200 1300 1400 1500 1600

BADESP

Fig. 2.5 The OLS Fit Minimizes the Sum of Squared Residuals

n = Y2i 
where the residual ri () xTi . In other words, let ri = ri () be the
n
OLS residuals. Then i=1 ri i=1 ri2 () for any p 1 vector , and the
equality holds i = if the n p design
n matrixX n p n. In
is of full rank
n
particular, if X has full rank p, then i=1 ri2 < i=1 ri2 () = i=1 e2i even
if the MLR model Y = X + e is a good approximation to the data.

Example 2.13. When a model depends on the predictors x only through

the linear combination xT , then xT is called a sucient predictor and
xT is called an estimated sucient predictor (ESP). For OLS the model is
Y = xT + e, and the tted value Y = ESP . To illustrate the OLS criterion
graphically, consider the Gladstone (1905) data where we used brain weight as
the response. A constant, x2 = age, x3 = sex, and x4 = (size)1/3 were used
as predictors after deleting ve infants from the data set. In Figure 2.5a, the
OLS response plot of the OLS ESP = Y versus Y is shown. The vertical devi-
ations from the identity line are the residuals, and OLS minimizes the sum of
squared residuals. If any other ESP xT is plotted versus Y , then the vertical
2.8 The OLS Criterion 55

deviations from the identity line are the residuals ri (). For this data, the OLS
estimator = (498.726, 1.597, 30.462, 0.696)T . Figure 2.5b shows the re-
sponse plot using the ESP xT where = (498.726, 1.597, 30.462, 0.796)T .
Hence only the coecient for x4 was changed; however, the residuals ri () in
the resulting plot are much larger in magnitude on average than the residuals
in the OLS response plot. With slightly larger changes in the OLS ESP, the
resulting will be such that the squared residuals are massive.

Proposition 2.10. The OLS estimator is the unique minimizer of the

OLS criterion if X has full rank p n.

Proof: Seber and Lee (2003, pp. 3637). Recall that the hat matrix H =
X(X T X)1 X T and notice that (I H)T = I H, that (I H)H = 0
and that HX = X. Let be any p 1 vector. Then

(Y X )T (X X) = (Y HY )T (HY HX) =

Y T (I H)H(Y X) = 0.
Thus QOLS () = Y X2 = Y X + X X2 =
Y X 2 + X X2 + 2(Y X )T (X X).
Hence
Y X2 = Y X 2 + X X2 . (2.21)

So
Y X2 Y X 2

with equality i
X( ) = 0

i = since X is full rank. 

Alternatively calculus can be used. Notice that ri () = Yi xi,1 1 xi,2 2

xi,p p . Recall that xTi is the ith row of X while v j is the jth column.
Since QOLS () =


n
(Yi xi,1 1 xi,2 2 xi,p p )2 ,
i=1

the jth partial derivative

QOLS () n
= 2 xi,j (Yi xi,1 1 xi,2 2 xi,p p ) = 2(v j )T (Y X)
j i=1
56 2 Multiple Linear Regression

for j = 1, . . . , p. Combining these equations into matrix form, setting the

derivative to zero and calling the solution gives

X T Y X T X = 0,

or
X T X = X T Y . (2.22)
Equation (2.22) is known as the normal equations. If X has full rank,
then = (X T X)1 X T Y . To show that is the global minimizer of the
OLS criterion, use the argument following Equation (2.21).

2.9 Two Important Special Cases

When studying a statistical model, it is often useful to try to understand

the model that contains a constant but no nontrivial predictors, then try to
understand the model with a constant and one nontrivial predictor, then the
model with a constant and two nontrivial predictors, and then the general
model with many predictors. In this text, most of the models are such that
Y is independent of x given xT , written

Y x|xT .

Then wi = xTi is a scalar, and trying to understand the model in terms

of xTi is about as easy as trying to understand the model in terms of one
nontrivial predictor. In particular, the response plot of xTi versus Yi is
essential.
For MLR, the two main benets of studying the MLR model with one
nontrivial predictor X are that the data can be plotted in a scatterplot of Xi
versus Yi and that the OLS estimators can be computed by hand with the
aid of a calculator if n is small.

2.9.1 The Location Model

The location model

Yi = + ei , i = 1, . . . , n (2.23)
is a special case of the multiple linear regression model where p = 1, X = 1,
and = 1 = . This model contains a constant but no nontrivial predictors.
In the location model, OLS = 1 = = Y . To see this, notice that
2.9 Two Important Special Cases 57


n
dQOLS () n
QOLS () = (Yi ) 2
and = 2 (Yi ).
i=1
d i=1
n
Setting the derivative equal to 0 and calling the solution gives i=1 Yi = n
or = Y . The second derivative

d2 QOLS ()
= 2n > 0,
d 2
hence is the global minimizer.

2.9.2 Simple Linear Regression

The simple linear regression (SLR) model is

Yi = 1 + 2 Xi + ei = + Xi + ei

where the ei are iid with E(ei ) = 0 and VAR(ei ) = 2 for i = 1, . . . , n.

The Yi and ei are random variables while the Xi are treated as known
constants. The parameters 1 , 2 , and 2 are unknown constants that
need to be estimated. (If the Xi are random variables, then the model is
conditional on the Xi s provided that the errors ei are independent of the
Xi . Hence the Xi s are still treated as constants.)
The SLR model is a special case of the MLR model with p = 2, xi,1 1,
and xi,2 = Xi . The normal SLR model adds the assumption that the ei are
iid N(0, 2 ). That is, the error distribution is normal with zero mean and
constant variance 2 . The response variable Y is the variable that you want
to predict while the predictor variable X is the variable used to predict the
response. For SLR, E(Yi ) = 1 + 2 Xi and the line E(Y ) = 1 + 2 X is the
regression function. VAR(Yi ) = 2 .
For SLR, the least squares n estimators 1 and 2 minimize the least
squares criterion Q(1 , 2 ) = i=1 (Yi 1 2 Xi )2 . For a xed 1 and 2 ,
Q is the sum of the squared vertical deviations from the line Y = 1 + 2 X.
The least squares (OLS) line is Y = 1 + 2 X where the slope
n
(Xi X)(Yi Y )
2 = i=1 n
i=1 (Xi X)
2

and the intercept 1 = Y 2 X.

By the chain rule,

Q n
= 2 (Yi 1 2 Xi )
1 i=1
58 2 Multiple Linear Regression

and
2Q
= 2n.
12
Similarly,
Q n
= 2 Xi (Yi 1 2 Xi )
2 i=1

and
2Q n
=2 Xi2 .
22 i=1

Setting the rst partial derivatives to zero and calling the solutions 1 and
2 shows that the OLS estimators 1 and 2 satisfy the normal equations:

n 
n
Yi = n1 + 2 Xi and
i=1 i=1


n 
n 
n
Xi Yi = 1 Xi + 2 Xi2 .
i=1 i=1 i=1

The rst equation gives 1 = Y 2 X.

There are several equivalent formulas for the slope 2 .
n n n n
i=1 (Xi X)(Yi Y ) i=1 Xi Yi n (
1
i=1 Xi )( i=1 Yi )
2 = n = n n
i=1 (Xi X) i=1 Xi n (
2 2 1 2
i=1 Xi )
n n
(Xi X)Yi Xi Yi nX Y
= i=1
n = i=1
n = sY /sX .
i=1 (Xi X) i=1 Xi n(X)
2 2 2

Here the sample correlation (X, Y ) = corr(X, Y ) =

n n
i=1 (Xi X)(Yi Y ) (Xi X)(Yi Y )
=  i=1 n
(n 1)sX sY n
(X X)2 (Y Y )2
i=1 i i=1 i

where the sample standard deviation



 1  n
sW =  (Wi W )2
n 1 i=1

for W = X, Y. Notice that the term n 1 that occurs in the denominator of

, s2Y , and s2X can be replaced by n as long as n is used in all 3 quantities.
n
Also notice that the slope 2 = i=1 ki Yi where the constants
2.10 The No Intercept MLR Model 59

Xi X
ki =  n . (2.24)
j=1 (Xj X)
2

2.10 The No Intercept MLR Model

The no intercept MLR model, also known as regression through the origin, is
still Y = X+e, but there is no intercept in the model, so X does not contain
a column of ones 1. Hence the intercept term 1 = 1 (1) is replaced by 1 xi1 .
Software gives output for this model if the no intercept or intercept = F
option is selected. For the no intercept model, the assumption E(e) = 0 is
important, and this assumption is rather strong.
Many of the usual MLR results still hold: OLS = (X T X)1 X T Y , the
vector of predicted tted values Y = X OLS = HY where the hat matrix
H = X(X T X)1 X T provided the inverse exists, and the vector of residuals
is r = Y Y . The response plot and residual plot are made in the same way
and should be made before performing inference.
The main dierence in the output is the ANOVA table. The ANOVA F
test in Section 2.4 tests Ho : 2 = = p = 0. The test in this section tests
Ho : 1 = = p = 0 Ho : = 0. The following denition and test
follows Guttman (1982, p. 147) closely.

Denition 2.25. Assume that Y = X + e where the ei are iid. Assume

that it is desired to test Ho : = 0 versus HA : = 0.
a) The uncorrected total sum of squares


n
SST = Yi2 . (2.25)
i=1

b) The model sum of squares


n
SSM = Yi2 . (2.26)
i=1

c) The residual sum of squares or error sum of squares is


n 
n
SSE = (Yi Yi ) =
2
ri2 . (2.27)
i=1 i=1

d) The degrees of freedom (df) for SSM is p, the df for SSE is n p and
the df for SST is n. The mean squares are MSE = SSE/(n p) and MSM =
SSM/p.

The ANOVA table given for the no intercept or intercept = F option

is below.
60 2 Multiple Linear Regression

Summary Analysis of Variance Table

Source df SS MS F p-value
Model p SSM MSM Fo=MSM/MSE for Ho:
Residual n-p SSE MSE =0

The 4 step no intercept ANOVA F test for = 0 is below.

i) State the hypotheses Ho: = 0, Ha: = 0.
ii) Find the test statistic Fo = M SM/M SE or obtain it from output.
iii) Find the pval from output or use the F table: pval = P (Fp,np > Fo ).
iv) State whether you reject Ho or fail to reject Ho. If Ho is rejected, conclude
that there is an MLR relationship between Y and the predictors x1 , . . . , xp . If
you fail to reject Ho, conclude that there is not an MLR relationship between
Y and the predictors x1 , . . . , xp . (Or there is not enough evidence to conclude
that there is an MLR relationship between Y and the predictors.)

Warning: Several important models can be cast in the no intercept MLR

form, but often a dierent test than Ho : = 0 is desired. For example, when
the generalized or weighted least squares models of Chapter 4 are transformed
into no intercept MLR form, the test of interest is Ho: 2 = = p = 0.
The one way ANOVA model of Chapter 5 is equivalent to the cell means
model, which is in no intercept MLR form, but the test of interest is Ho :
1 = = p .

Proposition 2.11. Suppose Y = X + e where X may or may not con-

tain a column of ones. Then the partial F test of Section 2.6 can be used for
inference.

Example 2.14. Consider the Gladstone (1905) data described in Exam-

ple 2.5. If the le of data sets lregdata is downloaded into R, then the ANOVA
F statistic for testing 2 = = 4 = 0 can be found with the following
commands. The command lst adds a column of ones to x which contains
the variables size, sex, breadth, and circumference. Three of these predictor
variables are head measurements. Then the response Y is brain weight, and
the model contains a constant (intercept).

> y <- cbrainy

> x <- cbrainx[,c(11,10,3,6)]
> ls.print(lsfit(x,y))
F-statistic (df=4, 262)=196.2433

The ANOVA F test can also be found with the no intercept model by
adding a column of ones to the R matrix x and then performing the partial
F test with the full model and the reduced model that only uses the column
of ones. Notice that the intercept=F option needs
to be used to t both
models. The residual standard error = RSE = M SE. Thus SSE = (n
k)(RSE)2 where n k is the denominator degrees of freedom for the F test
2.11 Summary 61

and k is the numerator degrees of freedom = number of variables in the

model. The column of ones xone is counted as a variable. The last line of
output computes the partial F statistic and is again 196.24.

> xone <- 1 + 0*1:267

> x <- cbind(xone,x)
> ls.print(lsfit(x,y,intercept=F))
Residual Standard Error=82.9175
F-statistic (df=5, 262)=12551.02

Estimate Std.Err t-value Pr(>|t|)

xone 99.8495 171.6189 0.5818 0.5612
size 0.2209 0.0358 6.1733 0.0000
sex 22.5491 11.2372 2.0066 0.0458
breadth -1.2464 1.5139 -0.8233 0.4111
circum 1.0255 0.4719 2.1733 0.0307

> ls.print(lsfit(x[,1],y,intercept=F))
Residual Standard Error=164.5028
F-statistic (df=1, 266)=15744.48

Estimate Std.Err t-value Pr(>|t|)

X 1263.228 10.0674 125.477 0

((266*(164.5028)^2 - 262*(82.9175)^2)/4)/(82.9175)^2
[1] 196.2435

2.11 Summary

1) The response variable is the variable that you want to predict. The pre-
dictor variables are the variables used to predict the response variable.
2) Regression is the study of the conditional distribution Y |x.
3) The MLR model is

Yi = xi,1 1 + xi,2 2 + + xi,p p + ei = xTi + ei

for i = 1, . . . , n. Here n is the sample size and the random variable ei is the ith
error. Assume that the errors are iid with E(ei ) = 0 and VAR(ei ) = 2 < .
Assume that the errors are independent of the predictor variables xi . The
unimodal MLR model assumes that the ei are iid from a unimodal distribution
that is not highly skewed. Usually xi,1 1.

4) In matrix notation, these n equations become

62 2 Multiple Linear Regression

Y = X + e,

where Y is an n 1 vector of dependent variables, X is an n p matrix

of predictors, is a p 1 vector of unknown coecients, and e is an n 1
vector of unknown errors.
T 1 T 2
n5) The OLS estimators are OLS = (X X) X Y and = M SE =
i=1 ri /(n p). Thus =
2
M SE. The vector of predicted or tted values

Y OLS = X OLS = HY where the hat matrix H = X(X T X)1 X T . The
ith tted value Yi = xTi . The ith residual ri = Yi Yi and the vector of
residuals r = Y Y = (I H)Y . The least squares regression equation for
a model containing a constant is Y = 1 + 2 x2 + + p xp .
6) Always make the response plot of Y versus Y and residual plot of
Y versus r for any MLR analysis. The response plot is used to visualize the
MLR model, that is, to visualize the conditional distribution of Y |xT . If the
unimodal MLR model of 3) is useful, then i) the plotted points in the response
plot should scatter about the identity line with no other pattern, and ii) the
plotted points in the residual plot should scatter about the r = 0 line with no
other pattern. If either i) or ii) is violated, then the unimodal MLR model is
not sustained. In other words, if the plotted points in the residual plot show
some type of dependency, e.g. increasing variance or a curved pattern, then
the multiple linear regression model may be inadequate.
7) Use xf max hi for valid predictions.
8) If the
MLR model contains  a constant, then SSTO = SSE n + SSR where
n n
SST
n 2 O = i=1 (Y i Y ) 2
, SSR = i=1 ( Y i Y ) 2
and SSE = i=1 (Yi Yi ) =
2

i=1 ri .
9) If the MLR model contains a constant, then R2 = [corr(Yi , Yi )]2 =
SSR SSE
=1 .
SSTO SSTO

Source df SS MS F p-value
Regression p-1 SSR MSR Fo=MSR/MSE for Ho:
Residual n-p SSE MSE 2 = = p = 0

10) Be able to perform the 4 step ANOVA F test of hypotheses.

i) State the hypotheses Ho: 2 = = p = 0 Ha: not Ho.
ii) Find the test statistic Fo = M SR/M SE or obtain it from output.
iii) Find pval, the estimated pvalue, from output or use the F table:
pval = P (Fp1,np > Fo ).
iv) State whether you reject Ho or fail to reject Ho. If Ho is rejected, conclude
that there is an MLR relationship between Y and the predictors x2 , . . . , xp .
If you fail to reject Ho, conclude that there is a not an MLR relationship
between Y and the predictors x2 , . . . , xp .
2.11 Summary 63

11) The large sample 100 (1 )% CI for E(Yf |xf ) = xTf = E(Yf ) is
Yf tnp,1/2 se(Yf ) where P (T tnp, ) = if T has a t distribution with
n p degrees of freedom.
12) The classical 100 (1 )% PI for Yf is Yf tnp,1/2 se(pred), but
should be replaced with the asymptotically optimal PI (2.20).
Full model

Source df SS MS Fo and p-value

Regression p 1 SSR MSR Fo=MSR/MSE
Residual dfF = n p SSE(F) MSE(F) for Ho:2 = = p = 0

Reduced model

Source df SS MS Fo and p-value

Regression q 1 SSR MSR Fo=MSR/MSE
Residual dfR = n q SSE(R) MSE(R) for Ho: 2 = = q = 0

13) Be able to perform the 4 step partial F test of hypotheses. i) State

the hypotheses Ho: the reduced model is good Ha: use the full model.
ii) Find the test statistic FR =
 
SSE(R) SSE(F )
/M SE(F )
dfR dfF

iii) Find the pval = P(FdfR dfF ,dfF > FR ). (On exams typically an F table is
used. Here dfR dfF = pq = number of parameters set to 0, and dfF = np,
while pval is the estimated pvalue.)
iv) State whether you reject Ho or fail to reject Ho. Reject Ho if the pval
and conclude that the full model should be used. Otherwise, fail to reject Ho
and conclude that the reduced model is good.

Label Estimate Std. Error t-value p-value

Constant 1 se(1 ) to,1 for Ho: 1 = 0

x2 2 se(2 ) to,2 = 2 /se(2 ) for Ho: 2 = 0
..
.
xp p se(p ) to,p = p /se(p ) for Ho: p = 0

14) The 100 (1 ) % CI for k is k tnp,1/2 se(k ). If the degrees

of freedom d = n p 30, the N(0,1) cuto z1/2 may be used.
15) The corresponding 4 step ttest of hypotheses has the following steps.
i) State the hypotheses Ho: k = 0 Ha: k = 0.
ii) Find the test statistic to,k = k /se(k ) or obtain it from output.
iii) Find the pval from output or use the ttable: pval =
64 2 Multiple Linear Regression

2P (tnp < |to,k |) = 2P (tnp > |to,k |).

Use the normal table or the d = Z line in the ttable if the degrees of freedom
d = n p 30.
iv) State whether you reject Ho or fail to reject Ho and give a nontechnical
sentence restating your conclusion in terms of the story problem. If Ho is
rejected, then conclude that xk is needed in the MLR model for Y given that
the other predictors are in the model. If you fail to reject Ho, then conclude
that xk is not needed in the MLR model for Y given that the other predictors
are in the model.
n n
16) Given i=1 (Xi X)(Yi Y ), i=1 (Xi X)2 , X, and Y , nd the
least squares line Y = 1 + 2 X where
n
(Xi X)(Yi Y )
2 = i=1 n
i=1 (Xi X)
2

and 1 = Y 2 X.
17) Given , sX , sY , X, and Y , nd the least squares line Y = 1 + 2 X
where 2 = sY /sX and 1 = Y 2 X.

2.12 Complements

The Least Squares Central Limit Theorem 2.8 is often a good approximation
if n 10p and the error distribution has light tails, i.e. the probability of
an outlier is nearly 0 and the tails go to zero at an exponential rate or faster.
For error distributions with heavier tails, much larger samples are needed,
and the assumption that the variance 2 exists is crucial, e.g. Cauchy errors
are not allowed. Norman and Streiner (1986, p. 63) recommend n 5p.
The classical MLR prediction interval does not work well and should be re-
placed by the Olive (2007) asymptotically optimal PI (2.20). Lei and Wasser-
man (2014) provide an alternative: use the Lei et al. (2013) PI [rL , rU ] on the
residuals, then the PI for Yf is

[Yf + rL , Yf + rU ]. (2.28)

Bootstrap PIs need more theory and instead of using B = 1000 samples, use
B = max(1000, n). See Olive (2014, pp. 279285).
For the additive error regression model Y = m(x) + e, the response plot
of Y = m(x) vs. Y , with the identity line added as a visual aid, is used
like the MLR response plot. We want n 10 df where df is the degrees of
freedom from tting m. Olive (2013a) provides PIs for this model, including
the location model. These PIs are large sample PIs provided that the sample
quantiles of the residuals are consistent estimators of the population quantiles
2.12 Complements 65

of the errors. The response plot and PIs could also be used for methods
described in James et al. (2013) such as ridge regression, lasso, principal
components regression, and partial least squares. See Pelawa Watagoda and
Olive (2017) if n is not large compared to p.
In addition to large sample theory, we want the PIs to work well on a
single data set as future observations are gathered, but only have the training
data (x1 , Y1 ), . . . , (xn , Yn ). Much like k-fold cross validation for discriminant
analysis, randomly divide the data set into k = 5 groups of approximately
equal size. Compute the model from 4 groups and use the 5th group as a
validation set: compute the PI for xf = xj for each j in the 5th group.
Repeat so each of the 5 groups is used as a validation set. Compute the
proportion of times Yi was in its PI for i = 1, . . . , n as well as the average
length of the n PIs. We want the proportion near the nominal proportion
and short average length if two or more models or PIs are being considered.
Following Chapter 11, under the regularity conditions, much of the infer-
ence that is valid for the normal MLR model is approximately valid for the
unimodal MLR model when the sample size is large. For example, condence
intervals for i are asymptotically correct, as are t tests for i = 0 (see Li
and Duan (1989, p. 1035)), the MSE is an estimator of 2 by Theorems 2.6
and 2.7, and variable selection procedures perform well (see Chapter 3 and
Olive and Hawkins 2005).
Algorithms for OLS are described in Datta (1995), Dongarra et al. (1979),
and Golub and Van Loan (1989). See Harter (1974a,b, 1975a,b,c, 1976) for
a historical account of multiple linear regression. Draper (2002) provides a
bibliography of more recent references.
Cook and Weisberg (1997, 1999a: ch. 17) call a plot that emphasizes model
agreement a model checking plot. Anscombe (1961) and Anscombe and Tukey
(1963) suggested graphical methods for checking multiple linear regression
and experimental design methods that were the state of the art at the
time.
The rules of thumb given in this chapter for residual plots are not perfect.
Cook (1998, pp. 46) gives an example of a residual plot that looks like a
right opening megaphone, but the MLR assumption that was violated was
linearity, not constant variance. Ghosh (1987) gives an example where the
residual plot shows no pattern even though the constant variance assumption
is violated. Searle (1988) shows that residual plots will have parallel lines if
several cases take on each of the possible values of the response variable, e.g.
if the response is a count.
Several authors have suggested using the response plot to visualize the co-
ecient of determination R2 in multiple linear regression. See, for example,
Chambers et al. (1983, p. 280). Anderson-Sprecher (1994) provides an ex-
cellent discussion about R2 . Kachigan (1982, pp. 174177) also gives a good
explanation of R2 . Also see Kvalseth (1985), and Freedman (1983).
Hoaglin and Welsh (1978) discuss the hat matrix H, and Brooks et al.
(1988) recommend using xf < max hi for valid predictions. Simultaneous
66 2 Multiple Linear Regression

prediction intervals are given by Sadooghi-Alvandi (1990). Olive (2007) sug-

gests three large sample prediction intervals for MLR that are valid under
the unimodal MLR model. Also see Schoemoyer (1992).
Sall (1990) discusses the history of added variable plots while Darlington
(1969) provides an interesting proof that minimizes the OLS criterion.

2.12.1 Lack of Fit Tests

Label Estimate Std. Error t-value p-value

Constant 1 se(1 ) to,1 for Ho: 1 = 0

x2 2 se(2 ) to,2 = 2 /se(2 ) for Ho: 2 = 0
..
.
xp p se(p ) to,p = p /se(p ) for Ho: p = 0

R Squared: R2
Sigma hat: M SE
Number of cases: n
Degrees of Freedom : np

Source df SS MS F p-value
Regression p-1 SSR MSR Fo=MSR/MSE for Ho:
Residual n-p SSE MSE 2 = = p = 0

The typical relevant OLS output has the form given above, but occa-
sionally software also includes output for a lack of t test as shown below.

Source df SS MS Fo
Regression p1 SSR MSR Fo=MSR/MSE
Residual np SSE MSE
lack of t cp SSLF MSLF FLF = MSLF/MSPE
pure error nc SSPE MSPE

The lack of t test assumes that

Yi = m(xi ) + ei (2.29)

where E(Yi |xi ) = m(xi ), m is some possibly nonlinear function, and that
the ei are iid N (0, 2 ). Notice that the MLR model is the special case with
m(xi ) = xTi . The lack of t test needs at least one replicate: 2 or more Ys
with the same value of predictors x. Then there a c replicate groups with
nj observations in the jth group. Each group has the vector of predictors xj ,
2.12 Complements 67
c
say, and at least one nj > 1. Also, j=1 nj = n. Denote the Ys in the jth
group by Yij , and let the sample mean of the Ys in the jth group be Y j .
Then
1 
nj
(Yij Y j )2
nj 1 i=1

is an estimator of 2 for each group with nj > 1. Let


c 
nj
SSP E = (Yij Y j )2 .
j=1 i=1

Then M SP E = SSP E/(n c) is an unbiased estimator of 2 when

model (2.29) holds, regardless of the form of m. The PE in SSPE stands
for pure error. c
Now SSLF = SSE SSP E = j=1 nj (Y j Yj )2 . Notice that Y j is an
unbiased estimator of m(xj ) while Yj is an estimator of m if the MLR model
is appropriate: m(xj ) = xTj . Hence SSLF and MSLF can be very large if
the MLR model is not appropriate.
The 4 step lack of t test is i) Ho: no evidence of MLR lack of t, HA :
there is lack of t for the MLR model.
ii) FLF = M SLF/M SP E.
iii) The pval = P (Fcp,nc > FLF ).
iv) Reject Ho if pval and state the HA claim that there is lack of t.
Otherwise, fail to reject Ho and state that there is not enough evidence to
conclude that there is MLR lack of t.

Although the lack of t test seems clever, examining the response plot and
residual plot is a much more eective method for examining whether or not
the MLR model ts the data well provided that n 10p. A graphical version
of the lack of t test would compute the Y j and see whether they scatter
about the identity line in the response plot. When there are no replicates,
the range of Y could be divided into several narrow nonoverlapping intervals
called slices. Then the mean Y j of each slice could be computed and a step
function with step height Y j at the jth slice could be plotted. If the step
function follows the identity line, then there is no evidence of lack of t.
However, it is easier to check whether the Yi are scattered about the identity
line. Examining the residual plot is useful because it magnies deviations
from the identity line that may be dicult to see until the linear trend is
removed. The lack of t test may be sensitive to the assumption that the
errors are iid N (0, 2 ).
When Y x|xT , then the response plot of the estimated sucient pre-
dictor (ESP) xT versus Y is used to visualize the conditional distribution of
Y |xT , and will often greatly outperform the corresponding lack of t test.
When the response plot can be combined with a good lack of t plot such as
68 2 Multiple Linear Regression

a residual plot, using a one number summary of lack of t such as the test
statistic FLF makes little sense.
Nevertheless, the literature for lack of t tests for various statistical meth-
ods is enormous. See Joglekar et al. (1989), Pena and Slate (2006), and Su
and Yang (2006) for references.

For the following homework problems, Cody and Smith (2006) is useful
for SAS, while Cook and Weisberg (1999a) is useful for Arc. Becker et al.
(1988) and Crawley (2013) are useful for R.

2.13 Problems

Problems with an asterisk * are especially important.

Output for Problem 2.1

Full Model Summary Analysis of Variance Table
Source df SS MS F p-value
Regression 6 265784. 44297.4 172.14 0.0000
Residual 67 17240.9 257.327

Reduced Model Summary Analysis of Variance Table

Source df SS MS F p-value
Regression 1 264621. 264621. 1035.26 0.0000
Residual 72 18403.8 255.608

2.1. Assume that the response variable Y is height, and the explanatory
variables are X2 = sternal height, X3 = cephalic index, X4 = nger to ground,
X5 = head length, X6 = nasal height, and X7 = bigonal breadth. Suppose that
the full model uses all 6 predictors plus a constant (= X1 ) while the reduced
model uses the constant and sternal height. Test whether the reduced model
can be used instead of the full model using the output above. The data set
had 74 cases.

Output for Problem 2.2

Full Model Summary Analysis of Variance Table
Source df SS MS F p-value
Regression 9 16771.7 1863.52 1479148.9 0.0000
Residual 235 0.29607 0.00126

Reduced Model Summary Analysis of Variance Table

Source df SS MS F p-value
Regression 2 16771.7 8385.85 6734072.0 0.0000
Residual 242 0.301359 0.0012453
2.13 Problems 69

Coefficient Estimates, Response = y, Terms = (x2 x2^2)

Label Estimate Std. Error t-value p-value
Constant 958.470 5.88584 162.843 0.0000
x2 -1335.39 11.1656 -119.599 0.0000
x2^2 421.881 5.29434 79.685 0.0000

2.2. The above output, starting on the previous page, comes from the
Johnson (1996) STATLIB data set bodyfat after several outliers are deleted.
It is believed that Y = 1 + 2 X2 + 3 X22 + e where Y is the persons bodyfat
and X2 is the persons density. Measurements on 245 people were taken. In
addition to X2 and X22 , 7 additional measurements X4 , . . . , X10 were taken.
Both the full and reduced models contain a constant X1 1.

a) Predict Y if X2 = 1.04. (Use the reduced model Y = 1 + 2 X2 +

3 X22 + e.)

b) Test whether the reduced model can be used instead of the full model.

Output for Problem 2.3

Label Estimate Std. Error t-value p-value
Constant -5.07459 1.85124 -2.741 0.0076
log[H] 1.12399 0.498937 2.253 0.0270
log[S] 0.573167 0.116455 4.922 0.0000

R Squared: 0.895655 Sigma hat: 0.223658, n = 82

(log[H] log[S]) (4 5)
Prediction = 2.2872, s(pred) = 0.467664,
Estimated population mean value = 2.287, s = 0.410715

2.3. The above output was produced from the le mussels.lsp in Arc. See
Cook and Weisberg (1999a). Let Y = log(M) where M is the muscle mass
of a mussel. Let X1 1, X2 = log(H) where H is the height of the shell,
and let X3 = log(S) where S is the shell mass. Suppose that it is desired to
predict Yf if log(H) = 4 and log(S) = 5, so that xTf = (1, 4, 5). Assume that
se(Yf ) = 0.410715 and that se(pred) = 0.467664.

a) If xTf = (1, 4, 5) nd a 99% condence interval for E(Yf ).

b) If xTf = (1, 4, 5) nd a 99% prediction interval for Yf .

Problem 2.4 Output, Coef. Estimates Response = height
Label Estimate Std. Error t-value p-value
Constant 227.351 65.1732 3.488 0.0008
sternal height 0.955973 0.0515390 18.549 0.0000
finger to ground 0.197429 0.0889004 2.221 0.0295
70 2 Multiple Linear Regression

R Squared: 0.879324 Sigma hat: 22.0731

Summary Analysis of Variance Table

Source df SS MS F p-value
Regression 2 259167. 129583. 265.96 0.0000
Residual 73 35567.2 487.222

2.4. The above output, starting on the previous page, is from the multiple
linear regression of the response Y = height on the two nontrivial predictors
sternal height = height at shoulder, and nger to ground = distance from the
tip of a persons middle nger to the ground.

a) Consider the plot with Yi on the vertical axis and the least squares
tted values Yi on the horizontal axis. Sketch how this plot should look if the
multiple linear regression model is appropriate.

b) Sketch how the residual plot should look if the residuals ri are on the
vertical axis and the tted values Yi are on the horizontal axis.

c) From the output, are sternal height and nger to ground useful for
predicting height? (Perform the ANOVA F test.)

2.5. Suppose that it is desired to predict the weight of the brain (in
grams) from the cephalic index measurement. The output below uses data
from 267 people.
predictor coef Std. Error t-value p-value
Constant 865.001 274.252 3.154 0.0018
cephalic 5.05961 3.48212 1.453 0.1474
Do a 4 step test for 2 = 0.

2.6. Suppose that the scatterplot of X versus Y is strongly curved rather

than ellipsoidal. Should you use simple linear regression to predict Y from
X? Explain.

2.7. Suppose that the 95% condence interval for 2 is [17.457, 15.832].
In the simple linear regression model, is X a useful linear predictor for Y ? If
your answer is no, could X be a useful predictor for Y ? Explain.

2.8. Suppose it is desired to predict the yearly return from the stock
market from the return in January. Assume that the correlation = 0.496.
Using the table below, nd the least squares line Y = 1 + 2 X.

variable mean X or Y standard deviation s

January return 1.75 5.36
yearly return 9.07 15.35
2.13 Problems 71

2.9. Suppose that (Xi X)(Yi Y ) = 70690.0,

(Xi X) = 19800.0, X = 70.0, and Y = 312.28.
2

a) Find the least squares slope 2 .

b) Find the least squares intercept 1 .
c) Predict Y if X = 80.

xi yi xi x yi y (xi x)(yi y) (xi x)2

38 41

56 63

59 70

64 72

74 84

2.10. In the above table, xi is the length of the femur and yi is the length
of the humerus taken from ve dinosaur fossils (Archaeopteryx) that preserved
both bones. See Moore (2000, p. 99).
a) Complete the table and nd the least squares estimators 1 and 2 .

b) Predict the humerus length if the femur length is 60.

2.11. Suppose that the regression model is Yi = 7 + Xi + ei for i =

1, . . . , n where the ei are iid N (0, 2 ) random variables. The least squares
n
criterion is Q() = (Yi 7 Xi )2 .
i=1

a) What is E(Yi )?

b) Find the least squares estimator of by setting the rst derivative

d
Q() equal to zero.
d

c) Show that your is the global minimizer of the least squares criterion
d2
Q by showing that the second derivative Q() > 0 for all values of .
d 2
2.12. The location model is Yi = +ei for i = 1, . . . , n where the ei are iid
with mean E(ei ) = 0 and constant variance VAR(ei ) = 2 . The least squares
 n
estimator of minimizes the least squares criterion Q() = (Yi )2 .
i=1
To nd the least squares estimator, perform the following steps.
72 2 Multiple Linear Regression

d
a) Find the derivative Q, set the derivative equal to zero and solve for
d
. Call the solution .

b) To show that the solution was indeed the global minimizer of Q, show
d2
that Q > 0 for all real . (Then the solution is a local min and Q is
d 2
convex, so is the global min.)
2.13. The normal error model for simple linear regression through the
origin is
Yi = Xi + ei
for i = 1, . . . , n where e1 , . . . , en are iid N (0, 2 ) random variables.

a) Show that the least squares estimator for is

n
Xi Y i
= i=1
n 2
.
i=1 Xi

b) Find E().

c) Find VAR().
n
(Hint: Note that = i=1 ki Yi where the ki depend on the Xi which are
treated as constants.)
2.14. Suppose that the regression model is Yi = 10+2Xi2 +3 Xi3 +ei for
i = 1, . . . , n where the ei are iid N (0, 2 ) random variables. The least squares
n
criterion is Q(3 ) = (Yi 10 2Xi2 3 Xi3 )2 . Find the least squares es-
i=1
d
timator 3 of 3 by setting the rst derivative Q(3 ) equal to zero. Show
d3
that your 3 is the global minimizer of the least squares criterion Q by show-
d2
ing that the second derivative Q(3 ) > 0 for all values of 3 .
d32
Minitab Problems
Double click means press the rightmost mouse button twice in rapid
succession. Drag means hold the mouse button down. This technique is
used to select menu options.
After your computer is on, get into Minitab, often by searching programs
and then double clicking on the icon marked Student Minitab.
i) In a few seconds, the Minitab session and worksheet windows ll the screen.
At the top of the screen there is a menu. The upper left corner has the menu
option File. Move your cursor to File and drag down the option Open
Worksheet. A window will appear. Double click on the icon Student. This
will display a large number of data sets.
2.13 Problems 73

ii) In the middle of the screen there is a scroll bar, a gray line with left and
right arrow keys. Use the right arrow key to make the data le Prof.mtw
appear. Double click on Prof.mtw. A window will appear. Click on OK.
iii) The worksheet window will now be lled with data. The top of the screen
has a menu. Go to Stat and drag down Regression. Another window will
appear: drag down Regression (write this as Stat>Regression>Regression).
iv) A window will appear with variables to the left and the response variable
and predictors (explanatory variables) to the right. Double click on instru-
crs to make it the response. Double click on manner to make it the (pre-
dictor) explanatory variable. Then click on OK.
v) The required output will appear in the session window. You can view the
output by using the vertical scroll bar on the right of the screen.
vi) Copy and paste the output into Word, or to print your single page of
output, go to File, and drag down the option Print Session Window. A
window will appear. Click on ok. Then get your output from the printer.

Use the F3 key to clear entries from a dialog window if you make a mistake
or want a new plot.

To get out of Minitab, move your cursor to the x in the upper right
corner of the screen. When asked whether to save changes, click on no.

2.15. (Minitab problem.) See the above instructions for using Minitab.
Get the data set prof.mtw. Assign the response variable to be instrucr (the
instructor rating from course evaluations) and the explanatory variable (pre-
dictor) to be manner (the manner of the instructor). Run a regression on
these variables.

a) Place the computer output into Word.

b) Write the regression equation.

c) Predict instrucr if manner = 2.47.

d) To get residual and response plots you need to store the residuals and
tted values. Use the menu commands Stat>Regression>Regression to get
the regression window. Put instrucr in the Response and manner in the
Predictors boxes. The click on Storage. From the resulting window click
on Fits and Residuals. Then click on OK twice.
To get a response plot, use the commands Graph>Plot, (double click)
place instrucr in the Y box, and Fits1 in the X box. Then click on OK. Print
the plot by clicking on the graph and then clicking on the printer icon.
e) To make a residual plot, use the menu commands Graph>Plot to get
a window. Place Resi1 in the Y box and Fits1 in the X box. Then click
on OK. Print the plot by clicking on the graph and then clicking on the
printer icon.
74 2 Multiple Linear Regression

2.16. a) Enter the following data on the Minitab worksheet:

x y
30 73
20 50
60 128
80 170
40 87
50 108
60 135
30 60
70 148
60 132
To enter the data click on the C1 column header and enter x. Then click on
the C2 header and enter y. Then enter the data. Or copy the data from Prob-
lem 2.17 obtained from (http://lagrange.math.siu.edu/Olive/lreghw.
txt).
Then in Minitab, use the menu commands Edit>Paste Cells and click on
OK. Obtain the regression output from Minitab with the menu commands
Stat>Regression>Regression.
b) Place the output into Word.
c) Write down the least squares equation.
To save your output on your ash drive (J, say), use the Word menu
commands File > Save as. In the Save in box select Removable Disk
(J:), and in the File name box enter HW2d16.doc. To get a Word printout,
click on the printer icon or use the menu commands File>Print.
d) To get residual and response plots you need to store the residuals and
tted values. Use the menu commands Stat>Regression>Regression to get
the regression window. Put Y in the Response and X in the Predictors
boxes. The click on Storage. From the resulting window click on Fits and
Residuals. Then click on OK twice.
To make a response plot, use the menu commands Graph>Plot to get
a window. Place Y in the Y box and Fits1 in the X box. Then click on
OK. Print the plot by clicking on the graph and then clicking on the printer
icon.
e) To make a residual plot of the tted values versus the residuals, use
the menu commands Graph>Plot to get a window. Place Resi1 in the Y
box and Fits1 in the X box. Then click on OK. Print the plot by clicking
on the graph and then clicking on the printer icon.

f) To save your Minitab data on your ash drive, use the menu commands
File>Save Current Worksheet as. In the resulting dialog window, the top
box says Save in and there is an arrow icon to the right of the top box.
Click several times on the arrow icon until the Save in box reads My com-
2.13 Problems 75

puter, then click on Removable Disk (J:). In the File name box, enter
H2d16.mtw. Then click on OK.
SAS Problems
Copy and paste the SAS programs for problems 2.17 and 2.18
from (http://lagrange.math.siu.edu/Olive/lreghw.txt), or enter
the SAS program in Notepad or Word.
SAS is a statistical software package widely used in industry. You will need
a ash dive. Referring to the program for Problem 2.17, the semicolon ;
is used to end SAS commands and the options ls = 70; command makes
the output readable. (An * can be used to insert comments into the SAS
program. Try putting an * before the options command and see what it does
to the output.) The next step is to get the data into SAS. The command data
wcdata; gives the name wcdata to the data set. The command input x
y; says the rst entry is variable x and the 2nd variable y. The command
cards; means that the data is entered below. Then the data is entered
and the isolated semicolon indicates that the last case has been entered. The
command proc print; prints out the data. The command proc corr; will
give the correlation between x and y. The commands proc plot; plot y*x;
makes a scatterplot of x and y. The commands proc reg; model y=x; output
out = a p =pred r =resid; tells SAS to perform a simple linear regression
with y as the response variable. The output data set is called a and contains
the tted values and residuals. The command proc plot data = a; tells SAS
to make plots from data set a rather than data set wcdata. The command
plot resid*(pred x); will make a residual plot of the tted values versus the
residuals and a residual plot of x versus the residuals. The next plot command
makes a response plot.
To use SAS on windows (PC), use the following steps.
i) Get into SAS, often by double clicking on an icon for programs such as a
Math Progs icon and then double clicking on a SAS icon. If your computer
does not have SAS, go to another computer.

ii) A window should appear with 3 icons. Double click on The SAS System
for . . . .

iii) Like Minitab, a window with a split screen will open. The top screen
says Log-(Untitled) while the bottom screen says Editor-Untitled1. Press the
spacebar and an asterisk appears: Editor-Untitled1*.

2.17. a) Copy and paste the program for this problem from
(http://lagrange.math.siu.edu/Olive/lreghw.txt), or enter the SAS
program in Notepad or Word. The ls stands for linesize so l is a lowercase L,
not the number one.
When you are done entering the program, you may want to save the pro-
gram as h2d17.sas on your ash drive (J: drive, say). (On the top menu of
the editor, use the commands File > Save as. A window will appear. Use
76 2 Multiple Linear Regression

the upper right arrow to locate Removable Disk (J:) and then type the le
name in the bottom box. Click on OK.)
b) Get back into SAS, and from the top menu, use the File> Open
command. A window will open. Use the arrow in the upper right corner
of the window to navigate to Removable Disk (J:). (As you click on the
arrow, you should see My Documents, C: etc, then Removable Disk (J:).)
Double click on h2d17.sas. (Alternatively cut and paste the program into the
SAS editor window.) To execute the program, use the top menu commands
Run>Submit. An output window will appear if successful.

If you were not successful, look at the log window for hints on errors.
A single typo can cause failure. Reopen your le in Word or Notepad and
make corrections. Occasionally you cannot nd your error. Then nd your
instructor or wait a few hours and reenter the program.

c) To copy and paste relevant output into Word or Notepad, click on the
output window and use the top menu commands Edit>Select All and then
the menu commands Edit>Copy.
In Notepad use the commands Edit>Paste. Then use the mouse to high-
light the relevant output. Then use the commands Edit>Copy.
Finally, in Word, use the command Paste. You can also cut output from
Word and paste it into Notepad.

You may want to save your SAS output as the le HW2d17.doc on your
ash drive.

d) To save your output on your ash drive, use the Word menu commands
File > Save as. In the Save in box select Removable Disk (J:) and in
the File name box enter HW2d17.doc. To get a Word printout, click on the
printer icon or use the menu commands File>Print.
Save the output giving the least squares coecients in Word.
e) Predict Y if X = 40.
f) What is the residual when X = 40?

2.18. This problem shows how to use SAS for MLR. The data are from
Kutner et al. (2005, problem 6.5). The response is brand liking, a measure-
ment for whether the consumer liked the brand. The variable X1 is moisture
content and the variable X2 is sweetness. Copy and paste the program for
this problem from (http://lagrange.math.siu.edu/Olive/lreghw.txt).
a) Execute the SAS program and copy the output le into Notepad. Scroll
down the output that is now in Notepad until you nd the regression coe-
cients and ANOVA table. Then cut and paste this output into Word.
b) Do the 4 step ANOVA F test.
You should scroll through your SAS output to see how it made the re-
sponse plot and various residual plots, but cutting and pasting these plots is
2.13 Problems 77

tedious. So we will use Minitab to get these plots. Find the program for this
problem from (http://lagrange.math.siu.edu/Olive/lreghw.txt). Then
copy and paste the numbers (between cards; and the semicolon ;) into
Minitab. Use the mouse commands Edit>Paste Cells. This should enter
the data in the Worksheet (bottom part of Minitab). Under C1 enter Y and
under C2 enter X1 under C3 enter X2. Use the menu commands
Stat>Regression>Regression to get a dialog window. Enter Y as the re-
sponse variable and X1 and X2 as the predictor variable. Click on Storage
then on Fits, Residuals, and OK OK.
c) To make a response plot, enter the menu commands Graph>Plot
and place Y in the Ybox and FITS1 in the Xbox. Click on OK. Then
use the commands Edit>Copy Graph to copy the plot. Include the plot in
Word with the commands Edit> Paste. If these commands fail, click on
the graph and then click on the printer icon.
d) Based on the response plot, does a linear model seem reasonable?
e) To make a residual plot, enter the menu commands Graph>Plot and
place RESI 1 in the Ybox and FITS1 in the Xbox. Click on OK. Then
use the commands Edit>Copy Graph to copy the plot. Include the plot in
Word with the commands Edit> Paste. If these commands fail, click on
the graph and then click on the printer icon.
f) Based on the residual plot does a linear model seem reasonable?
Problems using ARC

To quit Arc, move the cursor to the x in the upper right corner and click.
Warning: Some of the following problems uses data from the
books webpage (http://lagrange.math.siu.edu/Olive/lregbk.htm).
Save the data les on a ash drive G, say. Get in Arc and use the menu
commands File > Load and a window with a Look in box will appear. Click
on the black triangle and then on Removable Disk (G:). Then click twice on
the data set name.

2.19 . (Scatterplot in Arc.) Get cbrain.lsp as described above. (Activate

the cbrain.lsp dataset with the menu commands File > Load > Removable
Disk (G:) > cbrain.lsp.) Scroll up the screen to read the data description.

a) Make a plot of age versus brain weight brnweight. The commands

Graph&Fit > Plot of will bring down a menu. Put age in the H box
and brnweight in the V box. Put sex in the Mark by box. Click OK. Make
the lowess bar on the plot read .1. Open Word.

In Arc, use the menu commands Edit > Copy. In Word, use the menu
command Paste. This should copy the graph into the Word document.

b) For a given age, which gender tends to have larger brains?

c) At what age does the brain weight appear to be decreasing?

78 2 Multiple Linear Regression

2.20. (SLR in Arc.) Activate cbrain.lsp as in Problem 2.19. Brain weight

and the cube root of size should be linearly related. To add the cube root of
size to the data set, use the menu commands cbrain > Transform. From
the window, select size and enter 1/3 in the p: box. Then click OK. Get some
output with commands Graph&Fit > Fit linear LS. In the dialog window,
put brnweight in Response, and (size)1/3 in terms.

a) Cut and paste the output (from Coecient Estimates to Sigma hat)
into Word. Write down the least squares equation Y = 1 + 2 x.

b) If (size)1/3 = 15, what is the estimated brnweight?

c) Make a residual plot of the tted values versus the residuals. Use
the commands Graph&Fit > Plot of and put L1:Fit-values in H and
L1:Residuals in V. Put sex in the Mark by box. Move the OLS bar to 1.
Put the plot into Word. Does the plot look ellipsoidal with zero mean?

d) Make a response plot of the tted values versus Y = brnweight. Use

the commands Graph&Fit > Plot of and put L1:Fit-values in H and
brnweight in V. Put sex in Mark by. Move the OLS bar to 1. Put the plot
into Word. Does the plot look linear?

2.21. In Arc enter the menu commands File>Load>Data and open the
le mussels.lsp. This data set is from Cook and Weisberg (1999a).
The response variable Y is the mussel muscle mass M, and the explanatory
variables are X2 = S = shell mass, X3 = H = shell height, X4 = L = shell
length, and X5 = W = shell width.
Enter the menu commands Graph&Fit>Fit linear LS and t the model:
enter S, H, L, W in the Terms/Predictors box, M in the Response box
and click on OK.

a) To get a response plot, enter the menu commands

Graph&Fit>Plot of and place L1:Fit-Values in the Hbox and M in the
Vbox. Copy the plot into Word.

b) Based on the response plot, does a linear model seem reasonable?

c) To get a residual plot, enter the menu commands Graph&Fit>Plot of

and place L1:Fit-Values in the Hbox and L1:Residuals in the Vbox. Copy
the plot into Word.

d) Based on the residual plot, what MLR assumption seems to be violated?

e) Include the regression output in Word.

2.13 Problems 79

f) Ignoring the fact that an important MLR assumption seems to have

been violated, do any of predictors seem to be needed given that the other
predictors are in the model?

g) Ignoring the fact that an important MLR assumption seems to have

been violated, perform the ANOVA F test.

2.22. Get cyp.lsp as described above Problem 2.19. You can open the
le in Notepad and then save it on a ash drive G, say, using the Notepad
menu commands File>Save As and clicking the top checklist then click
Removable Disk (G:). You could also save the le on the desktop, load it
in Arc from the desktop, and then delete the le (sending it to the Recycle
Bin).
a) In Arc enter the menu commands File>Load>Removable Disk (G:)
and open the le cyp.lsp. This data set consists of various measurements
taken on men from Cyprus around 1920. Let the response Y = height and
X = cephalic index = 100(head breadth)/(head length). Use Arc to get the
least squares output and include the relevant output in Word.

b) Intuitively, the cephalic index should not be a good predictor for a

persons height. Perform a 4 step test of hypotheses with Ho: 2 = 0.

2.23. a) In Arc open the le cyp.lsp (obtained as in Problem 2.22).

The response variable Y is height, and the explanatory variables are a
constant, X2 = sternal height (probably height at shoulder), and X3 = nger
to ground.
Enter the menu commands Graph&Fit>Fit linear LS and t the model:
enter sternal height and nger to ground in the Terms/Predictors box,
height in the Response box and click on OK.
Include the output in Word. Your output should certainly include the lines
from Response = height to the ANOVA table.

b) Predict Y if X2 = 1400 and X3 = 650.

c) Perform a 4 step ANOVA F test of the hypotheses with

Ho: 2 = 3 = 0.

d) Find a 99% CI for 2 .

e) Find a 99% CI for 3 .

f) Perform a 4 step test for 2 = 0.

g) Perform a 4 step test for 3 = 0.

h) What happens to the conclusion in g) if = 0.01?

80 2 Multiple Linear Regression

i) The Arc menu L1 should have been created for the regression. Use
the menu commands L1>Prediction to open a dialog window. Enter 1400
650 in the box and click on OK. Include the resulting output in Word.

j) Let Xf,2 = 1400 and Xf,3 = 650 and use the output from i) to nd a
95% CI for E(Yf ). Use the last line of the output, that is, se = S(Yf ).

k) Use the output from i) to nd a 95% PI for Yf . Now se(pred) = s(pred).

2.24. In Arc enter the menu commands File>Load>Removable Disk

(G:) and open the le cyp.lsp (obtained as in Problem 2.22).
The response variable Y is height, and the explanatory variables are
X2 = sternal height (probably height at shoulder), and X3 = nger to ground.

Enter the menu commands Graph&Fit>Fit linear LS and t the model:

enter sternal height and nger to ground in the Terms/Predictors box,
height in the Response box and click on OK.

a) To get a response plot, enter the menu commands

Graph&Fit>Plot of and place L1:Fit-Values in the Hbox and height in
the Vbox. Copy the plot into Word.

b) Based on the response plot, does a linear model seem reasonable?

c) To get a residual plot, enter the menu commands Graph&Fit>Plot of

and place L1:Fit-Values in the Hbox and L1:Residuals in the Vbox. Copy
the plot into Word.

d) Based on the residual plot, does a linear model seem reasonable?

2.25. In Arc enter the menu commands File>Load>Removable Disk

(G:) and open the le cyp.lsp (obtained as in Problem 2.22).
The response variable Y is height, and the explanatory variables are X2 =
sternal height, X3 = nger to ground, X4 = bigonal breadth, X5 = cephalic
index, X6 = head length, and X7 = nasal height. Enter the menu commands
Graph&Fit>Fit linear LS and t the model: enter the 6 predictors (in
order: X2 1st and X7 last) in the Terms/Predictors box, height in the
Response box and click on OK. This gives the full model. For the reduced
model, only use predictors 2 and 3.

a) Include the ANOVA tables for the full and reduced models in Word.

b) Use the menu commands Graph&Fit>Plot of. . . to get a dialog win-

dow. Place L2:Fit-Values in the Hbox and L1:Fit-Values in the Vbox. Place
the resulting plot in Word.
2.13 Problems 81

c) Use the menu commands Graph&Fit>Plot of. . . to get a dialog win-

dow. Place L2:Residuals in the Hbox and L1:Residuals in the Vbox. Place
the resulting plot in Word.

d) Both plots should cluster tightly about the identity line if the reduced
model is about as good as the full model. Is the reduced model good?

e) Perform the 4 step partial F test (of Ho: the reduced model is good)
using the 2 ANOVA tables from part a).
2.26. a) Activate the cbrain.lsp data set in ARC. Fit least squares with
age, sex, size1/3 , and headht as terms and brnweight as the response. See
Problem 2.20. Assume that the multiple linear regression model is appro-
priate. (This may be a reasonable assumption, 5 infants appear as outliers
but the data set has hardly any cases that are babies. If age was uniformly
represented, the babies might not be outliers anymore.) Assuming that ARC
makes the menu L1 for this regression, select AVP-All 2D. A window will
appear. Move the OLS slider bar to 1 and click on the zero line box. The
window will show the added variable plots for age, sex, size1/3 , and headht
as you move along the slider bar that is below case deletions. Include all 4
added variable plots in Word.
b) What information do the 4 plots give? For example, which variables do
not seem to be needed?
(If it is clear that the zero and OLS lines intersect at the origin, then the
variable is probably needed, and the point cloud should be tilted away from
the zero line. If it is dicult to see where the two lines intersect since they
nearly coincide near the origin, then the variable may not be needed, and the
point cloud may not tilt away from the zero line.)
R Problems

Use the command source(G:/lregpack.txt) to download the func-

tions and the command source(G:/lregdata.txt) to download the data.
See Preface or Section 14.1. Typing the name of the lregpack function,
e.g. piplot, will display the code for the function. Use the args command,
e.g. args(pisim), to display the needed arguments for the function. For some
of the following problems, the R commands can be copied and pasted from
(http://lagrange.math.siu.edu/Olive/lreghw.txt) into R.
2.27. a) Download the data into R as described above.
For the Buxton (1920) data suppose that the response Y = height and the
predictors were a constant, head length, nasal height, bigonal breadth, and
cephalic index. There are 87 cases.
Type the following commands
zbux <- cbind(buxx,buxy)
zbux <- as.data.frame(zbux)
zfull <- lm(buxy~len+nasal+bigonal+cephalic,data=zbux)
82 2 Multiple Linear Regression

zred <- lm(buxy~len+nasal,data=zbux)

anova(zred,zfull)
b) Include the output in Word: press the Ctrl and c keys as the same
time. Then use the menu commands Paste in Word (or copy and paste the
output: hit the Ctrl and v keys at the same time).
c) Use the output to perform the partial F test where the full model is
described in a) and the reduced model uses a constant, head length, and
nasal height. The output from the anova(zred,zfull) command produces
the correct partial F statistic.
d) Use the following commands to make the response plot for the reduced
model. Include the plot in Word.

plot(zred$fit,buxy)
abline(0,1)

e) Use the following command to make the residual plot for the reduced
model. Include the plot in Word.

plot(zred$fit,zred$resid)

f) The plots look bad because of 5 massive outliers. The following com-
mands remove the outliers. Include the output in Word.

zbux <- zbux[-c(60,61,62,63,64,65),]

zfull <- lm(buxy~len+nasal+bigonal+cephalic,data=zbux)
zred <- lm(buxy~len+nasal,data=zbux)
anova(zred,zfull)

g) Redo the partial F test.

h) Use the following commands to make the response plot for the reduced
model without the outliers. Include the plot in Word.

plot(zred$fit,zbux[,5])
abline(0,1)

i) Use the following command to make the residual plot for the reduced
model without the outliers. Include the plot in Word.

plot(zred$fit,zred$resid)

j) Do the plots look ok?

2.28. Get the R commands for this problem. The data is such that Y =
2 + x2 + x3 + x4 + e where the zero mean errors are iid [exponential(2) -
2]. Hence the residual and response plots should show high skew. Note that
= (2, 1, 1, 1)T . The R code uses 3 nontrivial predictors and a constant, and
the sample size n = 1000.
2.13 Problems 83

a) Copy and paste the commands for part a) of this problem into R. Include
the response plot in Word. Is the lowess curve fairly close to the identity line?
b) Copy and paste the commands for part b) of this problem into R.
Include the residual plot in Word: press the Ctrl and c keys as the same time.
Then use the menu command Paste in Word. Is the lowess curve fairly
close to the r = 0 line?
c) The output out$coef gives . Write down . Is close to ?

2.29. a) Download the R functions piplot and pisim from lregpack.

b) The command pisim(n=100, type = 1) will produce the mean length

of the classical, semiparametric, asymptotically conservative, and asymptot-
ically optimal PIs when the errors are normal, as well as the coverage pro-
portions. Give the simulated lengths and coverages. Note: pimenlen gives the
four lengths (classical, semi, ac, aopt). Make table with headers classical,
semi, ac, and aopt.

c) Repeat b) using the command pisim(n=100, type = 3). Now the er-
rors are EXP(1) - 1.

d) Download lregdata.txt and type the command

piplot(cbrainx,cbrainy). This command gives the semiparametric PI lim-
its for the Gladstone data. Include the plot in Word.

e) The infants are in the lower left corner of the plot. Do the PIs seem to
be better for the infants or the bulk of the data. Explain briey.

2.30. Use the function MLRsim as described in Rule of Thumb 2.1 to

generate 10 pairs of response and residual plots. Right click Stop twenty
times, and include the last plot in Word.
Chapter 3
Building an MLR Model

Building a multiple linear regression (MLR) model from data is one of the
most challenging regression problems. The nal full model will have re-
sponse variable Y = t(Z), a constant x1 , and predictor variables x2 =
t2 (w2 , . . . , wr ), . . . , xp = tp (w2 , . . . , wr ) where the initial data consists of
Z, w2 , . . . , wr . Choosing t, t2 , . . . , tp so that the nal full model is a useful
MLR approximation to the data can be dicult.

Model building is an iterative process. Given the problem and data but
no model, the model building process can often be aided by graphs that
help visualize the relationships between the dierent variables in the data.
Then a statistical model can be proposed. This model can be t and inference
performed. Then diagnostics from the t can be used to check the assumptions
of the model. If the assumptions are not met, then an alternative model can
be selected. The t from the new model is obtained, and the cycle is repeated.
This chapter provides some tools for building a good full model.

Warning: Researchers often have a single data set and tend to expect
statistics to provide far more information from the single data set than is
reasonable. MLR is an extremely useful tool, but MLR is at its best when the
nal full model is known before collecting and examining the data. However,
it is very common for researchers to build their nal full model by using
the iterative process until the nal model ts the data well. Researchers
should not expect that all or even many of their research questions can be
answered from such a full model. If the nal MLR full model is built from
a single data set in order to t that data set well, then typically inference
from that model will not be valid. The model may be useful for describing
the data, but may perform very poorly for prediction of a future response.
The model may suggest that some predictors are much more important than
others, but a model that is chosen prior to collecting and examining the data
is generally much more useful for prediction and inference. A single data

Springer International Publishing AG 2017 85

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 3
86 3 Building an MLR Model

set is a great place to start an analysis, but can be a terrible way

to end the analysis.

Often a nal full model is built after collecting and examining the data.
This procedure is called data snooping, and such models cannot be ex-
pected to be reliable. If possible, spend about 1/8 of the budget to collect
data and build an initial MLR model. Spend another 1/8 of the budget to
collect more data to check the initial MLR model. If changes are necessary,
continue this process until no changes from the previous step are needed,
resulting in a tentative MLR model. Then spend between 1/2 and 3/4 of the
budget to collect data assuming that the tentative model will be useful.
Alternatively, if the data set is large enough, use a training set of a
random sample of k of the n cases to build a model where 10p n/2 k
0.9n. Then use validation set of the other n k cases to conrm that the
model built with the training set is good. This technique may help reduce
biases, but needs n 20p.

After obtaining a nal full model, researchers will typically nd a nal

submodel after performing variable selection. Even if the nal full model was
selected before collecting data, the nal submodel, obtained after performing
variable selection, may be hard to use.

Rule of thumb 3.1. If the MLR model is built using the variable se-
lection methods from Section 3.4, then the nal submodel can be used for
description. If the full model was found after collecting the data, the model
may not be useful for inference and prediction. If the full model was selected
before collecting the data, then the prediction region method of bootstrap-
ping the variable selection model, described in Section 3.4.1, may be useful.

The remainder of this chapter considers interactions, predictor transfor-

mations, variable selection, and diagnostics. These techniques are useful for a
wide variety of regression models, including those covered in Chapters 12 and
13. This chapter also gives a graphical method for response transformations
which can be extended to additive error regression models.

3.1 Predictor Transformations

As a general rule, inferring about the distribution of Y |X from a lower

dimensional plot should be avoided when there are strong nonlinearities
among the predictors.
Cook and Weisberg (1999b, p. 34)
Predictor transformations are used to remove gross nonlinearities in the
predictors, and this technique is often very useful. Power transformations are
particularly eective, and the techniques of this section are often useful for
3.1 Predictor Transformations 87

general regression problems, not just for multiple linear regression. A power
transformation has the form x = t (w) = w for = 0 and x = t0 (w) =
log(w) for = 0. Often L where

L = {1, 1/2, 1/3, 0, 1/3, 1/2, 1} (3.1)

is called the ladder of powers. Often when a power transformation is needed,

a transformation that goes down the ladder, e.g. from = 1 to = 0 will
be useful. If the transformation goes too far down the ladder, e.g. if = 0
is selected when = 1/2 is needed, then it will be necessary to go back up
the ladder. Additional powers such as 2 and 3 can always be added.
Denition 3.1. A scatterplot of x versus Y is used to visualize the con-
ditional distribution of Y |x. A scatterplot matrix is an array of scatter-
plots. It is used to examine the marginal relationships of the predictors and
response.
In this section we will only make a scatterplot matrix of the predictors.
Often nine or ten variables can be placed in a scatterplot matrix. The names
of the variables appear on the diagonal of the scatterplot matrix. The software
Arc gives two numbers, the minimum and maximum of the variable, along
with the name of the variable. The R software labels the values of each
variable in two places, see Example 3.2 below. Let one of the variables be W .
All of the marginal plots above and below W have W on the horizontal axis.
All of the marginal plots to the left and the right of W have W on the vertical
axis.

There are several rules of thumb that are useful for visually selecting a
power transformation to remove nonlinearities from the predictors. Let a
plot of X1 versus X2 have X2 is on the vertical axis and X1 on the horizontal
axis.
Rule of thumb 3.2. a) If strong nonlinearities are apparent in the scat-
terplot matrix of the predictors w2 , . . . , wp , it is often useful to remove the
nonlinearities by transforming the predictors using power transformations.

b) Use theory if available.

c) Suppose the plot of X1 versus X2 is nonlinear. The unit rule says that
if X1 and X2 have the same units, then try the same transformation for both
X1 and X2 .

Assume that all values of X1 and X2 are positive. Then the following six
rules are often used.

d) The log rule states that a positive predictor that has the ratio between
the largest and smallest values greater than ten should be transformed to logs.
So X > 0 and max(X)/ min(X) > 10 suggests using log(X).
88 3 Building an MLR Model

e) The range rule states that a positive predictor that has the ratio be-
tween the largest and smallest values less than two should not be transformed.
So X > 0 and max(X)/ min(X) < 2 suggests keeping X.

f) The bulging rule states that changes to the power of X2 and the power
of X1 can be determined by the direction that the bulging side of the curve
points. If the curve is hollow up (the bulge points down), decrease the power
of X2 . If the curve is hollow down (the bulge points up), increase the power
of X2 . If the curve bulges towards large values of X1 increase the power of
X1 . If the curve bulges towards small values of X1 decrease the power of X1 .
See Tukey (1977, pp. 173176).

g) The ladder rule appears in (Cook and Weisberg 1999a, p. 86).

To spread small values of a variable, make smaller.
To spread large values of a variable, make larger.

h) If it is known that X2 X1 and the ranges of X1 and X2 are such that

this relationship is one to one, then
1/
X1 X2 and X2 X1 .
1/
Hence either the transformation X1 or X2 will linearize the plot. Note
that log(X2 ) log(X1 ), so taking logs of both variables will also linearize
the plot. This relationship frequently occurs if there is a volume present. For
example, let X2 be the volume of a sphere and let X1 be the circumference
of a sphere.

i) The cube root rule says that if X is a volume measurement, then cube
root transformation X 1/3 may be useful.
In the literature, it is sometimes stated that predictor transformations
that are made without looking at the response are free. The reasoning is
that the conditional distribution of Y |(x2 = a2 , . . . , xp = ap ) is the same
as the conditional distribution of Y |[t2 (x2 ) = t2 (a2 ), . . . , tp (xp ) = tp (ap )]:
is simply a change of labelling. Certainly if Y |x = 9 N (0, 1), then
there
Y | x = 3 N (0, 1). To see that Rule of thumb 3.2a does not always work,
suppose that Y = 1 +2 x2 + +p xp +e where the xi are iid lognormal(0,1)
random variables. Then wi = log(xi ) N (0, 1) for i = 2, . . . , p and the
scatterplot matrix of the wi will be linear while the scatterplot matrix of the
xi will show strong nonlinearities if the sample size is large. However, there is
an MLR relationship between Y and the xi while the relationship between Y
and the wi is nonlinear: Y = 1 + 2 ew2 + + p ewp + e = T w + e. Given
Y and the wi with no information of the relationship, it would be dicult to
nd the exponential transformation and to estimate the i . The moral is that
predictor transformations, especially the log transformation, can and often
do greatly simplify the MLR analysis, but predictor transformations can turn
a simple MLR analysis into a very complex nonlinear analysis.
3.1 Predictor Transformations 89

Theory, if available, should be used to select a transformation. Frequently

more than one transformation will work. For example, if W = weight and X1
1/3
= volume = (X2 )(X3 )(X4 ), then W versus X1 and log(W ) versus log(X1 ) =
log(X2 )+log(X3 )+log(X4 ) may both work. Also if W is linearly related with
X2 , X3 , X4 and these three variables all have length units mm, say, then the
1/3
units of X1 are (mm)3 . Hence the units of X1 are mm.
Suppose that all values of the variable w to be transformed are positive.
The log rule says use log(w) if max(wi )/ min(wi ) > 10. This rule often works
wonders on the data and the log transformation is the most used (modied)
power transformation. If the variable w can take on the value of 0, use
log(w + c) where c is a small constant like 1, 1/2, or 3/8.
To use the ladder rule, suppose you have a scatterplot of two variables
x1 1 versus x2 2 where both x1 > 0 and x2 > 0. Also assume that the plotted
points follow a nonlinear one to one function. Consider the ladder of powers

L = {1, 1/2, 1/3, 0, 1/3, 1/2, 1}.

To spread small values of the variable, make i smaller. To spread large values
of the variable, make i larger.
For example, if both variables are right skewed, then there will be many
more cases in the lower left of the plot than in the upper right. Hence small
values of both variables need spreading. Figures 13.3 b) and 13.16 have this
shape.

Consider the ladder of powers. Often no transformation ( = 1) is best,

then the log transformation, then the square root transformation, then the
reciprocal transformation.

Example 3.1. Examine Figure 3.1. Let X1 = w and X2 = x. Since w is on

the horizontal axis, mentally add a narrow vertical slice to the plot. If a large
amount of data falls in the slice at the left of the plot, then small values need
spreading. Similarly, if a large amount of data falls in the slice at the right of
the plot (compared to the middle and left of the plot), then large values need
spreading. For the variable on the vertical axis, make a narrow horizontal
slice. If the plot looks roughly like the northwest corner of a square, then
small values of the horizontal and large values of the vertical variable need
spreading. Hence in Figure 3.1a, small values of w need spreading. Notice that
the plotted points bulge up towards small values of the horizontal variable.
If the plot looks roughly like the northeast corner of a square, then large
values of both variables need spreading. Hence in Figure 3.1b, large values
of x need spreading. Notice that the plotted points bulge up towards large
values of the horizontal variable. If the plot looks roughly like the southwest
corner of a square, as in Figure 3.1c, then small values of both variables
need spreading. Notice that the plotted points bulge down towards small
values of the horizontal variable. If the plot looks roughly like the southeast
corner of a square, then large values of the horizontal and small values of the
90 3 Building an MLR Model

a) b)

1.4

1 2 3 4 5 6 7
1.0
x

x
0.6
0.2

0.0 0.5 1.0 1.5 2.0 2.5 0.0 0.5 1.0 1.5 2.0 2.5
w w
c) d)

6 8 10 12
60
40
x

x
20

2 4
0

0.0 0.5 1.0 1.5 2.0 2.5 0.0 0.5 1.0 1.5 2.0 2.5
w w

Fig. 3.1 Plots to Illustrate the Bulging and Ladder Rules

vertical variable need spreading. Hence in Figure 3.1d, small values of x need
spreading. Notice that the plotted points bulge down towards large values of
the horizontal variable.

Example 3.2: Mussel Data. Cook and Weisberg (1999a, pp. 351,
433, 447) gave a data set on 82 mussels sampled o the coast of New
Zealand. The response is muscle mass M in grams, and the predictors are
a constant, the length L and height H of the shell in mm, the shell width
W , and the shell mass S. Figure 3.2 shows the scatterplot matrix of the
predictors L, W , H, and S. Examine the variable length. Length is on the
vertical axis on the three top plots and the right of the scatterplot matrix
labels this axis from 150 to 300. Length is on the horizontal axis on the
three leftmost marginal plots, and this axis is labelled from 150 to 300 on the
bottom of the scatterplot matrix. The marginal plot in the bottom left corner
has length on the horizontal and shell on the vertical axis. The marginal
plot that is second from the top and second from the right has height on the
horizontal and width on the vertical axis. If the data is stored in x, the plot
can be made with the following command in R.

pairs(x,labels=c("length","width","height","shell"))
3.1 Predictor Transformations 91

Nonlinearity is present in several of the plots. For example, width and

length seem to be linearly related while length and shell have a nonlin-
earrelationship. The minimum value of shell is 10 while the max is 350. Since

20 30 40 50 60 0 100 200 300

150 200 250 300

length
20 30 40 50 60

width

140
height

80 100
100 200 300

shell
0

150 200 250 300 80 100 120 140 160

Fig. 3.2 Scatterplot Matrix for Original Mussel Data Predictors

350/10 = 35 > 10, the log rule suggests that log S may be useful. If log S
replaces S in the scatterplot matrix, then there may be some nonlinearity
present in the plot of log S versus W with small values of W needing spread-
ing. Hence the ladder rule suggests reducing from 1 and we tried log(W ).
Figure 3.3 shows that taking the log transformations of W and S results in
a scatterplot matrix that is much more linear than the scatterplot matrix of
Figure 3.2. Notice that the plot of W versus L and the plot of log(W ) versus
L both appear linear. This plot can be made with the following commands.
z <- x; z[,2] <- log(z[,2]); z[,4] <- log(z[,4])
pairs(z,labels=c("length","Log W","height","Log S"))
The plot of shell versus height in Figure 3.2 is nonlinear, and small values
of shell need spreading since if the plotted points were projected on the
horizontal axis, there would be too many points at values of shell near 0.
Similarly, large values of height need spreading.
92 3 Building an MLR Model

3.0 3.4 3.8 4.2 3 4 5 6

150 200 250 300

length
4.2
3.8

Log W
3.4
3.0

140
height

80 100
6
5

Log S
4
3

150 200 250 300 80 100 120 140 160

Fig. 3.3 Scatterplot Matrix for Transformed Mussel Data Predictors

3.2 Graphical Methods for Response Transformations

If the ratio of largest to smallest value of y is substantial, we usually begin

by looking at log y.
Mosteller and Tukey (1977, p. 91)
The applicability of the multiple linear regression model can be expanded
by allowing response transformations. An important class of response trans-
formation models adds an additional unknown transformation parameter o ,
such that
(o )
Yi = to (Zi ) Zi = E(Yi |xi ) + ei = xTi + ei . (3.2)

If o was known, then Yi = to (Zi ) would follow a multiple linear regression

model with p predictors including the constant. Here, is a p 1 vector
of unknown coecients depending on o , x is a p 1 vector of predictors
that are assumed to be measured with negligible error, and the errors ei are
assumed to be iid with zero mean.
Denition 3.2. Assume that all of the values of the response Zi are
positive. A power transformation has the form Y = t (Z) = Z for = 0
and Y = t0 (Z) = log(Z) for = 0 where
L = {1, 1/2, 1/3, 0, 1/3, 1/2, 1}.
3.2 Graphical Methods for Response Transformations 93

Denition 3.3. Assume that all of the values of the response Zi are
positive. Then the modied power transformation family

() Zi 1
t (Zi ) Zi = (3.3)

(0)
for = 0 and Zi = log(Zi ). Generally where is some interval such
as [1, 1] or a coarse subset such as L . This family is a special case of the
response transformations considered by Tukey (1957).

A graphical method for response transformations rets the model using

the same tting method: changing only the response from Z to t (Z).
Compute the tted values Wi using Wi = t (Zi ) as the response. Then
a transformation plot of Wi versus Wi is made for each of the seven values of
L with the identity line added as a visual aid. Vertical deviations from
the identity line are the residuals ri = Wi Wi . Then a candidate response
transformation Y = t (Z) is reasonable if the plotted points follow the iden-
tity line in a roughly evenly populated band if the unimodal MLR model is
reasonable for Y = W and x. See Denition 2.6. Curvature from the identity
line suggests that the candidate response transformation is inappropriate.

By adding the response Z to the scatterplot matrix, the methods of

the previous section can also be used to suggest good values of , and it is
usually a good idea to use predictor transformations to remove nonlinearities
from the predictors before selecting a response transformation. Check that
the scatterplot matrix with the transformed variables is better than the scat-
terplot matrix of the original variables. Notice that the graphical method is
equivalent to making response plots for the seven values of W = t (Z),
and choosing the best response plot where the MLR model seems most
reasonable. The seven response plots are called transformation plots be-
low. Our convention is that a plot of X versus Y means that X is on the
horizontal axis and Y is on the vertical axis.

Warning: The Rule of thumb 3.2 does not always work. For example, the
log rule may fail. If the relationships in the scatterplot matrix are already lin-
ear or if taking the transformation does not increase the linearity (especially
in the row containing the response), then no transformation may be better
than taking a transformation. For the Arc data set evaporat.lsp, the log
rule suggests transforming the response variable Evap, but no transformation
works better.

Denition 3.4. A transformation plot is a plot of W versus W with the

identity line added as a visual aid.

There are several reasons to use a coarse grid of powers. First, several of the
powers correspond to simple transformations such as the log, square root, and
cube root. These powers are easier to interpret than = 0.28, for example.
94 3 Building an MLR Model

a) lambda = 1 b) lambda = 0.5

sqrt(Z)
2000

40
Z

10
0
500 1000 10 30 50
TZHAT TZHAT

c) lambda = 0 d) lambda = 1

0.008
log(Z)
7

1/Z

0.000
5

5 6 7 8 0.002 0.002 0.006

TZHAT TZHAT

Fig. 3.4 Four Transformation Plots for the Textile Data

According to Mosteller and Tukey (1977, p. 91), the most commonly used
power transformations are the = 0 (log), = 1/2, = 1, and = 1/3
transformations in decreasing frequency of use. Secondly, if the estimator n
can only take values in L , then sometimes n will converge (e.g., in prob-
ability) to L . Thirdly, Tukey (1957) showed that neighboring power
transformations are often very similar, so restricting the possible powers to
a coarse grid is reasonable. Note that powers can always be added to the
grid L . Useful powers are 1/4, 2/3, 2, and 3. Powers from numerical
methods can also be added.

Application 3.1. This graphical method for selecting a response trans-

formation is very simple. Let Wi = t (Zi ). Then for each of the seven values
of L , perform OLS on (Wi , xi ) and make the transformation plot of
Wi versus Wi . If the plotted points follow the identity line for , then take
o = , that is, Y = t (Z) is the response transformation. (Note that this
procedure can be modied to create a graphical diagnostic for a numerical
estimator of o by adding to L . OLS can be replaced by other methods.)

If more than one value of L gives a linear plot, take the simplest or
most reasonable transformation or the transformation that makes the most
sense to subject matter experts. Also check that the corresponding residual
plots of W versus W W look reasonable. The values of in decreasing order
of importance are 1, 0, 1/2, 1, and 1/3. So the log transformation would be
chosen over the cube root transformation if both transformation plots look
equally good.
3.2 Graphical Methods for Response Transformations 95

After selecting the transformation, the usual checks should be made. In

particular, the transformation plot for the selected transformation is the re-
sponse plot, and a residual plot should also be made. The following example
illustrates the procedure, and the plots show W = t (Z) on the vertical axis.
The label TZHAT of the horizontal axis are the tted values W that
result from using W = t (Z) as the response in the OLS software.
Example 3.3: Textile Data In their pioneering paper on response trans-
formations, Box and Cox (1964) analyze data from a 33 experiment on the
behavior of worsted yarn under cycles of repeated loadings. The response
Z is the number of cycles to failure and a constant is used along with the
three predictors length, amplitude, and load. Using the normal prole log
likelihood for o , Box and Cox determine o = 0.06 with approximate 95
percent condence interval 0.18 to 0.06. These results give a strong indi-
cation that the log transformation may result in a relatively simple model,
as argued by Box and Cox. Nevertheless, the numerical BoxCox transfor-
mation method provides no direct way of judging the transformation against
the data.
Shown in Figure 3.4 are transformation plots of W versus W = Z for
four values of except log(Z) is used if = 0. The plots show how the trans-
formations bend the data to achieve a homoscedastic linear trend. Perhaps
more importantly, they indicate that the information on the transformation
is spread throughout the data in the plot since changing causes all points
along the curvilinear scatter in Figure 3.4a to form along a linear scatter in
Figure 3.4c. Dynamic plotting using as a control seems quite eective for
judging transformations against the data and the log response transformation
does indeed seem reasonable.
Note the simplicity of the method: Figure 3.4a shows that a response trans-
formation is needed since the plotted points follow a nonlinear curve while
Figure 3.4c suggests that Y = log(Z) is the appropriate response transforma-
tion since the plotted points follow the identity line. If all 7 plots were made
for L , then = 0 would be selected since this plot is linear. Also, Fig-
ure 3.4a suggests that the log rule is reasonable since max(Z)/ min(Z) > 10.

The essential point of the next example is that observations that inuence
the choice of the usual BoxCox numerical power transformation are often
easily identied in the transformation plots. The transformation plots are
especially useful if the bivariate relationships of the predictors, as seen in the
scatterplot matrix of the predictors, are linear.
Example 3.4: Mussel Data. Consider the mussel data of Example 3.2
where the response is muscle mass M in grams, and the predictors are the
length L and height H of the shell in mm, the logarithm log W of the shell
width W, the logarithm log S of the shell mass S, and a constant. With this
starting point, we might expect a log transformation of M to be needed
96 3 Building an MLR Model

a) lambda = 1 b) lambda = 0

50

4
log(Z)
2
Z
20
8
48

0
0
10 10 30 1.0 2.0 3.0 4.0
TZHAT TZHAT

c) lambda = 0.28 d) lambda = 1

3.0

48
Z**(0.28)

0.6
1/Z
2.0

8
8
1.0

0.0
48
1.5 2.0 2.5 3.0 0.00 0.15 0.30
TZHAT TZHAT
Fig. 3.5 Transformation Plots for the Mussel Data

because M and S are both mass measurements and log S is being used as
a predictor. Using log M would essentially reduce all measurements to the
scale of length. The BoxCox likelihood method gave 0 = 0.28 with ap-
proximate 95 percent condence interval 0.15 to 0.4. The log transformation
is excluded under this inference leading to the possibility of using dierent
transformations of the two mass measurements.
Shown in Figure 3.5 are transformation plots for four values of . A striking
feature of these plots is the two points that stand out in three of the four
plots (cases 8 and 48). The BoxCox estimate = 0.28 is evidently inuenced
by the two outlying points and, judging deviations from the identity line in
Figure 3.5c, the mean function for the remaining points is curved. In other
words, the BoxCox estimate is allowing some visually evident curvature
in the bulk of the data so it can accommodate the two outlying points.
Recomputing the estimate of o without the highlighted points gives o =
0.02, which is in good agreement with the log transformation anticipated
at the outset. Reconstruction of the transformation plots indicated that now
the information for the transformation is consistent throughout the data on
the horizontal axis of the plot.
Note that in addition to helping visualize against the data, the transfor-
mation plots can also be used to show the curvature and heteroscedasticity in
the competing models indexed by L . Example 3.4 shows that the plot
can also be used as a diagnostic to assess the success of numerical methods
such as the BoxCox procedure for estimating o .
3.3 Main Eects, Interactions, and Indicators 97

Example 3.5: Mussel Data Again. Return to the mussel data, this
time considering the regression of M on a constant and the four untrans-
formed predictors L, H, W , and S. Figure 3.2 shows the scatterplot matrix
of the predictors L, H, W , and S. Again nonlinearity is present. Figure 3.3
shows that taking the log transformations of W and S results in a linear
scatterplot matrix for the new set of predictors L, H, log W , and log S. Then
the search for the response transformation can be done as in Example 3.4.

3.3 Main Eects, Interactions, and Indicators

Section 1.4 explains interactions, factors, and indicator variables in an ab-

stract setting when Y x|xT where xT is the sucient predictor (SP).
MLR is such a model. The Section 1.4 interpretations given in terms of the
SP can be given in terms of E(Y |x) for MLR since E(Y |x) = xT = SP for
MLR.

Denition 3.5. Suppose that the explanatory variables have the form
x2 , . . . , xk , xjj = x2j , xij = xi xj , x234 = x2 x3 x4 , et cetera. Then the variables
x2 , . . . , xk are main eects. A product of two or more dierent main eects
is an interaction. A variable such as x22 or x37 is a power. An x2 x3 interaction
will sometimes also be denoted as x2 : x3 or x2 x3 .

Denition 3.6. A factor W is a qualitative random variable. Suppose W

has c categories a1 , . . . , ac . Then the factor is incorporated into the MLR
model by using c 1 indicator variables xW j = 1 if W = aj and xW j = 0
otherwise, where one of the levels aj is omitted, e.g. use j = 1, . . . , c1. Each
indicator variable has 1 degree of freedom. Hence the degrees of freedom of
the c 1 indicator variables associated with the factor is c 1.

Rule of thumb 3.3. Suppose that the MLR model contains at least one
power or interaction. Then the corresponding main eects that make up the
powers and interactions should also be in the MLR model.

Rule of thumb 3.3 suggests that if x23 and x2 x7 x9 are in the MLR model,
then x2 , x3 , x7 , and x9 should also be in the MLR model. A quick way to check
whether a term like x23 is needed in the model is to t the main eects models
and then make a scatterplot matrix of the predictors and the residuals, where
the residuals r are on the top row. Then the top row shows plots of xk versus
r, and if a plot is parabolic, then x2k should be added to the model. Potential
predictors wj could also be added to the scatterplot matrix. If the plot of
wj versus r shows a positive or negative linear trend, add wj to the model.
If the plot is quadratic, add wj and wj2 to the model. This technique is for
quantitative variables xk and wj .
98 3 Building an MLR Model

The simplest interaction to interpret is the interaction between a quanti-

tative variable x2 and a qualitative variable x3 with 2 levels. Suppose that
x3 = 1 for level a2 and x3 = 0 for level a1 . Then a rst order model with
interaction is SP = E(Y |x) = 1 + 2 x2 + 3 x3 + 4 x2 x3 . This model yields
two unrelated lines in the conditional expectation depending on the value of
x3 : E(Y |x) = 1 + 3 + (2 + 4 )x2 if x3 = 1, and E(Y |x) = 1 + 2 x2 if
x3 = 0. If 4 = 0, then there are two parallel lines: E(Y |x) = 1 + 3 + 2 x2
if x3 = 1, and E(Y |x) = 1 + 2 x2 if x3 = 0. If 3 = 4 = 0, then the two
lines are coincident: E(Y |x) = 1 + 2 x2 for both values of x3 . If 3 = 0,
then the two lines have the same intercept: E(Y |x) = 1 + (2 + 4 )x2 if
x3 = 1, and E(Y |x) = 1 + 2 x2 if x3 = 0.
Notice that 4 = 0 corresponds to no interaction. The estimated slopes
of the two lines will not be exactly identical, so the two estimated lines will
not be parallel even if there is no interaction. If the two estimated lines
have similar slopes and do not cross, there is evidence of no interaction,
while crossing lines is evidence of interaction provided that the two lines
are not nearly coincident. Two lines with very dierent slopes also suggests
interaction. In general, as factors have more levels and interactions have more
terms, e.g. x2 x3 x4 x5 , the interpretation of the model rapidly becomes very
complex.

Example 3.6. Two varieties of cement that replace sand with coal waste
products were compared to a standard cement mix. The response Y was the
compressive strength of the cement measured after 7, 28, 60, 90, or 180 days

a) Response Plot b) Residual Plot

500
6000

RESID
Y

500
3000

4000 6000 4000 6000

FIT FIT
Fig. 3.6 Plots to Illustrate Interaction for the Cement Data

of curing time = x2 . This cement was intended for sidewalks and barriers but
not for construction. The data is likely from small batches of cement prepared
in the lab, and is likely correlated; however, MLR can be used for exploratory
and descriptive purposes. Actually using the dierent cement mixtures in the
eld (e.g., as sidewalks) would be very expensive. The factor mixture had 3
levels: 2 for the standard cement, and 0 and 1 for the coal based cements.
3.4 Variable Selection 99

A plot of x2 versus Y (not shown but see Problem 3.15) resembled the left
half of a quadratic Y = c(x2 180)2 . Hence x2 and x22 were added to the
model.
Figure 3.6 shows the response plot and residual plots from this model.
The standard cement mix uses the symbol + while the coal based mixes
use an inverted triangle and square. OLS lines based on each mix are added
as visual aids. The lines from the two coal based mixes do not intersect,
suggesting that there may not be an interaction between these two mixes.
There is an interaction between the standard mix and the two coal mixes
since these lines do intersect. All three types of cement become stronger with
time, but the standard mix has the greater strength at early curing times
while the coal based cements become stronger than the standard mix at the
later times. Notice that the interaction is more apparent in the residual plot.
Problem 3.15 adds a factor F x3 based on mix as well as the x2 F x3 and
x22 F x3 interactions. The resulting model is an improvement, but there is
still some curvature in the residual plot, and one case is not t very well.

3.4 Variable Selection

Variable selection, also called subset or model selection, is the search for a
subset of predictor variables that can be deleted without important loss of
information. A model for variable selection in multiple linear regression can
be described by

Y = xT + e = T x + e = xTS S + xTE E + e = xTS S + e (3.4)

where e is an error, Y is the response variable, x = (xTS , xTE )T is a p 1

vector of predictors, xS is a kS 1 vector, and xE is a (p kS ) 1 vector.
Given that xS is in the model, E = 0 and E denotes the subset of terms
that can be eliminated given that the subset S is in the model.

Since S is unknown, candidate subsets will be examined. Let xI be the

vector of k terms from a candidate subset indexed by I, and let xO be the
vector of the remaining predictors (out of the candidate submodel). Then

Y = xTI I + xTO O + e. (3.5)

Denition 3.7. The model Y = xT + e that uses all of the predictors is

called the full model. A model Y = xTI I +e that only uses a subset xI of the
predictors is called a submodel. The full model is always a submodel. The
sucient predictor (SP) is the linear combination of the predictor variables
used in the model. Hence the full model has SP = xT and the submodel
has SP = xTI I .
100 3 Building an MLR Model

The estimated sucient predictor (ESP) is xT , and the following re-

marks suggest that a submodel I is worth considering if the correlation
corr(ESP, ESP (I)) 0.95. Suppose that S is a subset of I and that
model (3.4) holds. Then

SP = xT = xTS S = xTS S + xTI/S (I/S) + xTO 0 = xTI I (3.6)

where xI/S denotes the predictors in I that are not in S. Since this is true
regardless of the values of the predictors, O = 0 and the sample correlation
corr(xTi , xTI,i I ) = 1.0 for the population model if S I.

All too often, variable selection is performed and then the researcher tries
to use the nal submodel for inference as if the submodel was selected before
gathering data. At the other extreme, it could be suggested that variable se-
lection should not be done because classical inferences after variable selection
are not valid. Neither of these two extremes is useful.
Ideally the model is known before collecting the data. After the data is
collected, the MLR assumptions are checked and then the model is used
for inference. Alternatively, a preliminary study can be used to collect data.
Then the predictors and response can be transformed until a full model is
built that seems to be a useful MLR approximation of the data. Then variable
selection can be performed, suggesting a nal model. Then this nal model is
the known model used before collecting data for the main part of the study.
See the two paragraphs above the paragraph above Rule of thumb 3.1. If
the full model is known, inference with the bootstrap prediction region
method and prediction intervals of Section 3.4.1 may be useful.
In practice, the researcher often has one data set, builds the full model,
and performs variable selection to obtain a nal submodel. In other words, an
extreme amount of data snooping was used to build the nal model. A major
problem with the nal MLR model (chosen after variable selection or data
snooping) is that it is not valid for inference in that the p-values for the OLS
t-tests and ANOVA F test are likely to be too small, while the p-value for the
partial F test that uses the nal model as the reduced model is likely to be
too high. Similarly, the actual coverage of the nominal 100(1)% prediction
intervals tends to be too small and unknown (e.g., the nominal 95% PIs may
only contain 83% of the future responses Yf ). Thus the model is likely to t
the data set from which it was built much better than future observations.
Call the data set from which the MLR model was built the training data,
consisting of cases (Yi , xi ) for i = 1, . . . , n. Then the future predictions tend
to be poor in that |Yf Yf | tends to be larger on average than |Yi Yi |.
To summarize, a nal MLR model selected after variable selection can be
useful for description and exploratory analysis: the tests and intervals can
be used for exploratory purposes, but the nal model is usually not valid for
inference.
Generally the research paper should state that the model was built with
one data set, and is useful for description and exploratory purposes, but
3.4 Variable Selection 101

should not be used for inference. The research paper should only suggest
that the model is useful for inference if the model has been shown to be
useful on data collected after the model was built. For example, if
the researcher can collect new data and show that the model produces valid
inferences (e.g., 97 out of 100 95% prediction intervals contained the future
response Yf ), then the researcher can perhaps claim to have found a model
that is useful for inference.
Other problems exist even if the full MLR model Y = xT + e is good.
Let I {1, . . . , p} and let xI be the nal vector of predictors. If xI is missing
important predictors contained in the full model, sometimes called undert-
ting, then the nal model Y = xTI I + e may be a very poor approximation
to the data, in particular the full model may be linear while the nal model
may be nonlinear. Similarly the full model may satisfy V (ei ) = 2 while the
constant variance assumption is violated by the submodel: V (ei ) = i2 . These
two problems are less severe if the joint distribution of (Y, xT )T is multivari-
ate normal, since then Y = xTI I + e satises the constant variance MLR
model regardless of the subset I used. See Problem 10.10.

In spite of these problems, if the researcher has a single data set with
many predictors, then usually variable selection must be done. Let p 1 be
the number of nontrivial predictors and assume that the model also contains
a constant. Also assume that n 10p. If the MLR model found after variable
selection has good response and residual plots, then the model may be very
useful for descriptive and exploratory purposes.
Simpler models are easier to explain and use than more complicated mod-
els, and there are several other important reasons to perform variable selec-
tion. First, an MLR model with unnecessary predictors has a mean square
error for prediction that is too large. Let xS contain the necessary predictors,
let x be the full model, and let xI be a submodel. If (3.4) holds and S I,
then E(Y |xI ) = xTI I = xTS S = xT . Hence OLS applied to Y and xI
yields an unbiased estimator I of I . If (3.4) holds, S I, S is a k 1
vector, and I is a j 1 vector with j > k, then

1 1
n n
2 j 2 k
V (YIi ) = > = V (YSi ). (3.7)
n i=1 n n n i=1

In particular, the full model has j = p. Hence having unnecessary predic-

tors decreases the precision for prediction. Fitting unnecessary predictors is
sometimes called tting noise or overtting. As an extreme case, suppose
that the full model contains p = n predictors, including a constant, so that
the hat matrix H = I n , the n n identity matrix. Then Y = Y so that
VAR(Y |x) = VAR(Y ).
To see that (3.7) holds, assume that the model includes all p possible
terms so may overt but does not undert. Then Y = HY and Cov(Y ) =
2 HIH T = 2 H. Thus
102 3 Building an MLR Model

1
n
1 2 2 p
V (Yi ) = tr( 2 H) = tr((X T X)1 X T X) =
n i=1 n n n

where tr(A) is the trace operation. Replacing p by j and k and replacing H

by H I and H S implies Equation (3.7). Hence if only k parameters are needed
1
n
and p >> k, then serious overtting occurs and increases V (Yi ).
n i=1
Secondly, often researchers are interested in examining the eects of certain
predictors on the response. Recall that i measures the eect of xi given that
all of the other predictors x1 , . . . , xi1 , xi+1 , . . . , xp are in the model. If some
of the predictors are highly correlated, then these predictors may not be
needed in the MLR model given that the other predictors are in the model.
Hence it will not be possible to examine the eects of these predictors on the
response unless the MLR model is changed.
Thirdly, there may be an extremely expensive predictor xp that researchers
would like to omit. If xp is not needed in the MLR model given that
x1 , . . . , xp1 are in the model, then xp can be removed from the model, saving
money.

A major assumption before performing variable selection is that the full

model is good. A factor with c levels can be incorporated into the full model by
creating c 1 indicator variables. Sometimes the categories can be combined
into fewer categories. For example, if the factor is race with levels white,
black, and other, new levels white and nonwhite may be useful for some data
sets. Two rules of thumb are useful for building a full model. Notice that Rule
of thumb 3.4 uses data snooping. Hence the full model and the submodels
chosen after variable selection can be used for description and exploratory
analysis, but should not be used for inference.
Rule of thumb 3.4. Remove strong nonlinearities from the predictors by
making scatterplot matrices of the predictors and the response. If necessary,
transform the predictors and the response using methods from Sections 3.1
and 3.2. Do not transform indicator variables. Each scatterplot matrix should
contain the response entered as the last variable. Do not use more than 10 or
11 variables per scatterplot matrix. Hence if there are 90 predictor variables,
make 10 scatterplot matrices. The rst will contain x1 , . . . , x9 , Y and the last
will contain x82 , . . . , x90 , Y.
Often a variable xi does not need to be transformed if the transformation
does not increase the linearity of the plot of xi versus Y . If the plot of xi
versus xj is nonlinear for some xj , try to transform one or both of xi and xj
in order to remove the nonlinearity, but be careful that the transformations
do not cause a nonlinearity to appear in the plots of xi and xj versus Y .

Rule of thumb 3.5. Let xw1 , . . . , xw,c1 correspond to the indicator

variables of a factor W. Either include all of the indicator variables in the
3.4 Variable Selection 103

model or exclude all of the indicator variables from the model. If the model
contains powers or interactions, also include all main eects in the model
(see Section 3.3).

Next we suggest methods for nding a good submodel. We make the sim-
plifying assumptions that the full model is good, that all predictors have the
same cost, that each submodel contains a constant, and that there is no the-
ory requiring that a particular predictor must be in the model. Also assume
that n 10p, and that the response and residual plots of the full model
are good. Rule of thumb 3.5 should be used for the full model and for all
submodels.

The basic idea is to obtain tted values from the full model and the can-
didate submodel. If the candidate model is good, then the plotted points in
a plot of the submodel tted values versus the full model tted values should
follow the identity line. In addition, a similar plot should be made using the
residuals.
A problem with this idea is how to select the candidate submodel from
the nearly 2p potential submodels. One possibility would be to try to order
the predictors in importance, say x1 , . . . , xp . Then let the kth model contain
the predictors x1 , x2 , . . . , xk for k = 1, . . . , p. If the predicted values from the
submodel are highly correlated with the predicted values from the full model,
then the submodel is good. All subsets selection, forward selection, and
backward elimination can be used (see Section 1.3), but criteria to separate
good submodels from bad are needed.
Two important summaries for submodel I are R2 (I), the proportion of
the variability of Y explained by the nontrivial predictors in the model, and
M SE(I) = I2 , the estimated error variance. See Denitions 2.15 and 2.16.
Suppose that model I contains k predictors, including a constant. Since
adding predictors does not decrease R2 , the adjusted RA 2
(I) is often used,
where
n n
RA2
(I) = 1 (1 R2 (I)) = 1 M SE(I) .
nk SST
See Seber and Lee (2003, pp. 400401). Hence the model with the maximum
2
RA (I) is also the model with the minimum M SE(I).

For multiple linear regression, recall that if the candidate model of xI

has k terms (including the constant), then the partial F statistic for testing
whether the p k predictor variables in xO can be deleted is
 
SSE(I) SSE SSE n p SSE(I)
FI = / = 1
(n k) (n p) n p pk SSE

where SSE is the error sum of squares from the full model, and SSE(I) is the
error sum of squares from the candidate submodel. An extremely important
criterion for variable selection is the Cp criterion.
104 3 Building an MLR Model

Denition 3.8.
SSE(I)
Cp (I) = + 2k n = (p k)(FI 1) + k
M SE
where MSE is the error mean square for the full model.
From Section 1.3, recall that all subsets selection, forward selection, and
backward elimination produce one or more submodels of interest for k =
2, . . . , p where the submodel contains k predictors including a constant. The
following proposition helps explain why Cp is a useful criterion and suggests
that for subsets I with k terms, submodels with Cp (I) min(2k, p) are
especially interesting. Olive and Hawkins (2005) show that this interpretation
of Cp can be generalized to 1D regression models with a linear predictor T x,
such as generalized linear models. Denote the residuals and tted values from
the full model by ri = Yi xTi = Yi Yi and Yi = xTi , respectively.
Similarly, let I be the estimate of I obtained from the regression of Y on xI
and denote the corresponding residuals and tted values by rI,i = Yi xTI,i I
and YI,i = xTI,i I where i = 1, . . . , n.
Proposition 3.1. Suppose that a numerical variable selection method
suggests several submodels with k predictors, including a constant, where
2 k p.
a) The model I that minimizes Cp (I) maximizes corr(r, rI ).

p
b) Cp (I) 2k implies that corr(r, rI ) 1 .
n
c) As corr(r, rI ) 1,

I I ) = corr(ESP, ESP(I)) = corr(Y, YI ) 1.

corr(xT , xT

Proof. These results are a corollary of Proposition 3.2 below. 

Remark 3.1. Consider the model Ii that deletes the predictor xi . Then
the model has k = p 1 predictors including the constant, and the test
statistic is ti where
t2i = FIi .
Using Denition 3.8 and Cp (If ull ) = p, it can be shown that

Cp (Ii ) = Cp (If ull ) + (t2i 2).

Using the screen Cp (I) min(2k, p) suggests that the predictor xi should
not be deleted if
|ti | > 2 1.414.

If |ti | < 2, then the predictor can probably be deleted since Cp decreases.
The literature suggests using the Cp (I) k screen, but this screen eliminates
too many potentially useful submodels.
3.4 Variable Selection 105

More generally, it can be shown that Cp (I) 2k i

p
FI .
pk
Now k is the number of terms in the model including a constant while p k
is the number of terms set to 0. As k 0, the partial F test will reject Ho:
O = 0 (i.e., say that the full model should be used instead of the submodel I)
unless FI is not much larger than 1. If p is very large and p k is very small,
then the partial F test will tend to suggest that there is a model I that is
about as good as the full model even though model I deletes p k predictors.

Denition 3.9. The tt or FF plot is a plot of YI,i versus Yi while

a residualresidual or RR plot is a plot rI,i versus ri . A response plot is a
plot of YI,i versus Yi . An EE plot is a plot of ESP(I) versus ESP. For MLR,
the EE and FF plots are equivalent.

Six graphs will be used to compare the full model and the candidate sub-
model. Let be the estimate of obtained from the regression of Y on all
of the terms x. Many numerical methods such as forward selection, back-
ward elimination, stepwise, and all subsets methods using the Cp (I) criterion
(Jones 1946; Mallows 1973) have been suggested for variable selection. We
will use the FF plot, RR plot, the response plots from the full and submodel,
and the residual plots (of the tted values versus the residuals) from the full
and submodel. These six plots will contain a great deal of information about
the candidate subset provided that Equation (3.4) holds and that a good
estimator (such as OLS) for and I is used.
For these plots to be useful, it is crucial to verify that a multiple linear
regression (MLR) model is appropriate for the full model. Both the re-
sponse plot and the residual plot for the full model need to be
used to check this assumption. The plotted points in the response plot
should cluster about the identity line (that passes through the origin with
unit slope) while the plotted points in the residual plot should cluster about
the horizontal axis (the line r = 0). Any nonlinear patterns or outliers in
either plot suggest that an MLR relationship does not hold. Similarly, be-
fore accepting the candidate model, use the response plot and the residual
plot from the candidate model to verify that an MLR relationship holds for
the response Y and the predictors xI . If the submodel is good, then the
residual and response plots of the submodel should be nearly identical to the
corresponding plots of the full model. Assume that all submodels contain a
constant.

Application 3.2. To visualize whether a candidate submodel using pre-

dictors xI is good, use the tted values and residuals from the submodel and
full model to make an RR plot of the rI,i versus the ri and an FF plot of YI,i
versus Yi . Add the OLS line to the RR plot and identity line to both plots as
106 3 Building an MLR Model

visual aids. The subset I is good if the plotted points cluster tightly about
the identity line in both plots. In particular, the OLS line and the identity
line should nearly coincide so that it is dicult to tell that the two lines
intersect at the origin in the RR plot.

To verify that the six plots are useful for assessing variable selection,
the following notation will be useful. Suppose that all submodels include
a constant and that X is the full rank n p design matrix for the full
model. Let the corresponding vectors of OLS tted values and residuals
be Y = X(X T X)1 X T Y = HY and r = (I H)Y , respectively.
Suppose that X I is the n k design matrix for the candidate submodel
and that the corresponding vectors of OLS tted values and residuals are
Y I = X I (X TI X I )1 X TI Y = H I Y and r I = (I H I )Y , respectively.

A plot can be very useful if the OLS line can be compared to a reference
line and if the OLS slope is related to some quantity of interest. Suppose that
a plot of w versus z places w on the horizontal axis and z on the vertical axis.
Then denote the OLS line by z = a + bw. The following proposition shows
that the plotted points in the FF, RR, and response plots will cluster about
the identity line. Notice that the proposition is a property of OLS and holds
even if the data does not follow an MLR model. Let corr(x, y) denote the
correlation between x and y.

Proposition 3.2. Suppose that every submodel contains a constant and

that X is a full rank matrix.
Response Plot: i) If w = YI and z = Y , then the OLS line is the identity
line.
2
ii) If w = Y and z = YI , then the OLS line has slope
n b = [corr(Y, Y2I )] =
R (I) and intercept a = Y (1 R (I)) where Y = i=1 Yi /n and R (I) is
2 2

the coecient of multiple determination from the candidate model.

FF or EE Plot: iii) If w = YI and z = Y , then the OLS line is the identity
line. Note that ESP (I) = YI and ESP = Y .
iv) If w = Y and z = YI , then the OLS line has slope b = [corr(Y , YI )]2 =
SSR(I)/SSR and intercept a = Y [1 (SSR(I)/SSR)] where SSR is the
regression sum of squares.
RR Plot: v) If w = r and z = rI , then the OLS line is the identity line.
vi) If w = rI and z = r, then a = 0 and the OLS slope b = [corr(r, rI )]2 and
  
SSE np np
corr(r, rI ) = = = .
SSE(I) Cp (I) + n 2k (p k)FI + n p

Proof: Recall that H and H I are symmetric idempotent matrices and

that HH I = H I . The mean of OLS tted values is equal to Y and the
mean of OLS residuals is equal to 0. If the OLS line from regressing z on w
is z = a + bw, then a = z bw and
3.4 Variable Selection 107

(wi w)(zi z) SD(z)
b=  = corr(z, w).
(wi w) 2 SD(w)

Also recall that the OLS line passes through the means of the two variables
(w, z).
(*) Notice that the OLS slope from regressing z on w is equal to one if
and only if the OLS slope from regressing w on z is equal to [corr(z, w)]2 .
  2 T
i) The slope b = 1 if YI,i Yi = YI,i . This equality holds since Y I Y =
T
Y T H I Y = Y T H I H I Y = Y I Y I . Since b = 1, a = Y Y = 0.

ii) By (*), the slope


(YI,i Y )2
b = [corr(Y, YI )] = R (I) = 
2 2
= SSR(I)/SST O.
(Yi Y )2

The result follows since a = Y bY .

  2
iii) The slope b = 1 if YI,i Yi = YI,i . This equality holds since
T T
Y Y I = Y T HH I Y = Y T H I Y = Y I Y I . Since b = 1, a = Y Y = 0.

iv) From iii),

SD(Y )
1= [corr(Y , YI )].
SD(YI )
Hence
SD(YI )
corr(Y , YI ) =
SD(Y )
and the slope
SD(YI )
b= corr(Y , YI ) = [corr(Y , YI )]2 .
SD(Y )
Also the slope 
(YI,i Y )2
b=  = SSR(I)/SSR.
(Yi Y )2
The result follows since a = Y bY .

v) The OLS line passes through the origin. Hence a = 0. The slope b =
r T r I /r T r. Since r T r I = Y T (I H)(I H I )Y and (I H)(I H I ) =
I H, the numerator r T r I = r T r and b = 1.

vi) Again a = 0 since the OLS line passes through the origin. From v),

SSE(I)
1= [corr(r, rI )].
SSE
108 3 Building an MLR Model

Hence 
SSE
corr(r, rI ) =
SSE(I)
and the slope

SSE
b= [corr(r, rI )] = [corr(r, rI )]2 .
SSE(I)

Algebra shows that

 
np np
corr(r, rI ) = = . 
Cp (I) + n 2k (p k)FI + n p

A standard model selection procedure will often be needed to suggest

models. For example, forward selection or backward elimination could be
used. If p < 30, Furnival and Wilson (1974) provide a technique for selecting
a few candidate subsets after examining all possible subsets.

Remark 3.2. Daniel and Wood (1980, p. 85) suggest using Mallows
graphical method for screening subsets by plotting k versus Cp (I) for models
close to or under the Cp = k line. Proposition 3.2 vi) implies that if Cp (I) k
or FI < 1, then corr(r, rI ) and corr(ESP, ESP (I)) both go to 1.0 as n .
Hence models I that satisfy the Cp (I) k screen will contain the true model
S with high probability when n is large. This result does not guarantee that
the true model S will satisfy the screen, but overt is likely. Let d be a lower
bound on corr(r, rI ). Proposition 3.2 vi) implies that if
 
1 p
Cp (I) 2k + n 2 1 2 ,
d d

then corr(r, rI ) d. The simple screen Cp (I) 2k corresponds to


p
d dn = 1 .
n

To avoid excluding too many good submodels, consider models I with

Cp (I) min(2k, p). Models under both the Cp = k line and the Cp = 2k line
are of interest.

Rule of thumb 3.6. a) After using a numerical method such as forward

selection or backward elimination, let Imin correspond to the submodel with
the smallest Cp . Find the submodel II with the fewest number of predictors
such that Cp (II ) Cp (Imin ) + 1. Then II is the initial submodel that should
be examined. It is possible that II = Imin or that II is the full model. Do
not use more predictors than model II to avoid overtting.
3.4 Variable Selection 109

b) Models I with fewer predictors than II such that Cp (I) Cp (Imin ) + 4

are interesting and should also be examined.
c) Models I with k predictors, including a constant and with fewer predic-
tors than II such that Cp (Imin ) + 4 < Cp (I) min(2k, p) should be checked
but often undert: important predictors are deleted from the model. Under-
t is especially likely to occur if a predictor with one degree of freedom is
deleted (recall that if the c 1 indicator variables corresponding to a factor
are deleted, then the factor has c 1 degrees of freedom) and the jump in Cp
is large, greater than 4, say.
d) If there are no models I with fewer predictors than II such that Cp (I)
min(2k, p), then model II is a good candidate for the best subset found by
the numerical procedure.

Rule of thumb 3.7. Assume that the full model has good response and
residual plots and that n 10p. Let subset I have k predictors, including a
constant. Know how to nd good models from output. The following rules of
thumb (roughly in order of decreasing importance) may be useful. It is often
not possible to have all 10 rules of thumb to hold simultaneously. Let Imin be
the minimum Cp model and let II be the model with the fewest predictors
satisfying Cp (II ) Cp (Imin ) + 1. Do not use more predictors than model II
to avoid overtting. Then the submodel I is good if
i) the response and residual plots for the submodel looks like the response
and residual plots for the full model,
ii) corr(ESP,ESP(I)) = corr(Y, YI ) 0.95.
iii) The plotted points in the FF plot (= EE plot for MLR) cluster tightly
about the identity line.
iv) Want the p-value 0.01 for the partial F test that uses I as the reduced
model.
v) The plotted points in the RR plot cluster tightly about the identity line.
vi) Want R2 (I) > 0.9R2 and R2 (I) > R2 0.07 (recall that R2 (I) R2 =
R2 (f ull) since adding predictors to I does not decrease R2 (I)).
vii) Want Cp (Imin ) Cp (I) min(2k, p) with no big jumps in Cp (the
increase should be less than four) as variables are deleted.
viii) Want hardly any predictors with p-values > 0.05.
ix) Want few predictors with p-values between 0.01 and 0.05.
x) Want MSE(I) to be smaller than or not much larger than the MSE from
the full model.
(If n 5p, use the above rules, but we want n 10k.)

The following description of forward selection and backward elimina-

tion modies the description of Section 1.3 slightly. Criterion such as AIC,
2
M SE(I), or RA (I) are sometimes used instead of Cp . For forward selection,
the numerical method may add the predictor not yet in the model that has
the smallest pvalue for the t test. For backward elimination, the numerical
method may delete the variable in the model (that is not a constant) that
has the largest pvalue for the t test.
110 3 Building an MLR Model

Forward selection Step 1) k = 1: Start with a constant w1 = x1 . Step

2) k = 2: Compute Cp for all models with k = 2 containing a constant and a
single predictor xi . Keep the predictor w2 = xj , say, that minimizes Cp .
Step 3) k = 3: Fit all models with k = 3 that contain w1 and w2 . Keep the
predictor w3 that minimizes Cp . . . .
Step j) k = j: Fit all models with k = j that contains w1 , w2 , . . . , wj1 . Keep
the predictor wj that minimizes Cp . . . .
Step p): Fit the full model.

Backward elimination: All models contain a constant = u1 . Step 0)

k = p: Start with the full model that contains x1 , . . . , xp . We will also say
that the full model contains u1 , . . . , up where u1 = x1 but ui need not equal
xi for i > 1.
Step 1) k = p 1: Fit each model with k = p 1 predictors including a
constant. Delete the predictor up , say, that corresponds to the model with
the smallest Cp . Keep u1 , . . . , up1 .
Step 2) k = p 2: Fit each model with p 2 predictors including a con-
stant. Delete the predictor up1 corresponding to the smallest Cp . Keep
u1 , . . . , up2 . . . .
Step j) k = p j: t each model with p j predictors including a con-
stant. Delete the predictor upj+1 corresponding to the smallest Cp . Keep
u1 , . . . , upj . . . .
Step p 2) k = 2. The current model contains u1 , u2 , and u3 . Fit the model
u1 , u2 and the model u1 , u3 . Assume that model u1 , u2 minimizes Cp . Then
delete u3 , and keep u1 and u2 .

Heuristically, backward elimination tries to delete the variable that will

increase Cp the least. An increase in Cp greater than 4 (if the predictor has 1
degree of freedom) may be troubling in that a good predictor may have been
deleted. In practice, the backward elimination program may use some other
criterion: e.g. delete the variable such that the submodel I with j predictors
has a) the smallest Cp (I) or b) the biggest pvalue in the test Ho i = 0
versus HA i = 0 where the model with j + 1 terms from the previous step
(using the j predictors in I and the variable xj+1 ) is treated as the full model.
Heuristically, forward selection tries to add the variable that will decrease
Cp the most. A decrease in Cp less than 4 (if the predictor has 1 degree of
freedom) may be troubling in that a bad predictor may have been added.
In practice, the forward selection program may use some other criterion, e.g.
add the variable such that the submodel I with j nontrivial predictors has
a) the smallest Cp (I) or b) the smallest pvalue in the test Ho i = 0 versus
HA i = 0 where the current model with j terms plus the predictor xi is
treated as the full model (for all variables xi not yet in the model).
Suppose that the full model is good and is stored in M1. Let M2, M3,
M4, and M5 be candidate submodels found after forward selection, backward
elimination, et cetera. Recall that ESP (I) = YI . Make a scatterplot matrix
of the ESPs for M1, M2, M3, M4, M5, and Y . Good candidates should have
3.4 Variable Selection 111

estimated sucient predictors that are highly correlated with the full model
ESP (the correlation should be at least 0.9 and preferably greater than 0.95).
Similarly, make a scatterplot matrix of the residuals for M1, M2, M3, M4,
and M5.

To summarize, the nal submodel should have few predictors, few variables
with large OLS t test pvalues (0.01 to 0.05 is borderline), good response and
residual plots, and an FF plot (= EE plot) that clusters tightly about the
identity line. If a factor has c 1 indicator variables, either keep all c 1
indicator variables or delete all c 1 indicator variables, do not delete some
of the indicator variables.

Example 3.7. The pollution data of McDonald and Schwing (1973) can
be obtained from STATLIB or the texts website. The response Y = mort
is the mortality rate, and most of the independent variables were related
to pollution. A scatterplot matrix of the rst 9 predictors and Y was made
and then a scatterplot matrix of the remaining predictors with Y . The log
rule suggested making the log transformation with 4 of the variables. The
summary output is shown below and on the following page. The response
and residual plots were good. Notice that p = 16 and n = 60 < 5p. Also
many p-values are too high.

Response = MORT
Label Estimate Std. Error t-value p-value
Constant 1881.11 442.628 4.250 0.0001
DENS 0.00296 0.00397 0.747 0.4588
EDUC -19.6669 10.7005 -1.838 0.0728
log[HC] -31.0112 15.5615 -1.993 0.0525
HOUS -0.40107 1.64372 -0.244 0.8084
HUMID -0.44540 1.06762 -0.417 0.6786
JANT -3.58522 1.05355 -3.403 0.0014
JULT -3.84292 2.12079 -1.812 0.0768
log[NONW] 27.2397 10.1340 2.688 0.0101
log[NOX] 57.3041 15.4764 3.703 0.0006
OVR65 -15.9444 8.08160 -1.973 0.0548
POOR 3.41434 2.74753 1.243 0.2206
POPN -131.823 69.1908 -1.905 0.0633
PREC 3.67138 0.77814 4.718 0.0000
log[SO] -10.2973 7.38198 -1.395 0.1700
WWDRK 0.88254 1.50954 0.585 0.5618

R Squared: 0.787346 Sigma hat: 33.2178

Number of cases: 60 Degrees of freedom: 44

Summary Analysis of Variance Table

Source df SS MS F p-value
112 3 Building an MLR Model

Regression 15 179757. 11983.8 10.86 0.0000

Residual 44 48550.5 1103.42
Shown below this paragraph is some output from forward selection. The
minimum Cp model had Cp = 7.353 with 7 predictors, including a con-
stant. Deleting JANT from this model increased Cp to 17.763, suggesting
that JANT is an important predictor. Notice that Cp > 2k = 12 for the
model that deletes JANT.
Base terms: (log[NONW] EDUC log[SO] PREC)
df RSS | k C_I
Add: log[NOX] 54 72563.9 | 6 17.763
Add: JANT 54 72622. | 6 17.815
Add: HOUS 54 74884.8 | 6 19.866
Add: POPN 54 75350.2 | 6 20.288
Add: log[HC] 54 75373.4 | 6 20.309
Add: JULT 54 75405.8 | 6 20.338
Add: OVR65 54 75692.2 | 6 20.598
Add: HUMID 54 75747.4 | 6 20.648
Add: DENS 54 75872.1 | 6 20.761
Add: POOR 54 75938.4 | 6 20.821
Add: WWDRK 54 75971.8 | 6 20.851

Base terms: (log[NONW] EDUC log[SO] PREC log[NOX])

df RSS | k C_I
Add: JANT 53 58871. | 7 7.353
Add: log[HC] 53 69233.3 | 7 16.744
Add: HOUS 53 70774.1 | 7 18.141
Add: POPN 53 71424.7 | 7 18.730
Add: POOR 53 72049.4 | 7 19.296
Add: OVR65 53 72337.1 | 7 19.557
Add: JULT 53 72348.6 | 7 19.568
Add: WWDRK 53 72483.1 | 7 19.690
Add: DENS 53 72494.9 | 7 19.700
Add: HUMID 53 72563.9 | 7 19.763
Output for backward elimination is shown below, and the minimum Cp
model had Cp = 6.284 with 6 predictors, including a constant. Deleting
EDUC increased Cp to 10.800 > 2k = 10. Since Cp increased by more than
4, EDUC is probably important.
Current terms:(EDUC JANT log[NONW] log[NOX] OVR65 PREC)
df RSS | k C_I
Delete: OVR65 54 59897.9 | 6 6.284
Delete: EDUC 54 66809.3 | 6 12.547
Delete: log[NONW] 54 73178.1 | 6 18.319
Delete: JANT 54 76417.1 | 6 21.255
3.4 Variable Selection 113

Delete: PREC 54 83958.1 | 6 28.089

Delete: log[NOX] 54 86823.1 | 6 30.685

Current terms: (EDUC JANT log[NONW] log[NOX] PREC)

df RSS | k C_I
Delete: EDUC 55 67088.1 | 5 10.800
Delete: JANT 55 76467.4 | 5 19.300
Delete: PREC 55 87206.7 | 5 29.033
Delete: log[NOX] 55 88489.6 | 5 30.196
Delete: log[NONW] 55 95327.5 | 5 36.393

Taking the minimum Cp model from backward elimination gives the out-
put shown below. The response and residual plots were OK although the
2
correlation in the RR and FF plots was not real high. The R in the sub-
model decreased from about 0.79 to 0.74 while = M SE was 33.22 for
the full model and 33.31 for the submodel. Removing nonlinearities from the
predictors by using two scatterplots and the log rule, and then using back-
ward elimination and forward selection, seems to be very eective for nding
the important predictors for this data set. See Problem 13.17 in order to
reproduce this example with the essential plots.
Response = MORT
Label Estimate Std. Error t-value p-value
Constant 943.934 82.2254 11.480 0.0000
EDUC -15.7263 6.17683 -2.546 0.0138
JANT -1.86899 0.48357 -3.865 0.0003
log[NONW] 33.5514 5.93658 5.652 0.0000
log[NOX] 21.7931 4.29248 5.077 0.0000
PREC 2.92801 0.59011 4.962 0.0000

R Squared: 0.737644 Sigma hat: 33.305

Number of cases: 60 Degrees of freedom: 54

Summary Analysis of Variance Table

Source df SS MS F p-value
Regression 5 168410. 33681.9 30.37 0.0000
Residual 54 59897.9 1109.22

Example 3.8. The FF and RR plots can be used as a diagnostic for

whether a given numerical method is including too many variables. Glad-
stone (1905) attempts to estimate the weight of the human brain (measured
in grams after the death of the subject) using simple linear regression with
a variety of predictors including age in years, height in inches, head height
in mm, head length in mm, head breadth in mm, head circumference in mm,
and cephalic index. The sex (coded as 0 for females and 1 for males) of each
subject was also included. The variable cause was coded as 1 if the cause of
114 3 Building an MLR Model

death was acute, 3 if the cause of death was chronic, and coded as 2 oth-
erwise. A variable ageclass was coded as 0 if the age was under 20, 1 if the
age was between 20 and 45, and as 3 if the age was over 45. Head size, the
product of the head length, head breadth, and head height, is a volume mea-
surement, hence (size)1/3 was also used as a predictor with the same physical
dimensions as the other lengths. Thus there are 11 nontrivial predictors and
one response, and all models will also contain a constant. Nine cases were
deleted because of missing values, leaving 267 cases.
Figure 3.7 shows the response plots and residual plots for the full model
and the nal submodel that used a constant, size1/3 , age, and sex. The ve
cases separated from the bulk of the data in each of the four plots correspond
to ve infants. These may be outliers, but the visual separation reects the
small number of infants and toddlers in the data. A purely numerical variable
selection procedure would miss this interesting feature of the data. We will
rst perform variable selection with the entire data set, and then examine the
eect of deleting the ve cases. Using forward selection and the Cp statistic
on the Gladstone data suggests the subset I5 containing a constant, (size)1/3 ,
age, sex, breadth, and cause with Cp (I5 ) = 3.199. The pvalues for breadth
and cause were 0.03 and 0.04, respectively. The subset I4 that deletes cause
has Cp (I4 ) = 5.374 and the pvalue for breadth was 0.05. Figure 3.8d shows
the RR plot for the subset I4 . Note that the correlation of the plotted points
is very high and that the OLS and identity lines nearly coincide.

a) Full Response Plot b) Full Residual Plot

1200

100
FRES
Y

200
400

400 800 1200 400 800 1200

FFIT FFIT

c) Sub Response Plot d) Sub Residual Plot

1200

200 100
SRES3
Y
400

400 800 1200 400 800 1200

SFIT3 SFIT3

Fig. 3.7 Gladstone data: comparison of the full model and the submodel.

A scatterplot matrix of the predictors and response suggests that (size)1/3

might be the best single predictor. First we regressed Y = brain weight on the
eleven predictors described above (plus a constant) and obtained the residuals
ri and tted values Yi . Next, we regressed Y on the subset I containing
(size)1/3 and a constant and obtained the residuals rI,i and the tted values
3.4 Variable Selection 115

yI,i . Then the RR plot of rI,i versus ri , and the FF plot of YI,i versus Yi were
constructed.

a) RR Plot for (size)^(1/3) b) FF Plot for (size)^(1/3)

200

1200
100
FRES

FFIT
0

400 600 800

-200

-200 -100 0 100 200 400 600 800 1000 1200 1400
SRES1 SFIT1

c) RR Plot for 2 Predictors d) RR Plot for 4 Predictors

100 200

200
100
FRES

FRES
0
0
-200

-200

-200 -100 0 100 200 -200 -100 0 100 200

SRES2 SRES4

Fig. 3.8 Gladstone data: submodels added (size)1/3 , sex, age, and nally breadth.

For this model, the correlation in the FF plot (Figure 3.8b) was very
high, but in the RR plot the OLS line did not coincide with the identity
line (Figure 3.8a). Next sex was added to I, but again the OLS and identity
lines did not coincide in the RR plot (Figure 3.8c). Hence age was added
to I. Figure 3.9a shows the RR plot with the OLS and identity lines added.
These two lines now nearly coincide, suggesting that a constant plus (size)1/3 ,
sex, and age contains the relevant predictor information. This subset has
Cp (I) = 7.372, R2 (I) = 0.80, and I = 74.05. The full model which used
11 predictors and a constant has R2 = 0.81 and = 73.58. Since the Cp
criterion suggests adding breadth and cause, the Cp criterion may be leading
to an overt.
Figure 3.9b shows the FF plot. The ve cases in the southwest corner cor-
respond to ve infants. Deleting them leads to almost the same conclusions,
although the full model now has R2 = 0.66 and = 73.48 while the submodel
has R2 (I) = 0.64 and I = 73.89.
116 3 Building an MLR Model

a) RR Plot b) FF Plot

1400
200

1200
100

1000
FRES

FFIT
0

800
-100

600
-200

400

-200 -100 0 100 200 400 600 800 1000 1200 1400
SRES3 SFIT3

Fig. 3.9 Gladstone data with Predictors (size)1/3 , sex, and age

a) RR Plot b) FF plot
0.10

0.50
0.05

0.45
full$residual

ffit
0.40
0.0 -0.05

0.35
-0.10

0.30

-0.10 -0.05 0.0 0.05 0.10 0.30 0.35 0.40 0.45 0.50
sub$residual sfit

Fig. 3.10 RR and FF Plots for Rat Data

Example 3.9. Cook and Weisberg (1999a, pp. 261, 371) describe a data
set where rats were injected with a dose of a drug approximately proportional
to body weight. The data set is included as the le rat.lsp in the Arc software
3.4 Variable Selection 117

and can be obtained from the website (www.stat.umn.edu/arc/). The re-

sponse Y is the fraction of the drug recovered from the rats liver. The three
predictors are the body weight of the rat, the dose of the drug, and the liver
weight. The experimenter expected the response to be independent of the
predictors, and 19 cases were used. However, the Cp criterion suggests using
the model with a constant, dose, and body weight, both of whose coecients
were statistically signicant. The RR and FF plots are shown in Figure 3.10.
The identity line was added to both plots and the OLS line was added to the
RR plot. The upper right corner of the FF plot shows one outlier, the third
case, that is clearly separated from the rest of the data.
We deleted this case and again searched for submodels. The Cp statistic
is less than one for all three simple linear regression models, and the RR and
FF plots look the same for all submodels containing a constant. Figure 2.2
shows the RR plot where the residuals from the full model are plotted against
Y Y , the residuals from the model using no nontrivial predictors. This plot
suggests that the response Y is independent of the nontrivial predictors.

The point of this example is that a subset of outlying cases can cause
numeric second-moment criteria such as Cp to nd structure that does not
exist. The FF and RR plots can sometimes detect these outlying cases, allow-
ing the experimenter to run the analysis without the inuential cases. The
example also illustrates that global numeric criteria can suggest a model with
one or more nontrivial terms when in fact the response is independent of the
predictors.

Numerical variable selection methods for MLR are very sensitive to in-
uential cases such as outliers. Olive and Hawkins (2005) show that a plot
of the residuals versus Cooks distances (see Section 3.5) can be used to de-
tect inuential cases. Such cases can also often be detected from response,
residual, RR, and FF plots.
Warning: deleting inuential cases and outliers will often lead to
better plots and summary statistics, but the cleaned data may no
longer represent the actual population. In particular, the resulting
model may be very poor for both prediction and description.
Multiple linear regression data sets with cases that inuence numerical
variable selection methods are common. Table 3.1 shows results for seven
interesting data sets. The rst two rows correspond to the Ashworth (1842)
data, the next 2 rows correspond to the Gladstone data in Example 3.8, and
the next 2 rows correspond to the Gladstone data with the 5 infants deleted.
Rows 7 and 8 are for the Buxton (1920) data, while rows 9 and 10 are for
the Tremearne (1911) data. These data sets are available from the books
website. Results from the nal two data sets are given in the last 4 rows. The
last 2 rows correspond to the rat data described in Example 3.9. Rows 11
and 12 correspond to the ais data that comes with Arc (Cook and Weisberg
1999a).
118 3 Building an MLR Model

The full model used p predictors, including a constant. The nal submodel
I also included a constant, and the nontrivial predictors are listed in the
second column of Table 3.1. For a candidate submodel I, let Cp (I, c) denote
the value of the Cp statistic for the clean data that omits inuential cases and
outliers. The third column lists p, Cp (I), andCp (I, c) while the rst column

Table 3.1 Summaries for Seven Data Sets

inuential cases submodel I p, Cp (I), Cp (I, c)
le, response transformed predictors
14, 55 log(x2 ) 4, 12.665, 0.679
pop, log(y) log(x1 ), log(x2 ), log(x3 )
118, 234, 248, 258 (size)1/3 , age, sex 10, 6.337, 3.044
cbrain,brnweight (size)1/3
118, 234, 248, 258 (size)1/3 , age, sex 10, 5.603, 2.271
cbrain-5,brnweight (size)1/3
11, 16, 56 sternal height 7, 4.456, 2.151
cyp,height none
3, 44 x2 , x5 6, 0.793, 7.501
major,height none
11, 53, 56, 166 log(LBM), log(Wt), sex 12, 1.701, 0.463
ais,%Bfat log(Ferr), log(LBM), log(Wt), Ht
3 no predictors 4, 6.580, 1.700
rat,y none

gives the set of inuential cases. Two rows are presented for each data set. The
second row gives the response variable and any predictor transformations.
For example, for the Gladstone data p = 10 since there were 9 nontrivial
predictors plus a constant. Only the predictor size was transformed, and the
nal submodel is the one given in Example 3.8. For the rat data, the nal
submodel is the one given in Example 3.9: none of the 3 nontrivial predictors
was used.
Table 3.1 and simulations suggest that if the subset I has k predictors,
then using the Cp (I) min(2k, p) screen is better than using the conven-
tional Cp (I) k screen. The major and ais data sets show that deleting the
inuential cases may increase the Cp statistic. Thus interesting models from
the entire data set and from the clean data set should be examined.

Example 3.10. Conjugated linoleic acid (CLA) occurs in beef and dairy
products and appears to have many human health benets. Joanne Numrich
provided four data sets where the response was the amount of CLA (or re-
lated compounds), and the explanatory variables were feed components from
the cattle diet. The data was to be used for descriptive and exploratory pur-
poses. Several data sets had outliers with unusually high levels of CLA. These
outliers were due to one researcher and may be the most promising cases in
the data set. However, to describe the bulk of the data with OLS MLR,
the outliers were omitted. In one of the data sets there are 33 cases and 25
3.4 Variable Selection 119

predictors, including a constant. Regressing Y on all of the predictors gave

R2 = 0.84 and an ANOVA F test p-value of 0.223, suggesting that none of the
predictors are useful. From Proposition 2.5, an R2 > (p 1)/(n 1) = 0.75
is not very surprising. Remarks above Theorem 2.7 help explain why R2 can
be high with a high ANOVA F test p-value.
Of course just tting the data to the collected variables is a poor way
to proceed. Only variables x1 , x2 , x5 , x6 , x20 , and x21 took on more than a
few values. Taking log(Y ) and using variables x2 , x9 , x23 , and x24 seemed to
result in an adequate model, although the number of distinct tted values
was rather small. See Problem 3.18 for more details.

3.4.1 Bootstrapping Variable Selection

The bootstrap will be described and then applied to variable selection. Sup-
pose there is data w1 , . . . , wn collected from a distribution with cdf F into
an n p matrix W . The empirical distribution, with cdf Fn , gives each ob-
served data case wi probability 1/n. Let the statistic Tn = t(W ) = t(Fn )
be computed from the data. Suppose the statistic estimates = t(F ). Let
t(W ) = t(Fn ) = Tn indicate that t was computed from an iid sample from
the empirical distribution Fn : a sample of size n was drawn with replacement
from the observed sample w1 , . . . , wn .
Some notation is needed to give the Olive (2013a) prediction region used
to bootstrap a hypothesis test. Suppose w1 , . . . , wn are iid p 1 random
vectors with mean and nonsingular covariance matrix w . Let a future
test observation wf be independent of the wi but from the same distribution.
Let (w, S) be the sample mean and sample covariance matrix where

1 1 
n n
w= wi and S = S w = (wi w)(wi w)T . (3.8)
n i=1 n1
i=1

Then the ith squared sample Mahalanobis distance is the scalar

2
Dw 2
= Dw (w, S) = (w w)T S 1 (w w). (3.9)

Let Di2 = Dw 2
i
for each observation wi . Let D(c) be the cth order statistic of
D1 , . . . , Dn . Consider the hyperellipsoid

An = {w : Dw
2
(w, S) D(c)
2
} = {w : Dw (w, S) D(c) }. (3.10)

If n is large, we can use c = kn = n(1 ). If n is not large, using c =

Un where Un decreases to kn , can improve small sample performance. Olive
(2013a) showed that (3.10) is a large sample 100(1 )% prediction region
for a large class of distributions, although regions with smaller volumes may
exist. Note that the result follows since if w and S are nonsingular, then the
120 3 Building an MLR Model

Mahalanobis distance is a continuous function of (w, S). Let D = D(, w ).

D D
Then Di D and Di2 D2 . Hence the sample percentiles of the Di are
consistent estimators of the population percentiles of D at continuity points
of the cumulative distribution function (cdf) of D. Prediction region (3.10)
estimates the highest density region for a large class of elliptically contoured
distributions. Some of the above terms appear in Chapter 10.
Denition 3.10. Given training data w1 , . . . , wn , a large sample 100(1
)% prediction region for a future test value wf is a set An such that P (wf
An ) 1 as n , while a large sample condence region for a parameter
is a set An such that P ( An ) 1 as n .
When teaching condence intervals, it is often noted that by the cen-
tral limit theorem, the probability that Y n is within two standard devia-
tions (2SD(Y n ) = 2/ n) of is about 95%. Hence the probability that
is within twostandard deviations
of Y n is about 95%. Thus the inter-
val [ 1.96S/ n, + 1.96S/ n ] is a large sample 95% prediction in-
terval for a sample mean Y n,f if is known, while
future value of the
[Y n 1.96S/ n, Y n + 1.96S/ n ] is a large sample 95% condence interval
for the population mean . Note that the lengths of the two intervals are
the same. Where the interval is centered determines whether the interval is a
condence or a prediction interval. Here S is the sample standard deviation.
The following theorem shows that the hyperellipsoid Rc centered at the
statistic Tn is a large sample 100(1 )% condence region for , but the hy-
perellipsoid Rp centered at known is a large sample 100(1 )% prediction
region for a future value of the statistic Tf,n .
Theorem 3.3. Let the 100(1 )th percentile D1
2
be a continuity point
D D
of the distribution of D2 . Assume that D
2
(Tn , T ) D2 , D
2
(Tn , T )
P
D2 , and D1
2
D1
2
where P (D2 D12
) = 1 . i) Then Rc = {w :
Dw (Tn , T ) D1 } is a large sample 100(1 )% condence region for ,
2 2

and if is known, then Rp = {w : Dw 2

(, T ) D1
2
} is a large sample
100(1)% prediction region for a future value of the statistic Tf,n . ii) Region
Rc contains i region Rp contains Tn .
2
Proof: i) Note that D (Tn , T ) = DT2 n (, T ). Thus the probability
that Rc contains is P (D 2
(Tn , T ) D1
2
) 1 , and the probability
that Rp contains Tf,n is P (D (Tf,n , T ) D1
2 2
) 1 , as n .
ii) D (Tn , T ) D1 i DTn (, T ) D1 . 
2 2 2 2

Hence if there was an iid sample T1,n , . . . , TB,n of the statistic, the Olive
(2013a) large sample 100(1 )% prediction region {w : D2 (T , S T ) D(c) 2
}
for Tf,n contains E(Tn ) = with asymptotic coverage 1 . To make the
asymptotic coverage equal to 1, use the large sample 100(1)% condence
region {w : D2 (T1,n , S T ) D(c)
2
}. The prediction region method bootstraps
3.4 Variable Selection 121

this procedure by using a bootstrap sample of the statistic T1,n , . . . , TB,n .

Centering the region at T1,n instead of T is not needed since the bootstrap
sample is centered near Tn : the distribution of n(Tn ) is approximated

by the distribution of n(T Tn ) or by the distribution of n(T T ).

Consider testing H0 : = c versus H1 : = c where c is a known r 1

vector. If a condence region can be constructed for c, then fail to reject
H0 if 0 is in the condence region, and reject H0 if 0 is not in the condence
region.
The prediction region method makes a bootstrap sample wi = i c
for i = 1, . . . , B. Make the prediction region (3.10) for the wi , and reject H0
if 0 is not in the prediction region. As shown below, the prediction region
method is a special case of the percentile method, and a special case of
bootstrapping a test statistic.
For p = 1, the percentile method uses an interval that contains UB

kB = B(1 ) of the Ti,n from a bootstrap sample T1,n , . . . , TB,n where
the statistic Ti,n is an estimator of based on a sample of size n. Often
the n is suppressed. Here x is the smallest integer x, e.g. 7.8 = 8.

Let T(1) , T(2) , . . . , T(B) be the order statistics of the bootstrap sample. Then
one version of the percentile method discards the largest and smallest B/2
order statistics, resulting in an interval [LB , RB ] that is a large sample 100(1
)% condence interval (CI) for , and also a large sample 100(1 )%

prediction interval (PI) for a future bootstrap value Tf,n .
Olive (2016a,b, 2014: p. 283) recommended using the shorth(c) estimator
for the percentile method. The shorth interval tends to be shorter than the
interval that deletes the smallest and largest B/2 observations Wi when
the Wi do not come from a symmetric distribution. Frey (2013) showed that
 B and iid data, the shorth(kB ) PI has maximum undercoverage
for large
1.12 /B, and used the shorth(c) estimator as the large sample 100(1 )%
prediction interval where c = min(B, B[1 + 1.12 /B ] ). Hence if
B = 1000, there may be about 1% undercoverage using c = kB = B(1 ).
Consider testing H0 : = c versus H1 : = c, and the statistic Ti = c.
If E(Ti ) = and Cov(Ti ) = T were known, then the squared Maha-
lanobis distance Di2 (, T ) = (Ti )T 1 T (Ti ) would be a natural
2
statistic to use if the percentile D1 (, T ) was known. The prediction
region method bootstraps the squared Mahalanobis distances, forming the
bootstrap sample wi = Ti = i c and the squared Mahalanobis distances
1 
B
Di2 = Di2 (T , S T ) = (Ti T )T [S T ]1 (Ti T ) where T = T and
B i=1 i
1 
B
S T = (T T )(Ti T )T are the sample mean and sample co-
B 1 i=1 i
variance matrix of T1 , . . . , TB . Then the percentile method that contains the
smallest UB B(1 ) distances is used to get the closed interval [0, D(UB ) ].
122 3 Building an MLR Model

T
2
If H0 is true and E[] = c, then = 0. Let D0 = T [S T ]1 T and fail to re-
ject H0 if D0 D(UB ) and reject H0 if D0 > D(UB ) . This percentile method
is equivalent to computing the prediction region (3.10) on the wi = Ti and
checking whether 0 is in the prediction region.
Methods for bootstrapping the multiple linear regression model are well
known. The estimated covariance matrix of the (ordinary) least squares esti-
mator is
 OLS ) = M SE(X T X)1 .
Cov(
The residual bootstrap computes the least squares estimator and obtains the
n residuals and tted values r1 , . . . , rn and Y1 , . . . , Yn . Then a sample of size

n is selected with replacement from the residuals resulting in r11 , . . . , rn1 .

Hence the empirical distribution of the residuals is used. Then a vector Y 1 =
(Y11 T
, . . . , Yn1 ) is formed where Yi1 = Yi + ri1

. Then Y 1 is regressed on X

resulting in the estimator 1 . This process is repeated B times resulting in

the estimators 1 , . . . , B . This method should have n 10p so that the
residuals ri are close to the errors ei .
Efron (1982, p. 36) notes that for the residual bootstrap, the sample co-

variance matrix of the i is estimating the population bootstrap matrix
np
M SE(X T X)1 as B . Hence the residual bootstrap standard
n 
np
error SE(i ) SE(i,OLS ).
n
T T
If the z i = (Yi , xi ) are iid observations from some population, then a
sample of size n can be drawn with replacement from z 1 , . . . , z n . Then the
response and predictor variables can be formed into vector Y 1 and design

matrix X 1 . Then Y 1 is regressed on X 1 resulting in the estimator 1 . This

process is repeated B times resulting in the estimators 1 , . . . , B . If the
z i are the rows of a matrix Z, then this nonparametric bootstrap uses the
empirical distribution of the z i .
Following Seber and Lee (2003, p. 100), the classical test statistic for test-
ing H0 : A = c, where A is a full rank r p matrix, is

(A c)T [M SE A(X T X)1 AT ]1 (A c)

FR = ,
r
D
and when H0 is true, rFR 2r for a large class of error distributions. The

sample covariance matrix S w of the wi = A i c is estimating
np
M SE A(X T X)1 AT ,
n
and w 0 when H0 is true. Thus under H0 , the squared distance Di2 =
(wi w)T S 1
w (wi w)
3.4 Variable Selection 123

n
(A c)T [M SE A(X T X)1 AT ]1 (A c),
np
2
and we expect D(U B)
n 2
np r,1 , for large n and B, and p << n.

Returning to variable selection, suppose model I is selected. Then least

squares output for the model Y = X I I + e can be obtained, but the least
squares output is not correct for inference. In particular, M SE(I)(X TI X I )1
is not the correct estimated covariance matrix of I . The selected model tends
to t the data too well, so SE(i ) from the incorrect estimated covariance
matrix tends to be too small. Hence the condence intervals for i are too
short, and hypothesis tests reject H0 : i = 0 too often.
Hastie et al. (2009, p. 57) note that variable selection is a shrinkage es-
timator: the coecients are shrunk to 0 for the omitted variables. Suppose
n 10p. If I is k 1, form I,0 from I by adding 0s corresponding to
the omitted variables. Then I,0 is a nonlinear estimator of , and the resid-
ual bootstrap method can be applied. For example, suppose = Imin ,0
is formed from model Imin that minimizes Cp from some variable selection
method such as forward selection, backward elimination, stepwise selection,
or all subsets variable selection. Instead of computing the least squares esti-
mator from regressing Y i on X, perform variable selection on Y i and X,
t the model that minimizes the criterion, and add 0s corresponding to the

omitted variables, resulting in estimators 1 , . . . , B .
Suppose the variable selection method, such as forward selection or all
subsets, produces K models. Let model Imin be the model that minimizes
the criterion, e.g. Cp (I) or AIC(I). Following Seber and Lee (2003, p. 448)
and Nishi (1984), the probability that model Imin from Cp or AIC underts
goes to zero as n . Since there are a nite number of regression models
I that contain the true model, and each model gives a consistent estimator
I,0 of , the probability that Imin picks one of these models goes to one as
n . Hence Imin ,0 is a consistent estimator of under model (3.4).
D
Note that if S I, and Y = X I I + eI , then n( I I )
Nk (0, I2 W I ) under mild regularity conditions where n(X TI X I )1 W I .
D
Hence n( I,0 ) Np (0, I2 W I,0 ) where the W I,0 has a column and
row of zeroes added for each variable not in I. Note that W I,0 is singular
unless I corresponds to the full model. For example, if p = 3 and model I
uses a constant and x3 with

  W11 0 W12
W11 W12
WI = , then W I,0 = 0 0 0 .
W21 W22
W21 0 W22
D
Hence it is reasonable to conjecture that n( Imin ,0 ) U where
124 3 Building an MLR Model


K
U= i Np (0, I2i W Ii ,0 ),
i=1

K
0 i 1, i=1 i = 1, and K is the number of subsets Ii that contain S.
Inference techniques for the variable selection model have not had much
success. Efron (2014) lets t(Z) be a scalar valued statistic, based on all of the
data Z, that estimates a parameter of interest . Form a bootstrap sample
1 
n
Z i and t(Z i ) for i = 1, . . . , B. Then = s(Z) = t(Z i ), a boot-
B i=1
strap smoothing or bagging estimator. In the regression setting with vari-
able selection, Z i can be formed with the nonparametric or residual boot-
strap using the full model. The prediction region method can also be applied
to t(Z). For example, when A is 1 p, the prediction region method uses

= A c, t(Z) = A c and T = . Efron (2014) used the condence

interval T z1 SE(T ) which is symmetric about T . The prediction re-

gion method uses T ST D(UB ) which is also a symmetric interval centered

at T . If both the prediction region method and Efrons method are large
sample condence intervalsfor , then they have the same asymptotic length
(scaled by multiplying by n), since otherwise the shorter interval will have
lower asymptotic coverage. Since the prediction region interval is a percentile
interval, the shorth(c) interval could have much shorter length than both the
Efron interval and the prediction region interval if the bootstrap distribution
is not symmetric.
The prediction region method can be used for vector valued statistics and
parameters, and may not need the statistic to be asymptotically normal.
These features are likely useful for variable selection models. Prediction in-
tervals and regions can have higher than the nominal coverage 1 if the
distribution is discrete or a mixture of a discrete distribution and some other
distribution. In particular, coverage can be high if the wi distribution is a
mixture of a point mass at 0 and the method checks whether 0 is in the
prediction region. Such a mixture often occurs for variable selection meth-

ods. The bootstrap sample for the Wi = ij can contain many zeroes and be
highly skewed if the jth predictor is weak. Then the computer program may

fail because S w is singular, but if all or nearly all of the ij = 0, then there
is strong evidence that the jth predictor is not needed given that the other
predictors are in the variable selection method.

As an extreme simulation case, suppose ij = 0 for i = 1, . . . , B and for
each run in the simulation. Consider testing H0 : j = 0. Then regardless of
the nominal coverage 1 , the closed interval [0,0] will contain 0 for each
run and the observed coverage will be 1 > 1 . Using the open interval
(0,0) would give observed coverage 0. Also intervals [0, b] and [a, 0] correctly
suggest failing to reject j = 0, while intervals (0, b) and (a, 0) incorrectly
suggest rejecting H0 : j = 0. Hence closed regions and intervals make sense.
3.4 Variable Selection 125

Olive (2016a) showed that applying the prediction region method results
in a large sample 100(1 )% condence region for for a wide variety of
problems, and used the method for variable selection where = .

Example 3.11. Cook and Weisberg (1999a, pp. 351, 433, 447) gives a
data set on 82 mussels sampled o the coast of New Zealand. Let the response
variable be the logarithm log(M ) of the muscle mass, and the predictors are
the length L and height H of the shell in mm, the logarithm log(W ) of the shell
width W, the logarithm log(S) of the shell mass S and a constant. Inference
for the full model is shown along with the shorth(c) nominal 95% condence
intervals for i computed using the nonparametric and residual bootstraps.
As expected, the residual bootstrap intervals are close to the classical least
squares condence intervals i 2SE(i ).
The minimum Cp model from all subsets variable selection uses a constant,
H, and log(S). The shorth(c) nominal 95% condence intervals for i using
the residual bootstrap are shown. Note that the interval for H is right skewed
and contains 0 when closed intervals are used instead of open intervals. The
least squares output is also shown, but should only be used for inference if
the model was selected before looking at the data.
It was expected that log(S) may be the only predictor needed, along with
a constant, since log(S) and log(M ) are both log(mass) measurements and
likely highly correlated. Hence we want to test H0 : 2 = 3 = 4 = 0 with
the Imin model selected by all subsets variable selection. (Of course this test
would be easy to do with the full model using least squares theory.) Then
H0 : A = (2 , 3 , 4 )T = 0. Using the prediction region method  with the
full model gave an interval [0,2.930] with D0 = 1.641. Note that 23,0.95 =
2.795. So fail to reject H0 . Using the prediction region method with the Imin
variable selection model had [0, D(UB ) ] = [0, 3.293] while D0 = 1.134. So fail
to reject H0 .

large sample full model inference

Est. SE t Pr(>|t|) rowboot resboot
i -1.249 0.838 -1.49 0.14 [-2.93,-0.048][-3.138,0.194]
L -0.001 0.002 -0.28 0.78 [-0.005,0.003][-0.005,0.004]
W 0.130 0.374 0.35 0.73 [-0.384,0.827][-0.555,0.971]
H 0.008 0.005 1.50 0.14 [-0.002,0.018][-0.003,0.017]
S 0.640 0.169 3.80 0.00 [ 0.188,1.001][ 0.276,0.955]
output and shorth intervals for the min Cp submodel
Est. SE t Pr(>|t|) 95% shorth CI
int -0.9573 0.1519 -6.3018 0.0000 [-2.769, 0.460]
L 0 [-0.004, 0.004]
W 0 [-0.595, 0.869]
H 0.0072 0.0047 1.5490 0.1254 [ 0.000, 0.016]
S 0.6530 0.1160 5.6297 0.0000 [ 0.324, 0.913]
126 3 Building an MLR Model

The R code used to produce the above output is shown below.

library(leaps)
y <- log(mussels[,5]); x <- mussels[,1:4]
x[,4] <- log(x[,4]); x[,2] <- log(x[,2])
out <- regboot(x,y,B=1000)
tem <- rowboot(x,y,B=1000)
outvs <- vselboot(x,y,B=1000) #get bootstrap CIs,
apply(out$betas,2,shorth3);
apply(tem$betas,2,shorth3);
apply(outvs$betas,2,shorth3)
ls.print(outvs$full)
ls.print(outvs$sub)
#test if beta_2 = beta_3 = beta_4 = 0
Abeta <- out$betas[,2:4]
#prediction region method with residual bootstrap
predreg(Abeta)
Abeta <- outvs$betas[,2:4]
#prediction region method with Imin
predreg(Abeta)

Example 3.12. Consider the Gladstone (1905) data set where the vari-
ables are as in Problem 3.6. Output is shown below for the full model and the
bootstrapped minimum Cp forward selection estimator. Note that the shorth
intervals for length and sex are quite long. These variables are often in and
often deleted from the bootstrap forward selection model. Output for II is
also shown. For this data set, II = Imin .

large sample full model inference for Ex. 3.12

Estimate SE t Pr(>|t|) resboot
Int -3021.255 1701.070 -1.77 0.077 [-6549.8,322.79]
age -1.656 0.314 -5.27 0.000 [ -2.304,-1.050]
breadth -8.717 12.025 -0.72 0.469 [-34.229,14.458]
cephalic 21.876 22.029 0.99 0.322 [-20.911,67.705]
circum 0.852 0.529 1.61 0.109 [ -0.065, 1.879]
headht 7.385 1.225 6.03 0.000 [ 5.138, 9.794]
height -0.407 0.942 -0.43 0.666 [ -2.211, 1.565]
len 13.475 9.422 1.43 0.154 [ -5.519,32.605]
sex 25.130 10.015 2.51 0.013 [ 6.717,44.19]
output and shorth intervals for the min Cp submodel
Estimate SE t Pr(>|t|) 95% shorth CI
Int -1764.516 186.046 -9.48 0.000 [-6151.6,-415.4]
age -1.708 0.285 -5.99 0.000 [ -2.299,-1.068]
breadth 0 [-32.992, 8.148]
cephalic 5.958 2.089 2.85 0.005 [-10.859,62.679]
circum 0.757 0.512 1.48 0.140 [ 0.000, 1.817]
3.4 Variable Selection 127

headht 7.424 1.161 6.39 0.000 [ 5.028, 9.732]

height 0 [ -2.859, 0.000]
len 6.716 1.466 4.58 0.000 [ 0.000,30.508]
sex 25.313 9.920 2.55 0.011 [ 0.000,42.144]
output and shorth for I_I model
Estimate SE t Pr(>|t|) 95% shorth CI
Int -1764.516 186.046 -9.48 0.000 [-6104.9,-778.2]
age -1.708 0.285 -5.99 0.000 [ -2.259,-1.003]
breadth 0 [-31.012, 6.567]
cephalic 5.958 2.089 2.85 0.005 [ -6.700,61.265]
circum 0.757 0.512 1.48 0.140 [ 0.000, 1.866]
headht 7.424 1.161 6.39 0.000 [ 5.221,10.090]
height 0 [ -2.173, 0.000]
len 6.716 1.466 4.58 0.000 [ 0.000,28.819]
sex 25.313 9.920 2.55 0.011 [ 0.000,42.847]
The R code used to produce the above output is shown below. The last
four commands are useful for examining the variable selection output.
x<-cbrainx[,c(1,3,5,6,7,8,9,10)]
y<-cbrainy
library(leaps)
out <- regboot(x,y,B=1000)
outvs <- fselboot(x,cbrainy) #get bootstrap CIs,
apply(out$betas,2,shorth3);
apply(outvs$betas,2,shorth3)
ls.print(outvs$full)
ls.print(outvs$sub)
outvs <- modIboot(x,cbrainy) #get bootstrap CIs,
apply(out$betas,2,shorth3);
apply(outvs$betas,2,shorth3)
ls.print(outvs$sub)
tem<-regsubsets(x,y,method="forward")
tem2<-summary(tem)
tem2$which
tem2$cp
A small simulation study was done in R using B = max(1000, n, 20p) and
5000 runs. The regression model used = (1, 1, 0, 0)T with n = 100, p = 4,
and various zero mean iid error distributions. The design matrix X consisted
of iid N(0,1) random variables. Hence the full model least
squares condence
intervals for i should have length near 2t96,0.975 / n 2(1.96)/10 =
0.392 when the iid zero mean errors have variance 2 . The simulation com-
puted the shorth(c) interval for each i and used the prediction region method
to test H0 : 3 = 4 = 0. The nominal coverage was 0.95 with = 0.05. Ob-
served coverage between 0.94 and 0.96 would suggest coverage is close to the
nominal value.
128 3 Building an MLR Model

The regression models used the residual bootstrap on the full model least
squares estimator and on the all subsets variable selection estimator for the
model Imin . The residuals were from least squares applied to the full model
in both cases. Results are shown for when the iid errors ei N (0, 1). Ta-
ble 3.2 shows two rows for each model giving the observed condence interval
coverages and average lengths of the condence intervals. The term reg is
for the full model regression, and the term vs is for the all subsets variable
selection. The column for the test gives the length and coverage = P(fail
to reject H0 ) for the interval [0, D(UB ) ] where D(UB ) is the cuto for the
condence region. The volume of the condence
 region will decrease to 0 as
n . The cuto will often be near r,0.95 if the statistic T is asymp-
2

totically normal. Note that 22,0.95 = 2.448 is very close to 2.449 for the
full model regression bootstrap test. The coverages were near 0.95 for the
regression bootstrap on the full model. For Imin the coverages were near 0.95
for 1 and 2 , but higher for the other 3 tests since zeroes often occurred for
j for j = 3, 4. The average lengths and coverages were similar for the full
model and all subsets variable selection Imin for 1 and 2 , but the lengths
were shorter for Imin for 3 and 4 .

Table 3.2 Bootstrapping Regression and Variable Selection

model cov/len 1 2 3 4 test
reg cov 0.9496 0.9430 0.9440 0.9454 0.9414
len 0.3967 0.3996 0.3997 0.3997 2.4493
vs cov 0.9482 0.9486 0.9974 0.9974 0.9896
len 0.3965 0.3990 0.3241 0.3257 2.6901

The R code for the simulation is shown below.

regbootsim(nruns=5000) #takes a while
library(leaps)
vsbootsim(nruns=5000) #takes a long while
vsbootsim2(nruns=5000) #bootstraps forwards selection
Remark 3.3. Predictor transformations can be done as long as the re-
sponse variable is not used. Suppose the p predictors are selected and variable
selection is done. Use the prediction region method for exploratory testing.
Olive (2013a) gives prediction intervals for models of the form Y = m(x) + e.
The variable selection model is such a model, so use the Olive (2013a) PI
after automated variable selection using Cp or AIC.
The Olive (2013a) PI has
 
15 n + 2p
bn = 1 + . (3.11)
n np

Let qn = min(1 + 0.05, 1 + p/n) for > 0.1 and

3.5 Diagnostics 129

qn = min(1 /2, 1 + 10p/n), otherwise. (3.12)

If 1 < 0.999 and qn < 1 + 0.001, set qn = 1 .

Let c = nqn . Compute r(c) r(1) , r(c+1) r(2) , . . . , r(n) r(nc+1) . Let
(r(d) , r(d+c1) ) = (1 , 12 ) correspond to the interval with the smallest
length. Then the asymptotically optimal 100 (1 )% large sample PI for
Yf is
(m(xf ) + bn 1 , m(xf ) + bn 12 ), (3.13)
Let Y = xT + e where is p 1, but let m(x) = xTImin Imin using
the minimum Cp model Imin from forward selection. The lregpack function
vspisim simulates (3.13) when = (1, 1, . . . , 1, 0, . . . , 0)T where the rst
k + 1 entries of the p 1 vector are 1s, for various error distributions using
the nominal 95% PI. This simulation is similar to the full model simulation
done under Remark 2.8. With 5000 runs, p = 4, k = 1, and N (0, 1) errors,
the asymptotic length is 3.92 = 2(1.96). With n = 40, the coverage was
0.9858 with average length 7.7557. With n = 80, the coverage was 0.979 with
average length 5.0278. With n = 200, the coverage was 0.966 with average
length 4.2852. With n = 400, the coverage was 0.958 with average length
4.081. After variable selection, coverage starts to be good for n 10p, but
the PI length was not near the optimal asymptotic length until n 100p.
Use the following R code.
library(leaps)
vspisim(n=40,p=4,k=1,type=1,nruns=5000)

3.5 Diagnostics

Automatic or blind use of regression models, especially in exploratory work,

all too often leads to incorrect or meaningless results and to confusion
rather than insight. At the very least, a user should be prepared to make and
study a number of plots before, during, and after tting the model.
Chambers et al. (1983, p. 306)
Diagnostics are used to check whether model assumptions are reasonable.
This section focuses on diagnostics for the unimodal MLR model Yi = xTi +ei
for i = 1, . . . , n where the errors are iid from a unimodal distribution that is
not highly skewed with E(ei ) = 0 and VAR(ei ) = 2 . See Denition 2.6.
It is often useful to use notation to separate the constant from the non-
trivial predictors. Assume that xi = (1, xi,2 , . . . , xi,p )T (1, uTi )T where the
(p 1) 1 vector of nontrivial predictors ui = (xi,2 , . . . , xi,p )T . In matrix
form,
Y = X + e,
X = [X1 , X2 , . . . , Xp ] = [1, U ],
130 3 Building an MLR Model

1 is an n 1 vector of ones, and U = [X2 , . . . , Xp ] is the n (p 1) matrix

of nontrivial predictors. The kth column of U is the n 1 vector of the jth
predictor Xj = (x1,j , . . . , xn,j )T where j = k + 1. The sample mean and
covariance matrix of the nontrivial predictors are

1
n
u= ui (3.14)
n i=1

and
1 
n
C = Cov(U ) = (ui u)(ui u)T , (3.15)
n 1 i=1

respectively, where uTi is the ith row of U .

Some important numerical quantities that are used as diagnostics measure


the distance of ui from u and the inuence of case i on the OLS t 
OLS .
  =
The ith residual ri = Yi Yi , and the vector of tted values is Y = X
T 1 T
X(X X) X Y = HY where H is the hat matrix. Case (or leave one out
or deletion) diagnostics are computed by omitting the ith case from the OLS
regression. Following Cook and Weisberg (1999a, p. 357), let


Y (i) = X (3.16)
(i)

denote the n 1 vector of tted values from estimating with OLS without
the ith case. Denote the jth element of Y (i) by Y(i),j . It can be shown that
the variance of the ith residual VAR(ri ) = 2 (1 hi ). The usual estimator
of the error variance is n 2
r
 = i=1 i .
2
np
The (internally) studentized residual
r
ei = i
 1 hi

has zero mean and approximately unit variance.

Denition 3.11. The ith leverage hi = H ii is the ith diagonal element

of the hat matrix H. The ith squared (classical) Mahalanobis distance

MD2i = (ui u)T C 1 (ui u).

The ith Cooks distance

 )
(  T X T X(
 ) (Y (i) Y )T (Y (i) Y )
(i) (i)
CDi = = (3.17)
2
p 2
p
3.5 Diagnostics 131

1  
n
= (Y(i),j Yj )2 .
2 j=1
p

Proposition 3.4. a) (Rousseeuw and Leroy 1987, p. 225)

1 1
hi = MD2i + .
n1 n
b) (Cook and Weisberg 1999a, p. 184)
1
hi = xTi (X T X)1 xi = (xi x)T (U T U )1 (xi x) + .
n
c) (Cook and Weisberg 1999a, p. 360)

ri2 hi e2i hi
CDi = = .
2 (1 hi ) 1 hi
p p 1 hi

When the statistics CDi , hi , and MDi are large, case i may be an outlier or
inuential case. Examining a stem plot or dot plot of these three statistics for
unusually large values can be useful for agging inuential cases. Cook and
Weisberg (1999a, p. 358) suggest examining cases with CDi > 0.5 and that
cases with CDi > 1 should always be studied. Since H = H T and H = HH,
the hat matrix is symmetric andnidempotent. Hence the eigenvalues of H are
zero or one, and trace(H) = i=1 hi = p. It can be shown that 0 hi 1.
Rousseeuw and Leroy (1987, pp. 220, 224) suggest using hi > 2p/n and
MD2i > 2p1,0.95 as benchmarks for leverages and Mahalanobis distances
where 2p1,0.95 is the 95th percentile of a chisquare distribution with p 1
degrees of freedom.
Note that Proposition 3.4c) implies that Cooks distance is the product
of the squared residual and a quantity that becomes larger the farther ui
is from u. Hence inuence is roughly the product of leverage and distance
of Yi from Yi (see Fox 1991, p. 21). Mahalanobis distances and leverages
both dene hyperellipsoids based on a metric closely related to the sample
covariance matrix of the nontrivial predictors. All points ui on the same
hyperellipsoidal contour are the same distance from u and have the same
leverage (or the same Mahalanobis distance).
Cooks distances, leverages, and Mahalanobis distances can be eective for
nding inuential cases when there is a single outlier, but can fail if there
are two or more outliers. Nevertheless, these numerical diagnostics combined
with response and residual plots are probably the most eective techniques for
detecting cases that eect the tted values when the multiple linear regression
model is a good approximation for the bulk of the data.
A scatterplot of x versus y (recall the convention that a plot of x versus y
means that x is on the horizontal axis and y is on the vertical axis) is used to
132 3 Building an MLR Model

visualize the conditional distribution y|x of y given x (see Cook and Weisberg
1999a, p. 31). For the simple linear regression model (with one nontrivial
predictor x2 ), the most eective technique for checking the assumptions of
the model is to make a scatterplot of x2 versus Y and a residual plot of x2
versus ri . Departures from linearity in the scatterplot suggest that the simple
linear regression model is not adequate. The points in the residual plot should
scatter about the line r = 0 with no pattern. If curvature is present or if the
distribution of the residuals depends on the value of x2 , then the simple linear
regression model is not adequate.
In general there is more than one nontrivial predictor and in this setting
two plots are crucial for any multiple linear regression analysis, re-
gardless of the regression estimator (e.g., OLS, L1 etc.). The rst plot is the
residual plot of the tted values Yi versus the residuals ri , and the second
plot is the response plot of the tted values Yi versus the response Yi .

Recalling Denitions 2.11 and 2.12, residual and response plots are plots of
wi = xTi versus ri and Yi , respectively, where is a known p 1 vector. The
most commonly used residual and response plots takes = .  Plots against
the individual predictors xj and potential predictors are also used. If the
residual plot is not ellipsoidal with zero slope, then the unimodal MLR model
(with iid errors from a unimodal distribution that is not highly skewed) is
not sustained. In other words, if the variables in the residual plot show some
type of dependency, e.g. increasing variance or a curved pattern, then the
multiple linear regression model may be inadequate. Proposition 2.1 showed
that the response plot simultaneously displays the tted values, response, and
residuals. The plotted points in the response plot should scatter about the
identity line if the multiple linear regression model holds. Recall that residual
plots magnify departures from the model while the response plot emphasizes
how well the model ts the data.

When the bulk of the data follows the MLR model, the following rules of
thumb are useful for nding inuential cases and outliers from the response
and residual plots. Look for points with large absolute residuals and for points
far away from Y . Also look for gaps separating the data into clusters. To
determine whether small clusters are outliers or good leverage points, give
zero weight to the clusters, and t an MLR estimator to the bulk of the
data. Denote the weighted estimator by w . Then plot Yw versus Y using
the entire data set. If the identity line passes through the bulk of the data
but not the cluster, then the cluster points may be outliers. In Figure 3.7,
the 5 infants are good leverage points in that the t to the bulk of the data
passes through the cluster of infants. For the Buxton (1920) data, the cluster
of cases far from the bulk of the data in Figure 3.11 are outliers.
To see why gaps are important, recall that the coecient of determination
R2 is equal to the squared correlation (corr(Y, Y ))2 . R2 over emphasizes the
3.6 Outlier Detection 133

strength of the MLR relationship when there are two clusters of data since
much of the variability of Y is due to the smaller cluster.

Information from numerical diagnostics can be incorporated into the re-

sponse plot by highlighting cases that have large absolute values of the di-
agnostic. For example, the Cooks distance CDi for the ith case tends to be
large if Yi is far from the sample mean Y and if the corresponding absolute
residual |ri | is not small. If Yi is close to Y , then CDi tends to be small
unless |ri | is large. Thus cases with large Cooks distances can often be found
by examining the response and residual plots. An exception to these rules
of thumb occurs if a group of cases form a cluster and the OLS t passes
through the cluster. Then the CDi s corresponding to these cases tend to be
small even if the cluster is far from Y .

Example 3.13. Tremearne (1911) presents a data set of about 17 mea-

surements on 115 people of Hausa nationality. We deleted 3 cases because
of missing values and used height as the response variable Y . The ve pre-
dictor variables used were height when sitting, height when kneeling, head
length, nasal breadth, and span (perhaps from left hand to right hand). Fig-
ure 2.1 presents the OLS residual and response plots for this data set. Points
corresponding to cases with Cooks distance > min(0.5, 2p/n) are shown as
highlighted squares (cases 3, 44, and 63). The 3rd person was very tall while
the 44th person was rather short. From the plots, the standard deviation of
the residuals appears to be around 10. Hence cases 3 and 44 are certainly
worth examining. Two other cases have residuals near fty.
Data sets like this one are very common. The majority of the cases seem to
follow a multiple linear regression model with iid Gaussian errors, but a small
percentage of cases seem to come from an error distribution with heavier tails
than a Gaussian distribution.

3.6 Outlier Detection

Do not attempt to build a model on a set of poor data! In human surveys,

one often nds 14inch men, 1000pound women, students with no lungs,
and so on. In manufacturing data, one can nd 10,000 pounds of material
in a 100 pound capacity barrel, and similar obvious errors. All the planning,
and training in the world will not eliminate these sorts of problems. . . . In
our decades of experience with messy data, we have yet to nd a large
data set completely free of such quality problems.
Draper and Smith (1981, p. 418)
134 3 Building an MLR Model

There is an enormous literature on outlier detection in multiple linear

regression. Typically a numerical measure such as Cooks distance or a resid-
ual plot based on resistant ts is used. The following terms are frequently
encountered.

Denition 3.12. Outliers are cases that lie far from the bulk of the data.
Hence Y outliers are cases that have unusually large vertical distances from
the MLR t to the bulk of the data while x outliers are cases with predictors
x that lie far from the bulk of the xi . Suppose that some analysis to detect
outliers is performed. Masking occurs if the analysis suggests that one or
more outliers are in fact good cases. Swamping occurs if the analysis suggests
that one or more good cases are outliers.

The residual and response plots are very useful for detecting outliers. If
there is a cluster of cases with outlying Y s, the identity line will often pass
through the outliers. If there are two clusters with similar Y s, then the two
plots may fail to show the clusters. Then using methods to detect x outliers
may be useful.
Let the q continuous predictors in the MLR model be collected into vectors
ui for i = 1, . . . , n. Let the n q matrix W have n rows uT1 , . . . , uTn . Let the
q 1 column vector T (W ) be a multivariate location estimator, and let the
q q symmetric positive denite matrix C(W ) be a covariance estimator.
Often q = p 1 and only the constant is omitted from xi to create ui .

Denition 3.13. The ith squared Mahalanobis distance is

Di2 = Di2 (T (W ), C(W )) = (ui T (W ))T C 1 (W )(ui T (W )) (3.18)

for each point ui . Notice that Di2 is a random variable (scalar valued).

The classical Mahalanobis distance corresponds to the sample mean and

sample covariance matrix

1
n
T (W ) = u = ui ,
n i=1

and
1 
n
C(W ) = S = (ui T (W ))(ui T (W ))T
n 1 i=1

and will bedenoted by M Di . When T (W ) and C(W ) are robust estima-

tors, Di = Di2 will sometimes be denoted by RDi . We suggest using the
Olive (2008) RFCH or RMVNestimator as the robust estimator. The sample
Mahalanobis distance Di = Di2 is an analog of the absolute value of the
sample z-score |zi | = |(Yi Y )/|. Also notice that the Euclidean distance of
ui from the estimate of center T (W ) is Di (T (W ), I q ) where I q is the q q
identity matrix. Plot the M Di versus the RDi to detect outlying u.
3.6 Outlier Detection 135

Denition 3.14: Rousseeuw and Van Driessen (1999). The DD plot

is a plot of the classical Mahalanobis distances MDi versus robust Maha-
lanobis distances RDi . The DD plot is best for n 20p.

Olive (2002) shows that the plotted points in the DD plot will follow the
identity line with zero intercept and unit slope if the predictor distribution
is multivariate normal (MVN), and will follow a line with zero intercept but
nonunit slope if the distribution is elliptically contoured with nonsingular
covariance matrix but not MVN. (Such distributions have linear scatterplot
matrices. See Chapter 10.) Hence if the plotted points in the DD plot follow
some line through the origin, then there is some evidence that outliers and
strong nonlinearities have been removed from the predictors.

Response Plot
500 1000 1500
Y

61
62
0

0 500 1000 1500

FIT
Residual Plot
-50 0 50 100

61
RES

64
63
65 62
-150

0 500 1000 1500

FIT

Fig. 3.11 Residual and Response Plots for Buxton Data

Example 3.14. Buxton (1920, pp. 2325) gives 20 measurements of 88

men. We chose to predict stature using an intercept, head length, nasal height,
bigonal breadth, and cephalic index. Observation 9 was deleted since it had
missing values. Five individuals, numbers 6266, were reported to be about
0.75 inches tall with head lengths well over ve feet! This appears to be a
clerical error; these individuals stature was recorded as head length and the
integer 18 or 19 given for stature, making the cases massive outliers with
enormous leverage.
136 3 Building an MLR Model

Figure 3.11 shows the response plot and residual plot for the Buxton
data. Although an index plot of Cooks distance CDi may be useful for
agging inuential cases, the index plot provides no direct way of judg-
ing the model against the data. As a remedy, cases in the response plot
with CDi > min(0.5, 2p/n) were highlighted. Notice that the OLS t passes
through the outliers, but the response plot is resistant to Y outliers since Y
is on the vertical axis. Also notice that although the outlying cluster is far
from Y , only two of the outliers had large Cooks distance. Hence masking
occurred for both Cooks distances and for OLS residuals, but not for OLS
tted values.

a) DD Plot with Outliers b) DD Plot without Outliers

5
250

4
150
RD

RD
3
2
50

1
0

1 2 3 4 5 0.5 1.5 2.5 3.5

MD MD

Fig. 3.12 DD Plots for Buxton Data

Figure 3.12a shows the DD plot made from the four predictors head length,
nasal height, bigonal breadth, and cephalic index. The ve massive outliers cor-
respond to head lengths that were recorded to be around 5 feet. Figure 3.12b
is the DD plot computed after deleting these points and suggests that the
predictor distribution is now much closer to a multivariate normal distribu-
tion.
High leverage outliers are a particular challenge to conventional numer-
ical MLR diagnostics such as Cooks distance, but can often be visualized
using the response and residual plots. The following techniques are useful for
detecting outliers when the multiple linear regression model is appropriate.
1. Find the OLS residuals and tted values and make a response plot and
a residual plot. Look for clusters of points that are separated from the
bulk of the data and look for residuals that have large absolute values.
Beginners frequently label too many points as outliers. Try to estimate
the standard deviation of the residuals in both plots. In the residual plot,
3.6 Outlier Detection 137

look for residuals that are more than 5 standard deviations away from
the r = 0 line. The identity line and r = 0 line may pass right through a
cluster of outliers, but the cluster of outliers can often be detected because
there is a large gap between the cluster and the bulk of the data, as in
Figure 3.11.
2. Make a DD plot of the predictors that take on many values (the continuous
predictors).
3. Make a scatterplot matrix of several diagnostics such as leverages, Cooks
distances, and studentized residuals.

Detecting outliers is much easier than deciding what to do with them.

After detection, the investigator should see whether the outliers are recording
errors. The outliers may become good cases after they are corrected. But
frequently there is no simple explanation for why the cases are outlying.
Typical advice is that outlying cases should never be blindly deleted and that
the investigator should analyze the full data set including the outliers as well
as the data set after the outliers have been removed (either by deleting the
cases or the variables that contain the outliers).
Typically two methods are used to nd the cases (or variables) to delete.
The investigator computes OLS diagnostics and subjectively deletes cases,
or a resistant multiple linear regression estimator is used that automatically
gives certain cases zero weight. A third, much more eective method is to use
the response and residual plots.
Suppose that the data has been examined, recording errors corrected, and
impossible cases deleted. For example, in the Buxton (1920) data, 5 people
with heights of 0.75 inches were recorded. For this data set, these heights
could be corrected. If they could not be corrected, then these cases should
be discarded since they are impossible. If outliers are present even after cor-
recting recording errors and discarding impossible cases, then we can add an
additional rough guideline.
If the purpose is to display the relationship between the predictors and the
response, make a response plot using the full data set (computing the tted
values by giving the outliers weight zero) and using the data set with the
outliers removed. Both plots are needed if the relationship that holds for
the bulk of the data is obscured by outliers. The outliers are removed from
the data set in order to get reliable estimates for the bulk of the data. The
identity line should be added as a visual aid and the proportion of outliers
should be given.
138 3 Building an MLR Model

3.7 Summary

1) Suppose you have a scatterplot of two variables x1 1 versus x2 2 with

x1 , x2 > 0, and that the plotted points follow a nonlinear one to one func-
tion. Consider the ladder of powers 1, 0.5, 1/3, 0, 1/3, 0.5, and 1. The
ladder rule says to spread small values of the variable, make i smaller. To
spread large values of the variable, make i larger.
2) Suppose w is positive. The log rule says use log(w) if
max(wi )/ min(wi ) > 10.
3) There are several guidelines for choosing power transformations. First,
see rules 1) and 2) above. Suppose that all values of the variable w to be
transformed are positive. The log rule often works wonders on the data.
If the variable w can take on the value of 0, use log(w + c) where c is a
small constant like 1, 1/2, or 3/8. The unit rule says that if Xi and y
have the same units, then use the same transformation of Xi and y. The
cube root rule says that if w is a volume measurement, then the cube root
transformation w1/3 may be useful. Consider the ladder of powers given in
point 1). No transformation ( = 1) is best, then the log transformation,
then the square root transformation. Theory, if available, should be used to
select a transformation. Frequently more than one transformation will work.
1/3
For example, if y = weight and X1 = volume = X2 X3 X4 , then y vs. X1
or log(y) vs. log(X1 ) = log(X2 ) + log(X3 ) + log(X4 ) may both work. Also if
y is linearly related with X2 , X3 , X4 and these three variables all have length
1/3
units mm, say, then the units of X1 are (mm)3 . Hence the units of X1 are
mm.
4) To nd a response transformation, make the transformation plots
and choose a transformation such that the transformation plot is linear.
5) A factor (with c levels a1 , . . . , ac ) is incorporated into the MLR model
by using c 1 indicator variables xW j = 1 if W = aj and xW j = 0 otherwise,
where one of the levels aj is omitted, e.g. use j = 1, . . . , c 1.
6) For variable selection, the model Y = xT + e that uses all of the
predictors is called the full model. A model Y = xTI I + e that only uses a
subset xI of the predictors is called a submodel. The full model is always
a submodel. The full model has SP = xT and the submodel has SP =
xTI I .
7) Make scatterplot matrices of the predictors and the response. Then re-
move strong nonlinearities from the predictors using power trans-
formations. The log rule is very useful.
8) Either include all of the indicator variables for a factor in the model or
exclude all of them. If the model contains powers or interactions, also include
all main eects in the model.
9) After selecting a submodel I, make the response and residual plots for
the full model and the submodel. Make the RR plot of rI,i versus ri and the
FF plot of YI,i versus Yi . The submodel is good if the plotted points in the
3.7 Summary 139

FF and RR plots cluster tightly about the identity line. In the RR plot, the
OLS line and identity line can be added to the plot as visual aids. It should
be dicult to see that the OLS and identity lines intersect at the origin, so
the two lines should nearly coincide at the origin. If the FF plot looks good
but the RR plot does not, the submodel may be good if the main goal of the
analysis is for prediction.
10) Forward selection Step 1) k = 1: Start with a constant w1 = x1 .
Step 2) k = 2: Compute Cp for all models with k = 2 containing a constant
and a single predictor xi . Keep the predictor w2 = xj , say, that minimizes Cp .
Step 3) k = 3: Fit all models with k = 3 that contain w1 and w2 . Keep the
predictor w3 that minimizes Cp . . . .
Step j) k = j: Fit all models with k = j that contains w1 , w2 , . . . , wj1 . Keep
the predictor wj that minimizes Cp . . . .
Step p): Fit the full model.

Backward elimination: All models contain a constant = u1 . Step 0)

k = p: Start with the full model that contains x1 , . . . , xp . We will also say
that the full model contains u1 , . . . , up where u1 = x1 but ui need not equal
xi for i > 1.
Step 1) k = p 1: Fit each model with k = p 1 predictors including a
constant. Delete the predictor up , say, that corresponds to the model with
the smallest Cp . Keep u1 , . . . , up1 .
Step 2) k = p 2: Fit each model with p 2 predictors including a con-
stant. Delete the predictor up1 corresponding to the smallest Cp . Keep
u1 , . . . , up2 . . . .
Step j) k = p j: t each model with p j predictors including a con-
stant. Delete the predictor upj+1 corresponding to the smallest Cp . Keep
u1 , . . . , upj . . . .
Step p 2) k = 2. The current model contains u1 , u2 , and u3 . Fit the model
u1 , u2 and the model u1 , u3 . Assume that model u1 , u2 minimizes Cp . Then
delete u3 and keep u1 and u2 .

11) Let Imin correspond to the submodel with the smallest Cp . Find
the submodel II with the fewest number of predictors such that Cp (II )
Cp (Imin ) + 1. Then II is the initial submodel that should be examined. It
is possible that II = Imin or that II is the full model. Models I with fewer
predictors than II such that Cp (I) Cp (Imin ) + 4 are interesting and should
also be examined. Models I with k predictors, including a constant and with
fewer predictors than II such that Cp (Imin ) + 4 < Cp (I) min(2k, p) should
be checked.
12) There are several guidelines for building an MLR model. Suppose
that variable Z is of interest and variables W1 , . . . , Wr have been collected
along with Z. Make a scatterplot matrix of W1 , . . . , Wr and Z. (If r is large,
several matrices may need to be made. Each one should include Z.) Remove
or correct any gross outliers. It is often a good idea to transform the Wi
to remove any strong nonlinearities from the predictors. Eventually
140 3 Building an MLR Model

you will nd a response variable Y = tZ (Z) and nontrivial predictor variables

X2 , . . . , Xp for the full model. Interactions such as Xk = Wi Wj and powers
such as Xk = Wi2 may be of interest. Indicator variables are often used in
interactions, but do not transform an indicator variable. The response plot
for the full model should be linear, and the residual plot should be ellipsoidal
with zero trend. Find the LS output. Often want the number of predictors
k in the submodel to be small. We will almost always include a constant in
the submodel. If the submodel seems to be good, make the response plot and
residual plot for the submodel. They should be linear and ellipsoidal with
zero trend, respectively. From the output, see if any terms can be eliminated
(look for predictors Xi such that the pvalue > 0.01, 0.05, or 0.1 for Ho:
i = 0). Also see point 13) below.
13) Assume that the full model has good response and residual plots and
than n 10p. Let subset I have k predictors, including a constant. The
following rules of thumb may be useful, but may not all hold simultaneously.
Let Imin be the minimum Cp model and let II be the model with the fewest
predictors satisfying Cp (II ) Cp (Imin ) + 1. Do not use more predictors than
model II to avoid overtting. Then the submodel I is good if
i) the response and residual plots for the submodel looks like the response
and residual plots for the full model,
ii) corr(ESP,ESP(I)) = corr(Y, YI ) 0.95.
iii) The plotted points in the FF plot cluster tightly about the identity line.
iv) Want the p-value 0.01 for the partial F test that uses I as the reduced
model.
v) The plotted points in the RR plot cluster tightly about the identity line.
vi) Want R2 (I) > 0.9R2 and R2 (I) > R2 0.07 (recall that R2 (I) R2 (f ull)
since adding predictors to I does not decrease R2 (I)).
vii) Want Cp (Imin ) Cp (I) min(2k, p) with no big jumps in Cp (the
increase should be less than four) as variables are deleted.
viii) Want hardly any predictors with p-values > 0.05.
ix) Want few predictors with p-values between 0.01 and 0.05.
14) Always check that the full model is good. If the candidate model seems
to be good, the usual MLR checks should still be made. In particular, the
response plot and residual plot need to be made for the submodel.
15) Inuence is roughly (leverage)(discrepancy). The leverages hi are
the diagonal elements of the hat matrix H and measure how far xi is from
the sample mean of the predictors. Cooks distance is widely used, but the
response plot and residual plot are the most eective tools for detecting
outliers and inuential cases.
3.8 Complements 141

3.8 Complements

With one data set, OLS is a great place to start but a bad place to end. If
n = 5kp where k > 2, it may be useful to take a random sample of n/k cases
to build the MLR model. Then check the model on the full data set.

Predictor Transformations

One of the most useful techniques in regression is to remove

gross nonlinearities in the predictors by using predictor transfor-
mations. The log rule is very useful for transforming highly skewed
predictors. The linearizing of the predictor relationships could be done by
using marginal power transformations or by transforming the joint distribu-
tion of the predictors towards an elliptically contoured distribution. The lin-
earization might also be done by using simultaneous power transformations
= (2 , . . . , p )T of the predictors so that the vector w = (x2 2 , . . . , xp p )T
( ) ( )

of transformed predictors is approximately multivariate normal. A method

for doing this was developed by Velilla (1993). (The basic idea is the same
as that underlying the likelihood approach of Box and Cox for estimating a
power transformation of the response in regression, but the likelihood comes
from the assumed multivariate normal distribution of w .) The Cook and
Nachtsheim (1994) procedure can cause the distribution to be closer to ellip-
tical symmetry. Marginal Box-Cox transformations also seem to be eective.
Power transformations can also be selected with slider bars in Arc.
Suppose that it is thought that the model Y = xT + e could be improved
by transforming xj . Let xT = uT + j xj where uT = x1 1 + +
xj1 j1 + xj+1 j+1 + + xp p . Let (xj ) denote the unknown transfor-
mation.

Denition 3.15. Consider the OLS residuals ri (j) = Yi uTi obtained

from the OLS regression of Y on u. A partial residual plot or component plus
residual plot or ceres plot with linear augmentation is a plot of the ri (j) versus
xj and is used to visualize .

Cook (1993) shows that partial residual plots are useful for visualizing
provided that the plots of xi versus xj are linear. More general ceres plots, in
particular ceres plots with smooth augmentation, can be used to visualize
if Y = uT + (x j )p + e but the linearity condition fails. Fitting the additive
model Y = 1 + j=2 Sj (xj ) + e or Y = 1 + 2 x2 + + j1 xj1 + S(xj ) +
j+1 xj+1 + + p xp + e and plotting S(xj ) can be useful. Similar ideas are
also useful for GLMs. See Chapter 13 and Olive (2013b) which also discusses
response plots for many regression models.

The assumption that all values of x1 and x2 are positive for power trans-
formation can be removed by using the modied power transformations of
Yeo and Johnson (2000).
142 3 Building an MLR Model

Response Transformations

Application 3.1 was suggested by Olive (2004b, 2013b) for additive error
regression models Y = m(x) + e. An advantage of this graphical method is
that it works for linear models: that is, for multiple linear regression and for
many experimental design models. Notice that if the plotted points in the
transformation plot follow the identity line, then the plot is also a response
plot. The method is also easily performed for MLR methods other than least
squares.
A variant of the method would plot the residual plot or both the response
and the residual plot for each of the seven values of . Residual plots are also
useful, but they no not distinguish between nonlinear monotone relationships
and nonmonotone relationships. See Fox (1991, p. 55).
Cook and Olive (2001) also suggest a graphical method for selecting and
assessing response transformations under model (3.2). Cook and Weisberg
(1994) show that a plot of Z versus xT (swap the axis on the transformation
plot for = 1) can be used to visualize t if Y = t(Z) = xT + e, suggesting
that t1 can be visualized in a plot of xT versus Z.
If there is nonlinearity present in the scatterplot matrix of the nontrivial
predictors, then transforming the predictors to remove the nonlinear-
ity will often be a useful procedure. More will be said about response
transformations for experimental designs in Section 5.4.

There has been considerable discussion on whether the response transfor-

mation parameter should be selected with maximum likelihood (see Bickel
and Doksum 1981), or selected by maximum likelihood and then rounded to
a meaningful value on a coarse grid L (see Box and Cox 1982 and Hinkley
and Runger 1984). Suppose that no strong nonlinearities are present among
the predictors x and that if predictor transformations were used, then the
transformations were chosen without examining the response. Also assume
that
Y = to (Z) = xT + e.
Suppose that a transformation t is chosen without examining the response.
Results in Li and Duan (1989), Chen and Li (1998), and Chang and Olive
(2010) suggest that if x has an approximate elliptically contoured distribu-
tion, then the OLS ANOVA F , partial F , and Wald t tests will have the
correct level asymptotically, even if = o .
Now assume that the response is used to choose . For example, assume
that the numerical Box Cox method is used. Then is likely to be variable
unless the sample size is quite large, and considerable bias can be introduced,
as observed by Bickel and Doksum (1981). Now assume that is chosen with
the graphical method (and assume that ties are broken by using theory or by
using the following list in decreasing order of importance 1, 0, 1/2, 1, and
1/3 so that the log transformation is chosen over the cube root transformation
if both look equally good). Then will often rapidly converge in probability
3.8 Complements 143

to a value L . Hence for moderate sample sizes, it may be reason-

able to assume that the OLS tests have approximately the correct level. Let
W = t (Z) and perform the OLS regression of W on x. If the response and
residual plots suggest that the MLR model is appropriate, then the response
transformation from the graphical method will be useful for description and
exploratory purposes, and may be useful for prediction and inference. If a
numerical method is used to choose , perhaps in an interval or on a coarse
grid, the Olive (2016a) bootstrap tests for H0 : A = c may be useful.
The MLR assumptions always need to be checked after making a response
transformation. Since the graphical method uses a response plot to choose
the transformation, the graphical method should be much more reliable than
a numerical method. Transformation plots should be made if a numerical
method is used, but numerical methods are not needed to use the graphical
method.

Variable Selection and Multicollinearity

The literature on numerical methods for variable selection in the OLS

multiple linear regression model is enormous. Three important papers are
Jones (1946), Mallows (1973), and Furnival and Wilson (1974). Chatterjee
and Hadi (1988, pp. 4347) give a nice account on the eects of overtting
on the least squares estimates. Ferrari and Yang (2015) give a method for
testing whether a model is undertting. Section 3.4.1 followed Olive (2016a)
closely. See Olive (2016b) for more on prediction regions. Also see Claeskins
and Hjort (2003), Hjort and Claeskins (2003), and Efron et al. (2004). Texts
include Burnham and Anderson (2002), Claeskens and Hjort (2008), and
Linhart and Zucchini (1986).
Cook and Weisberg (1999a, pp. 264265) give a good discussion of the ef-
fect of deleting predictors on linearity and the constant variance assumption.
Walls and Weeks (1969) note that adding predictors increases the variance
of a predicted response. Also R2 gets large. See Freedman (1983).
Discussion of biases introduced by variable selection and data snooping
include Hurvich and Tsai (1990), Leeb and Potscher (2006), Selvin and Stuart
(1966), and Hjort and Claeskins (2003). This theory assumes that the full
model is known before collecting the data, but in practice the full model is
often built after collecting the data. Freedman (2005, pp. 192195) gives an
interesting discussion on model building and variable selection.
The predictor variables can be transformed if the response is not used,
and then inference can be done for the linear model. Suppose the p predictor
variables are xed so Y = t(Z) = X+e, and the computer program outputs
, after doing an automated response transformation and automated variable
selection. Then the nonlinear estimator can be bootstrapped. See Olive
(2016a). If data snooping, such as using graphs, is used to select the response
transformation and the submodel from variable selection, then strong, likely
unreasonable assumptions are needed for valid inference for the nal nonlinear
model.
144 3 Building an MLR Model

Olive and Hawkins (2005) discuss inuential cases in variable selection,

as do Leger and Altman (1993). The interpretation of Mallows Cp given in
Proposition 3.2 is due to Olive and Hawkins (2005), who show that the Cp
statistic can be used for variable selection for many 1D regression models,
such as GLMs, where SP = T x. Other interpretations of the Cp statistic
specic to MLR can be given. See Gilmour (1996). The Cp statistic is due to
Jones (1946). Also see Kenard (1971).
The AIC(I) statistic is often used instead of Cp (I). The full model and
the model Imin found with the smallest AIC are always of interest. Burnham
and Anderson (2004) suggest that if (I) = AIC(I) AIC(Imin ), then
models with (I) 2 are good, models with 4 (I) 7 are borderline,
and models with (I) > 10 should not be used as the nal submodel. Find
the submodel II with the smallest number of predictors such that (II ) 2.
Then II is the initial submodel to examine, and often II = Imin . Also examine
submodels I with fewer predictors than II with (I) 7. See Chapter 13.
When there are strong linear relationships among the predictors, multi-
collinearity is present. Let Rk2 be the coecient of multiple determination
when xk is regressed on the remaining predictor variables, including a con-
stant. The variance ination factor is VIF(k) = 1/(1 Rk2 ). Both Rk2 and
VIF(k) are large when multicollinearity is present. Following Cook and Weis-
berg (1999a, p. 274), if sk is the sample standard deviation of xk , then the
standard error of k is

M SE 1 M SE 
se(k ) = = V IF (k).
sk n 1 1 Rk
2
sk n 1

Hence k becomes more dicult to estimate when multicollinearity is present.

Variable selection is a useful way to reduce multicollinearity, and alternatives
such as ridge regression are discussed in Gunst and Mason (1980). See James
et al. (2013, ch. 6) for more information on variable selection, ridge regression,
and lasso. Belsley (1984) shows that centering the data before diagnosing the
data for multicollinearity is not necessarily a good idea.
We note that the pollution data of Example 3.7 has been heavily analyzed
in the ridge regression literature, but this data was easily handled by the log
rule combined with variable selection. The pollution data can be obtained
from this texts website, or from the STATLIB website:
(http://lib.stat.cmu.edu/).
The leaps function in R and Proc Rsquare in SAS can be used to perform
all subsets variable selection with the Cp criterion. The step and regsubsets
functions in R can be used for forward selection and backward elimination.
See Problem 3.6. Get more information on these R functions with the follow-
ing commands.
?step
library(leaps)
?leaps
3.8 Complements 145

?regsubsets
Bootstrap

Olive (2016a,b,c) showed that the prediction region method for creating
a large sample 100(1 )% condence region for an r 1 parameter vector
is a special case of the percentile method when r = 1, and gave sucient
conditions for r > 1. The shorth method gives the shortest percentile method
intervals, asymptotically, and should be used when B 1000. Efron (2014)
reviews some alternative methods for variable selection inference.
Consider the residual bootstrap, and let r W denote an n1 random vector
of elements selected with replacement from the n residuals r1 , . . . , rn . Then
there are K = nn possible values for r W . Let r W W
1 , . . . , r K be the possible
W
values of r . These values are equally likely, so are selected with probability
= 1/K. Note that r W has a discrete distribution. Then


E(r1j )
..
E(r W
j )= . .

E(rnj )

Now the marginal distribution of rij takes on the n values r1 , . . . , rn with
the same probability 1/n. So each of the n marginal  distributions is the
n
empirical distribution of the residuals. Hence E(rij ) = i=1 ri /n = r, and
r = 0 for least squares residuals for multiple linear regression when there

is a constant in the model. So for least squares, E(r Wj ) = 0, and E( j ) =
(X T X)1 X T E(Y + r W T
j ) = (X X)
1
X T Y = (X T X)1 X T HY =

(X T X)1 X T Y = = n

since HX = X and X T H = X T . Here j = 1, . . . , B.

Diagnostics

Excellent introductions to OLS diagnostics include Fox (1991) and Cook

and Weisberg (1999a, pp. 161163, 183184, section 10.5, section 10.6, ch. 14,
ch. 15, ch. 17, ch. 18, and section 19.3). More advanced works include Belsley
et al. (1980), Cook and Weisberg (1982), Atkinson (1985), and Chatterjee and
Hadi (1988). Hoaglin and Welsh (1978) examine the hat matrix while Cook
(1977) introduces Cooks distance. Also see Velleman and Welsch (1981).
Cook and Weisberg (1997, 1999a: ch. 17) call a plot that emphasizes model
agreement a model checking plot.

Outliers

Olive (2008) is an introduction to outlier detection and robust regression.

Also see Olive (2005) and Olive and Hawkins (2011). Some useful properties
of the DD plot are given in Olive (2002). Theory for the FCH, RFCH, and
146 3 Building an MLR Model

RMVN estimators is given in Olive (2008: ch. 10, 2016c: ch. 4) and Olive and
Hawkins (2010). These three estimators are also used in Zhang et al. (2012).
Lasso and Other Variable Selection Techniques
Response plots, prediction intervals, and the bootstrap prediction region
method are also useful for other variable selection techniques such as lasso
and ridge regression. If n 400 and p 3000, Bertsimas et al. (2016) give a
fast all subsets variable selection method.
Recent theory for lasso assumes that is selected before looking at the
data, rather than being estimated using kfold cross validation. See Hastie
et al. (2015). The prediction region method appears to be useful when n >> p
if none of the i = 0, but (in 2016) it takes a long time to simulate lasso with
kfold cross validation.
Lasso seems to work under ASSUMPTION L: assume the predictors are
uncorrelated or the number of active predictors (predictors with nonzero
coecients) is not much larger than 20. When n is xed and p increases, the
lasso prediction intervals increase in length slowly provided that assumption
L held. Methods are being developed that should work under more reasonable
assumptions. See Pelawa Watagoda (2017) and Pelawa Watagoda and Olive
(2017).

3.9 Problems

Problems with an asterisk * are especially important.

Output for problem 3.1. Current terms:

(finger to ground nasal height sternal height)
df RSS | k C_I
Delete: nasal height 73 35567.2 | 3 1.617
Delete: finger to ground 73 36878.8 | 3 4.258
Delete: sternal height 73 186259. | 3 305.047

3.1. From the above output from backward elimination, what terms
should be used in the MLR model to predict Y ? (You can tell that the non-
trivial variables are nger to ground, nasal height, and sternal height from
the delete lines. DONT FORGET THE CONSTANT!)

3.2. The table on the following page gives summary statistics for 4 MLR
models considered as nal submodels after performing variable selection. The
response plot and residual plot for the full model L1 was good. Model L3 was
the minimum Cp model found. Which model should be used as the nal
submodel? Explain briey why each of the other 3 submodels should not be
used.
3.9 Problems 147

Output for Problem 3.2.

L1 L2 L3 L4
# of predictors 10 6 4 3
# with 0.01 p-value 0.05 0 0 0 0
# with p-value > 0.05 6 2 0 0
R2 (I) 0.774 0.768 0.747 0.615
corr(Y , YI ) 1.0 0.996 0.982 0.891
C
p (I) 10.0 3.00 2.43 22.037
M SE 63.430 61.064 62.261 75.921
p-value for partial F test 1.0 0.902 0.622 0.004

Output for Problem 3.3.

L1 L2 L3 L4
# of predictors 10 5 4 3
# with 0.01 p-value 0.05 0 1 0 0
# with p-value > 0.05 8 0 0 0
R2 (I) 0.655 0.650 0.648 0.630
corr(Y , YI ) 1.0 0.996 0.992 0.981
C
p (I) 10.0 4.00 5.60 13.81
M SE 73.548 73.521 73.894 75.187
p-value for partial F test 1.0 0.550 0.272 0.015

3.3. The above table gives summary statistics for 4 MLR models consid-
ered as nal submodels after performing variable selection. The response plot
and residual plot for the full model L1 was good. Model L2 was the minimum
Cp model found.
a) Which model is II , the initial submodel to look at?
b) What other model or models, if any, should be examined?

3.4. The output below and on the following page is from software that
does all subsets variable selection. The data is from Ashworth (1842). The
predictors were A = log(1692 property value), B = log(1841 property value),
and C = log(percent increase in value), while the response variable is Y =
log(1841 population).
a) The top output corresponds to data with 2 small outliers. From this
output, what is the best model? Explain briey.
b) The bottom output corresponds to the data with the 2 outliers removed.
From this output, what is the best model? Explain briey.

Output for Problem 3.4.

ADJ 99 cases 2 outliers
k CP R SQ R SQ RESID SS VARIABLES
-- ----- ---- ---- -------- -------------
1 760.7 0.0000 0.0000 185.928 INTERCEPT ONLY
2 12.7 0.8732 0.8745 23.3381 B
148 3 Building an MLR Model

2 335.9 0.4924 0.4976 93.4059 A

2 393.0 0.4252 0.4311 105.779 C
3 12.2 0.8748 0.8773 22.8088 B C
3 14.6 0.8720 0.8746 23.3179 A B
3 15.7 0.8706 0.8732 23.5677 A C
4 4.0 0.8857 0.8892 20.5927 A B C

ADJ 97 cases after deleting 2 outliers

k CP R SQ R SQ RESID SS VARIABLES
-- ----- ---- ------ -------- -------------
1 903.5 0.0000 0.0000 183.102 INTERCEPT ONLY
2 0.7 0.9052 0.9062 17.1785 B
2 406.6 0.4944 0.4996 91.6174 A
2 426.0 0.4748 0.4802 95.1708 C
3 2.1 0.9048 0.9068 17.0741 A C
3 2.6 0.9043 0.9063 17.1654 B C
3 2.6 0.9042 0.9062 17.1678 A B
4 4.0 0.9039 0.9069 17.0539 A B C
R Problems

Use the command source(G:/lregpack.txt) to download the func-

tions and the command source(G:/lregdata.txt) to download the data.
See Preface or Section 14.1. Typing the name of the lregpack function,
e.g. tplot, will display the code for the function. Use the args command,
e.g. args(tplot), to display the needed arguments for the function. For some
of the following problems, the R commands can be copied and pasted from
(http://lagrange.math.siu.edu/Olive/lreghw.txt) into R.
3.5 . a) Download the R function tplot that makes the transformation
plots for L .
b) Use the following R command to make a 100 3 matrix. The columns
of this matrix are the three nontrivial predictor variables.
nx <- matrix(rnorm(300),nrow=100,ncol=3)
Use the following command to make the response variable Y.
y <- exp( 4 + nx%*%c(1,1,1) + 0.5*rnorm(100) )
This command means the MLR model log(Y ) = 4 + X2 + X3 + X4 + e will
hold where e N (0, 0.25).
To nd the response transformation, you need the program tplot given
in a). Type ls() to see if the programs were downloaded correctly.
3.9 Problems 149

c) To make the transformation plots type the following command.

tplot(nx,y)
The rst plot will be for = 1. Move the cursor to the plot and hold
the rightmost mouse key down and highlight Stop to go to the next plot.
Repeat these mouse operations to look at all of the plots. The identity line is
included in each plot. When you get a plot where the plotted points cluster
about the identity line with no other pattern, include this transformation
plot in Word by pressing the Ctrl and c keys simultaneously. This will copy
the graph. Then in Word use the menu command Paste. You should get
the log transformation.

d) Type the following commands.

out <- lsfit(nx,log(y))
ls.print(out)
Use the mouse to highlight the created output and include the output in
Word.
) using the output in d).
e) Write down the least squares equation for log(Y

3.6. Download cbrainx and cbrainy into R.

The data is the brain weight data from Gladstone (1905). The response Y
is brain weight while the predictors are age, breadth, cephalic, circum, headht,
height, len, sex, and a constant. The step function can be used to perform
forward selection and backward elimination in R.
a) Copy and paste the commands for this problem into R. The commands
t the full model, display the LS output, and perform backward elimination
using the AIC criterion. Copy and paste the output for backward elimination
into Word (one page of output).

zx <- cbrainx[,c(1,3,5,6,7,8,9,10)]
zbrain <- as.data.frame(cbind(cbrainy,zx))
zfull <- lm(cbrainy~.,data=zbrain)
summary(zfull)
back <- step(zfull)

b) We want low AIC and as few predictors as possible. Backward elimina-

tion starts with the full model then deletes one nontrivial predictor at a time.
The term <None> corresponds to the current model that does not eliminate
any terms. The terms listed above <None> correspond to models that have
smaller AIC than the current model. R stops when eliminating terms makes
the AIC higher than the current model. Which terms, including a constant,
were in this minimum AIC model?
c) Copy and paste the commands for this problem into R. The commands
t the null model that only contains a constant. Forward selection starts at
150 3 Building an MLR Model

the null model (corresponding to lower) and considers 8 nontrivial predictors

(given by upper).
Copy and paste the output for forward selection into Word (two pages of
output).
zint <- lm(cbrainy~1,data=zbrain)
forw <- step(zint,scope=list(lower=~1,
upper=~age+breadth+cephalic+circum+headht+height
+len+sex),direction="forward")
d) Forward selection in R starts with the null model and then adds a
predictor circum to the model. Forward selection in R allows you to consider
models with fewer predictors than the minimum AIC model (unlike backward
elimination). Which terms, including a constant, were in the minimum AIC
model?
e) The following code can be used to do all subsets regression. When k = 6
there is a Cp = 6.009241 corresponding to the 39th model that looks good.
From the output, this model that contains a constant and variables 1 = x2 , 3
= x4 , 5 = x6 , 7 = x8 , and 8 = x9 . Note that R labels the nontrivial predictors
from 1 to 8, so variable j = xj+1 .
library(leaps)
out<-leaps(x=zx,y=cbrainy)
out
plot(out$size,out$Cp)
tem<-1:length(out$size)
tem[out$Cp < 6.01]
for(i in 2:max(out$size))
print( c(i, min(out$Cp[out$size==i])))
out$which[39,]
1 2 3 4 5 6 7 8
TRUE FALSE TRUE FALSE TRUE FALSE TRUE TRUE
zx[1,]
age breadth cephalic circum headht height len sex
39.0 149.5 81.9 550.0 137.0 68.0 182.5 1.0
Problems using ARC

To quit Arc, move the cursor to the x in the northeast corner and click.
Problems 3.73.11 use data sets that come with Arc (Cook and Weisberg
1999a).
3.7 . a) In Arc enter the menu commands File>Load>Data and open
the le big-mac.lsp. Next use the menu commands Graph&Fit> Plot of to
obtain a dialog window. Double click on TeachSal and then double click on
BigMac. Then click on OK. These commands make a plot of x = TeachSal =
primary teacher salary in thousands of dollars versus y = BigMac = minutes
of labor needed to buy a Big Mac and fries. Include the plot in Word.
3.9 Problems 151

Consider transforming y with a (modied) power transformation

"
() (y 1)/, = 0
y =
log(y), = 0

b) Should simple linear regression be used to predict y from x? Explain.

c) In the plot, = 1. Which transformation will increase the linearity of

the plot, log(y), or y (2) ? Explain.

3.8 . In Arc enter the menu commands File>Load>Data and open the
le mussels.lsp. Use the commands Graph&Fit>Scatterplot Matrix of. In
the dialog window select H, L, W, S, and M (so select M last). Click on OK
and include the scatterplot matrix in Word. The response M is the edible part
of the mussel while the 4 predictors are shell measurements. Are any of the
marginal predictor relationships nonlinear? Is E(M |H) linear or nonlinear?

3.9 . The le wool.lsp has data from a 33 experiment on the behavior of

worsted yarn under cycles of repeated loadings. The response Y is the number
of cycles to failure and the three predictors are the length, amplitude, and
load. Make ve transformation plots by using the following commands.
From the menu Wool select transform and double click on Cycles.
Select modied power and use p = 1, 0.5, 0, and 0.5. Use the menu
commands Graph&Fit>Fit linear LS to obtain a dialog window. Next t LS
ve times. Use Amp, Len, and Load as the predictors for all 5 regressions, but
use Cycles1 , Cycles0.5 , log[Cycles], Cycles0.5 , and Cycles as the response.
Use the menu commands Graph&Fit>Plot of to create a dialog window.
Double click on L5:Fit-Values and double click on Cycles, double click on
L4:Fit-Values and double click on Cycles0.5 , double click on L3:Fit-Values
and double click on log[Cycles], double click on L2:Fit-Values and double click
on Cycles0.5 , double click on L1:Fit-Values and double click on Cycles1 .
a) You may stop when the resulting plot in linear. Let Z = Cycles. Include
the plot of Y versus Y = Z () that is linear in Word. Move the OLS slider
bar to 1. What response transformation do you end up using?
b) Use the menu commands Graph&Fit>Plot of and put L5:Fit-Values
in the H box and L3:Fit-Values in the V box. Is the plot linear?

3.10. In Arc enter the menu commands File>Load>Data and open

the le bcherry.lsp. The menu Trees will appear. Use the menu commands
Trees>Transform and a dialog window will appear. Select terms Vol, D,
and Ht. Then select the log transformation. The terms log Vol, log D, and log
Ht should be added to the data set. If a tree is shaped like a cylinder or a
cone, then V ol D2 Ht and taking logs results in a linear model.

a) Fit the full model with Y = log V ol, X1 = log D, and X2 = log Ht.
Add the output that has the LS coecients to Word.
152 3 Building an MLR Model

b) Fitting the full model will result in the menu L1. Use the commands
L1>AVPAll 2D. This will create a plot with a slider bar at the bottom
that says log[D]. This is the added variable plot for log(D). To make an added
variable plot for log(Ht), click on the slider bar. Add the OLS line to the AV
plot for log(Ht) by moving the OLS slider bar to 1, and add the zero line by
clicking on the Zero line box. Include the resulting plot in Word.

c) Fit the reduced model that drops log(Ht). Make an RR plot with the
residuals from the full model on the V axis and the residuals from the sub-
model on the H axis. Add the LS line and the identity line as visual aids.
(Click on the Options menu to the left of the plot and type y=x in the
resulting dialog window to add the identity line.) Include the plot in Word.

d) Similarly make an FF plot using the tted values from the two models.
Add the OLS line which is the identity line. Include the plot in Word.

e) Next put the residuals from the submodel on the V axis and log(Ht)
on the H axis. Move the OLS slider bar to 1, and include this residual plot
in Word.

f) Next put the residuals from the submodel on the V axis and the tted
values from the submodel on the H axis. Include this residual plot in Word.

g) Next put log(Vol) on the V axis and the tted values from the submodel
on the H axis. Move the OLS slider bar to 1, and include this response plot
in Word.

h) Does log(Ht) seem to be an important term? If the only goal is to

predict volume, will much information be lost if log(Ht) is omitted? Beside
each of the 6 plots, remark on the information given by the plot.
(Some of the plots will suggest that log(Ht) is needed while others will suggest
that log(Ht) is not needed.)

3.11 . a) In this problem we want to build an MLR model to predict

Y = t(BigM ac) where t is some power transformation. In Arc enter the
menu commands File>Load>Data and open the le big-mac.lsp. Make a
scatterplot matrix of the variables, except City, and include the plot in
Word.
b) The log rule makes sense for the BigMac data. From the scatterplot
matrix, use the Transformations menu and select Transform to logs.
Include the resulting scatterplot matrix in Word.
c) From the Mac menu, select Transform. Then select all 10 variables
and click on the Log transformations button. Then click on OK. From the
3.9 Problems 153

Graph&Fit menu, select Fit linear LS. Use log[BigMac] as the response
and the other 9 log variables as the Terms. This model is the full model.
Include the output in Word.
d) Make a response plot (L1:Fit-Values in H and log(BigMac) in V) and
residual plot (L1:Fit-Values in H and L1:Residuals in V), and include both
plots in Word.
e) Using the L1 menu, select Examine submodels and try forward
selection and backward elimination. Using the Cp min(2k, p) rule suggests
that the submodel using log[service], log[TeachSal], and log[TeachTax] may be
good. From the Graph&Fit menu, select Fit linear LS, t the submodel
and include the output in Word.
f) Make a response plot (L2:Fit-Values in H and log(BigMac) in V) and
residual plot (L2:Fit-Values in H and L2:Residuals in V) for the submodel,
and include the plots in Word.
g) Make an RR plot (L2:Residuals in H and L1:Residuals in V) and FF plot
(L2:Fit-Values in H and L1:Fit-Values in V) for the submodel, and include
the plots in Word. Move the OLS slider bar to 1 in each plot to add the
identity line. For the RR plot, click on the Options menu then type y = x in
the long horizontal box near the bottom of the window and click on OK to
add the identity line.
h) Do the plots and output suggest that the submodel is good? Explain.

Warning: The following problems use data from the books web-
page (http://lagrange.math.siu.edu/Olive/lregbk.htm). Save the
data les on a ash drive G, say. Get in Arc and use the menu commands
File > Load and a window will appear. Click on Removable Disk (G:).
Then click twice on the data set name.

3.12 . The following data set has 5 babies that are good leverage
points: they look like outliers but should not be deleted because they follow
the same model as the bulk of the data.
a) In Arc enter the menu commands File>Load>Removable Disk (G:)
and open the le cbrain.lsp. Select transform from the cbrain menu, and add
size1/3 using the power transformation option (p = 1/3). From
Graph&Fit, select Fit linear LS. Let the response be brnweight and as terms
include everything but size and Obs. Hence your model will include size1/3 .
This regression will add L1 to the menu bar. From this menu, select Examine
submodels. Choose forward selection. You should get models including k = 2
to 12 terms including the constant. Find the model with the smallest Cp (I) =
CI statistic and include all models with the same k as that model in Word.
That is, if k = 2 produced the smallest CI , then put the block with k = 2
into Word. Next go to the L1 menu, choose Examine submodels and choose
Backward Elimination. Find the model with the smallest CI and include all
of the models with the same value of k in Word.
154 3 Building an MLR Model

b) What was the minimum Cp model was chosen by forward selection?

c) What was the minimum Cp model was chosen by backward elimination?

d) Which minimum Cp model do you prefer? Explain.

e) Give an explanation for why the two models are dierent.

f) Pick a submodel and include the regression output in Word.

g) For your submodel in f), make an RR plot with the residuals from the
full model on the V axis and the residuals from the submodel on the H axis.
Add the OLS line and the identity line y=x as visual aids. Include the RR
plot in Word.

h) Similarly make an FF plot using the tted values from the two models.
Add the OLS line which is the identity line. Include the FF plot in Word.

i) Using the submodel, include the response plot (of Y versus Y ) and
residual plot (of Y versus the residuals) in Word.

j) Using results from f)-i), explain why your submodel is a good model.

3.13. Activate the cyp.lsp data set. Choosing no more than 3 nonconstant
terms, try to predict height with multiple linear regression. Include a plot with
the tted values on the horizontal axis and height on the vertical axis. Is your
model linear? Also include a plot with the tted values on the horizontal axis
and the residuals on the vertical axis. Does the residual plot suggest that the
linear model may be inappropriate? (There may be outliers in the plot. These
could be due to typos or because the error distribution has heavier tails than
the normal distribution.) State which model you use.

3.14. Activate the insulation data, contributed by Elizabeth Spector,

with the commands File>Load>Removable Disk (G:)>insulation.lsp.
The data description should appear in the Listener window.
Then go to the Graph&Fit menu and choose Plot of . . . and select
time for the H box, y for the V box, and type for the Mark by
box. Then click on OK and a window with a plot should open.
a) The OLS popdown menu is the triangle below OLS. Select Fit by
marksgeneral and then use the cursor to move the small black box to 2 on
the OLS slider bar. Then copy and paste the plot to Word. This command
ts least squares quadratic functions to the data from each of the 5 types of
insulation.
b) If there is no interaction, then the 5 curves will be roughly parallel and
will not cross. The curves will cross if there is interaction. Is there interaction?
3.9 Problems 155

c) The top curve corresponds to no insulation, and the temperature rapidly

rose and then rapidly cooled o. Corn pith corresponds to curve 2. Is corn
pith comparable to the more standard types of insulation 35?

3.15. Activate the cement.lsp data, contributed by Alyass Hossin. Act

as if 20 dierent samples were used to collect this data. If 5 measurements
on 4 dierent samples were used, then experimental design with repeated
measures or longitudinal data analysis may be a better way to analyze this
data.
a) From Graph&Fit select Plot of, place x1 in H, y in V, and x2 in the
Mark by box. From the OLS menu, select Fit by marksgeneral and move the
slider bar to 2. Include the plot in Word.
b) A quadratic seems to be a pretty good MLR model. From the cement
menu, select Transform, select x1, and place a 2 in the p box. This should
add x12 to the data set. From Graph&Fit select Fit linear LS, select x1 and
x12 as the terms and y as the response. Include the output in Word.
c) Make the response plot. Again from the OLS menu, select Fit by marks
general and move the slider bar to 1. Include the plot in Word. This plot
suggests that there is an interaction: the CM cement is stronger for low curing
times and weaker for higher curing times. The plot suggests that there may
not be an interaction between the two new types of cement.
d) Place the residual plot in Word. (Again from the OLS menu, select Fit
by marksgeneral and move the slider bar to 1.) The residual plot is slightly
fan shaped.
e) From the cement menu, select Make factors and select x2. From the
cement menu, select Make interactions and select x1 and (F)x2. Repeat,
selecting x12 and (F)x2. From Graph&Fit select Fit linear LS, select x1, x12 ,
(F)x2, x1*(F)x2, and x12 *(F)x2 as the terms and y as the response. Include
the output in Word.
f) Include the response plot and residual plot in Word.
g) Next delete the standard cement in order to compare the two coal
based cements. From Graph&Fit select Scatterplotmatrix of, then select x1,
x2, and y. Hold down the leftmost mouse button and highlight the x2 = 2
cases. Then from the Case deletions menu, select Delete selection from data
set. From Graph&Fit select Fit linear LS, select x1, x12 , x2 as the terms and
y as the response. Include the output in Word. The output suggests that the
MA brand is about 320 psi less strong than the ME brand. (May need to add
x2*x1 and x2*x12 interactions.)
h) Include the response plot and residual plot in Word. The residual plot
is not particularly good.

3.16. This problem gives a slightly simpler model than Problem 3.15 by
using the indicator variable x3 = 1 if standard cement (if x2 = 2) and x3 =
0 otherwise (if x2 is 0 or 1). Activate the cement.lsp data.
156 3 Building an MLR Model

a) From the cement menu, select Transform, select x1, and place a 2 in the
p box. This should add x12 to the data set. From the cement menu, select
Make interactions and select x1 and x3.
b) From Graph&Fit select Fit linear LS, select x1, x12 , x3, and x1*x3 as
the terms and y as the response. Include the output in Word.
c) Make the response and residual plots. When making these plots, place
x2 in the Mark by box. Include the plots in Word. Does the model seem ok?

3.17 . Get the McDonald and Schwing (1973) data pollution.lsp from
(http://lagrange.math.siu.edu/Olive/lregbk.htm), and save the le on
a ash drive. Activate the pollution.lsp dataset with the menu commands
File > Load > Removable Disk (G:) > pollution.lsp. Scroll up the screen
to read the data description. Often simply using the log rule on the predictors
with max(x)/ min(x) > 10 works wonders.
a) Make a scatterplot matrix of the rst nine predictor variables and the re-
sponse Mort. The commands Graph&Fit > Scatterplot-Matrix of will bring
down a Dialog menu. Select DENS, EDUC, HC, HOUS, HUMID, JANT,
JULT, NONW, NOX, and MORT. Then click on OK.
A scatterplot matrix with slider bars will appear. Move the slider bars for
NOX, NONW, and HC to 0, providing the log transformation. In Arc, the
diagonals have the min and max of each variable, and these were the three
predictor variables satisfying the log rule. Open Word.
In Arc, use the menu commands Edit > Copy. In Word, use the menu
command Paste. This should copy the scatterplot matrix into the Word
document. Print the graph.
b) Make a scatterplot matrix of the last six predictor variables and the
response Mort. The commands Graph&Fit > Scatterplot-Matrix of will
bring down a Dialog menu. Select OVR65, POOR, POPN, PREC, SO,
WWDRK, and MORT. Then click on OK. Move the slider bar of SO to 0
and copy the plot into Word. Print the plot as described in a).
c) Click on the pollution menu and select Transform. Click on the log
transformations button and select HC, NONW, NOX, and SO. Click on OK.
Then t the full model with the menu commands Graph&Fit > Fit lin-
ear LS. Select MORT for the response. For the terms, select DENS, EDUC,
log[HC], HOUS, HUMID, JANT, JULT, log[NONW], log[NOX], OVR65,
POOR, POPN, PREC, log[SO], and WWDRK. Click on OK.
This model is the full model. To make the response plot use the menu
commands Graph&Fit >Plot of. Select MORT for the V-box and L1:Fit-
Values for the H-box. Click on OK. When the graph appears, move the OLS
slider bar to 1 to add the identity line. Copy the plot into Word.
To make the residual plot use the menu commands Graph&Fit >Plot
of. Select L1:Residuals for the V-box and L1:Fit-Values for the H-box. Click
on OK. Copy the plot into Word. Print the two plots.
3.9 Problems 157

d) Using the L1 menu, select Examine submodels and try forward

selection. Using the L1 menu, select Examine submodels and try back-
ward elimination. You should get a lot of output including that shown in
Example 3.7.
Fit the submodel with the menu commands Graph&Fit > Fit linear
LS. Select MORT for the response. For the terms, select EDUC, JANT,
log[NONW], log[NOX], and PREC. Click on OK.
This model is the submodel suggested by backward elimination. To make
the response plot use the menu commands Graph&Fit >Plot of. Select
MORT for the V-box and L2:Fit-Values for the H-box. Click on OK. When
the graph appears, move the OLS slider bar to 1 to add the identity line.
Copy the plot into Word.
To make the residual plot use the menu commands Graph&Fit >Plot
of. Select L2:Residuals for the V-box and L2:Fit-Values for the H-box. Click
on OK. Copy the plot into Word. Print the two plots.
e) To make an RR plot use the menu commands Graph&Fit >Plot of.
Select L1:Residuals for the V-box and L2:Residuals for the H-box. Click on
OK. Move the OLS slider bar to one. On the window for the plot, click on
Options. A window will appear. Type y = x and click on OK to add the
identity line. Copy the plot into Word. Print the plot.
f) To make an FF plot use the menu commands Graph&Fit >Plot of.
Select L1:Fit-Values for the V-box and L2:Fit-Values for the H-box. Click on
OK. Move the OLS slider bar to one and click on OK to add the identity
line. Copy the plot into Word.
g) Using the response and residual plots from the full model and submodel
along with the RR and FF plots, does the submodel seem ok?

3.18. Get the Joanne Numrich data c12.lsp from

(http://lagrange.math.siu.edu/Olive/lregbk.htm), and save the le on
a ash drive. Activate the c12.lsp dataset with the menu commands File >
Load > Removable Disk (G:) > c12.lsp. Scroll up the screen to read the
data description. This data set is described in Example 3.10.
a) A bad model uses Y1 and all 24 nontrivial predictors. There are many
indicator variables. Click on the CLA menu and select Transform. Click on
the log transformations button and select y1 . Click on OK.
b) Use the menu commands Graph&Fit > Fit linear LS. Select log[y1]
for the response. For the terms, select x1, x2, x8, x9, x10, x11, x18, x20, x23,
and x24. Click on OK.
This model will be used as the full model. To make the response plot use
the menu commands Graph&Fit >Plot of. Select log[y1] for the V-box and
L1:Fit-Values for the H-box. Click on OK. When the graph appears, move
the OLS slider bar to 1 to add the identity line. Copy the plot into Word.
To make the residual plot use the menu commands Graph&Fit >Plot
of. Select L1:Residuals for the V-box and L1:Fit-Values for the H-box. Click
on OK. Copy the plot into Word. Print the two plots.
158 3 Building an MLR Model

c) As in Problem 13.17, use forward selection, backward elimination, and

plots to nd a good submodel.
Using material learned in Chapters 23, analyze the data sets described in
Problems 3.193.29. Assume that the response variable Y = t(Z) and that
the predictor variables X2 , . . . , Xp are functions of the remaining variables
W2 , . . . , Wr . Unless told otherwise, the full model Y, X1 , X2 , . . . , Xp (where
X1 1) should use functions of every variable W2 , . . . , Wr (and often p = r).
(In practice, often some of the variables and some of the cases are deleted,
but we will use all variables and cases, unless told otherwise, primarily so
that the instructor has some hope of grading the problems in a reasonable
amount of time.)
Read the description of the data provided by Arc. Once you have a
good full model, perform forward selection and backward elimination. Find
the model Imin that minimizes Cp (I), nd the model II with the fewest
number of predictors such that Cp (II ) Cp (Imin ) + 1 (it is possible that
II = Imin ), and nd the smallest value of k such that Cp (I) min(p, 2k).
Model II often has too many terms while the 2nd model often has too few
terms.
a) Give the output for your full model, including Y = t(Z) and R2 . If it
is not obvious from the output what your full model is, then write down the
full model. Include a response plot for the full model. (This plot should be
linear.) Also include a residual plot.
b) Give the output for your nal submodel. If it is not obvious from the
output what your submodel is, then write down the nal submodel.
c) Give between 3 and 5 plots that justify that your multiple linear regres-
sion submodel is reasonable. Below or beside each plot, give a brief explana-
tion for how the plot gives support for your model.
3.19. For the le bodfat.lsp, described in Problem 2.2, use Z = Y =
bodyf at but do not use X1 = density as a predictor in the full model. You
may use the remaining 13 nontrivial predictor variables. Do parts a), b), and
c) above.
3.20 . For the le boston2.lsp use Z = (y =) CRIM. Do parts a), b), and
c) above Problem 3.19.
Note: Y = log(CRIM ), X4 , X8 , is an interesting submodel, but more pre-
dictors are probably needed. The data set comes from Harrison and Rubinfeld
(1978).
3.21 . For the le major.lsp, described in Example 2.3, use Z = Y . Do
parts a), b), and c) above Problem 3.19.
Note: there are 1 or more outliers that aect numerical methods of variable
selection.
3.22. For the le marry.lsp, described in Example 2.12, use Z = Y . This
data set comes from Hebbler (1847). The census takers were not always willing
3.9 Problems 159

to count a womans husband if he was not at home. Do not use the predictor
X2 in the full model. Do parts a), b), and c) above Problem 3.19.

3.23 . For the le museum.lsp, described below, use Z = Y . Do parts a),

b), and c) above Problem 3.19.
This data set consists of measurements taken on skulls at a museum and
was extracted from tables in Schaahausen (1878). There are at least three
groups of data: humans, chimpanzees, and gorillas. The OLS t obtained from
the humans passes right through the chimpanzees. Since Arc numbers cases
starting at 0, cases 4759 are apes. These cases can be deleted by highlighting
the cases with small values of Y in the scatterplot matrix and using the
case deletions menu. (You may need to maximize the window containing the
scatterplot matrix in order to see this menu.)
i) Try variable selection using all of the data.
ii) Try variable selection without the apes.
perhaps only X1 , X2 , and X3 should be used
If all of the cases are used,
in the full model. Note that Y and X2 have high correlation.

3.24 . For the le pop.lsp, described below, use Z = Y . Do parts a), b),
and c) above Problem 3.19.
This data set comes from Ashworth (1842). Try transforming all variables
to logs. Then the added variable plots show two outliers. Delete these two
cases. Notice the eect of these two outliers on the pvalues for the coecients
and on numerical methods for variable selection.
Note: then log(Y ) and log(X2 ) make a good submodel.

3.25 . For the le pov.lsp, described below, use i) Z = f lif e and ii)
Z = gnp2 = gnp + 2. This data set comes from Rounceeld (1995). Making
loc into a factor may be a good idea. Use the commands poverty>Make factors
and select the variable loc. For ii), try transforming to logs and deleting the 6
cases with gnp2 = 0. (These cases had missing values for gnp. The le povc.lsp
has these cases deleted.) Try your nal submodel on the data that includes
the 6 cases with gnp2 = 0. Do parts a), b), and c) above Problem 3.19.

3.26 . For the le skeleton.lsp, described below, use Z = y.

This data set is also from Schaahausen (1878). At one time I heard or
read a conversation between a criminal forensics expert with his date. It went
roughly like If you wound up dead and I found your femur, I could tell what
your height was to within an inch. Two things immediately occurred to me.
The rst was no way and the second was that the man must not get many
dates! The les cyp.lsp and major.lsp have measurements including height,
but their R2 0.9. The skeleton data set has at least four groups: stillborn
babies, newborns and children, older humans, and apes.
a) Take logs of each variable and t the regression of log(Y) on log(X1 ),
. . . , log(X13 ). Make a residual plot and highlight the case with the smallest
residual. From the Case deletions menu, select Delete selection from data
160 3 Building an MLR Model

set. Go to Graph&Fit and again t the regression of log(Y) on log(X1 ), . . . ,

log(X13 ) (you should only need to click on OK). The output should say that
case 37 has been deleted. Include this output for the full model in Word.
b) Do part b) above Problem 3.19.
c) Do part c) above Problem 3.19.

3.27. Activate big-mac.lsp in Arc. Assume that a multiple linear regres-

sion model holds for t(y) and some terms (functions of the predictors) where
y is BigMac = hours of labor to buy Big Mac and fries. Using techniques you
have learned in class nd such a model. (Hint: Recall from Problem 3.11
that transforming all variables to logs and then using the model constant,
log(service), log(TeachSal) and log(TeachTax) was ok but the residuals did
not look good. Try adding a few terms from the minimal Cp model.)
a) Write down the full model that you use (e.g., a very poor full model is
exp(BigM ac) = 1 + 2 exp(EngSal) + 3 (T eachSal)3 + e) and include a
response plot for the full model. (This plot should be linear.) Give R2 for the
full model.

b) Write down your nal model (e.g., a very poor nal model is
exp(BigM ac) = 1 + 2 exp(EngSal) + 3 (T eachSal)3 + e).
c) Include the least squares output for your model and between 3 and 5
plots that justify that your multiple linear regression model is reasonable.
Below or beside each plot, give a brief explanation for how the plot gives
support for your model.

3.28. This is like Problem 3.27 with the BigMac data. Assume that a
multiple linear regression model holds for Y = t(Z) and for some terms
(usually powers or logs of the predictors). Using the techniques learned in
class, nd such a model. Give output for the full model, output for the nal
submodel and use several plots to justify your choices. These data sets, as
well as the BigMac data set, come with Arc. See Cook and Weisberg (1999a).
(INSTRUCTOR: Allow 2 hours for each part.)

file "response" Z
a) allomet.lsp BRAIN
b) casuarin.lsp W
c) evaporat.lsp Evap
d) hald.lsp Y
e) haystack.lsp Vol
f) highway.lsp rate
(From the menu Highway, select "Add a variate" and
type sigsp1 = sigs + 1. Then you can transform
sigsp1.)
g) landrent.lsp Y
h) ozone.lsp ozone
i) paddle.lsp Weight
3.9 Problems 161

j) sniffer.lsp Y
k) water.lsp Y

i) Write down the full model that you use and include the full model
residual plot and response plot in Word. Give R2 for the full model.

ii) Write down the nal submodel that you use.

iii) Include the least squares output for your model and between 3 and
5 plots that justify that your multiple linear regression model is reasonable.
Below or beside each plot, give a brief explanation for how the plot gives
support for your model.
3.29 . a) Activate buxton.lsp (you need to download the le onto your
ash drive Removable Disk (G:)). From the Graph&Fit menu, select Fit
linear LS. Use height as the response variable and bigonal breadth, cephalic
index, head length, and nasal height as the predictors. Include the output in
Word.
b) Make a response plot (L1:Fit-Values in H and height in V) and residual
plot (L1:Fit-Values in H and L1:Residuals in V) and include both plots in
Word.
c) In the residual plot use the mouse to move the cursor just above and
to the left of the outliers. Hold the leftmost mouse button down and move
the mouse to the right and then down. This will make a box on the residual
plot that contains the outliers. Go to the Case deletions menu and click
on Delete selection from data set. From the Graph&Fit menu, select Fit
linear LS and t the same model as in a) (the model should already be
entered, just click on OK). Include the output in Word.
d) Make a response plot (L2:Fit-Values in H and height in V) and residual
plot (L2:Fit-Values in H and L2:Residuals in V) and include both plots in
Word.
e) Explain why the outliers make the MLR relationship seem much
stronger than it actually is. (Hint: look at R2 .)
Variable Selection in SAS

3.30. Copy and paste the SAS program for this problem into the SAS
editor. Then perform the menu commands Run>Submit to obtain about
15 pages of output. Do not print out the output.
The data is from SAS Institute (1985, pp. 695704, 717718). Aerobic
tness is being measured by the ability to consume oxygen. The response
Y = Oxygen (uptake rate) is expensive to measure, and it is hoped that
the OLS Y can be used instead. The variables are Age in years, Weight in
kg, RunTime = time in minutes to run 1.5 miles, RunPulse = heart rate
when Y is measured, RestPulse = heart rate while running, and MaxPulse =
maximum heart rate recorded while running.
162 3 Building an MLR Model

The selection commands do forward selection, backward elimination, and

all subsets selection where the best ten models with the lowest Cp are
recorded. The proc rsquare command also does all subsets regression with
the Cp criterion.
The plots give the response and residual plots for the full model and the
submodel that used Age, RunTime, RunPulse, MaxPulse, and a constant.
a) Was the above plot for the minimum Cp model?
b) Do the plots suggest that the submodel was good?

Variable Selection in Minitab

3.31. Get the data set prof.mtb as described in Problem 2.15. The data is
described in McKenzie and Goldman (1999, pp. ED-22-ED-23). Assign the re-
sponse variable to be instrucr (the instructor rating from course evaluations)
and the predictors to be interest in the course, manner of the instructor, and
course = rating of the course.
a) To get residual and response plots you need to store the residuals and
tted values. Use the menu commands Stat>Regression>Regression to get
the regression window. Put instrucr in the Response and interest, manner,
and course in the Predictors boxes. The click on Storage. From the result-
ing window click on Fits and Residuals. Then click on OK twice.
b) To get a response plot, use the commands Graph>Plot, (double click)
place instrucr in the Y box, and Fits1 in the X box. Then click on OK. Print
the plot by clicking on the graph and then clicking on the printer icon.
c) To make a residual plot, use the menu commands Graph>Plot to get
a window. Place Resi1 in the Y box and Fits1 in the X box. Then click
on OK. Print the plot by clicking on the graph and then clicking on the
printer icon.
d) To perform all subsets regression, use the menu commands
Stat>Regression>Best Subsets to get the regression window. Put instrucr
in the Response and interest, manner, and course in the Free predictors
boxes. Which submodel is good?
Chapter 4
WLS and Generalized Least Squares

4.1 Random Vectors

The concepts of a random vector, the expected value of a random vector, and
the covariance of a random vector are needed before covering generalized least
squares. Recall that for random variables Yi and Yj , the covariance of Yi and
Yj is Cov(Yi , Yj ) i,j = E[(Yi E(Yi ))(Yj E(Yj )] = E(Yi Yj )E(Yi )E(Yj )
provided the second moments of Yi and Yj exist.

Denition 4.1. Y = (Y1 , . . . , Yn )T is an n 1 random vector if Yi is

a random variable for i = 1, . . . , n. Y is a discrete random vector if each Yi
is discrete, and Y is a continuous random vector if each Yi is continuous.
A random variable Y1 is the special case of a random vector with n = 1.

Denition 4.2. The population mean of a random n 1 vector Y =

(Y1 , . . . , Yn )T is
E(Y ) = (E(Y1 ), . . . , E(Yn ))T
provided that E(Yi ) exists for i = 1, . . . , n. Otherwise the expected value
does not exist. The n n population covariance matrix

Cov(Y ) = E[(Y E(Y ))(Y E(Y ))T ] = (i,j )

where the ij entry of Cov(Y ) is Cov(Yi , Yj ) = i,j provided that each i,j
exists. Otherwise Cov(Y ) does not exist.

The covariance matrix is also called the variancecovariance matrix and

variance matrix. Sometimes the notation Var(Y ) is used. Note that Cov(Y )
is a symmetric positive semidenite matrix. If Z and Y are n 1 random
vectors, a a conformable constant vector, and A and B are conformable
constant matrices, then

E(a + Y ) = a + E(Y ) and E(Y + Z) = E(Y ) + E(Z) (4.1)

Springer International Publishing AG 2017 163

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 4
164 4 WLS and Generalized Least Squares

and
E(AY ) = AE(Y ) and E(AY B) = AE(Y )B. (4.2)
Also
Cov(a + AY ) = Cov(AY ) = ACov(Y )AT . (4.3)

Example 4.1. Consider the OLS model Y = X + e where the ei are

iid with mean 0 and variance 2 . Then Y and e are random vectors while
a = X is a constant vector. Notice that E(e) = 0. Thus

E(Y ) = X + E(e) = X.

Since the ei are iid,

Cov(Y ) = Cov(e) = 2 I n (4.4)
where I n is the n n identity matrix. This result makes sense because the
Yi are independent with Yi = xTi + ei . Hence VAR(Yi ) = VAR(ei ) = 2 .
Recall that OLS = (X T X)1 X T Y . Hence

E( OLS ) = (X T X)1 X T E(Y ) = (X T X)1 X T X = .

That is, OLS is an unbiased estimator of . Using (4.3) and (4.4),

Cov( OLS ) = (X T X)1 X T Cov(Y )X(X T X)1

= 2 (X T X)1 X T X(X T X)1 = 2 (X T X)1 .

Recall that Y OLS = X OLS = X(X T X)1 X T Y = HY . Hence

E(Y OLS ) = X(X T X)1 X T E(Y ) = X(X T X)1 X T X = X = E(Y ).

Using (4.3) and (4.4),

Cov(Y OLS ) = HCov(Y )H T = 2 H

since H T = H and HH = H.
Recall that the vector of residuals r OLS = (I H)Y = Y Y OLS . Hence
E(r OLS ) = E(Y ) E(Y OLS ) = E(Y ) E(Y ) = 0. Using (4.3) and (4.4),

Cov(r OLS ) = (I H)Cov(Y )(I H)T = 2 (I H)

since I H is symmetric and idempotent: (I H)T = I H and

(I H)(I H) = I H.
4.2 GLS, WLS, and FGLS 165

4.2 GLS, WLS, and FGLS

Denition 4.3. Suppose that the response variable and at least one of the
predictor variables is quantitative. Then the generalized least squares (GLS)
model is
Y = X + e, (4.5)
where Y is an n 1 vector of dependent variables, X is an n p matrix
of predictors, is a p 1 vector of unknown coecients, and e is an n 1
vector of unknown errors. Also E(e) = 0 and Cov(e) = 2 V where V is a
known n n positive denite matrix.
Denition 4.4. The GLS estimator

GLS = (X T V 1 X)1 X T V 1 Y . (4.6)

The tted values are Y GLS = X GLS .

Denition 4.5. Suppose that the response variable and at least one of the
predictor variables is quantitative. Then the weighted least squares (WLS)
model with weights w1 , . . . , wn is the special case of the GLS model where
V is diagonal: V = diag(v1 , . . . , vn ) and wi = 1/vi . Hence

Y = X + e, (4.7)

E(e) = 0, and Cov(e) = 2 diag(v1 , . . . , vn ) = 2 diag(1/w1 , . . . , 1/wn ).

Denition 4.6. The WLS estimator

W LS = (X T V 1 X)1 X T V 1 Y . (4.8)

The tted values are Y W LS = X W LS .

Denition 4.7. The feasible generalized least squares (FGLS) model is
the same as the GLS estimator except that V = V () is a function of an
unknown q 1 vector of parameters . Let the estimator of V be V = V ().
Then the FGLS estimator
1 1
F GLS = (X T V X)1 X T V Y. (4.9)

The tted values are Y F GLS = X F GLS . The feasible weighted least squares
(FWLS) estimator is the special case of the FGLS estimator where V =
V () is diagonal. Hence the estimated weights wi = 1/vi = 1/vi (). The
FWLS estimator and tted values will be denoted by F W LS and Y F W LS ,
respectively.
Notice that the ordinary least squares (OLS) model is a special case of
GLS with V = I n , the n n identity matrix. It can be shown that the GLS
estimator minimizes the GLS criterion
166 4 WLS and Generalized Least Squares

QGLS () = (Y X)T V 1 (Y X).

Notice that the FGLS and FWLS estimators have p + q + 1 unknown param-
eters. These estimators can perform very poorly if n < 10(p + q + 1).
The GLS and WLS estimators can be found from the OLS regression
(without an intercept) of a transformed model. Typically there will be a
constant in the model: the rst column of X is a vector of ones. Following
Seber and Lee (2003, pp. 6668), there is a nonsingular n n matrix K
such that V = KK T . Let Z = K 1 Y , U = K 1 X, and  = K 1 e. This
method uses the fast, but rather unstable, Cholesky decomposition.

Proposition 4.1. a)
Z = U +  (4.10)
2
follows the OLS model since E() = 0 and Cov() = I n .
b) The GLS estimator GLS can be obtained from the OLS regression
(without an intercept) of Z on U .
c) For WLS, Yi = xTi + ei . The corresponding OLS model Z = U + 
is equivalent to Zi = uTi + i for i = 1, . . . , n where uTi is the ith row of U .

Then Zi = wi Yi and ui = wi xi . Hence W LS can be obtained from the

OLS regression (without an intercept) of Zi = wi Yi on ui = wi xi .

Proof. a) E() = K 1 E(e) = 0 and

Cov() = K 1 Cov(e)(K 1 )T = 2 K 1 V (K 1 )T

= 2 K 1 KK T (K 1 )T = 2 I n .
Notice that OLS without an intercept needs to be used since U does not
contain a vector of ones. The rst column of U is K 1 1 = 1.
b) Let ZU denote the OLS estimator obtained by regressing Z on U .
Then

ZU = (U T U )1 U T Z = (X T (K 1 )T K 1 X)1 X T (K 1 )T K 1 Y

and the result follows since V 1 = (KK T )1 = (K T )1 K 1 =

(K 1 )T K 1 .

c) The result follows from b) if Zi = wi Yi and ui = wi xi . But

for WLS, V = diag(v1 , . . . , vn ) and hence K = K T = diag( v1 , . . . , vn ).
Hence

K 1 = diag(1/ v1 , . . . , 1/ vn ) = diag( w1 , . . . , wn )
4.2 GLS, WLS, and FGLS 167

and Z = K 1 Y has ith element Zi = wi Yi . Similarly, U = K 1 X has

ith row uTi = wi xTi . 

Following Johnson and Wichern (1988, p. 51) and Freedman (2005, p. 54),
there is a symmetric, nonsingular n n square root matrix R = V 1/2 such
that V = RR. Let Z = R1 Y , U = R1 X and  = R1 e. This method
uses the spectral theorem (singular value decomposition) and has better com-
putational properties than transformation based on the Cholesky decompo-
sition.

Proposition 4.2. a)
Z = U +  (4.11)
2
follows the OLS model since E() = 0 and Cov() = I n .
b) The GLS estimator GLS can be obtained from the OLS regression
(without an intercept) of Z on U .
c) For WLS, Yi = xTi + ei . The corresponding OLS model Z = U + 
is equivalent to Zi = uTi + i for i = 1, . . . , n where uTi is the ith row of U .

Then Zi = wi Yi and ui = wi xi . Hence W LS can be obtained from the

OLS regression (without an intercept) of Zi = wi Yi on ui = wi xi .

Proof. a) E() = R1 E(e) = 0 and

Cov() = R1 Cov(e)(R1 )T = 2 R1 V (R1 )T

= 2 R1 RR(R1 ) = 2 I n .
Notice that OLS without an intercept needs to be used since U does not
contain a vector of ones. The rst column of U is R1 1 = 1.
b) Let ZU denote the OLS estimator obtained by regressing Z on U .
Then

ZU = (U T U )1 U T Z = (X T (R1 )T R1 X)1 X T (R1 )T R1 Y

and the result follows since V 1 = (RR)1 = R1 R1 = (R1 )T R1 .

c) The result follows from b) if Zi = wi Yi and ui = wi xi . But for

WLS, V = diag(v1 , . . . , vn ) and hence R = diag( v1 , . . . , vn ). Hence

R1 = diag(1/ v1 , . . . , 1/ vn ) = diag( w1 , . . . , wn )

and Z = R1 Y has ith element Zi = wi Yi . Similarly, U = R1 X has ith

row uTi = wi xTi . 

Remark 4.1. Standard software produces WLS output and the ANOVA
F test and Wald t tests are performed using this output.
168 4 WLS and Generalized Least Squares

Remark 4.2. The FGLS estimator can also be found from the OLS
regression (without an intercept) of Z on U where V () = RR. Similarly
the FWLS estimator can be found from the OLS regression (without an in-
tercept) of Zi = wi Yi on ui = wi xi . But now U is a random matrix
instead of a constant matrix. Hence these estimators are highly nonlinear.
OLS output can be used for exploratory purposes, but the pvalues are gen-
erally not correct. The Olive (2016a,b) nonparametric bootstrap tests may
be useful for FGLS and FWLS. The nonparametric bootstrap could also be
applied to the OLS estimator.
Under regularity conditions, the OLS estimator OLS is a consistent esti-
mator of when the GLS model holds, but GLS should be used because it
generally has higher eciency.
Denition 4.8. Let ZU be the OLS estimator from regressing Z on U .
The vector of tted values is Z = U ZU and the vector of residuals is
r ZU = Z Z. Then ZU = GLS for GLS, ZU = F GLS for FGLS,
ZU = W LS for WLS, and ZU = F W LS for FWLS. For GLS, FGLS,
WLS, and FWLS, a residual plot is a plot of Zi versus rZU,i and a response
plot is a plot of Zi versus Zi .
Notice that the residual and response plots are based on the OLS output
from the OLS regression without intercept of Z on U . If the model is good,
then the plotted points in the response plot should follow the identity line
in an evenly populated band while the plotted points in the residual plot
should follow the line rZU,i = 0 in an evenly populated band (at least if the
distribution of  is not highly skewed).
Plots based on YGLS = X ZU and on ri,GLS = Yi Yi,GLS should be
similar to those based on OLS . Although the plot of Yi,GLS versus Yi should
be linear, the plotted points will not scatter about the identity line in an
evenly populated band. Hence this plot cannot be used to check whether
the GLS model with V is a good approximation to the data. Moreover, the
ri,GLS and Yi,GLS may be correlated and usually do not scatter about the
r = 0 line in an evenly populated band. The plots in Denition 4.8 are both
a check on linearity and on whether the model using V (or V ) gives a good
approximation of the data, provided that n > k(p + q + 1) where k 5 and
preferably k 10.
For GLS and WLS (and for exploratory purposes for FGLS and FWLS),
plots and model building and variable selection should be based on Z and U .
Form Z and U and then use OLS software for model selection and variable
selection. If the columns of X are v 1 , . . . , v p , then the columns of U are
U1 , . . . , Up where Uj = R1 v j corresponds to the jth predictor Xj . For
example, the analog of the OLS residual plot of jth predictor versus the
residuals is the plot of the jth predictor Uj versus rZU . The notation is
confusing but the idea is simple: form Z and U , then use OLS software and
the OLS techniques from Chapters 2 and 3 to build the model.
4.2 GLS, WLS, and FGLS 169

a) OLS Response Plot b) OLS Residual Plot

12

2
RESID
1
Y
6
2

4
2 4 6 8 2 4 6 8
FIT FIT

c) WLS Response Plot d) WLS Residual Plot

15

ZRESID
0
Z
5

0 4 8 12 2 0 4 8 12
ZFIT ZFIT
Fig. 4.1 Plots for Draper and Smith Data

Example 4.2. Draper and Smith (1981, pp. 112114) present an FWLS
example with n = 35 and p = 2. Hence Y = 1 + 2 x + e. Let vi =
vi () = 1.5329 0.7334xi + 0.0883x2i . Thus = (1.5329, 0.7334, 0.0883)T .
Figure 4.1a and b shows the response and residual plots based on the OLS
regression of Y on x. The residual plot has the shape of the right opening
megaphone, suggesting that the variance is not constant. Figure 4.1c and d
shows the response and residual plots based on FWLS with weights wi = 1/vi .
See Problem 4.2 to reproduce these plots. Software meant for WLS needs the
weights. Hence FWLS can be computed using WLS software with the es-
timated weights, but the software may print WLS instead of FWLS, as in
Figure 4.1c and d.

Warning. A problem with the response and residual plots for GLS and
FGLS given in Denition 4.8 is that some of the transformed cases (Zi , uTi )T
can be outliers or high leverage points.

Remark 4.3. If the response Yi is the sample mean or sample median of

ni cases where the ni are not all equal, then use WLS with weights wi = ni .
See Sheather (2009, p. 121).
170 4 WLS and Generalized Least Squares

4.3 Inference for GLS

Inference for the GLS model Y = X + e can be performed by using the

partial F test for the equivalent no intercept OLS model Z = U + .
Following Section 2.10, create Z and U , t the full and reduced model using
the no intercept or intercept = F option. Let pval be the estimated
pvalue.
The 4 step partial F test of hypotheses: i) State the hypotheses Ho:
the reduced model is good Ha: use the full model
ii) Find the test statistic FR =
 
SSE(R) SSE(F )
/M SE(F )
dfR dfF

iii) Find the pval = P(FdfR dfF ,dfF > FR ). (On exams often an F table is
used. Here dfR dfF = pq = number of parameters set to 0, and dfF = np.)
iv) State whether you reject Ho or fail to reject Ho. Reject Ho if pval
and conclude that the full model should be used. Otherwise, fail to reject Ho
and conclude that the reduced model is good.

Assume that the GLS model contains a constant 1 . The GLS ANOVA F
test of Ho : 2 = = p versus Ha: not Ho uses the reduced model that
contains the rst column of U . The GLS ANOVA F test of Ho : i = 0
versus Ho : i = 0 uses the reduced model with the ith column of U deleted.
For the special case of WLS, the software will often have a weights option
that will also give correct output for inference.

Example 4.3. Suppose that the data from Example 4.2 has valid weights,
so that WLS can be used instead of FWLS. The R commands below per-
form WLS.

> ls.print(lsfit(dsx,dsy,wt=dsw))
Residual Standard Error=1.137
R-Square=0.9209
F-statistic (df=1, 33)=384.4139, p-value=0
Estimate Std.Err t-value Pr(>|t|)
Intercept -0.8891 0.3004 -2.9602 0.0057
X 1.1648 0.0594 19.6065 0.0000

Alternative R commands given below produce similar output.

zout<-lm(dsy~dsx,weights=dsw)
summary(zout)
anova(zout)
zoutr<-lm(dsy~1,weights=dsw)
anova(zoutr,zout)
4.3 Inference for GLS 171

The F statistic 384.4139 tests Ho : 2 = 0 since weights were used. The

WLS ANOVA F test for Ho : 2 = 0 can also be found with the no intercept
model by adding a column of ones to x, form U and Z and compute the
partial F test where the reduced model uses the rst column of U . Notice that
the intercept=F option needs to be used to t both models. The residual
standard error = RSE = M SE. Thus SSE = (n k)(RSE)2 where n k
is the denominator degrees of freedom for the F test and k is the numerator
degrees of freedom = number of variables in the model. The column of ones
xone is counted as a variable. The last line of output computes the partial F
statistic and is again 384.4.

> xone <- 1 + 0*1:35

> x <- cbind(xone,dsx)
> z <- as.vector(diag(sqrt(dsw))%*%dsy)
> u <- diag(sqrt(dsw))%*%x
> ls.print(lsfit(u,z,intercept=F))
Residual Standard Error=1.137, R-Square=0.9817
F-statistic (df=2, 33)=886.4982, p-value=0
Estimate Std.Err t-value Pr(>|t|)
xone -0.8891 0.3004 -2.9602 0.0057
dsx 1.1648 0.0594 19.6065 0.0000

> ls.print(lsfit(u[,1],z,intercept=F))
Residual Standard Error=3.9838, R-Square=0.7689
F-statistic (df=1, 34)=113.1055, p-value=0
Estimate Std.Err t-value Pr(>|t|)
X 4.5024 0.4234 10.6351 0
> ((34*(3.9838)^2-33*(1.137)^2)/1)/(1.137)^2
[1] 384.4006

The WLS t-test for this data has t = 19.6065 which corresponds to F =
t2 = 384.4 since this test is equivalent to the WLS ANOVA F test when there
is only one predictor. The WLS t-test for the intercept has F = t2 = 8.76.
This test statistic can be found from the no intercept OLS model by leaving
the rst column of U out of the model, then perform the partial F test as
shown below.

> ls.print(lsfit(u[,2],z,intercept=F))
Residual Standard Error=1.2601
F-statistic (df=1, 34)=1436.300
Estimate Std.Err t-value Pr(>|t|)
X 1.0038 0.0265 37.8985 0

> ((34*(1.2601)^2-33*(1.137)^2)/1)/(1.137)^2
[1] 8.760723
172 4 WLS and Generalized Least Squares

4.4 Complements

The theory for GLS and WLS is similar to the theory for the OLS MLR
model, but the theory for FGLS and FWLS is often lacking or huge sam-
ple sizes are needed. However, FGLS and FWLS are often used in practice
because usually V is not known and V must be used instead. Kariya and
Kurata (2004) is a PhD level text covering FGLS. Cook and Zhang (2015)
suggest an envelope method for WLS.
Shi and Chen (2009) describe numerical diagnostics for GLS. Long and
Ervin (2000) discuss methods for obtaining standard errors when the constant
variance assumption is violated.
Following Sheather (2009, ch. 9, ch. 10) many linear models with seri-
ally correlated errors (e.g. AR(1) errors) and many linear mixed models
can be t with FGLS. Both Sheather (2009) and Houseman et al. (2004)
use the Cholesky decomposition and make the residual plots based on the
Cholesky residuals Z Z where V () = KK T . We recommend plots based
on Z Z where V () = RR. In other words, use transformation corre-
sponding to Proposition 4.2 instead of the transformation corresponding to
Proposition 4.1.

4.5 Problems

Problems with an asterisk * are especially important.

R Problems

Use the command source(G:/lregpack.txt) to download the func-

tions and the command source(G:/lregdata.txt) to download the data.
See Preface or Section 14.1. Typing the name of the lregpack function,
e.g. wlsplot, will display the code for the function. Use the args command,
e.g. args(wlsplot), to display the needed arguments for the function. For some
of the following problems, the R commands can be copied and pasted from
(http://lagrange.math.siu.edu/Olive/lreghw.txt) into R.
4.1. Generalized and weighted least squares are each equivalent to a least
squares regression without intercept. Let V = diag(1, 1/2, 1/3, . . . , 1/9) =
diag(1/wi ) where n = 9 and the weights wi = i for i = 1, . . . , 9. Let xT =
(1, x1 , x2 , x3 ). Then the weighted least squares with weight vector wT =

(1, 2, . . . , 9) should be equivalent to the OLS regression of wi Yi = Zi on

u where uT = wi x = ( wi , wi x1 , wi x2 , wi x3 ). There is no intercept

because the vector of ones has been replaced by a vector of the wi s. Copy
and paste the commands for this problem into R, and include the output
from both lst commands. The coecients from both lst commands should
be the same.
4.5 Problems 173

4.2. Download the wlsplot function and the Draper and Smith (1981)
data dsx, dsy, dsw.
a) Enter the R command wlsplot(x=dsx, y = dsy, w = dsw) to re-
produce Figure 4.1. Once you have the plot you can print it out directly, but
it will generally save paper by placing the plots in the Word editor.
b) Activate Word (often by double clicking on a Word icon). Click on the
screen and type Problem 4.2. In R, click on the plot and then press the
keys Ctrl and c simultaneously. This procedure makes a temporary copy of
the plot. In Word, move the pointer to Edit and hold down the leftmost mouse
button. This will cause a menu to appear. Drag the pointer down to Paste.
In the future, these menu commands will be denoted by Edit>Paste. The
plot should appear on the screen. To save your output on your ash drive
(J, say), use the Word menu commands File > Save as. In the Save in box
select Removable Disk (J:) and in the File name box enter HW4d2.doc. To
exit from Word, click on the X in the upper right corner of the screen. In
Word a screen will appear and ask whether you want to save changes made
in your document. Click on No. To exit from R, type q() or click on the
X in the upper right corner of the screen and then click on No.

4.3. Download the fwlssim function. This creates WLS data if type
is 1 or 3 and FWLS data if type is 2 or 4. Let the sucient predictor
SP = 25 + 2x2 + + 2xp . Then Y = SP + |SP 25k|e where the xij and
ei are iid N (0, 1). Thus Y |SP N (SP, (SP 25k)2 2 ). If type is 1 or 2,
then k = 1/5, but k = 1 if type is 3 or 4. The default has 2 = 1.
The function creates the OLS response and residual plots and the FWLS
(or WLS) response and residual plots.
a) Type the following command several times. The OLS and WLS plots
tend to look the same.

fwlssim(type=1)

b) Type the following command several times. Now the FWLS plots often
have outliers.

fwlssim(type=2)

c) Type the following command several times. The OLS residual plots have
a saddle shape, but the WLS plots tend to have highly skewed tted values.

fwlssim(type=3)

d) Type the following command several times. The OLS residual plots
have a saddle shape, but the FWLS plots tend to have outliers and highly
skewed tted values.

fwlssim(type=4)
Chapter 5
One Way Anova

Chapters 59 consider experimental design models. These models are linear

models, and many of the techniques used for multiple linear regression can
be used for experimental design models. In particular, least squares, response
plots, and residual plots will be important. These models have been used
to greatly increase agricultural yield, greatly improve medicine, and greatly
improve the quality of manufactured goods. The models are also good for
screening out good ideas from bad ideas (e.g., for a medical treatment for
heart disease or for improving the gas mileage of a car).

Denition 5.1. Models in which the response variable Y is quantitative,

but all of the predictor variables are qualitative are called analysis of vari-
ance (ANOVA or Anova) models, experimental design models, or design of
experiments (DOE) models. Each combination of the levels of the predictors
gives a dierent distribution for Y . A predictor variable W is often called a
factor and a factor level ai is one of the categories W can take.

5.1 Introduction

Denition 5.2. A lurking variable is not one of the variables in the

study, but may aect the relationships among the variables in the study.
A unit is the experimental material assigned treatments, which are the
conditions the investigator wants to study. The unit is experimental if it was
randomly assigned to a treatment, and the unit is observational if it was not
randomly assigned to a treatment.

Denition 5.3. In an experiment, the investigators use randomiza-

tion to assign treatments to units. To assign p treatments to n = n1 + +np
experimental units, draw a random permutation of {1, . . . , n}. Assign the rst

Springer International Publishing AG 2017 175

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 5
176 5 One Way Anova

n1 units treatment 1, the next n2 units treatment 2, . . . , and the nal np units
treatment p.

Randomization allows one to do valid inference such as F tests of hy-

potheses and condence intervals. Randomization also washes out the eects
of lurking variables and makes the p treatment groups similar except for the
treatment. The eects of lurking variables are present in observational studies
dened in Denition 5.4.

Denition 5.4. In an observational study, investigators simply observe

the response, and the treatment groups need to be p random samples from p
populations (the levels) for valid inference.

Example 5.1. Consider using randomization to assign the following nine

people (units) to three treatment groups.

Carroll, Collin, Crawford, Halverson, Lawes,

Stach, Wayman, Wenslow, Xumong

Balanced designs have the group sizes the same: ni m = n/p. Label the
units alphabetically so Carroll gets 1, . . . , Xumong gets 9. The R function
sample can be used to draw a random permutation. Then the rst 3 numbers
in the permutation correspond to group 1, the next 3 to group 2, and the nal
3 to group 3. Using the output shown below gives the following 3 groups.

group 1: Stach, Wayman, Xumong

group 2: Lawes, Carroll, Halverson
group 3: Collin, Wenslow, Crawford

> sample(9)
[1] 6 7 9 5 1 4 2 8 3

Often there is a table or computer le of units and related measurements,

and it is desired to add the units group to the end of the table. The lregpack
function rand reports a random permutation and the quantity groups[i] =
treatment group for the ith person on the list. Since persons 6, 7, and 9 are in
group 1, groups[7] = 1. Since Carroll is person 1 and is in group 2, groups[1]
= 2, et cetera.

> rand(9,3)
$perm
[1] 6 7 9 5 1 4 2 8 3

$groups
[1] 2 3 3 2 2 1 1 3 1

Denition 5.5. Replication means that for each treatment, the ni re-
sponse variables Yi,1 , . . . , Yi,ni are approximately iid random variables.
5.2 Fixed Eects One Way Anova 177

Example 5.2. a) If ten students work two types of paper mazes three
times each, then there are 60 measurements that are not replicates. Each
student should work the six mazes in random order since speed increases
with practice. For the ith student, let Zi1 be the average time to complete
the three mazes of type 1, let Zi2 be the average time for mazes of type 2,
and let Di = Zi1 Zi2 . Then D1 , . . . , D10 are replicates.
b) Cobb (1998, p. 126) states that a student wanted to know if the shapes
of sponge cells depends on the color (green or white). He measured hundreds
of cells from one white sponge and hundreds of cells from one green sponge.
There were only two units so n1 = 1 and n2 = 1. The student should have
used a sample of n1 green sponges and a sample of n2 white sponges to get
more replicates.
c) Replication depends on the goals of the study. Box et al. (2005, pp. 215
219) describe an experiment where the investigator times how long it takes
him to bike up a hill. Since the investigator is only interested in his perfor-
mance, each run up a hill is a replicate (the time for the ith run is a sample
from all possible runs up the hill by the investigator). If the interest had been
on the eect of eight treatment levels on student bicyclists, then replication
would need n = n1 + + n8 student volunteers where ni ride their bike up
the hill under the conditions of treatment i.

5.2 Fixed Eects One Way Anova

The one way Anova model is used to compare p treatments. Usually there
is replication and Ho: 1 = 2 = = p is a hypothesis of interest. In-
vestigators may also want to rank the population means from smallest to
largest.
Denition 5.6. Let fZ (z) be the pdf of Z. Then the family of pdfs
fY (y) = fZ (y ) indexed by the location parameter , < < ,
is the location family for the random variable Y = + Z with standard
pdf fZ (z).
Denition 5.7. A one way xed eects Anova model has a single qualita-
tive predictor variable W with p categories a1 , . . . , ap . There are p dierent
distributions for Y , one for each category ai . The distribution of

Y |(W = ai ) fZ (y i )

where the location family has second moments. Hence all p distributions come
from the same location family with dierent location parameter i and the
same variance 2 .
Denition 5.8. The one way xed eects normal Anova model is the spe-
cial case where
178 5 One Way Anova

Y |(W = ai ) N (i , 2 ).

Example 5.3. The pooled 2 sample ttest is a special case of a one way
Anova model with p = 2. For example, one population could be ACT scores
for men and the second population ACT scores for women. Then W = gender
and Y = score.

Notation. It is convenient to relabel the response variable Y1 , . . . , Yn as

the vector Y = (Y11 , . . . , Y1,n1 , Y21 , . . . , Y2,n2 , . . . , Yp1 , . . . , Yp,np )T where the
Yij are independent and Yi1 , . . . , Yi,ni are iid. Here j = 1, . . . , ni where ni is
the number of cases from the ith level where i = 1, . . . , p. Thus n1 + +np =
n. Similarly use double subscripts on the errors. Then there will be many
equivalent parameterizations of the one way xed eects Anova model.

Denition 5.9. The cell means model is the parameterization of the one
way xed eects Anova model such that

Yij = i + eij

where Yij is the value of the response variable for the jth trial of the ith
factor level. The i are the unknown means and E(Yij ) = i . The eij are
iid from the location family with pdf fZ (z) and unknown variance 2 =
VAR(Yij ) = VAR(eij ). For the normal cell means model, the eij are iid
N (0, 2 ) for i = 1, . . . , p and j = 1, . . . , ni .

The cell means model is a linear model (without intercept) of the form
Y = X c c + e =

Y11 100 ... 0 e11
.. .. .. .. .. ..
. . . . . .

Y1,n1 1 0 0 ... 0 e1,n1
e
Y21 0 1 0 ... 0
1 21
.. .. .. .. .. ..
. . . . . .

2
.. + .
Y2,n2 = 0 1 0 ... 0 (5.1)
. e2,n2
. . . . .. .
.. .. .. .. . p ..

Yp,1 0 0 0 ... 1 ep,1

. . . . .. .
.. .. .. .. . .
.
Yp,np 0 0 0 ... 1 ep,np
ni
Notation. Let Yi0 = j=1 Yij and let

1 
ni
i = Y i0 = Yi0 /ni = Yij . (5.2)
ni j=1
5.2 Fixed Eects One Way Anova 179

Hence the dot notation  means

sum over the subscript corresponding to the
p ni
0, e.g. j. Similarly, Y00 = i=1 j=1 Yij is the sum of all of the Yij .

Let X c = [v 1 v 2 v p ], and notice that the indicator variables used

in the cell means model (5.1) are v hk = xhk = 1 if the hth case has W = ak ,
and v hk = xhk = 0, otherwise, for k = 1, . . . , p and h = 1, . . . , n. So Yij
has xhk = 1 only if i = k and j = 1, ..., ni . The model can use p indicator
variables for the factor instead of p 1 indicator variables because the model
does not contain an intercept. Also notice that (X Tc X c ) = diag(n1 , . . . , np ),

E(Y ) = X c c = (1 , . . . , 1 , 2 , . . . , 2 , . . . , p , . . . , p )T ,

and X Tc Y = (Y10 , . . . , Y10 , Y20 , . . . , Y20 , . . . , Yp0 , . . . , Yp0 )T . Hence (X Tc X c )1

= diag(1/n1 , . . . , 1/np ) and the OLS estimator

c = (X Tc X c )1 X Tc Y = (Y 10 , . . . , Y p0 )T = (1 , . . . , p )T .

Thus Y = X c c = (Y 10 , . . . , Y 10 , . . . , Y p0 , . . . , Y p0 )T . Hence the ijth tted

value is
Yij = Y i0 = i (5.3)
and the ijth residual is

rij = Yij Yij = Yij i . (5.4)

Since the cell means model is a linear model, there is an associated response
plot and residual plot. However, many of the interpretations of the OLS
quantities for Anova models dier from the interpretations for MLR models.
First, for MLR models, the conditional distribution Y |x makes sense even if
x is not one of the observed xi provided that x is not far from the xi . This
fact makes MLR very powerful. For MLR, at least one of the variables in x
is a continuous predictor. For the one way xed eects Anova model, the p
distributions Y |xi make sense where xTi is a row of X c .
Also, the OLS MLR ANOVA F test for the cell means model tests H0 :
c = 0 H0 : 1 = = p = 0, while the one way xed eects ANOVA F
test given after Denition 5.13 tests H0 : 1 = = p .
Denition 5.10. Consider the one way xed eects Anova model. The
response plot is a plot of Yij i versus Yij and the residual plot is a plot of
Yij i versus rij .
The points in the response plot scatter about the identity line and the
points in the residual plot scatter about the r = 0 line, but the scatter need
not be in an evenly populated band. A dot plot of Z1 , . . . , Zm consists of an
axis and m points each corresponding to the value of Zi . The response plot
consists of p dot plots, one for each value of i . The dot plot corresponding
to i is the dot plot of Yi1 , . . . , Yi,ni . The p dot plots should have roughly the
180 5 One Way Anova

same amount of spread, and each i corresponds to level ai . If a new level

af corresponding to xf was of interest, hopefully the points in the response
plot corresponding to af would form a dot plot at f similar in spread to
the other dot plots, but it may not be possible to predict the value of f .
Similarly, the residual plot consists of p dot plots, and the plot corresponding
to i is the dot plot of ri1 , . . . , ri,ni .
Assume that each ni 10. Under the assumption that the Yij are from
the same location family with dierent parameters i , each of the p dot plots
should have roughly the same shape and spread. This assumption is easier
to judge with the residual plot. If the response plot looks like the residual
plot, then a horizontal line ts the p dot plots about as well as the identity
line, and there is not much dierence in the i . If the identity line is clearly
superior to any horizontal line, then at least some of the means dier.
Denition 5.11. An outlier corresponds to a case that is far from the
bulk of the data. Look for a large vertical distance of the plotted point from
the identity line or the r = 0 line.
Rule of thumb 5.1. Mentally add 2 lines parallel to the identity line and
2 lines parallel to the r = 0 line that cover most of the cases. Then a case is
an outlier if it is well beyond these 2 lines.
This rule often fails for large outliers since often the identity line goes
through or near a large outlier so its residual is near zero. A response that is
far from the bulk of the data in the response plot is a large outlier (large
in magnitude). Look for a large gap between the bulk of the data and the
large outlier.
Suppose there is a dot plot of nj cases corresponding to level aj that is
far from the bulk of the data. This dot plot is probably not a cluster of bad
outliers if nj 4 and n 5p. If nj = 1, such a case may be a large outlier.

Rule of thumb 5.2. Often an outlier is very good, but more often an
outlier is due to a measurement error and is very bad.
The assumption of the Yij coming from the same location family with
dierent location parameters i and the same constant variance 2 is a big
assumption and often does not hold. Another way to check this assumption is
to make a box plot of the Yij for each i. The box in the box plot corresponds
to the lower, middle, and upper quartiles of the Yij . The middle quartile
is just the sample median of the data mij : at least half of the Yij mij
and at least half of the Yij mij . The p boxes should be roughly the same
length and the median should occur in roughly the same position (e.g., in
the center) of each box. The whiskers in each plot should also be roughly
similar. Histograms for each of the p samples could also be made. All of the
histograms should look similar in shape.
Example 5.4. Kuehl (1994, p. 128) gives data for counts of hermit crabs
on 25 dierent transects in each of six dierent coastline habitats. Let Z be
the count. Then the response variable Y = log10 (Z + 1/6). Although the
5.2 Fixed Eects One Way Anova 181

counts Z varied greatly, each habitat had several counts of 0 and often there
were several counts of 1, 2, or 3. Hence Y is not a continuous variable. The cell
means model was t with ni = 25 for i = 1, . . . , 6. Each of the six habitats
was a level. Figure 5.1a and b shows the response plot and residual plot.
There are 6 dot plots in each plot. Because several of the smallest values in
each plot are identical, it does not always look like the identity line is passing
through the six sample means Y i0 for i = 1, . . . , 6. In particular, examine the
dot plot for the smallest mean (look at the 25 dots furthest to the left that
fall on the vertical line FIT 0.36). Random noise (jitter) has been added to
the response and residuals in Figure 5.1c and d. Now it is easier to compare
the six dot plots. They seem to have roughly the same spread.

a) Response Plot b) Residual Plot

2

1
RESID
1
Y

0
-1
0

0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.4 0.5 0.6 0.7 0.8 0.9 1.0
FIT FIT

c) Jittered Response Plot d) Jittered Residual Plot

2

1
JR
JY
1

0
-1
0

0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.4 0.5 0.6 0.7 0.8 0.9 1.0
FIT FIT

Fig. 5.1 Plots for Crab Data

The plots contain a great deal of information. The response plot can be
used to explain the model, check that the sample from each population (treat-
ment) has roughly the same shape and spread, and to see which populations
have similar means. Since the response plot closely resembles the residual plot
in Figure 5.1, there may not be much dierence in the six populations. Lin-
earity seems reasonable since the samples scatter about the identity line. The
residual plot makes the comparison of similar shape and spread easier.
182 5 One Way Anova

Denition 5.12. a) The total sum of squares


p 
ni
SST O = (Yij Y 00 )2 .
i=1 j=1

b) The treatment sum of squares


p
SST R = ni (Y i0 Y 00 )2 .
i=1

c) The residual sum of squares or error sum of squares


p 
ni
SSE = (Yij Y i0 )2 .
i=1 j=1

Denition 5.13. Associated with each SS in Denition 5.12 is a degrees

of freedom (df) and a mean square = SS/df. For SSTO, df = n 1 and
M ST O = SST O/(n1). For SSTR, df = p1 and M ST R = SST R/(p1).
For SSE, df = n p and M SE = SSE/(n p).
ni
Let Si2 = j=1 (Yij Y i0 )2 /(ni 1) be the sample variance of the ith
group. Then the MSE is a weighted sum of the Si2 :

1  1 
p ni i p n
2 = M SE = 2
rij = (Yij Y i0 )2 =
n p i=1 j=1 n p i=1 j=1

1 
p
(ni 1)Si2 = Spool
2
n p i=1
2
where Spool is known as the pooled variance estimator.

The ANOVA F test tests whether the p means are equal. If Ho is not
rejected and the means are equal, then it is possible that the factor is unim-
portant, but it is also possible that the factor is important but the
level is not. For example, the factor might be type of catalyst. The yield
may be equally good for each type of catalyst, but there would be no yield if
no catalyst was used.
The ANOVA table is the same as that for MLR, except that SSTR re-
places the regression sum of squares. The MSE is again an estimator of 2 .
The ANOVA F test tests whether all p means i are equal. Shown below is
an ANOVA table given in symbols. Sometimes Treatment is replaced by
Between treatments, Between Groups, Model, Factor, or Groups.
Sometimes Error is replaced by Residual, or Within Groups. Some-
times p-value is replaced by P, P r(> F ), or PR > F. The p-value
is nearly always an estimated p-value, denoted by pval.
5.2 Fixed Eects One Way Anova 183

Summary Analysis of Variance Table

Source df SS MS F p-value
Treatment p-1 SSTR MSTR Fo=MSTR/MSE for Ho:
Error n-p SSE MSE 1 = = p

Be able to perform the 4 step xed eects one way ANOVA F

test of hypotheses.
i) State the hypotheses Ho: 1 = 2 = = p and Ha: not Ho.
ii) Find the test statistic Fo = M ST R/M SE or obtain it from output.
iii) Find the pval from output or use the F table: pval =

P (Fp1,np > Fo ).

iv) State whether you reject Ho or fail to reject Ho. If the pval , reject Ho
and conclude that the mean response depends on the factor level. (Hence not
all of the treatment means are equal.) Otherwise fail to reject Ho and conclude
that the mean response does not depend on the factor level. (Hence all of the
treatment means are equal, or there is not enough evidence to conclude that
the mean response depends on the factor level.) Give a nontechnical sentence.

Rule of thumb 5.3. If

max(S1 , . . . , Sp ) 2 min(S1 , . . . , Sp ),

then the one way ANOVA F test results will be approximately correct if the
response and residual plots suggest that the remaining one way Anova model
assumptions are reasonable. See Moore (2007, p. 634). If all of the ni 5,
replace the standard deviations by the ranges of the dot plots when exam-
ining the response and residual plots. The range Ri = max(Yi,1 , . . . , Yi,ni )
min(Yi,1 , . . . , Yi,ni ) = length of the ith dot plot for i = 1, . . . , p.

The assumption that the zero mean iid errors have constant variance
V (eij ) 2 is much stronger for the one way Anova model than for the mul-
tiple linear regression model. The assumption implies that the p population
distributions have pdfs from the same location family with dierent means
1 , . . . , p but the same variances 12 = = p2 2 . The one way ANOVA
F test has some resistance to the constant variance assumption, but con-
dence intervals have much less resistance to the constant varianceassumption.

Consider condence intervals for i such as Y i0 tni 1,1/2 M SE/ ni .
MSE is a weighted average of the Si2 . Hence MSE overestimates small i
2
2 2
and underestimates large i when the i are not equal. Hence using M SE
instead of Si will make the CI too long or too short, and Rule of thumb 5.3
does not apply to condence intervals based on MSE.
184 5 One Way Anova

Remark 5.1. If the units are a representative sample of some population

of interest, then randomization of units into groups makes the assumption
that Yi1 , . . . , Yi,ni are iid hold to a useful approximation for large sample
theory. Random sampling from populations also induces the iid assumption.
Linearity can be checked with the response plot, and similar shape and spread
of the location families can be checked with both the response and residual
plots. Also check that outliers are not present. If the p dot plots in the
response plot are approximately symmetric, then the sample sizes ni can
be smaller than if the dot plots are skewed.

Remark 5.2. When the assumption that the p groups come from the
same location family with nite variance 2 is violated, the one way ANOVA
F test may not make much sense because unequal means may not imply the
superiority of one category over another. Suppose Y is the time in minutes
until relief from a headache and that Y1j N (60, 1) while Y2j N (65, 2 ).
If 2 = 1, then the type 1 medicine gives headache relief 5 minutes faster, on
average, and is superior, all other things being equal. But if 2 = 100, then
many patients taking medicine 2 experience much faster pain relief than those
taking medicine 1, and many experience much longer time until pain relief.
In this situation, predictor variables that would identify which medicine is
faster for a given patient would be very useful.

Example 5.5. The output below represents grams of fat (minus 100
grams) absorbed by doughnuts using 4 types of fat. See Snedecor and Cochran
(1967, p. 259). Let i denote the mean amount of fati absorbed by doughnuts,
i = 1, 2, 3 and 4. a) Find 1 . b) Perform a 4 step ANOVA F test.
n1
Solution: a) 1c = 1 = Y 10 = Y10 /n1 = j=1 Y1j /n1 =
(64 + 72 + 68 + 77 + 56 + 95)/6 = 432/6 = 72.
b) i) H0 : 1 = 2 = 3 = 4 Ha : not H0
ii) F = 5.41
iii) pval = 0.0069
iv) Reject H0 , the mean amount of fat absorbed by doughnuts depends on
the type of fat.

fat1 fat2 fat3 fat4

64 78 75 55
72 91 93 66
68 97 78 49
77 82 71 64
56 85 63 70
95 77 76 68
One way Anova for Fat1 Fat2 Fat3 Fat4
Source DF SS MS F P
treatment 3 1636.5 545.5 5.41 0.0069
error 20 2018.0 100.9
5.2 Fixed Eects One Way Anova 185
p p
Denition 5.14. A contrast C = ki i where ki = 0. The
p i=1 i=1
estimated contrast is C = i=1 ki Y i0 .
If the null hypothesis of the xed eects one way ANOVA test is not true,
then not all of the means i are equal. Researchers will often have hypotheses,
before examining the data, that they desire to test. Often such a hypothesis
can be put in the form of a contrast. For example, the contrast C = i j
is used to compare the means of the ith and jth groups while the contrast
1 (2 + + p )/(p 1) is used to compare the last p 1 groups with
the 1st group. This contrast is useful when the 1st group corresponds to a
standard or control treatment while the remaining groups correspond to new
treatments.
Assume that the normal cell means model is a useful approximation to the
data. Then the Y i0 N (i , 2 /ni ) are independent, and
 
p p
ki2
C = ki Y i0 N C, 2
.
i=1
n
i=1 i

Hence the standard error



 p
ki2
SE(C) = M SE .
n
i=1 i

The degrees of freedom is equal to the MSE degrees of freedom p= n p.

Consider
p a family of null hypotheses for contrasts {Ho : i=1 ki i = 0
where i=1 ki = 0 and the ki may satisfy other constraints}. Let S denote
the probability of a type I error for a single test from the family where a
type I error is a false rejection. The family level F is an upper bound
on the (usually unknown) size T . Know how to interpret F T = P(of
making at least one type I error among the family of contrasts).
Two important families of contrasts are the family of all possible con-
trasts and the family of pairwise dierences Cij = i j where i = j. The
Schee multiple comparisons procedure has a F for the family of all possible
# $ the Tukey multiple comparisons procedure has a F for the
contrasts, while
family of all p2 pairwise contrasts.
To interpret output for multiple comparisons procedures, the underlined
means or blocks of letters besides groups of means indicate that the group of
means are not signicantly dierent.
Example 5.6. The output below uses data from SAS Institute (1985,
pp. 126129). The mean nitrogen content of clover depends on the strain
of clover (3dok1, 3dok5, 3dok7, compos, 3dok4, 3dok13). Recall that means
1 and 2 are signicantly dierent if you can conclude that 1 = 2 while
1 and 2 are not signicantly dierent if there is not enough evidence to
conclude that 1 = 2 (perhaps because the means are approximately equal
or perhaps because the sample sizes are not large enough).
186 5 One Way Anova

Notice that the strain of clover 3dok1 appears to have the highest mean
nitrogen content. There are 4 pairs of means that are not signicantly dier-
ent. The letter B suggests 3dok5 and 3dok7, the letter C suggests 3dok7 and
compos, the letter D suggests compos and 3dok4, while the letter E suggests
3dok4 and 3dok13 are not signicantly dierent.

Means with the same letter are not significantly

different.
Waller Grouping Mean N strain
A 28.820 5 3dok1

B 23.980 5 3dok5
B
C B 19.920 5 3dok7
C
C D 18.700 5 compos
D
E D 14.640 5 3dok4
E
E 13.260 5 3dok13

Remark 5.3. Two graphical methods can also be used. Recall from Chap-
ter 1 that a response plot is an estimated sucient summary plot. If n is not
too small, each ni 5, and the sample mean (where the dot plot crosses
the identity line) for one dot plot is below or above another dot plot, then
conclude that the population mean corresponding to the higher dot plot is
greater than the sample mean corresponding to the lower dot plot. As the
ni increase, the sample mean of one dot plot only needs to be above or be-
low most of the cases in the other dot plot. The p population means may or
may not be equal if all p of the dot plots have lots of overlap. This will hap-
pen, for example, if the response plot looks like the residual plot. Hence this
graphical method is inconclusive for Figure 5.1a. Remark 5.2 gives another
situation where this graphical method can fail. An advantage of this graphi-
cal method is that the p populations do not need to come from populations
with the same variance or from the same location scale family as long as OLS
gives a consistent estimator of . The second graphical method is given in
Denition 5.15.

Example 5.6, continued: Figure 5.2 shows the response and residual
plots for the clover data. The plots suggest the constant variance assumption
is not reasonable. The population means may or may not dier for the groups
with the two smallest sample means, but these two groups appear to have
smaller population means than the other groups. Similarly, the population
means may or may not dier for the two groups with sample means near
5.2 Fixed Eects One Way Anova 187

Response Plot

30
Y
20
10

15 20 25
FIT

Residual Plot
4
RES
2
8

15 20 25
FIT
Fig. 5.2 Response and Residual Plots for Clover Data

20, but these two groups appear to have population means that are smaller
than the two groups with the largest sample means. The population means
of these last two groups may or may not dier. Figure 5.2 was made with the
following commands, using the lregpack function aovplots.

x<-c(1,1,1,1,1,2,2,2,2,2,3,3,3,3,3,4,4,4,4,4,5,5,5,5,
5,6,6,6,6,6)

y<-c(19.4,32.6,27.0,32.1,33.0,17.7,24.8,27.9,25.2,
24.3,17.0,19.4,9.1,11.9,15.8,20.7,21.0,20.5,18.8,
18.6,14.3,14.4,11.8,11.6,14.2,17.3,19.4,19.1,16.9,
20.8)

x <- factor(x)
z <- aov(y~x)
aovplots(Y=y,FIT=fitted(z),RES=resid(z))
#right click stop twice

Denition 5.15. Graphical Anova for the one way model uses the
residuals as a reference set instead of a t, F , or normal distribution. The
scaled treatment deviations or scaled eect c(Y i0 Y 00 ) = c(i Y 00 )
are scaled to have the same variability as the residuals. A dot plot of the
scaled deviations is placed above the dot plot of the residuals. Assume that
188 5 One Way Anova

ni m = n/p for i = 1, . . . , p. For small n 40, suppose the distance be-

tween two scaled deviations (A and B, say) is greater than the range of the
residuals = max(rij ) min(rij ). Then declare A and B to be signicantly
dierent. If the distance is less than the range, do not declare A and B to
be signicantly dierent. Scaled deviations that lie outside the range of the
residuals are signicant (so signicantly dierent from the overall mean).

For n 100, let r(1) r(2) r(n) be the order statistics of the
residuals. Then instead of the range, use r(0.975n) r(0.025n) as the dis-
tance where x is the smallest integer x, e.g. 7.7 = 8. So eects outside
of the interval (r(0.025n) , r(0.975n) ) are signicant. See Box et al. (2005,

pp. 136, 166). A derivation of the scaling constant c = (n p)/(p 1) is
given in Section 5.6.

Scaled Treatment Deviations

20 18
graphicalanova
14 16
12
10

0.04 0.00 0.02 0.04 0.06

Residuals
Fig. 5.3 Graphical Anova

ganova(x,y)
smn 0.0296 0.0661 -0.0508 -0.0449
Treatments "A" "B" "C" "D"

Example 5.7. Cobb (1998) describes a one way Anova design used to
study the amount of calcium in the blood. For many animals, the bodys
ability to use calcium depends on the level of certain hormones in the blood.
The response was 1/(level of plasma calcium). The four groups were A: Fe-
male controls, B: Male controls, C: Females given hormone, and D: Males
5.3 Random Eects One Way Anova 189

given hormone. There were 10 birds of each gender, and ve from each gen-
der were given the hormone. The output above uses the lregpack function
ganova to produce Figure 5.3.

In Figure 5.3, the top dot plot has the scaled treatment deviations. From
left to right, these correspond to C, D, A, and B since the output shows that
the deviation corresponding to C is the smallest with value 0.050. Since the
deviations corresponding to C and D are much closer than the range of the
residuals, the C and D eects yielded similar mean response values. A and
B appear to be signicantly dierent from C and D. The distance between
the scaled A and B treatment deviations is about the same as the distance
between the smallest and largest residuals, so there is only marginal evidence
that the A and B eects are signicantly dierent.
Since all 4 scaled deviations lie outside of the range of the residuals, all
eects A, B, C, and D appear to be signicant.

5.3 Random Eects One Way Anova

Denition 5.16. For the random eects one way Anova, the levels of
the factor are a random sample of levels from some population of levels F .
The cell means model for the random eects one way Anova is Yij = i + eij
for i = 1, . . . , p and j = 1, . . . , ni . The i are randomly selected from some
population with mean and variance 2 , where i F is equivalent to
i . The eij and i are independent, and the eij are iid from a location
family with pdf f , mean 0, and variance 2 . The Yij |i f (y i ), the
location family with location parameter i and variance 2 . Unconditionally,
E(Yij ) = and V (Yij ) = 2 + 2 .

For the random eects model, the i are independent random variables
with E(i ) = and V (i ) = 2 . The cell means model for xed eects one
way Anova is very similar to that for the random eects model, but the i
are xed constants rather than random variables.

Denition 5.17. For the normal random eects one way Anova model,
N (, 2 ). Thus the i are independent N (, 2 ) random variables. The
eij are iid N (0, 2 ) and the eij and i are independent. For this model,
Yij |i N (i , 2 ) for i = 1, . . . , p. Note that the conditional variance 2 is
the same for each i . Unconditionally, Yij N (, 2 + 2 ).

The xed eects one way Anova tested Ho : 1 = = p . For the

random eects one way Anova, interest is in whether i for every i in
where the population is not necessarily nite. Note that if 2 = 0, then
i for all i . In the sample of p levels, the i will dier if 2 > 0.
190 5 One Way Anova

Be able to perform the 4 step random eects one way ANOVA

F test of hypotheses:
i) Ho : 2 = 0 Ha : 2 > 0
ii) Fo = M ST R/M SE is usually obtained from output.
iii) The pval = P (Fp1,np > Fo ) is usually obtained from output.
iv) If pval reject Ho, conclude that 2 > 0 and that the mean response
depends on the factor level. Otherwise, fail to reject Ho, conclude that 2 = 0
and that the mean response does not depend on the factor level. (Or there
is not enough evidence to conclude that the mean response depends on the
factor level.)

The ANOVA tables for the xed and random eects one way Anova models
are exactly the same, and the two F tests are very similar. The main dierence
is that the conclusions for the random eects model can be generalized to
the entire population of levels. For the xed eects model, the conclusions
only hold for the p xed levels. If Ho : 2 = 0 is true and the random eects
model holds, then the Yij are iid with pdf f (y ). So the F statistic for
the random eects test has an approximate Fp1,np distribution if the ni
are large by the results for the xed eects one way ANOVA test. For both
tests, the pval is an estimate of the population p-value.

Source df SS MS F P
brand 5 854.53 170.906 238.71 0.0000
error 42 30.07 0.716

Example 5.8. Data is from Kutner et al. (2005, problem 25.7). A re-
searcher is interested in the amount of sodium in beer. She selects 6 brands
of beer at random from 127 brands and the response is the average sodium
content measured from 8 cans of each brand.
a) State whether this is a random or xed eects one way Anova. Explain
briey.
b) Using the output above, perform the appropriate 4 step ANOVA F
test.
Solution: a) Random eects since the beer brands were selected at random
from a population of brands.
b) i) H0 : 2 = 0 Ha : 2 > 0
ii) F0 = 238.71
iii) pval = 0.0
iv) Reject H0 , so 2 > 0 and the mean amount of sodium depends on the
beer brand.

Remark 5.4. The response and residual plots for the random eects mod-
els are interpreted in the same way as for the xed eects model, except that
the dot plots are from a random sample of p levels instead of from p xed
levels.
5.4 Response Transformations for Experimental Design 191

5.4 Response Transformations for Experimental Design

A model for an experimental design is Yi = E(Yi ) + ei for i = 1, . . . , n where

the error ei = Yi E(Yi ) and E(Yi ) E(Yi |xi ) is the expected value of the
response Yi for a given vector of predictors xi . Many models can be t with
least squares (OLS or LS) and are linear models of the form

Yi = xi,1 1 + xi,2 2 + + xi,p p + ei = xTi + ei

for i = 1, . . . , n. Often xi,1 1 for all i. In matrix notation, these n equations

become
Y = X + e,
where Y is an n 1 vector of dependent variables, X is an n p design
matrix of predictors, is a p 1 vector of unknown coecients, and e is
an n 1 vector of unknown errors. If the tted values are Yi = xTi , then
Yi = Yi + ri where the residuals ri = Yi Yi .
The applicability of an experimental design model can be expanded by
allowing response transformations. An important class of response transfor-
mation models adds an additional unknown transformation parameter o ,
such that
( )
Yi = to (Zi ) Zi o = E(Yi ) + ei = xTi + ei .
If o was known, then Yi = to (Zi ) would follow the linear model for the
experimental design.

Denition 5.18. Assume that all of the values of the response Zi are
positive. A power transformation has the form Y = t (Z) = Z for = 0
and Y = t0 (Z) = log(Z) for = 0 where L = {1, 1/2, 0, 1/2, 1}.

A graphical method for response transformations computes the tted val-

ues Wi from the experimental design model using Wi = t (Zi ) as the re-
sponse. Then a plot of the W versus W is made for each of the ve values
of L . The plotted points follow the identity line in a (roughly) evenly
populated band if the experimental design model is reasonable for (W , W ).
An exception is the one way Anova model where there will be p dot plots
of roughly the same shape and spread that scatter about the identity line.
If more than one value of L gives a linear plot, consult subject matter
experts and use the simplest or most reasonable transformation. Also look
at the residual plots of the competing transformations. Note that L has 5
models, and the graphical method selects the model with the best response
plot. After selecting the transformation, the usual checks should be made. In
particular, the transformation plot is also the response plot, and a residual
plot should be made. The Equation (3.3) transformations could also be used.

Denition 5.19. A transformation plot is a plot of (W , W ) with the iden-

tity line added as a visual aid.
192 5 One Way Anova

In the following example, the plots show t (Z) on the vertical axis. The
label TZHAT of the horizontal axis are the tted values that result from
using t (Z) as the response in the software.

2.5
6

6
5

2.0

4
4

1/sqrt(Z)

LOG(Z)
1.5
1/Z
3

2
1.0
2

0
0.5
1
0

0.0

-2
1.2 1.4 1.6 1.8 2.0 2.2 0.7 0.8 0.9 1.0 1.1 1.0 1.5 2.0
TZHAT TZHAT TZHAT
20

400
15

300
sqrt(Z)

Z
10

200
100
5

0
0

3 4 5 10 20 30 40 50 60 70
TZHAT TZHAT

Fig. 5.4 Transformation Plots for Crab Data

For one way Anova models with ni m 5, look for a transformation

plot that satises the following conditions. i) The p dot plots scatter about
the identity line with similar shape and spread. ii) Dot plots with more skew
are worse than dot plots with less skew or dot plots that are approximately
symmetric. iii) Spread that increases or decreases with TZHAT is bad.
Example 5.4, continued. Following Kuehl (1994, p. 128), let C be the
count of crabs and let the response Z = C + 1/6. Figure 5.4 shows the
ve transformation plots. The transformation log(Z) results in dot plots that

have roughly the same shape and spread. The transformations 1/Z and 1/ Z
do not handle the 0 countswell, and the dot plots fail to cover the identity
line. The transformations Z and Z have variance that increases with the
mean. See Problem 5.13 to reproduce the plots.
Remark 5.5. The graphical method for response transformations can be
used for design models that are linear models, not just one way Anova models.
The method is nearly identical to that of Chapter 3, but L only has 5 values.
max(Zi )
The log rule states that if all of the Zi > 0 and if > 10, then the
min(Zi )
response transformation Y = log(Z) will often work.
5.5 Summary 193

5.5 Summary

1) The xed eects one way Anova model has one qualitative explanatory
variable called a factor and a quantitative response variable Yij . The factor
variable has p levels, E(Yij ) = i and V (Yij ) = 2 for i = 1, . . . , p and
j = 1, . . . , ni . Experimental units are randomly assigned to the treatment
levels.
2) Let n = n1 + + np . In an experiment, the investigators use random-
ization to randomly assign n units to treatments. Draw a random permutation
of {1, . . . , n}. Assign the rst n1 units to treatment 1, the next n2 units to
treatment 2, . . . , and the nal np units to treatment p. Use ni m = n/p if
possible. Randomization washes out the eect of lurking variables.
3) The 4 step xed eects one way ANOVA F test has steps
i) Ho: 1 = 2 = = p and Ha: not Ho.
ii) F o = MSTR/MSE is usually given by output.
iii) The pval = P(Fp1,np > F o) is usually given by output.
iv) If pval , reject Ho and conclude that the mean response depends
on the factor level. Otherwise fail to reject Ho and conclude that the mean
response does not depend on the factor level. (Hence all of the treatment
means are equal, or there is not enough evidence to conclude that the mean
response depends on the factor level.) Give a nontechnical sentence.

Summary Analysis of Variance Table

Source df SS MS F p-value
Treatment p-1 SSTR MSTR Fo=MSTR/MSE for Ho:
Error n-p SSE MSE 1 = = p

4) Shown is an ANOVA table given in symbols. Sometimes Treatment is

replaced by Between treatments, Between Groups, Model, Factor, or
Groups. Sometimes Error is replaced by Residual, or Within Groups.
Sometimes p-value is replaced by P, P r(> F ), or PR > F.
5) Boxplots and dot plots for each level are useful for this test. A dot plot
of Z1 , . . . , Zm consists of an axis and m points each corresponding to the
value of Zi . If all of the boxplots or dot plots are about the same for the
response plot, then the ANOVA F test may or may not fail to reject Ho. If
Ho is true, then Yij = + eij where the eij are iid with 0 mean and constant
variance 2 . Then = Y 00 and the factor levels do not help predict Yij .
6) Let fZ (z) be the pdf of Z. Then the family of pdfs fY (y) = fZ (y )
indexed by the location parameter , < < , is the location family
for the random variable Y = + Z with standard pdf fZ (y). A one way
xed eects Anova model has a single qualitative predictor variable W with
p categories a1 , . . . , ap . There are p dierent distributions for Y , one for each
category ai . The distribution of
194 5 One Way Anova

Y |(W = ai ) fZ (y i )

where the location family has second moments. Hence all p distributions
come from the same location family with dierent location parameter i and
the same variance 2 . The one way xed eects normal Anova model is the
special case where Y |(W = ai ) N (i , 2 ).
7) The response plot is a plot of Y versus Y . For the one way Anova model,
the response plot is a plot of Yij = i versus Yij . Often the identity line with
unit slope and zero intercept is added as a visual aid. Vertical deviations
from the identity line are the residuals rij = Yij Yij = Yij i . The plot
will consist of p dot plots that scatter about the identity line with similar
shape and spread if the xed eects one way Anova model is appropriate.
The ith dot plot is a dot plot of Yi,1 , . . . , Yi,ni . Assume that each ni 10. If
the response plot looks like the residual plot, then a horizontal line ts the p
dot plots about as well as the identity line, and there is not much dierence
in the i . If the identity line is clearly superior to any horizontal line, then
at least some of the means dier.
8) The residual plot is a plot of Y versus residual r = Y Y . The plot will
consist of p dot plots that scatter about the r = 0 line with similar shape
and spread if the xed eects one way Anova model is appropriate. The ith
dot plot is a dot plot of ri,1 , . . . , ri,ni . Assume that each ni 10. Under
the assumption that the Yij are from the same location family with dierent
parameters i , each of the p dot plots should have roughly the same shape
and spread. This assumption is easier to judge with the residual plot than
with the response plot.
9) Rule of thumb: If max(S1 , . . . , Sp ) 2 min(S1 , . . . , Sp ), then the one
way ANOVA F test results will be approximately correct if the response and
residual plots suggest that the remaining one way Anova model assumptions
are reasonable. Replace the Si by the ranges Ri of the dot plots in the residual
and response plots.
10) In an experiment, the investigators assign units to treatments. In
an observational study, investigators simply observe the response, and the
treatment groups need to be p random samples from p populations (the lev-
els). The eects of lurking variables are present in observational studies.
11) If a qualitative variable has c levels, represent it with c1 or c indicator
variables. Given a qualitative variable, know how to represent the data with
indicator variables.
12) The cell means model for the xed eects one way Anova is Yij =
i + eij where Yij is the value of the response variable for the jth trial
of the ith factor level for i = 1, . . . , p and j = 1, . . . , ni . The i are the
unknown means and E(Yij ) = i . The eij are iid from the location family
with pdf fZ (z), zero mean, and unknown variance 2 = V (Yij ) = V (eij ).
For the normal cell means model, the eij are iid N (0, 2 ). The estimator
ni
i = Y i0 = j=1 Yij /ni = Yij . The ith residual is rij = Yij Y i0 , and Y 00 is
5.5 Summary 195
p
the samplemeanof all of the Yij and n = i=1 ni . The total sum of squares
p ni
j=1 (Yij Y 00 ) , the treatment sum of squares SSTR =
2
SSTO =
p i=1
p ni
i=1 ni (Y i0 Y 00 ) , and the error sum of squares SSE = j=1 (Yij
2
i=1
2 2
Y i0 ) . The MSE is an estimator of . The ANOVA table is the same as
that for multiple linear regression, except that SSTR replaces the regression
sum of squares and that SSTO, SSTR, and SSE have n 1, p 1, and n p
degrees of freedom.
ni
13) Let Yi0 = j=1 Yij and let

1 
ni
i = Y i0 = Yi0 /ni = Yij .
ni j=1

Hence the dot notation  means sum over the subscript corresponding to the
p ni
0, e.g. j. Similarly, Y00 = i=1 j=1 Yij is the sum of all of the Yij . Be able
to nd i from data.
14) If the p treatment groups have the same pdf (so i in the location
family) with nite variance 2 , and if the one way ANOVA F test statistic is
n!
computed from all ways of assigning ni of the response variables
n1 ! np !
to treatment i, then the histogram of the F test statistic is approximately
Fp1,np for large ni .
15) For the one way Anova, the tted values Yij = Y i0 and the residuals
rij = Yij Yij .
16) Know that for the random eects one way Anova, the levels of
the factor are a random sample of levels from some population of levels F .
Assume the i are iid with mean and variance 2 . The cell means model
for the random eects one way Anova is Yij = i + eij for i = 1, . . . , p and
j = 1, . . . , ni . The sample size n = n1 + + np and often ni m so n = pm.
The i and eij are independent. The eij have mean 0 and variance 2 . The
Yij |i f (y i ), a location family with variance 2 while eij f (y). In
the test below, if H0 : 2 = 0 is true, then the Yij are iid with pdf f (y ),
so the F statistic Fp1,np if the ni are large.
17) Know that the 4 step random eects one way Anova test is
i) H0 : 2 = 0 HA : 2 > 0
ii) F0 = M ST R/M SE is usually obtained from output.
iii) The pval = P (Fp1,np > F0 ) is usually obtained from output.
iv) If pval reject Ho, conclude that 2 > 0 and that the mean response
depends on the factor level. Otherwise, fail to reject Ho, conclude that 2 = 0
and that the mean response does not depend on the factor level. (Or there
is not enough evidence to conclude that the mean response depends on the
factor level.)
18) Know how to tell whether the experiment is a xed or random eects
one way Anova. (Were the levels xed or a random sample from a population
of levels?)
196 5 One Way Anova

19) The applicability of a DOE (design of experiments) model can be ex-

panded by allowing response transformations. An important class of response
transformation models is

Y = to (Z) = E(Y ) + e = xT + e

where the subscripts (e.g., Yij ) have been suppressed. If o was known, then
Y = to (Z) would follow the DOE model. Assume that all of the values
of the response Z are positive. A power transformation has the form
Y = t (Z) = Z for = 0 and Y = t0 (Z) = log(Z) for = 0 where
L = {1, 1/2, 0, 1/2, 1}.
20) A graphical method for response transformations computes the tted
values W from the DOE model using W = t (Z) as the response for each
of the ve values of L . Let T = W = TZHAT and plot TZHAT vs.
t (Z) for {1, 1/2, 0, 1/2, 1}. These plots are called transformation
plots. The residual or error degrees of freedom used to compute the MSE
should not be too small. Choose the transformation Y = t (Z) that has the
best plot. Consider the one way Anova model with ni 5 for i = 1, . . . , p.
i) The dot plots should spread about the identity line with similar shape
and spread. ii) Dot plots that are approximately symmetric are better than
skewed dot plots. iii) Spread that increases or decreases with TZHAT (the
shape of the plotted points is similar to a right or left opening megaphone)
is bad.
21) The transformation plot for the selected transformation is also the
response plot for that model (e.g., for the model that uses Y = log(Z) as
the response). Make all of the usual checks on the DOE model (residual and
response plots) after selecting the response transformation.
22) The log rule says try Y = log(Z) if max(Z)/ min(Z) > 10 where
Z > 0 and the subscripts have been suppressed (so Z Zij for the one way
Anova model).
p p
23) A contrast C = i=1 ki i where i=1 ki = 0. The estimated contrast
p
is C = i=1 ki Y i0 .
p
24) Consider
p a family of null hypotheses for contrasts {Ho : i=1 ki i = 0
where i=1 ki = 0 and the ki may satisfy other constraints }. Let S denote
the probability of a type I error for a single test from the family. The family
level F is an upper bound on the (usually unknown) size T . Know how to
interpret F T = P(of making at least one type I error among the family
of contrasts) where a type I error is a false rejection.
25) Two important families of contrasts are the family of all possible con-
trasts and the family of pairwise dierences Cij = i j where i = j.
The Schee multiple comparisons procedure has a F for the family of all
possible contrasts, while
# $ the Tukey multiple comparisons procedure has a F
for the family of all p2 pairwise contrasts.
5.6 Complements 197

26) Know how to interpret output for multiple comparisons procedures.

Underlined means or blocks of letters besides groups of means indicates that
the group of means are not signicantly dierent.
27) Graphical Anova for the one way Anova model makes a dot plot
of scaled treatment deviations (eects) above a dot plot of the residuals. For
small n 40, suppose the distance between two scaled deviations (A and B,
say) is greater than the range of the residuals = max(rij ) min(rij ). Then
declare A and B to be signicantly dierent. If the distance is less than
the range, do not declare A and B to be signicantly dierent. Assume
the ni m for i = 1, . . . , p. Then the
 ith scaled deviation is c(Y i0 Y 00 ) =
 np
ci = i where c = dfe /dftreat = .
p1
28) The analysis of the response, not that of the residuals, is of primary
importance. The response plot can be used to analyze the response in the
background of the tted model. For linear models such as experimental de-
signs, the estimated mean function is the identity line and should be added
as a visual aid.
29) Assume that the residual degrees of freedom are large enough for test-
ing. Then the response and residual plots contain much information. Linearity
and constant variance may be reasonable if the p dot plots have roughly the
same shape and spread, and the dot plots scatter about the identity line.
The p dot plots of the residuals should have similar shape and spread, and
the dot plots scatter about the r = 0 line. It is easier to check linearity with
the response plot and constant variance with the residual plot. Curvature is
often easier to see in a residual plot, but the response plot can be used to
check whether the curvature is monotone or not. The response plot is more
eective for determining whether the signal to noise ratio is strong or weak,
and for detecting outliers or inuential cases.

5.6 Complements

Often the data does not consist of samples from p populations, but consists
of a group of n = mp units where m units are randomly assigned to each of
the p treatments. Then the Anova models can still be used to compare treat-
ments, but statistical inference to a larger population cannot be made. Of
course a nonstatistical generalization to larger populations can be made. The
nonstatistical generalization from the group of units to a larger population
is most compelling if several experiments are done with similar results. For
example, generalizing the results of an experiment for psychology students
to the population of all of the university students is less compelling than the
following generalization. Suppose one experiment is done for psychology stu-
dents, one for engineers, and one for English majors. If all three experiments
198 5 One Way Anova

give similar results, then generalize the results to the population of all of the
universitys students.
Four good tests on the design and analysis of experiments are Box et al.
(2005), Cobb (1998), Kuehl (1994), and Ledolter and Swersey (2007). Also
see Dean and Voss (2000), Kirk (2012), Maxwell and Delaney (2003), Mont-
gomery (2012), and Oehlert (2000).
A randomization test has H0 : the dierent treatments have no eect.
This null hypothesis is also true if all p pdfs Y |(W = ai ) fZ (y ) are
n!
the same. An impractical randomization test uses all M = n1 !n p!
ways of
assigning ni of the Yij to treatment i for i = 1, . . . , p. Let F0 be the usual
F statistic. The F statistic is computed for each of the M permutations and
H0 is rejected if the proportion of the M F statistics that are larger than
F0 is less than . The distribution of the M F statistics is approximately
Fp1,np for large n when H0 is true. The power of the randomization test is
also similar to that of the usual F test. See Hoeding (1952). These results
suggest that the usual F test is semiparametric: the pvalue is approximately
correct if n is large and if all p pdfs Y |(W = ai ) fZ (y ) are the same.
Let [x] be the integer part of x, e.g. [7.7] = 7. Olive (2014, section 9.3)
shows that practical randomization tests that use a random sample of
max(1000, [n log(n)]) permutations have level and power similar to the tests
that use all M possible permutations. See Ernst (2009) and the lregpack func-
tion rand1way for R code.
All of the parameterizations of the one way xed eects Anova model
yield the same predicted values, residuals, and ANOVA F test, but the inter-
pretations of the parameters dier. The cell means model is a linear model
(without intercept) of the form Y = X c c + e = that can be t using OLS.
The OLS MLR output gives the correct tted values and residuals but an
incorrect ANOVA table. An equivalent linear model (with intercept) with
correct OLS MLR ANOVA table as well as residuals and tted values can
be formed by replacing any column of the cell means model by a column of
ones 1. Removing the last column of the cell means model and making the
rst column 1 gives the model Y = 0 + 1 x1 + + p1 xp1 + e given in
matrix form by (5.5) on the following page.
It can be shown that the OLS estimators corresponding to (5.5) are 0 =
Y p0 = p , and i = Y i0 Y p0 = i p for i = 1, . . . , p 1. The cell means
model has i = i = Y i0 .
Wilcox (2012) gives an excellent discussion of the problems that outliers
and skewness can cause for the one and two sample tintervals, the ttest,
tests for comparing 2 groups, and the ANOVA F test. Wilcox (2012) replaces
ordinary population means by truncated population means and uses trimmed
means to create analogs of one way Anova and multiple comparisons.
5.6 Complements 199

110 ... 0
. . .
.. .. ..
..
.
Y11 e11
.. 110 ... 0 ..
. ... 0
.
101
Y1,n1
.. .. .. ..
e1,n1

Y21 . . . . e21
0
.. 1 01 ... 0
..
. .. 1 +
= .. .. ..
.
. .
Y2,n2 . . . . .. (5.5)
e 2,n2
. 1 00 ... 1 .
.. p1 ..
.
.. ... ... ..
Yp,1 . ep,1

. 100 ... 1 .
.. 1 0 0
. .
... 0
Yp,np . . . .. ep,np
.. .. .. .
1 0 0 ... 0
Graphical Anova uses scaled treatment eects = scaled treatment de-
viations di = cdi = c(Y i0 Y 00 ) for i = 1, . . . , p. Following Box et al.
(2005, p. 166), suppose ni m = n/p for i = 1, . . . , n. If Ho: 1 =
= p is true, want the sample variance of the scaled deviations to
be approximately
p equal to the sample variance of the residuals. So want
1 2 2 p
p i=1 c d i M ST R SST R/(p 1) md2i /(p 1)
1 1 n 2 = F0 = = = i=1n
i=1 ri M SE SSE/(n p) i=1 ri /(n p)
2
n
p p
since SST R = i=1 m(Y i0 Y 00 ) = i=1 mdi . So
2 2

p 2n 2
p m(np) 2
i=1 c di i=1 di
F0 = n p2 = n p12 .
i=1 ri i=1 ri

Equating numerators gives

mp (n p) (n p)
c2 = =
n (p 1) (p 1)

since mp/n = 1. Thus c = (n p)/(p 1).
For Graphical Anova, see Box et al. (2005, pp. 136, 150, 164, 166) and
Hoaglin et al. (1991). The R package granova, available from
(http://streaming.stat.iastate.edu/CRAN/), and authored by R.M.
Pruzek and J.E. Helmreich, may be useful.
The modied power transformation family

() Zi 1
Yi = t (Zi ) Zi =

for = 0 and t0 (Zi ) = log(Zi ) for = 0 where L .
200 5 One Way Anova

Box and Cox (1964) give a numerical method for selecting the response
transformation for the modied power transformations. Although the method
gives a point estimator o , often an interval of reasonable values is gen-
erated (either graphically or using a prole likelihood to make a condence
interval), and L is used if it is also in the interval.
There are several reasons to use a coarse grid L of powers. First, several of
the powers correspond to simple transformations such as the log, square root,
and reciprocal. These powers are easier to interpret than = 0.28, for exam-
ple. Secondly, if the estimator n can only take values in L , then sometimes
n will converge in probability to L . Thirdly, Tukey (1957) showed
that neighboring modied power transformations are often very similar, so
restricting the possible powers to a coarse grid is reasonable.
The graphical method for response transformations is due to Olive (2004b).
A variant of the method would plot the residual plot or both the response
and the residual plot for each of the ve values of . Residual plots are also
useful, but they do not distinguish between nonlinear monotone relationships
and nonmonotone relationships. See Fox (1991, p. 55). Alternative methods
are given by Cook and Olive (2001) and Box et al. (2005, p. 321).
An alternative to one way Anova is to use FWLS (see Chapter 4) on the
cell means model with 2 V = diag(12 , . . . , p2 ) where i2 occurs ni times
on the diagonal and i2 is the variance of the ith group for i = 1, . . . , p.
ni
Then V = diag(S12 , . . . , Sp2 ) where Si2 = ni11 j=1 (Yij Y i0 )2 is the sample
variance of the Yij . Hence the estimated weights for FWLS are wij wi =
1/Si2 . Then the FWLS cell means model has Y = X c c + e as in (5.1) except
Cov(e) = diag(12 , . . . , p2 ).
Hence Z = U c c + . Then U Tc U c = diag(n1 w1 , . . . , np wp ), (U Tc U c )1 =
1
diag(S12 /n1 , . . . , Sp2 /np ) = (X V X T )1 , and U Tc Z = (w1 Y10 , . . . , wp Yp0 )T .
Thus from Chapter 4,

F W LS = (Y 10 , . . . , Y p0 )T = c .

That is, the FWLS estimator equals the one way Anova estimator of based
on OLS applied to the cell means model. The ANOVA F test generalizes
the pooled t test in that the two tests are equivalent for p = 2. The FWLS
procedure is also known as the Welch one way Anova and generalizes the
Welch t test. The Welch t test is thought to be much better than the pooled
t test if n1 = n2 and 12 = 22 . See Brown and Forsythe (1974a,b), Kirk (1982),
pp. 100, 101, 121, 122), Olive (2014, pp. 278279), Welch (1947, 1951), and
Problem 5.11.
In matrix form Z = U c c +  becomes
5.6 Complements 201

w1 Y1,1 w1 0 0 ... 0 11
.. .. .. .. .. ..
. . .
. . .
w1 Y1,n1 w1 0 0 ... 0
1,n1
w2 Y21 0 w2 0 ... 0 21
1
.. .. .. .. .. ..
. . 2 .

. = . .
. + . (5.6)
w2 Y2,n2 0 0
w2 0 ... .. 2,n2
.. .. .. .. ..
. p ...
 . . .  .
wp Yp,1 0 0 0 ... wp p,1

.. . .. .. .. .
.. .
.
 . . . .
wp Yp,np 0 0 0 ... wp p,np

Four tests for Ho : 1 = = p can be used if Rule of Thumb 5.3:

max(S1 , . . . , Sp ) 2 min(S1 , . . . , Sp ) fails. Let Y = (Y1 , . . . , Yn )T , and let
Y(1) Y(2) Y(n) be the order statistics. Then the rank transformation
of the response is Z = rank(Y ) where Zi = j if Yi = Y(j) is the jth order
statistic. For example, if Y = (7.7, 4.9, 33.3, 6.6)T , then Z = (3, 1, 4, 2)T .
The rst test performs the one way ANOVA F test with Z replacing Y . See
Montgomery (1984, pp. 117118). Two of the next three tests are described
in Brown and Forsythe (1974b). Let x be the smallest integer x, e.g.
7.7 = 8. Then the Welch (1951) ANOVA F test uses test statistic
p
i=1 wi (Y i0 Y00 ) /(p 1)
2

2(p2) p
FW =
i=1 (1 u ) /(ni
wi 2
1+ p2 1 1)
p p
where wi = ni /Si2 , u = i=1 wi and Y00 = i=1 wi Y i0 /u. Then the test
statistic is compared to an Fp1,dW distribution where dW = f  and

3 % w i &2
p
1/f = 1 /(ni 1).
p 1 i=1
2 u

For the modied Welch (1947) test, the test statistic is compared to an
Fp1,dM W distribution where dM W = f  and
p p
(S 2 /ni )2 i=1 (1/wi )
2
f = p i=11 i 2 2
=  p 1 2
.
i=1 ni 1 (Si /ni ) i=1 ni 1 (1/wi )

Some software uses f instead of dW or dM W , and variants on the denominator

degrees of freedom dW or dM W are common.
The modied ANOVA F test uses test statistic
p
i=1 ni (Y i0 Y 00 )
2
FM =  p .
i=1 (1 n )Si
ni 2
202 5 One Way Anova

The test statistic is compared to an Fp1,dM distribution where dM = f 

and
p
1/f = c2i /(ni 1)
i=1

where  p 
% ni & 2  % ni & 2
ci = 1 Si / 1 Si .
n i=1
n
The lregpack function anovasim can be used to simulate and compare
the four tests with the usual one way ANOVA test. Some simulation results
are in Haenggi (2009).

5.7 Problems

Problems with an asterisk * are especially important.

Output for Problem 5.1.

A B C D E
9.8 9.8 8.5 7.9 7.6
10.3 12.3 9.6 6.9 10.6
13.6 11.1 9.5 6.6 5.6
10.5 10.6 7.4 7.6 10.1
8.6 11.6 7.6 8.9 10.5
11.1 10.9 9.9 9.1 8.6
Analysis of Variance for Time
Source DF SS MS F P
Design 4 44.88 11.22 5.82 0.002
Error 25 48.17 1.93
Total 29 93.05
5.1. In a psychology experiment on child development, the goal was to
study how dierent designs of mobiles vary in their ability to capture the
infants attention. Thirty 3-month-old infants were randomly divided into
ve groups of six each. Each group was shown a mobile with one of ve
designs A, B, C, D, or E. The time that each infant spent looking at the
design was recorded in the output above along with the ANOVA table. Data
is taken from McKenzie and Goldman (1999, p. 234). See the above output.
a) Find 2 = B .
b) Perform a 4 step ANOVA F test.
5.2. Moore (2007, p. 651): Nematodes are microscopic worms. A botanist
desired to learn how the presence of the nematodes aects tomato growth.
She used 16 pots each with a tomato seedling. Four pots got 0 nematodes, four
5.7 Problems 203

got 1000, four got 5000, and four got 10000. These four groups are denoted by
none, n1000, n5000, and n10000, respectively. The seedling growths
were all recorded and the table below gives the one way ANOVA results.
a) What is none ?

b) Do a four step test for whether the four mean growths are equal.
(So Ho: none = n1000 = n5000 = n10000 .)
c) Examine the Bonferroni comparison of means. Which groups of means
are not signicantly dierent?

Output for Problem 5.2.

Variable MEAN SAMPLE SIZE GROUP STD DEV
NONE 10.650 4 2.0535
N1000 10.425 4 1.4863
N5000 5.600 4 1.2437
N10000 5.450 4 1.7711
TOTAL 8.0312 16 1.6666

One Way Analysis of Variance Table

Source df SS MS F p-value
Treatments 2 100.647 33.549 12.08 0.0006
Error 28 33.328 2.777
Total 15 133.974

Bonferroni Comparison of Means

Homogeneous
Variable Mean Groups
----------------------------
NONE 10.650 I
N1000 10.425 I
N5000 5.600 .. I
N10000 5.450 .. I

5.3. According to Cobb (1998, p. 9) when the famous statistician W. G.

Cochran was starting his career, the experiment was to study rat nutrition
with two diets: ordinary rat food and rat food with a supplement. It was
thought that the diet with the supplement would be better. Cochran and his
coworker grabbed rats out of a cage, one at a time, and Cochran assigned
the smaller less healthy rats to the better diet because he felt sorry for them.
The results were as expected for the rats chosen by Cochrans coworker, but
the better diet looked bad for Cochrans rats.
a) What were the units?
b) Suppose rats were taken from the cage one at a time. How should the
rats have been assigned to the two diets?

5.4. Use the output from the command below

204 5 One Way Anova

> sample(11)
[1] 7 10 9 8 1 6 3 11 2 4 5

to assign the following 11 people to three groups of size n1 = n2 = 4 and

n3 = 3.
Anver, Arachchi, Field, Haenggi, Hazaimeh,
Liu, Pant, Tosun, Yi, Zhang, Zhou

5.5. Sketch a good response plot if there are 4 levels with Y 10 = 2,

Y 20 = 4, Y 30 = 6, Y 40 = 7, and ni = 5.

output for Problem 5.6 level

1 2 3 4 5
15 percent 20 percent 25 percent 30 percent 35 percent

y1 y5 y2 y3 y4
9.8 10.8 15.4 17.6 21.6

5.6. The tensile strength of a cotton nylon ber used to make womens
shirts is believed to be aected by the percentage of cotton in the ber. The
5 levels of cotton percentage that are of interest are tabled above. Also shown
is a (Tukey pairwise) comparison of means. Which groups of means are not
signicantly dierent? Data is from Montgomery (1984, pp. 51, 66).

output for Problem 5.7

Source df SS MS F P
color 2 7.60 3.80 0.390 0.684
error 12 116.40 9.70

5.7. A researcher is interested in whether the color (red, blue, or green)

of a paper maze eects the time to complete the maze.
a) State whether this is a random or xed eects one way Anova. Explain
briey.
b) Using the above output, perform the appropriate 4 step ANOVA F test.

A B C Output for Problem 5.8.

9.5 8.5 7.7
3.2 9.0 11.3
4.7 7.9 9.7
7.5 5.0 11.5
8.3 3.2 12.4 Analysis of Variance for Time
Source DF SS MS F P
Design 2 49.168 24.584 4.4625 0.0356
Error 12 66.108 5.509
5.7 Problems 205

5.8. Ledolter and Swersey (2007, p. 49) describe a one way Anova design
used to study the eectiveness of 3 product displays (A, B, and C). Fifteen
stores were used and each display was randomly assigned to 5 stores. The
response Y was the sales volume for the week during which the display was
present compared to the base sales for that store.
a) Find 2 = B using output on the previous page.
b) Perform a 4 step ANOVA F test.

Scaled Treatment Deviations

20
18
graphicalanova
16
14
12
10

2 0 2 4 6
Residuals
Fig. 5.5 Graphical Anova for Problem 5.9

ganova(x,y)
smn -3.2333 -3.0374 6.2710
Treatments "A" "B" "C"

5.9. Ledolter and Swersey (2007, p. 49) describe a one way Anova design
used to study the eectiveness of 3 product displays (A, B, and C). Fifteen
stores were used and each display was randomly assigned to 5 stores. The
response Y was the sales volume for the week during which the display was
present compared to the base sales for that store. Figure 5.5 is the Graphical
Anova plot found using the lregpack function ganova.
a) Which two displays (from A, B, and C) yielded similar mean sales
volume?
b) Which eect (from A, B, and C) appears to be signicant?
206 5 One Way Anova

Source df SS MS F P
treatment 3 89.19 29.73 15.68 0.0002
error 12 22.75 1.90

5.10. A textile factory weaves fabric on a large number of looms. They

would like to obtain a fabric of uniform strength. Four looms are selected
at random and four samples of fabric are obtained from each loom. The
strength of each fabric sample is measured. Data is from Montgomery (1984,
pp. 7475).
a) State whether this is a random or xed eects one way Anova. Explain
briey.
b) Using the output above, perform the appropriate 4 step ANOVA F test.

Problems using R.

Use the command source(G:/lregpack.txt) to download the func-

tions and the command source(G:/lregdata.txt) to download the data.
See Preface or Section 14.1. Typing the name of the lregpack function,
e.g. pcisim, will display the code for the function. Use the args command,
e.g. args(pcisim), to display the needed arguments for the function. For some
of the following problems, the R commands can be copied and pasted from
(http://lagrange.math.siu.edu/Olive/lreghw.txt) into R.

5.11. The pooled t procedures are a special case of one way Anova with
p = 2. Consider the pooled t CI for 1 2 . Let X1 , . . . , Xn1 be iid with
mean 1 and variance 12 . Let Y1 , . . . , Yn2 be iid with mean 2 and variance
22 . Assume that the two samples are independent (or that n1 + n2 units
were randomly assigned to two groups) and that ni for i = 1, 2 in
such a way that = n1n+n1
2
(0, 1). Let = 22 /12 , and let the pooled
sample variance Sp = [(n1 1)S12 + (n2 1)S22 ]/[n1 + n2 2] and 2 =
2

(1 + )/[ + (1 )]. Then

X Y (1 2 ) D
 2 N (0, 1) and
S1 S22
n1 + n2

S12 S22
n1 + n2 X Y (1 2 ) X Y (1 2 ) D
  2 2
=  N (0, 2 ).
1 1 S1 S2 1 1
Sp n1 + n2 n1 + n2
Sp n1 + n2

Now let = S22 /S12 and 2 = (1 + )/( + (1 ) ). Notice that

= 1 if = 1/2, and = 1 if = 1. The usual large sample (1 )100%
pooled t CI for (1 2 ) is

1 1
X Y tn1 +n2 2,1/2 Sp + (5.7)
n1 n2
5.7 Problems 207

is valid if = 1. The large sample (1 )100% modied pooled t CI for

(1 2 ) is

1 1
X Y tn1 +n2 4,1/2 Sp + . (5.8)
n1 n2
The large sample (1 )100% Welch CI for (1 2 ) is

S12 S2
X Y td,1/2 + 2 (5.9)
n1 n2

where d = max(1, [d0 ]), and

S2 S22 2
( n11 + n2 )
d0 = S12 2 S22 2
.
1 1
n1 1 ( n1 ) + n2 1 ( n2 )

Suppose n1 /(n1 + n2 ) . It can be shown that if the CI length is multi-

plied by n1 , 
then the scaled length of the pooled t CI converges in probabil-
2
ity to 2z1/2 + 22 while the scaled lengths of the modied pooled t
1 1


CI and Welch CI both converge in probability to 2z1/2 12 + 1 22 . The
pooled t CI should have coverage that is too low if

12 + 22 < 12 + 2 .
1 1 2

See Olive (2014, Example 9.23).

a) Download the function pcisim.
b) Type the command pcisim(n1=100,n2=200,var1=10,var2=1) to
simulate the CIs for N (i , i2 ) data for i = 1, 2. The terms pcov, mpcov, and
wcov are the simulated coverages for the pooled, modied pooled, and Welch
95% CIs. Record these quantities. Are they near 0.95?
5.12. From the end of Section 5.6, four tests for Ho : 1 = = p can
be used if Rule of Thumb: max(S1 , . . . , Sp ) 2 min(S1 , . . . , Sp ) fails. In R,
get the function anovasim. When H0 is true, the coverage = proportion of
times the test rejects H0 has a nominal value of 0.05. The terms faovcov is
for the usual F test, modfcov is for a modied F test, wfcov is for the Welch
test, mwfcov for the modied Welch test, and rfcov for the rank test. The
function generates 1000 data sets with p = 4, ni = ni = 20, mi = i and
sdi = i .
a) Get the coverages for the following command. Since the four population
means and the four population standard deviations are equal, we want the
coverages to be near or less than 0.05. Are they? anovasim(m1 = 0, m2 = 0,
m3 = 0, m4 = 0, sd1 = 1, sd2 = 1, sd3 = 1, sd4 = 1)
b) Get the coverages for the following command. The population means
are equal, but the population standard deviations are not. Are the coverages
208 5 One Way Anova

near or less than 0.05? anovasim(m1 = 0, m2 = 0, m3 = 0, m4 = 0, sd1 =

1, sd2 = 2, sd3 = 3, sd4 = 4)
c) Now use the following command where H0 is false: the four population
means are not all equal. We want the coverages near 1. Are they?
anovasim(m1 = 1, m2 = 0, m3 = 0, m4 = 1)
d) Now use the following command where H0 is false. We want the cov-
erages near 1. Since the i are not equal, the ANOVA F test is expected to
perform poorly. Is the ANOVA F test the best? anovasim(m4 = 1, sd4 = 9)

5.13. This problem uses data from Kuehl (1994, p. 128).

a) Get lregdata and lregpack into R. Type the following commands. Then
simultaneously press the Ctrl and c keys. In Word use the menu command
Paste. Print out the gure.

y <- ycrab+1/6
aovtplt(crabhab,y)

b) From the gure, what response

transformation should be used: Y =
1/Z, Y = 1/ Z, Y = log(Z), Y = Z, or Y = Z?

5.14. The following data set considers the number of warp breaks per
loom, where the factor is tension (low, medium, or high).
a) Copy and paste the commands for this problem into R.
Highlight the ANOVA table by pressing the left mouse key and dragging
the cursor over the ANOVA table. Then use the menu commands Edit>
Copy. Enter Word and use the menu command Paste. b) To place the
residual plot in Word, get into R and click on the plot, hit the Ctrl and c
keys at the same time. Enter Word and use the menu command Paste or
hit the Ctrl and v keys at the same time.
c) Copy and paste the commands for this part into R.
Click on the response plot, hit the Ctrl and c keys at the same time. Enter
Word and use the menu command Paste.

5.15. Obtain the Box et al. (2005, p. 134) blood coagulation data from
lregdata and the R program ganova from lregpack. The program does graph-
ical Anova for the one way Anova model.
a) Enter the following command and include the plot in Word by simulta-
neously pressing the Ctrl and c keys, then using the menu command Paste
in Word, or hit the Ctrl and v keys at the same time.

ganova(bloodx,bloody)

The scaled treatment deviations are on the top of the plot. As a rule
of thumb, if all of the scaled treatment deviations are within the spread of
the residuals, then population treatment means are not signicantly dierent
(they all give response near the grand mean). If some deviations are outside
of the spread of the residuals, then not all of the population treatment means
5.7 Problems 209

are equal. Box et al. (2005, p. 137) state The graphical analysis discourages
overreaction to high signicance levels and avoids underreaction to very
nearly signicant dierences.
b) From the output, which two treatments means were approximately the
same?

c) To perform a randomization F test in R, get the program rand1way

from lregpack, and type the following commands. The output z$rdist is the
randomization distribution, z$Fpval is the pvalue of the usual F test, and
z$randpval is the pvalue of the randomized F test.

z<-rand1way(y=bloody,group=bloodx,B=1000)
hist(z$rdist)
z$Fpval
z$randpval

d) Include the histogram in Word.

One Way Anova in SAS

To get into SAS, often you click on a SAS icon, perhaps something like
The SAS System for . . . . A window with a split screen will open. The top
screen says Log-(Untitled) while the bottom screen says Editor-Untitled1.
Press the spacebar and an asterisk appears: Editor-Untitled1*.
For Problem 5.16, consider saving your le as hw5d16.sas on your ash
drive (J, say). (On the top menu of the editor, use the commands File > Save
as. A window will appear. Use the upper right arrow to locate Removable
Disk (J:), and then type the le name in the bottom box. Click on OK.) From
the top menu in SAS, use the File> Open command. A window will open.
Use the arrow in the NE corner of the window to navigate to Removable
Disk (J:). (As you click on the arrow, you should see My Documents, C: etc,
then Removable Disk (J:).) Double click on hw5d16.sas.
This point explains the SAS commands. The semicolon ; is used to
end SAS commands and the options ls = 70; command makes the output
readable. (An * can be used to insert comments into the SAS program.
Try putting an * before the options command and see what it does to the
output.) The next step is to get the data into SAS. The command data
clover; gives the name clover to the data set. The command
input strain $ nitrogen @ @; says the rst entry is variable strain and
the $ means it is categorical, the second variable is nitrogen and the @@
means read 2 variables, then 2, . . . , until the end of the data. The command
cards; means that the data is entered below. Then the data is entered and
the isolated semicolon indicates that the last case has been entered.
The commands proc glm; class = strain; model nitrogen = strain; tells
SAS to perform one way Anova with nitrogen as the response variable and
strain as the factor.
210 5 One Way Anova

5.16. Cut and paste the SAS program for this problem into the SAS
Editor.
To execute the program, use the top menu commands Run>Submit. An
output window will appear if successful.
(If you were not successful, look at the log window for hints on errors.
A single typo can cause failure. Reopen your le in Word or Notepad and
make corrections. Occasionally you cannot nd your error. Then nd your
instructor or wait a few hours and reenter the program.)
Data is from SAS Institute (1985, pp. 126129). See Example 5.6.
a) In SAS, use the menu commands Edit>Select All then Edit>Copy.
In Word, use the menu command Paste. Highlight the rst page of output
and use the menu command Cut. (SAS often creates too much output.
These commands reduce the output from 4 pages to 3 pages.)
You may want to save your SAS output as the le HW5d16.doc on your
ash drive.
b) Perform the 4 step test for Ho: 1 = 2 = = 6 .
c) From the residual and response plots, does the assumption of equal
population standard deviations (i = for i = 1, . . . , 6) seem reasonable?

One Way Anova in ARC

5.17. To get in ARC, you need to nd the ARC icon. Suppose the ARC icon
is in a math progs folder. Move the cursor to the math progs folder, click the
right mouse button twice, move the cursor to ARC, double click, move the
cursor to ARC, double click. These menu commands will be written math
progs > ARC > ARC. To quit ARC, move cursor to the x in the northeast
corner and click.
This Cook and Weisberg (1999a, p. 289) data set contains IQ scores on
27 pairs of identical twins, one raised by foster parents IQf and the other
by biological parents IQb. C gives the social class of the biological parents:
C = 1 for upper class, 2 for middle class and 3 for lower class. Hence the
Anova test is for whether mean IQ depends on class.
a) Activate twins.lsp dataset with the menu commands
File > Load > Data > twins.lsp.
b) Use the menu commands Twins>Make factors, select C and click on
OK. The line {F}C Factor 27 Factorrst level dropped should appear on
the screen.
c) Use the menu commands Twins>Description to see a description of
the data.
d) Enter the menu commands Graph&Fit>Fit linear LS and select {F}C
as the term and IQb as the response. Highlight the output by pressing the
left mouse key and dragging the cursor over the output. Then use the menu
commands Edit> Copy. Enter Word and use the menu command Paste.
5.7 Problems 211

e) Enter the menu commands Graph&Fit>Boxplot of and enter IQb in

the selection box and C in the Condition on box. Click on OK. When the
boxplots appear, click on the Show Anova box. Click on the plot, hit the
Ctrl and c keys at the same time. Enter Word and use the menu command
Paste. Include the output in Word. Notice that the regression and Anova
F statistic and p-value are the same.
f) Residual plot: Enter the menu commands Graph&Fit>Plot of, select
L1:Fit-Values for the H box and L1:Residuals for the V box, and
click on OK. Click on the plot, hit the Ctrl and c keys at the same time.
Enter Word and use the menu command Paste.
g) Response plot: Enter the menu commands Graph&Fit>Plot of, select
L1:Fit-Values for the H box and IQb for the V box, and click on
OK. When the plot appears, move the OLS slider bar to 1 to add the
identity line. Click on the plot, hit the Ctrl and c keys at the same time.
Enter Word and use the menu command Paste.
h) Perform the 4 step test for Ho: 1 = 2 = 3 .
One Way Anova in Minitab
5.18. a) In Minitab, use the menu command File>Open Worksheet
and double click on Baby.mtw. A window will appear. Click on OK.

This McKenzie and Goldman (1999, p. T-234) data set has 30 three-month-
old infants randomized into ve groups of 6 each. Each infant is shown a
mobile of one of ve multicolored designs, and the goal of the study is to see
if the infant attention span varies with type of design of mobile. The times
that each infant spent watching the mobile are recorded.
b) Choose Stat>Basic Statistics>Display Descriptive Statistics, select
C1 Time as the Variable, click the By variable option and press Tab.
Select C2 Design as the By variable. c) From the window in b), click on
Graphs the Boxplots of data option, and OK twice. Click on the plot
and then click on the printer icon to get a plot of the boxplots.
d) Select Stat>ANOVA>One-way, select C1-time as the response and
C2-Design as the factor. Click on Store residuals and click on Store ts.
Then click on OK. Click on the output and then click on the printer icon.
e) To make a residual plot, select Graph>Plot. Select Resi1 for Y
and Fits1 for X and click on OK. Click on the plot and then click on
the printer icon to get the residual plot.
f) To make a response plot, select Graph>Plot. Select C1 Time for
Y and Fits1 for X and click on OK. Click on the plot and then click
on the printer icon to get the response plot.
g) Do the 4 step test for Ho: 1 = 2 = = 5 .
To get out of Minitab, move your cursor to the x in the NE corner of
the screen. When asked whether to save changes, click on no.
Chapter 6
The K Way Anova Model

For a K way Anova model, A1 , . . . , AK are the factors with li levels for
i = 1, . . . , K. Hence there are l1 l2 lK treatments where each treatment
uses exactly one level from each factor. First the two way Anova model is
discussed and then the model with K > 2. Interactions between the K factors
are important.

6.1 Two Way Anova

Denition 6.1. The xed eects two way Anova model has two factors
A and B plus a response Y . Factor A has a levels and factor B has b levels.
There are ab treatments.

Denition 6.2. The cell means model for two way Anova is Yijk =
ij + eijk where i = 1, . . . , a; j = 1, . . . , b; and k = 1, . . . , m. The sample size
n = abm. The ij are constants and the eijk are iid from a location family
with mean 0 and variance 2 . Hence the Yijk f (yij ) come from a location
family with location parameter ij . The tted values are Yijk = Y ij0 = ij
while the residuals rijk = Yijk Yijk .

For one way Anova models, the cell sizes ni need not be equal. For K way
Anova models with K 2 factors, the statistical theory is greatly simplied
if all of the cell sizes are equal. Such designs are called balanced designs.

Denition 6.3. A balanced design has all of the cell sizes equal: for the
two way Anova model, nij m.

In addition to randomization of units to treatments, another key princi-

ple of experimental design is factorial crossing. Factorial crossing allows for
estimation of main eects and interactions.

Springer International Publishing AG 2017 213

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 6
214 6 The K Way Anova Model

Denition 6.4. A two way Anova design uses factorial crossing if each
combination of an A level and a B level is used and called a treatment. There
are ab treatments for the two way Anova model.

Experimental two way Anova designs randomly assign m of the n = mab

units to each of the ab treatments. Observational studies take random samples
of size m from ab populations.

Denition 6.5. The main eects are A and B. The AB interaction is

not a main eect.

Remark 6.1. If A and B are factors, then there are 5 possible models.
i) The two way Anova model has terms A, B, and AB.
ii) The additive model or main eects model has terms A and B.
iii) The one way Anova model that uses factor A.
iv) The one way Anova model that uses factor B.
v) The null model does not use any of the three terms A, B, or AB. If the
null model holds, then Yijk f (y 00 ) so the Yijk form a random sample of
size n from a location family, and the distribution of the response is the same
for all ab treatments. For models i)iv), the distribution of the response is
not the same for all ab treatments.

Remark 6.2. The response plot, residual plot, and transformation plots
for response transformations are used in the same way as Chapter 5. The
plots work best if the MSE degrees of freedom max(10, n/5). The model
is overtting if 1 MSE df < max(10, n/5), and then the plots may only
be useful for detecting large deviations from the model. For the model that
contains A, B, and AB, there will be ab dot plots of size m, and we need
m 5 to check for similar shape and spread. For the additive model, the
response and residual plots often look like those for multiple linear regression.
Then the plotted points should scatter about the identity line or r = 0 line
in a roughly evenly populated band if the additive two way Anova model is
reasonable. We want n 5(number of parameters in the model) for inference.
So we want n 5ab or m 5 when all interactions and main eects are in
the two way Anova model.

Shown is an ANOVA table for the two way Anova model given in symbols.
Sometimes Error is replaced by Residual, or Within Groups. A and
B are the main eects while AB is the interaction. Sometimes p-value is
replaced by P, P r(> F ), or PR > F. The p-value corresponding to
FA is for Ho: 10 = = a0 . The p-value corresponding to FB is for Ho:
01 = = 0b . The p-value corresponding to FAB is for Ho: there is no
interaction. The sample p-value pval is an estimator of the population
p-value.
6.1 Two Way Anova 215

Source df SS MS F p-value
A a-1 SSA MSA FA =MSA/MSE pval
B b-1 SSB MSB FB =MSB/MSE pval
AB (a 1)(b 1) SSAB MSAB FAB =MSAB/MSE pval
Error n ab = ab(m 1) SSE MSE

Be able to perform the 4 step test for AB interaction:

i) Ho: no interaction HA : there is an interaction
ii) FAB is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that there is an interaction between A
and B, otherwise fail to reject Ho and conclude that there is no interaction
between A and B. (Or there is not enough evidence to conclude that there is
an interaction between A and B.)

Remark 6.3. i) Keep A and B in the model if there is an AB interaction.

The two tests for main eects (below) make the most sense if we fail to
reject the test for interaction. Rejecting Ho for main eects makes sense
when there is an AB interaction because the main eects tend to be larger
than the interaction eects. (Failing to reject Ho for main eects when there
is an AB interaction may not make sense.)
ii) The main eects tests are just like the F test for the xed eects one
way Anova model. If populations means are close, then larger sample sizes are
needed for the F test to reject Ho with high probability. If Ho is not rejected
and the means are equal, then it is possible that the factor is unimportant,
but it is also possible that the factor is important but the level
is not. For example, factor A might be type of catalyst. The yield may be
equally good for each type of catalyst, but there would be no yield if no
catalyst was used.

Be able to perform the 4 step test for A main eects:

i) Ho: 10 = = a0 HA : not Ho
ii) FA is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that the mean response depends on the
level of A, otherwise fail to reject Ho and conclude that the mean response
does not depend on the level of A. (Or there is not enough evidence to
conclude that the mean response depends on the level of A.)

Be able to perform the 4 step test for B main eects:

i) Ho: 01 = = 0b HA : not Ho
ii) FB is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that the mean response depends on the
level of B, otherwise fail to reject Ho and conclude that the mean response
216 6 The K Way Anova Model

does not depend on the level of B. (Or there is not enough evidence to
conclude that the mean response depends on the level of B.)

Remark 6.4. One could do a one way Anova on p = ab treatments, but

this procedure loses information about A, B, and the AB interaction.

Denition 6.6. An interaction plot is made by plotting the levels of one

factor (either 1, . . . , a or 1, . . . , b) versus the cell sample means Y ij0 . Typically
the factor with more levels (e.g., A if a > b) is used on the horizontal axis.
If the levels of A are on the horizontal axis, use line segments to join the a
means that have the same j. There will be b curves on the plot. If the levels
of B are on the horizontal axis, use line segments to join the b means that
have the same i. There will be a curves on the plot. If no interaction is
present, then the curves should be roughly parallel.

The interaction plot is rather hard to use, especially if the nij = m are
small. For small m, the curves can be far from parallel, even if there is no
interaction. The further the curves are from being parallel, the greater the
evidence of interaction. Intersection of curves suggests interaction unless the
two curves are nearly the same. The two curves may be nearly the same if
two levels of one factor give nearly the same mean response for each level of
the other factor. Then the curves could cross several times even though there
is no interaction. Software lls space. So the vertical axis needs to be checked
to see whether the  sample means for two curves are close with respect to
the standard error M SE/m for the means.
The interaction plot is the most useful if the conclusions for the plot agree
with the conclusions for the F test for no interaction.

Denition 6.7. The overparameterized two way Anova model has Yijk =
ij + eijk with ij = 00 + i + j + ()ij where the interaction parameters
()ij = ij i0 0j + 00 . The A main eects are i = i0 00 for
i = 1, . . . , a.The B main
  eects are j = 0j 00 for j= 1, . . . , b. Here
i i = 0, j j =
  i ()ij = 0 for j = 1, . . . , b and j ()ij = 0 for
0,
i = 1, . . . , a. Thus i j ()ij = 0.

The mean parameters have the following meaning. The parameter ij is

the population mean response for the ijth treatment. The means 0j =
a b
i=1 ij /a, and the means i0 = j=1 ij /b.
As was the case for multiple linear regression, interaction is rather dicult
to understand. Note that if all of the interaction parameters ()ij = 0, then
the factor eects are additive: ij = 00 + i + j . Hence no interaction
implies that the factor eects are additive while interaction implies that
the factor eects are not additive. When there is no interaction, 1j = 00 +
1 + j , 2j = 00 + 2 + j , . . . , aj = 00 + a + j . Consider a plot with
the ij on the vertical axis and the levels 1, 2, . . . , a of A on the horizontal
axis. If there is no interaction and if the ij with the same j are connected
6.1 Two Way Anova 217

1.5
2
1

1.4
mean of Y
1.3
1.2
1.1

1 2
A
Fig. 6.1 Interaction Plot for Example 6.1.

with line segments, then there will be b parallel curves with curve height
depending on j . If there is interaction, then not all of the p curves will be
parallel. The interaction plot replaces the ij by the ij = Y ij0 .

Example 6.1. Cobb (1998, pp. 200212) describes an experiment on

weight gain for baby pigs. The response Y was the average daily weight
gain in pounds for each piglet (over a period of time). Factor A consisted of
0 mg of an antibiotic or 40 mg an antibiotic, while factor B consisted of 0
mg of vitamin B12 or 5 mg of B12. Hence there were 4 diets (A, B) = (0,0),
(40,0), (0,5), or (40,5). Hence level 1 corresponds to 0 mg and level 2 to more
than 0 mg.
The interaction plot shown in Figure 6.1 suggests that there is an interac-
tion. If no vitamin B12 is given, then the pigs given the antibiotic have less
mean weight gain than the pigs not given the antibiotic. For pigs given the
diet with 5 mg of B12, the antibiotic was useful, with a mean gain near 1.6.
Pigs with A = 1 (no antibiotic in the diet) had similar mean weight gains,
but pigs with A = 2 (antibiotic in the diet) had greatly dierent mean weight
gains. The best diet had both vitamin B12 and the antibiotic, while the worst
diet had the antibiotic but no vitamin B12.

Example 6.2. The output below uses data from Kutner et al. (2005, prob-
lems 19.1415). The output is from an experiment on hay fever, and 36 vol-
unteers were given medicine. The two active ingredients (factors A and B)
in the medicine were varied at three levels each (low, medium, and high).
218 6 The K Way Anova Model

The response is the number of hours of relief. (The factor names for this
problem are A and B.)
a) Give a four step test for the A*B interaction.
b) Give a four step test for the A main eects.
c) Give a four step test for the B main eects.

Source DF SS MS F P
A 2 220.0200 110.0100 1827.86 0.000
B 2 123.6600 61.8300 1027.33 0.000
Interaction 4 29.4250 7.3562 122.23 0.000
Error 27 1.6250 0.0602

Solution: a) Ho: no interaction HA : there is an interaction

FAB = 122.23
pval = 0.0
Reject Ho, there is an interaction between the active ingredients A and B.
b) Ho: 10 = 20 = 30 HA : not Ho
FA = 1827.86
pval = 0.0
Reject Ho, the mean hours of relief depends on active ingredient A.
c) Ho: 01 = 02 = 03 HA : not Ho
FB = 1027.33
pval = 0.0
Reject Ho, the mean hours of relief depends on active ingredient B.

6.2 K Way Anova Models

Use factorial crossing to compare the eects (main eects, pairwise inter-
actions, . . . , K-fold interaction if there are K factors) of two or more factors.
If A1 , . . . , AK are the factors with li levels for i = 1, . . . , K; then there are
l1 l2 lK treatments where each treatment uses exactly one level from each
factor.

Source df SS MS F p-value
#K $ K main eects e.g. SSA = MSA FA pA
2 2 factor interactions e.g. SSAB = MSAB FAB pAB
#K $
3 3 factor interactions e.g. SSABC = MSABC FABC pABC
.. .. .. .. ..
# $ . . . . .
K
K1 K 1 factor interactions
the K factor interaction SSA L = MSA L FAL pAL
Error SSE MSE
6.3 Summary 219

On the previous page is a partial ANOVA table for a K way Anova design
with the degrees of freedom left blank. For A, use H0 : 100 = = l1 00 .
The other main eects have similar null hypotheses. For interaction, use H0 :
no interaction.
These models get complex rapidly as K and the number of levels li in-
crease. As K increases, there are a large number of models to consider. For
experiments, usually the 3 way and higher order interactions are #not $ signif-
icant. Hence a full model that includes all K main eects and K 2 2 way
interactions is a useful starting point for response, residual, and transforma-
tion plots. The higher order interactions can be treated as potential terms
and checked for signicance. As a rule of thumb, signicant interactions tend
to involve signicant main ' eects.
K
The sample size n = m i=1 li m 2K is minimized by taking li = 2 for
i = 1, . . . , K. Hence the sample size grows exponentially fast with K. Designs
that use the minimum number of levels 2 are discussed in Section 8.1.

6.3 Summary

1) The xed eects two way Anova model has two factors A and B plus a
response Y . Factor A has a levels and factor B has b levels. There are ab
treatments. The cell means model is Yijk = ij + eijk where i = 1, . . . , a; j =
1, . . . , b; and k = 1, . . . , m. The sample size n = abm. The ij are constants
and the eijk are iid with mean 0 and variance 2 . Hence the Yijk f (y ij )
come from a location family with location parameter ij . The tted values
are Yijk = Y ijo = ij while the residuals rijk = Yijk Yijk .
2) Know that the 4 step test for AB interaction is
i) Ho: no interaction HA : there is an interaction
ii) FAB is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho, and conclude that there is an interaction between A
and B, otherwise fail to reject Ho, and conclude that there is no interaction
between A and B.
3) Keep A and B in the model if there is an AB interaction.
4) Know that the 4 step test for A main eects is
i) Ho: 10 = = a0 HA : not Ho
ii) FA is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that the mean response depends on the
level of A, otherwise fail to reject Ho and conclude that the mean response
does not depend on the level of A.
5) Know that the 4 step test for B main eects is
i) Ho: 01 = = 0b HA : not Ho
ii) FB is obtained from output.
220 6 The K Way Anova Model

iii) The pval is obtained from output.

iv) If pval reject Ho and conclude that the mean response depends on the
level of B, otherwise fail to reject Ho and conclude that the mean response
does not depend on the level of B.
The tests for main eects (points 4) and 5)) do not always make sense if
the test for interactions is rejected.

6) Shown is an ANOVA table for the two way Anova model given in sym-
bols. Sometimes Error is replaced by Residual, or Within Groups. A
and B are the main eects while AB is the interaction. Sometimes p-value
is replaced by P, P r(> F ), or PR > F. The p-value corresponding to
FA is for Ho: 10 = = a0 . The p-value corresponding to FB is for Ho:
01 = = 0b . The p-value corresponding to FAB is for Ho: there is no
interaction.

Source df SS MS F p-value
A a-1 SSA MSA FA =MSA/MSE pval
B b-1 SSB MSB FB =MSB/MSE pval
AB (a 1)(b 1) SSAB MSAB FAB =MSAB/MSE pval
Error n ab = ab(m 1) SSE MSE

7) An interaction plot is made by plotting the levels of one factor (either

1, . . . , a or 1, . . . , b) versus the cell sample means Y ij0 . Typically the factor
with more levels (e.g., A if a > b) is used on the horizontal axis. If the levels
of A are on the horizontal axis, use line segments to join the a means that
have the same j. There will be b curves on the plot. If the levels of B are on
the horizontal axis, use line segments to join the b means that have the same
i. There will be a curves on the plot. If no interaction is present, then the
curves should be roughly parallel.
8) The interaction plot is rather hard to use, especially if the nij = m are
small. For small m, the curves could be far from parallel even if there is no
interaction, but the further the curves are from being parallel, the greater the
evidence of interaction. Intersection of curves suggests interaction unless the
two curves are nearly the same. The two curves may be nearly the same if
two levels of one factor give nearly the same mean response for each level of
the other factor. Then the curves could cross several times even though there
is no interaction. Software lls space. So the vertical axis needs to be checked
to see whether the  sample means for two curves are close with respect to
the standard error M SE/m for the means.
9) The interaction plot is the most useful if the conclusions for the plot
agree with the conclusions for the F test for no interaction.
10) The ij of the cell means model can be parameterized as ij = 00 +
i + j + ()  ij for i = 1, . . . , a and j = 1, . . . , b. Here the iare the A main
eects and i i = 0. The i are the B main  eects and j j = 0. The

()ij are the interaction eects and satisfy i ()ij = 0, j ()ij = 0
6.4 Complements 221
 
and i j ()ij = 0. The interaction  eect ()ij = ij i0 0j + 00 .
Here
 the row factor means
  i0 = j ij /b, the column factor means 0j =
i ij /a and 00 = i j ij /(ab).
11) If there is no interaction, then the factor eects are additive: ij =
00 + i + j .
12) If A and B are factors, then there are 5 possible models.
i) The two way Anova model has terms A, B, and AB.
ii) The additive model or main eects model has terms A and B.
iii) The one way Anova model that uses factor A.
iv) The one way Anova model that uses factor B.
v) The null model does not use any of the three terms A, B, or AB. If the
null model holds, then Yijk f (y 00 ) so the Yijk form a random sample of
size n from a location family, and the distribution of the response is the same
for all ab observed treatments.
13) A two way Anova model could be t as a one way Anova model with
k = ab treatments, but for balanced models where nij m, this procedure
loses information about A, B, and the interaction AB.
14) Response, residual, and transformation plots are used in the same way
for the two way Anova model as for the one way Anova model.

6.4 Complements

Four good texts on the design and analysis of experiments are mentioned
in the Complements of Chapter 5. The software for K way Anova is often
used to t block designs. Each block is entered as if it were a factor and the
main eects model is t. The one way block design treats the block like one
factor and the treatment factor as another factor and uses two way Anova
software without interaction to get the correct sum of squares, F statistic,
and p-value. The Latin square design treats the row block as one factor, the
column block as a second factor, and the treatment factor as another factor.
Then the three way Anova software for main eects is used to get the correct
sum of squares, F statistic, and p-value. These two designs are described in
Chapter 7. The K way software is also used to get output for the split plot
designs described in Chapter 9.
Consider nding a model using pretesting or variable selection, and then
acting as if that model was selected before examining the data. This method
does not lead to valid inference. See Fabian (1991) for results on the 2 way
Anova model. If the method can be automated, the bootstrap method of Olive
(2016a) is conjectured to be useful for inference. This bootstrap method may
also be useful for unbalanced designs where the nij are not all equal to m.
Gail (1996) explains why it took so long to use double blinded completely
randomized controlled experiments to test new vaccines.
222 6 The K Way Anova Model

An alternative method is to perform Anova on ranks. These rank tests

appear to work for main eects, but not for interactions. See Marden and
Muyot (1995).

6.5 Problems

Problems with an asterisk * are especially important.

Output for 6.1.

Source df SS MS F P
A 2 24.6 12.3 0.24 0.791
B 2 28.3 14.2 0.27 0.763
Interaction 4 1215.3 303.8 5.84 0.001
Error 36 1872.4 52.0
6.1. The above output uses data from Kutner et al. (2005, prob-
lems 19.1617). A study measured the number of minutes to complete a
repair job at a large dealership. The two explanatory variables were A =
technician and B = make of drive. The output is given above.
a) Give a four step test for no interaction.
b) Give a four step test for the B main eects.
6.2. Suppose A has 5 levels and B has 4 levels. Sketch an interaction plot
if there is no interaction.
Two Way Anova in SAS
In SAS, Y = A|B is equivalent to Y = A B A B. Thus the SAS model
statement could be written in either of the following two forms.
proc glm;
class material temp;
model mvoltage = material|temp;
output out =a p = pred r = resid;
proc glm;
class material temp;
model mvoltage = material temp material*temp;
output out =a p = pred r = resid;
6.3. Cut and paste the SAS program from
(http://lagrange.math.siu.edu/Olive/lreghw.txt) for 6.3 into the SAS
Editor.
To execute the program, use the top menu commands Run>Submit. An
output window will appear if successful. The data is from Montgomery (1984,
p. 198) and gives the maximum output voltage for a typical type of storage
battery. The two factors are material (1,2,3) and temperature (50, 65, 80 F).
6.5 Problems 223

a) Copy and paste the SAS program into SAS, use the le command
Run>Submit.
b) Click on the Graph1 window and scroll down to the second interaction
plot of tmp vs ymn. Press the printer icon to get the plot.
c) Is interaction present?
d) Click on the output window then click on the printer icon. This will
produce 5 pages of output, but only hand in the ANOVA table, response plot,
and residual plots.
(Cutting and pasting the output into Word resulted in bad plots. Using
Notepad gave better plots, but the printer would not easily put the ANOVA
table and two plots on one page each.)
e) Do the residual and response plots look ok?

Two Way Anova in Minitab

6.4. a) Copy the SAS data for problem 6.3 into Notepad. Then hit Enter
every three numbers so that the data is in 3 columns.

1 50 130
1 50 155
1 50 74
1 50 180
1 65 34
. . .
. . .
. . .
3 80 60

b) Copy and paste the data into Minitab using the menu commands
Edit>Paste Cells and click on OK. Right below C1 type material, below
C2 type temp and below C3 type mvoltage.
c) Select Stat>ANOVA>Two-way, select C3 mvoltage as the response
and C1 material as the row factor and C2 temp as the column factor.
Click on Store residuals and click on Store ts. Then click on OK.
Click on the output and then click on the printer icon.
d) To make a residual plot, select Graph>Plot. Select Resi1 for Y and
Fits1 for X and click on OK. Click on the printer icon to get a plot of
the graph.
e) To make a response plot, select Graph>Plot. Select C3 mvoltage for
Y and Fits1 for X and click on OK. Click on the printer icon to get
a plot of the graph.

f) Use the menu commands Stat>ANOVA>Interaction Plots. Enter

mvoltage in the Responses box and material and temp in the Factors
box. Click on OK and print the plot.
224 6 The K Way Anova Model

g) Use the menu commands Stat>ANOVA>Interaction Plots. Enter

mvoltage in the Responses box and temp and material in the Factors
box. Click on OK and print the plot.
h) Do the 4 step test for interaction.

R Problem

Use the command source(G:/lregpack.txt) to download the func-

tions and the command source(G:/lregdata.txt) to download the data.
See Preface or Section 14.1. Typing the name of the R function, e.g. aov,
will display the code for the function. Use the args command, e.g. args(aov),
to display the needed arguments for the function. For the following problem,
the R commands can be copied and pasted from
(http://lagrange.math.siu.edu/Olive/lreghw.txt) into R.
In R,
Y ~ A + B is equivalent to Y ~ . so the period
indicates use all main effects. Y ~ A:B is
equivalent to Y ~ A + B + A*B and Y ~ A*B and
Y ~ .^2 which means fit all main effects and all
two way interactions. A problem is that A and B
need to be of type factor.

6.5. The Box et al. (2005, p. 318) poison data has 4 types of treatments
(1,2,3,4) and 3 types of poisons (1,2,3). Each animal is given a poison and a
treatment, and the response is survival in hours. Get the poison data from
lregdata.
a) Type the following commands to see that the output for the three
models is the same. Print the output.

out1<-aov(stime~ptype*treat,poison)
summary(out1)
out2<-aov(stime~ptype + treat + ptype*treat,poison)
summary(out2)
out3<-aov(stime~.^2,poison)
summary(out3)
#The three models are the same.

b) Type the following commands to see the residual plot. Include the plot
in Word.

plot(fitted(out1),resid(out1))
title("Residual Plot")

c) Type the following commands to see the response plot. Include the plot
in Word.
6.5 Problems 225

FIT <- poison$stime - out1$resid

plot(FIT,poison$stime)
abline(0,1)
title("Response Plot")

d) Why is the two way Anova model inappropriate?

e) Now the response Y = 1/stime will be used. Type the following com-
mands to get the output. Copy the output into Word.

attach(poison)
out4 <- aov((1/stime)~ptype*treat,poison)
summary(out4)

f) Type the following commands to get the residual plot. Copy the plot
into Word.

plot(fitted(out4),resid(out4))
title("Residual Plot")

g) Type the following commands to get the response plot. Copy the plot
into Word.

FIT <- 1/poison$stime - out4$resid

plot(FIT,(1/poison$stime))
abline(0,1)
title("Response Plot")

h) Type the following commands to get the interaction plot. Copy the plot
into Word.

interaction.plot(treat,ptype,(1/stime))
detach(poison)

i) Test whether there is an interaction using the output from e).

Chapter 7
Block Designs

Blocks are groups of similar units and blocking can yield experimental designs
that are more ecient than designs that do not block. One way block designs
and Latin square designs will be discussed.
Denition 7.1. A block is a group of mk similar or homogeneous units.
In a block design, each unit in a block is randomly assigned to one of k
treatments with each treatment getting m units from the block. The meaning
of similar is that the units are likely to have similar values of the response
when given identical treatments.
In agriculture, adjacent plots of land are often used as blocks since adjacent
plots tend to give similar yields. Litter mates, siblings, twins, time periods
(e.g., dierent days), and batches of material are often used as blocks.
Following Cobb (1998, p. 247), there are 3 ways to get blocks. i) Sort units
into groups (blocks) of mk similar units. ii) Divide large chunks of material
(blocks) into smaller pieces (units). iii) Reuse material or subjects (blocks)
several times. Then the time slots are the units.
Example 7.1. For i), to study the eects of k dierent medicines, sort
n = bk people into b groups of size k according to similar age and weight. For
ii), suppose there are b plots of land. Divide each plot into k subplots. Then
each plot is a block and the subplots are units. For iii), give the k dierent
treatments to each person over k months. Then each person has a block of
time slots and the ith month = time slot is the unit.

7.1 One Way Block Designs

Suppose there are b blocks and n = kb. The one way Anova design randomly
assigns b of the units to each of the k treatments. Blocking places a constraint
on the randomization, since within each block of units, exactly one unit is
randomly assigned to each of the k treatments.

Springer International Publishing AG 2017 227

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 7
228 7 Block Designs

Hence a one way Anova design would use the R command sample(n) and
the rst b units would be assigned to treatment 1, the second b units to
treatment 2, . . . , and the last b units would be assigned to treatment k.
For the completely randomized block designs, described below, the com-
mand sample(k) is done b times: once for each block. The ith command is
for the units of the ith block. If k = 5 and the sample(5) command yields
2 5 3 1 4, then the 2nd unit in the ith block is assigned to treatment
1, the 5th unit to treatment 2, the 3rd unit to treatment 3, the 1st unit to
treatment 4, and the 4th unit to treatment 5.

Remark 7.1. Blocking and randomization often makes the iid error
assumption hold to a useful approximation.

For example, if grain is planted in n plots of land, yields tend to be similar

(correlated) in adjacent identically treated plots, but the yields from all of
the plots vary greatly, and the errors are not iid. If there are 4 treatments
and blocks of 4 adjacent plots, then randomized blocking makes the errors
approximately iid.

Denition 7.2. For the one way block design or completely ran-
domized block design (CRBD), there is a factor A with k levels and
there are b blocks. The CRBD model is

Yij = ij + eij = + i + j + eij

k
where i is the ith treatment eect and i=1 i = 0, j is the jth block eect
b
and j=1 j = 0. The indices i = 1, . . . , k and j = 1, . . . , b. Then

1
b
io
i = ( + i + j ) = + i .
b b j=1

So the i are all equal if the i are all equal. The errors eij are iid with 0
mean and constant variance 2 .

Notice that the CRBD model is additive: there is no block treatment in-
teraction. The ANOVA table for the CRBD is like the ANOVA table for a
two way Anova main eects model. Shown below is a CRBD ANOVA table in
symbols. Sometimes Treatment is replaced by Factor or Model. Some-
times Blocks is replaced by the name of the blocking variable. Sometimes
Error is replaced by Residual.
7.1 One Way Block Designs 229

Source df SS MS F p-value
Blocks b-1 SSB MSB Fblock pblock
Treatment k-1 SSTR MSTR F0 =MSTR/MSE pval for Ho
Error (k 1)(b 1) SSE MSE

Be able to perform the 4 step completely randomized block de-

sign ANOVA F test of hypotheses. This test is similar to the xed eects
one way ANOVA F test. As always, pval is the estimated pvalue.
i) Ho: 1 = 2 = = k and HA : not Ho.
ii) Fo = MSTR/MSE is usually given by output.
iii) The pval = P(Fk1,(k1)(b1) > Fo ) is usually given by output.
iv) If the pval , reject Ho and conclude that the mean response depends
on the factor level. Otherwise fail to reject Ho and conclude that the mean
response does not depend on the factor level. (Or there is not enough evidence
to conclude that the mean response depends on the factor level.) Give a
nontechnical sentence.

Rule of thumb 7.1. If pblock 0.1, then blocking was not useful. If
0.05 < pblock < 0.1, then the usefulness was borderline. If pblock 0.05,
then blocking was useful.

Remark 7.2. The response, residual, and transformation plots are used
almost in the same way as for the one and two way Anova model, but all of
the dot plots have sample size m = 1. Look for the plotted points falling in
roughly evenly populated bands about the identity line and r = 0 line. See
Problem 7.4 for these plots and the following plot.

Denition 7.3. The block response scatterplot plots blocks versus the
response. The plot will have b dot plots of size k with a symbol corresponding
to the treatment. Dot plots with clearly dierent means suggest that blocking
was useful. A symbol pattern within the blocks (e.g., symbols A and B are
always highest while C and D are always lowest) suggests that the response
depends on the factor level.

Denition 7.4. Graphical Anova for the CRBD model uses the resid-
uals as a reference
set instead of an F distribution. The scaled treatment
deviations b 1(Y i0 Y 00 ) have about the same
variability as the resid-
uals if Ho is true. The scaled block deviations k 1(Y 0j Y 00 ) also have
about the same variability as the residuals if blocking is ineective. A dot
plot of the scaled block deviations is placed above the dot plot of the scaled
treatment deviations which is placed above the dot plot of the residuals. For
small n 40, suppose the distance between two scaled deviations (A and
B, say) is greater than the range of the residuals = max(rij ) min(rij ).
Then declare A and B to be signicantly dierent. If the distance is less
than the range, do not declare A and B to be signicantly dierent. Scaled
230 7 Block Designs

deviations that lie outside the range of the residuals are signicant: the cor-
responding treatment means are signicantly dierent from the overall mean.
For n 100, let r(1) r(2) r(n) be the order statistics of the
residuals. Then instead of the range, use r(0.975n) r(0.025n) as the dis-
tance where x is the smallest integer x, e.g. 7.7 = 8. So eects out-
side of the interval (r(0.025n) , r(0.975n) ) are signicant. See Box et al. (2005,
pp. 150151).

Example 7.2. Ledolter and Swersey (2007, p. 60) give completely ran-
domized block design data. The block variable = market had 4 levels (1
Binghamton, 2 Rockford, 3 Albuquerque, 4 Chattanooga) while the treat-
ment factor had 4 levels (A no advertising, B $6 million, C $12 million, D
$18 million advertising dollars in 1973). The response variable was average
cheese sales (in pounds per store) sold in a 3-month period.
a) From the graphical Anova in Figure 7.1, were the blocks useful?
b) Perform an appropriate 4 step test for whether advertising helped cheese
sales.

Output for Example 7.2.

Df Sum Sq Mean Sq F value Pr(>F)
block 3 79308210 26436070 54.310 4.348e-06
treatment 3 1917416 639139 1.313 0.3292
Residuals 9 4380871 486763

> ganova2(x,block,y) scaled block deviations

-3790.377 4720.488 2881.483 -3811.594
block 1 2 3 4

scaled treatment deviations

-266.086 -833.766 733.307 366.545
Treatments "A" "B" "C" "D"

Solution: a) In Figure 7.1, the top dot plot is for the scaled block deviations.
The leftmost dot corresponds to blocks 4 and 1, the middle dot to block 3
and the rightmost dot to block 1 (see output from the lregpack function
ganova2). Yes, the blocks were useful since some (actually all) of the dots
corresponding to the scaled block deviations fall outside the range of the
residuals. This result also agrees with pblock = 4.348e06 < 0.05.
b) i) Ho: 1 = 2 = 3 = 4 HA : not Ho
ii) Fo = 1.313
iii) pval = 0.3292
iv) Fail to reject Ho, the mean sales does not depend on advertising level.
In Figure 7.1, the middle dot plot is for the scaled treatment deviations.
From left to right, these correspond to B, A, D, and C since the output shows
that the deviation corresponding to C is the largest with value 733.3. Since
7.1 One Way Block Designs 231

Scaled Block Deviations

30
25
Treatmentdevs
20
15
10

4000 2000 0 2000 4000

Residuals
Fig. 7.1 Graphical Anova for a One Way Block Design

the four scaled treatment deviations all lie within the range of the residuals,
the four treatments again do not appear to be signicant.

frate Block Response Plot for Example 7.3

-
- A
12.0+ A
- B A
- 2 D C
-
- E 2
8.0+ A C
- E B
- B D
-
- D E E B
4.0+ C
- D E
- B
-
-
+------+------+------+------+------+--block
1 2 3 4 5
232 7 Block Designs

Example 7.3. Snedecor and Cochran (1967, p. 300) give a data set with
5 types of soybean seed. The response frate = number of seeds out of 100
that failed to germinate. Five blocks were used. On the previous page is a
block response scatterplot where A, B, C, D, and E refer to seed type. The 2
in the second block indicates that A and C both had values 10. Which type
of seed has the highest germination failure rate?
a) A b) B c) C d) D e) E
Solution: a) A since A is on the top for blocks 25 and second for block 1.

Response Plot
10

5
Y
6
2

4 6 8 10 12
FIT

Residual Plot
5
2 4
RES
2

4 6 8 10 12
FIT
Fig. 7.2 One Way Block Design Does Not Fit All of the Data

Note: The response and residual plots in Figure 7.2 suggest that one case
is not t well by the model. The Bs and Es in the block response plot suggest
that there may be a block treatment interaction, which is not allowed by the
completely randomized block design. Figure 7.2 was made with the following
commands using the lregpack function aovplots.

block <- c(1,1,1,1,1,2,2,2,2,2,3,3,3,3,3,4,4,4,4,4,5,

5,5,5,5)
seed <- rep(1:5,5)
block <- factor(block)
seed <- factor(seed)
frate <- c(8,2,4,3,9,10,6,10,5,7,12,7,9,9,5,13,11,8,
10,5,11,5,10,6,3)
7.2 Blocking with the K Way Anova Design 233

soy <- data.frame(block,seed,frate)

rm(block,seed,frate)
attach(soy)
z <- aov(frate~block+seed,soy)
aovplots(Y=frate,FIT=fitted(z),RES=resid(z))
#right click Stop twice
detach(soy)

7.2 Blocking with the K Way Anova Design

Blocking is used to reduce the MSE so that inference such as tests and con-
dence intervals are more precise. Below is a partial ANOVA table for a k way
Anova design with one block where the degrees of freedom are left blank. For
A, use H0 : 100 = = l1 00 . The other main eects have similar null
hypotheses. For interaction, use H0 : no interaction.
These models get complex rapidly as k and the number of levels li in-
crease. As k increases, there are a large number of models to consider. For
experiments, usually the 3 way and higher order interactions are not signif-
icant.
# $ Hence a full model that includes the blocks, all k main eects, and
all k2 two way interactions is a useful starting point for response, residual,
and transformation plots. The higher order interactions can be treated as
potential terms and checked for signicance. As a rule of thumb, signicant
interactions tend to involve signicant main eects.

Source df SS MS F p-value
block SSblock MSblock Fblock pblock
#k $ k main eects e.g. SSA = MSA FA p A
2 2 way interactions e.g. SSAB = MSAB FAB pAB
#k $
3 3 way interactions e.g. SSABC = MSABC FABC pABC
.. .. .. .. ..
# $ . . . . .
k
k1 k 1 way interactions
the k way interaction SSA L = MSA L FAL pAL
Error SSE MSE

The following example has one block and 3 factors. Hence there are 3 two
way interactions and 1 three way interaction.

Example 7.4. Snedecor and Cochran (1967, pp. 361364) describe a block
design (2 levels) with three factors: food supplements Lysine (4 levels), Me-
thionine (3 levels), and Protein (2 levels). Male pigs were fed the supplements
in a 432 factorial arrangement and the response was average daily weight
gain. The ANOVA table is shown on the following page. The model could be
234 7 Block Designs

described as Yijkl = ijkl + eijkl for i = 1, 2, 3, 4; j = 1, 2, 3; k = 1, 2; and

l = 1, 2 where i, j, k are for L,M,P and l is for block. Note that i000 is the
mean corresponding to the ith level of L.
a) There were 24 pigs in each block. How were they assigned to the 24 =
4 3 2 runs (a run is an L,M,P combination forming a pig diet)?
b) Was blocking useful?
c) Perform a 4 step test for the signicant main eect.
d) Which, if any, of the interactions were signicant?

Solution: a) Randomly.
b) Yes, 0.0379 < 0.05.
c) H0 : 0010 = 0020 HA : not H0
FP = 19.47
pval = 0.0002
Reject H0 , the mean weight gain depends on the protein level.
d) None.

Source df SS MS F pvalue
block 1 0.1334 0.1334 4.85 0.0379
L 3 0.0427 0.0142 0.5164 0.6751
M 2 0.0526 0.0263 0.9564 0.3990
P 1 0.5355 0.5355 19.47 0.0002
LM 6 0.2543 0.0424 1.54 0.2099
LP 3 0.2399 0.0800 2.91 0.0562
MP 2 0.0821 0.0410 1.49 0.2463
LMP 6 0.0685 0.0114 0.4145 0.8617
error 23 0.6319 0.0275

Remark 7.3. There are 3 basic principles of DOE. Randomization, fac-

torial crossing, and blocking can be used to create many DOE models.
i) Use randomization to assign units to treatments.
ii) Use factorial crossing to compare the eects of 2 or more factors in
the same experiment: if A1 , A2 , . . . , Ak are the k factors where the ith factor
Ai has li levels, then there are (l1 )(l2 ) (lk ) treatments where a treatment
has one level from each factor.
iii) Use blocking to increase precision. Divide units into blocks of similar
homogeneous units where similar implies that the units are likely to have
similar values of the response if given the same treatment. Within each block,
randomly assign units to treatments.

7.3 Latin Square Designs

Latin square designs have a lot of structure. The design contains a row block
factor, a column block factor, and a treatment factor, each with a levels. The
7.3 Latin Square Designs 235

two blocking factors, and the treatment factor are crossed, but it is assumed
that there is no interaction. A capital letter is used for each of the a treatment
levels. So a = 3 uses A, B, C while a = 4 uses A, B, C, D.

Denition 7.5. In an a a Latin square, each letter appears exactly once

in each row and in each column. A standard Latin square has letters written
in alphabetical order in the rst row and in the rst column.

Five Latin squares are shown below. The rst, third, and fth are standard.
If a = 5, there are 56 standard Latin squares.
A B C A B C A B C D A B C D E A B C D E
B C A C A B B A D C E A B C D B A E C D
C A B B C A C D A B D E A B C C D A E B
D C B A C D E A B D E B A C
B C D E A E C D B A

Denition 7.6. The model for the Latin square design is

Yijk = + i + j + k + eijk

where i is the ith treatment eect, j is the jth row block eect, k is the
kth column block eect with i, j, and k = 1, . . . , a. The errors eijk are iid
with 0 mean and constant variance 2 . The ith treatment mean i = + i .

Shown below is an ANOVA table for the Latin square model given in
symbols. Sometimes Error is replaced by Residual, or Within Groups.
Sometimes rblocks and cblocks are replaced by the names of the blocking
factors. Sometimes p-value is replaced by P, P r(> F ), or PR > F.

Source df SS MS F p-value
rblocks a1 SSRB MSRB Frow prow
cblocks a1 SSCB MSCB Fcol pcol
treatments a1 SSTR MSTR Fo =MSTR/MSE pval
Error (a 1)(a 2) SSE MSE

Rule of thumb 7.2. Let pblock be prow or pcol . If pblock 0.1, then block-
ing was not useful. If 0.05 < pblock < 0.1, then the usefulness was borderline.
If pblock 0.05, then blocking was useful.

Be able to perform the 4 step ANOVA F test for the Latin square
design. This test is similar to the xed eects one way ANOVA F test.
i) Ho: 1 = 2 = = a and HA : not Ho.
ii) Fo = MSTR/MSE is usually given by output.
iii) The pval = P(Fa1,(a1)(a2) > Fo ) is usually given by output.
iv) If the pval , reject Ho and conclude that the mean response depends
on the factor level. Otherwise fail to reject Ho and conclude that the mean
response does not depend on the factor level. (Or there is not enough evidence
236 7 Block Designs

to conclude that the mean response depends on the factor level.) Give a
nontechnical sentence. Use = 0.05 if is not given.

Remark 7.4. The response, residual, and transformation plots are used
almost in the same way as for the one and two way Anova models, but all of
the dot plots have sample size m = 1. Look for the plotted points falling in
roughly evenly populated bands about the identity line and r = 0 line. See
Problem 7.5 and the following example.

Source df SS MS F P
rblocks 3 774.335 258.1117 2.53 0.1533
cblocks 3 133.425 44.4750 0.44 0.7349
fertilizer 3 1489.400 496.4667 4.87 0.0476
error 6 611.100 101.8500

Example 7.5. Dunn and Clark (1974, p. 129) examine a study of four
fertilizers on yields of wheat. The row blocks were 4 types of wheat. The
column blocks were 4 plots of land. Each plot was divided into 4 subplots
and a Latin square design was used. (To illustrate the inference for Latin
square designs, ignore the fact that the data had an outlier. Case 14 had a
yield of 64.5 while the next highest yield was 35.5. For the response plot in
Figure 7.3, note that both Y and Y are large for the high yield. Also note
that Y underestimates Y by about 10 for this case.)
a) Were the row blocks useful? Explain briey.
b) Were the column blocks useful? Explain briey.
c) Do an appropriate 4 step test.

Solution:
a) No, prow = 0.1533 > 0.1.
b) No, pcol = 0.7349 > 0.1.
c) i) H0 : 1 = 2 = 3 = 4 HA : not H0
ii) F0 = 4.87
iii) pval = 0.0476
iv) Reject H0 . The mean yield depends on the fertilizer level.

Figure 7.3 was made with the following commands using the lregpack func-
tion aovplots.
rblocks <- c(1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4)
cblocks <- c(1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4)
fertilizer <- c(1,2,3,4,2, 3, 4, 1, 3, 4, 1, 2, 4, 1, 2, 3)
yield <- c(35.5,24.5,14.7,35.5, 14.4, 6.2, 13.7, 24.5, 14.1,
16.2, 34.3, 19.7, 15.0, 64.5, 34.6, 19.0)
rblocks <- factor(rblocks)
cblocks <- factor(cblocks)
fertilizer <- factor(fertilizer)
dcls <- data.frame(yield,rblocks,cblocks,fertilizer)
rm(yield,rblocks,cblocks,fertilizer)
7.3 Latin Square Designs 237

Response Plot
14

10 30 50
Y

10 20 30 40 50
FIT

Residual Plot
10

14
RES
0
10

10 20 30 40 50
FIT
Fig. 7.3 Latin Square Data

attach(dcls)
z <- aov(yield~rblocks+cblocks+fertilizer)
summary(z)
aovplots(Y=yield,FIT=fitted(z),RES=resid(z))
#right click Stop twice, drag the plots to make them square
detach(dcls)

Remark 7.5. The Latin square model is additive, but the model is often
incorrectly used to study nuisance factors that can interact. Factorial or
fractional factorial designs should be used when interaction is possible.

Remark 7.6. The randomization is done in 3 steps. Draw 3 random per-

mutations of 1, . . . , a. Use the 1st permutation to randomly assign row block
levels to the numbers 1, . . . , a. Use the 2nd permutation to randomly as-
sign column block levels to the numbers 1, . . . , a. Use the 3rd permutation
to randomly assign treatment levels to the 1st a letters (A, B, C, and D
if a = 4).

Example 7.6. In the social sciences, often a blocking factor is time: the
levels are a time slots. Following Cobb (1998, p. 254), a Latin square design
was used to study the response Y = blood sugar level, where the row blocks
were 4 rabbits, the column blocks were 4 time slots, and the treatments were
4 levels of insulin. Label the rabbits as I, II, III, and IV; the dates as 1, 2, 3,
4; and the 4 insulin levels i1 < i2 < i3 < i4 as 1, 2, 3, 4. Suppose the random
permutation for the rabbits was 3, 1, 4, 2; the permutation for the dates 1,
4, 3, 2; and the permutation for the insulin levels was 2, 3, 4, 1. Then i2 is
238 7 Block Designs

treatment A, i3 is treatment B, i4 is treatment C, and i1 is treatment D.

Then the data are as shown below on the left. The data is rearranged for
presentation on the right.

raw data presentation data

date date
rabbit 4/23 4/27 4/26 4/25 rabbit 4/23 4/25 4/26 4/27
III 57A 45B 60C 26D I 24B 46C 34D 48A
I 24B 48A 34D 46C II 33D 58A 57B 60C
IV 46C 47D 61A 34B III 57A 26D 60C 45B
II 33D 60C 57B 58A IV 46C 34B 61A 47D

Example 7.7. Following Cobb (1998, p. 255), suppose there is a rectan-

gular plot divided into 5 rows and 5 columns to form 25 subplots. There
are 5 treatments which are 5 varieties of a plant, labelled 1, 2, 3, 4, 5; and
the response Y is yield. Adjacent subplots tend to give similar yields under
identical treatments, so the 5 rows form the row blocks and the 5 columns
form the column blocks. To perform randomization, three random permuta-
tions are drawn. Shown below are 3 Latin squares. The one on the left is an
unrandomized Latin square.
Suppose 2, 4, 3, 5, 1 is the permutation drawn for rows. The middle Latin
square with randomized rows has 1st row which is the 2nd row from the
original unrandomized Latin square. The middle square has 2nd row that is
the 4th row from the original, the 3rd row is the 3rd row from the original,
the 4th row is the 5th row from the original, and the 5th row is the 1st row
from the original.

unrandomized randomized rows randomized Latin square

rows columns rows columns rows columns
1 2 3 4 5 1 2 3 4 5 1 4 2 5 3
1 A B C D E 2 B C D E A 2 B E C A D
2 B C D E A 4 D E A B C 4 D B E C A
3 C D E A B 3 C D E A B 3 C A D B E
4 D E A B C 5 E A B C D 5 E C A D B
5 E A B C D 1 A B C D E 1 A D B E C

Suppose 1, 4, 2, 5, 3 is the permutation drawn for columns. Then the

randomized Latin square on the right has 1st column which is the 1st column
from the middle square, the 2nd column is the 4th column from the middle
square, the 3rd column is the 2nd column from the middle square, the 4th
column is the 5th column from the middle square, and the 5th column is the
3rd column from the middle square.
Suppose 3, 2, 5, 4, 1 is the permutation drawn for variety. Then variety 3
is treatment A, 2 is B, 5 is C, 4 is D, and variety 1 is E. Now sow each subplot
with the variety given by the randomized Latin square on the right. Hence
7.4 Summary 239

the northwest corner of the square gets B = variety 2, the northeast corner
gets D = variety 4, the southwest corner gets A = variety 3, the southeast
corner gets C = variety 5, et cetera.

7.4 Summary

1) A block is a group of similar (homogeneous) units in that the units in a

block are expected to give similar values of the response if given the same
treatment.
2) In agriculture, adjacent plots of land are often used as blocks since
adjacent plots tend to give similar yields. Litter mates, siblings, twins, time
periods (e.g., dierent days), and batches of material are often used as blocks.
3) The completely randomized block design (CRBD) with k treatments
and b blocks of k units uses randomization within each block to assign ex-
actly one of the blocks k units to each of the k treatments. This design is a
generalization of the matched pairs procedure used for k = 2.
4) The ANOVA F test for the completely randomized block design with
k treatments and b blocks is nearly the same as the xed eects one way
ANOVA F test.
i) Ho: 1 = 2 = = k and HA : not Ho.
ii) Fo = MSTR/MSE is usually given by output.
iii) The pval = P(Fk1,(k1)(b1) > Fo ) is usually given by output.
iv) If the pval , reject Ho and conclude that the mean response depends
on the factor level. Otherwise fail to reject Ho and conclude that the mean
response does not depend on the factor level. Give a nontechnical sentence.
5) Shown below is an ANOVA table for the completely randomized block
design.

Source df SS MS F p-value
Blocks b-1 SSB MSB Fblock pblock
Treatment k-1 SSTR MSTR F0 =MSTR/MSE pval for Ho
Error (k 1)(b 1) SSE MSE
6) Rule of thumb: If pblock 0.1, then blocking was not useful. If 0.05 <
pblock < 0.1, then the usefulness was borderline. If pblock 0.05, then blocking
was useful.
7) The response, residual, and transformation plots for CRBD models are
used almost in the same way as for the one and two way Anova model, but all
of the dot plots have sample size m = 1. Look for the plotted points falling
in roughly evenly populated bands about the identity line and r = 0 line.
8) The block response scatterplot plots blocks versus the response.
The plot will have b dot plots of size k with a symbol corresponding to the
treatment. Dot plots with clearly dierent means suggest that blocking was
useful. A symbol pattern within the blocks suggests that the response depends
on the factor level.
240 7 Block Designs

9) Shown is an ANOVA table for the Latin square model given in sym-
bols. Sometimes Error is replaced by Residual, or Within Groups.
Sometimes rblocks and cblocks are replaced by the blocking factor name.
Sometimes p-value is replaced by P, P r(> F ), or PR > F.

Source df SS MS F p-value
rblocks a1 SSRB MSRB Frow prow
cblocks a1 SSCB MSCB Fcol pcol
treatments a1 SSTR MSTR Fo =MSTR/MSE pval
Error (a 1)(a 2) SSE MSE

10) Let pblock be prow or pcol . Rule of thumb: If pblock 0.1, then blocking
was not useful. If 0.05 < pblock < 0.1, then the usefulness was borderline. If
pblock 0.05, then blocking was useful.
11) The ANOVA F test for the Latin square design with a treatments is
nearly the same as the xed eects one way ANOVA F test.
i) Ho: 1 = 2 = = a and HA : not Ho.
ii) Fo = MSTR/MSE is usually given by output.
iii) The pval = P(Fa1,(a1)(a2) > Fo ) is usually given by output.
iv) If the pval , reject Ho and conclude that the mean response depends
on the factor level. Otherwise fail to reject Ho and conclude that the mean
response does not depend on the factor level. Give a nontechnical sentence.
12) The response, residual, and transformation plots for Latin square de-
signs are used almost in the same way as for the one and two way Anova
models, but all of the dot plots have sample size m = 1. Look for the plotted
points falling in roughly evenly populated bands about the identity line and
r = 0 line.
13) The randomization is done in 3 steps. Draw 3 random permutations
of 1, . . . , a. Use the 1st permutation to randomly assign row block levels to
the numbers 1, . . . , a. Use the 2nd permutation to randomly assign column
block levels to the numbers 1, . . . , a. Use the 3rd permutation to randomly
assign treatment levels to the 1st a letters (A, B, C, and D if a = 4).
14) Graphical Anova for the completely randomizedblock de-
makes a dot plot of the scaled block deviations j = k 1j =
sign
k 1(y 0j0 y 000 ) on top, a dot plot of scaled treatment deviations (eects)

i = b 1i = b 1(y i00 y 000 ) in the middle, and a dot plot of the
residuals on the bottom. Here k is the number of treatments and b is the
number of blocks.
15) Graphical Anova uses the residuals as a reference distribution. Suppose
the dot plot of the residuals looks good. Rules of thumb: i) An eect is
marginally signicant if its scaled deviation is as big as the biggest residual
or as negative as the most negative residual. ii) An eect is signicant if it is
well beyond the minimum or maximum residual. iii) Blocking was eective
if at least one scaled block deviation is beyond the range of the residuals.
7.5 Complements 241

iv) The treatments are dierent if at least one scaled treatment eect is
beyond the range of the residuals. (These rules depend on the number of
residuals n. If n is very small, say 8, then the scaled eect should be well
beyond the range of the residuals to be signicant. If the n is 40, the value
of the minimum residual and the value of the maximum residual correspond
to a 1/40 + 1/40 = 1/20 = 0.05 critical value for signicance.)

7.5 Complements

Box et al. (2005, pp. 150156) explain Graphical Anova for the CRBD and
why randomization combined with blocking often makes the iid error assump-
tion hold to a reasonable approximation.
It is easier to see model deciencies if the response and residual plots are
square. In R, drag the plots so the plots look square. Matched pairs tests are
a special case of CRBD with k = 2.
The R package granova may be useful for graphical Anova. It is available
from (http://streaming.stat.iastate.edu/CRAN/) and authored by R.M.
Pruzek and J.E. Helmreich. Also see Hoaglin et al. (1991).
A randomization test has H0 : the dierent treatments have no eect.
This null hypothesis is also true if within each block, all k pdfs are from the
same location family. Let j = 1, . . . , b index the b blocks. There are b pdfs, one
for each block, that come from the same location family but possibly dierent
location parameters: fZ (y 0j ). Let A be the treatment factor with k levels
ai . Then Yij |(A = ai ) fZ (y 0j ) where j is xed and i = 1, . . . , k.
Thus the levels ai have no eect on the response, and the Yij are iid within
each block if H0 holds. Note that there are k! ways to assign Y1j , . . . Ykj
to the k treatments within each block. An impractical randomization test
uses all M = [k!]b ways of assigning responses to treatments. Let F0 be the
usual CRBD F statistic. The F statistic is computed for each of the M
permutations and H0 is rejected if the proportion of the M F statistics that
are larger than F0 is less than . The distribution of the M F statistics is
approximately Fk1,(k1)(b1) for large n under H0 . The randomization test
and the usual CBRD F test also have the same power, asymptotically. See
Hoeding (1952) and Robinson (1973). These results suggest that the usual
CRBD F test is semiparametric: the pvalue is approximately correct if n is
large and if all k pdfs Yij |(A = ai ) fZ (y 0j ) are the same for each block
where j is xed and i = 1, . . . , k. If H0 does not hold, then there are kb pdfs
Yij |(A = ai ) fZ (y ij ) from the same location family. Hence the location
parameter depends on both the block and treatment.
Olive (2014, section 9.3) shows that practical randomization tests that
use a random sample of max(1000, [n log(n)]) randomizations have level and
power similar to the tests that use all M possible randomizations. Here each
randomization uses b randomly drawn permutations of 1, . . . , k.
242 7 Block Designs

Hunter (1989) discusses some problems with the Latin square design.
Welch (1990) suggests that the ANOVA F test is not a good approxima-
tion for the permutation test for the Latin square design.

7.6 Problems

Problems with an asterisk * are especially important.

Output for 7.1.
source Df Sum Sq Mean Sq F value Pr(>F)
block 4 49.84 12.46 2.3031 0.10320
seed 4 83.84 20.96 3.8743 0.02189
Residuals 16 86.56 5.41
7.1. Snedecor and Cochran (1967, p. 300) give a data set with 5 types of
soybean seed. The response frate = number of seeds out of 100 that failed
to germinate. Five blocks were used. Assume the appropriate model can be
used (although this assumption may not be valid due to a possible interaction
between the block and the treatment).
a) Did blocking help? Explain briey.
b) Perform the appropriate 4 step test using the output above.
Output for 7.2.
Source df SS MS F P
blocks 3 197.004 65.668 9.12 0.001
treatment 5 201.316 40.263 5.59 0.004
error 15 108.008 7.201
7.2. Current nitrogen fertilization recommendations for wheat include
applications of specied amounts at specied stages of plant growth. The
treatment consisted of six dierent nitrogen application and rate schedules.
The wheat was planted in an irrigated eld that had a water gradient in one
direction as a result of the irrigation. The eld plots were grouped into four
blocks, each consisting of six plots, such that each block occurred in the same
part of the water gradient. The response was the observed nitrogen content
from a sample of wheat stems from each plot. The experimental units were
the 24 plots. Data is from Kuehl (1994, p. 263).
a) Did blocking help? Explain briey.

b) Perform the appropriate 4 step test using the output above.

7.3. An experimenter wanted to test 4 types of an altimeter. There were
eight helicopter pilots available for hire with from 500 to 3000 ight hours of
experience. The response variable was the altimeter reading error. Perform
the appropriate 4 step test using the output below. Data is from Kirk (1982,
p. 244).
7.6 Problems 243

Output for Problem 7.3

Source df SS MS F P
treatment 3 194.50 64.833 47.78 0.000
blocks 7 12.50 1.786 1.32
error 21 28.50 1.357

One way randomized block designs in SAS, Minitab, and R

7.4. This problem is for a one way block design and uses data from Box
et al. (2005, p. 146).
a) Copy and paste the SAS program for this problem from
(http://lagrange.math.siu.edu/Olive/lreghw.txt). Print out the out-
put but only turn in the ANOVA table, residual plot, and response plot.
b) Do the plots look ok?
c) Copy the SAS data into Minitab much as done for Problem 6.4. Right
below C1 type block, below C2 type treat, and below C3 type yield.
d) Select Stat>ANOVA>Two-way, select C3 yield as the response and
C1 block as the row factor and C2 treat as the column factor. Click on
Fit additive model, click on Store residuals, and click on Store ts.
Then click on OK.
e) block response scatterplot: Use le commands Edit>Command
Line Editor and write the following lines in the window.
GSTD
LPLOT yield vs block codes for treat

f) Click on the submit commands box and print the plot. Click on the
output and then click on the printer icon.
g) Copy (http://lagrange.math.siu.edu/Olive/lregdata.txt) into R.
Type the following commands to get the following ANOVA table.
z<-aov(yield~block+treat,pen)
summary(z)

Df Sum Sq Mean Sq F value Pr(>F)

block 4 264.000 66.000 3.5044 0.04075 *
treat 3 70.000 23.333 1.2389 0.33866
Residuals 12 226.000 18.833

h) Did blocking appear to help?

i) Perform a 4 step F test for whether yield depends on treatment.

Latin Square Designs in SAS and R

(Latin square designs can be t by Minitab, but not with the Students
version of Minitab.)
For some of the following problems, the R commands can be copied and
pasted from (http://lagrange.math.siu.edu/Olive/lreghw.txt) into R.

7.5. This problem is for a Latin square design and uses data from Box
et al. (2005, pp. 157160).
244 7 Block Designs

Copy and paste the SAS program for this problem from
(http://lagrange.math.siu.edu/Olive/lreghw.txt).
a) Click on the output and use the menu commands Edit>Select All
and Edit>Copy. In Word use the menu command Paste then use the
left mouse button to highlight the rst page of output. Then use the menu
command Cut. Then there should be one page of output including the
ANOVA table. Print out this page.
b) Copy the data for this problem from
(http://lagrange.math.siu.edu/Olive/lregdata.txt)
into R. Use the following commands to create a residual plot. Copy and paste
the plot into Word. (Click on the plot and simultaneously hit the Ctrl and c
buttons. Then go to Word and use the menu command Paste.)
z<-aov(emissions~rblocks+cblocks+additives,auto)
summary(z)
plot(fitted(z),resid(z))
title("Residual Plot")
abline(0,0)
c) Use the following commands to create a response plot. Copy and paste
the plot into Word. (Click on the plot and simultaneously hit the Ctrl and c
buttons. Then go to Word and use the menu command Paste.)
attach(auto)
FIT <- auto$emissions - z$resid
plot(FIT,auto$emissions)
title("Response Plot")
abline(0,1)
detach(auto)
d) Do the plots look ok?
e) Were the column blocks useful? Explain briey.
f) Were the row blocks useful? Explain briey.
g) Do an appropriate 4 step test.
7.6. Obtain the Box et al. (2005, p. 146) penicillin data from
(http://lagrange.math.siu.edu/Olive/lregdata.txt) and the R pro-
gram ganova2 from (http://lagrange.math.siu.edu/Olive/lregpack.
txt). The program does graphical Anova for completely randomized block
designs.
a) Copy and paste the R commands for this problem into R. Include the
plot in Word by simultaneously pressing the Ctrl and c keys, then using the
menu command Paste in Word.
b) Blocking seems useful because some of the scaled block deviations are
outside of the spread of the residuals. The scaled treatment deviations are in
the middle of the plot. Do the treatments appear to be signicantly dierent?
Chapter 8
Orthogonal Designs

Orthogonal designs for factors with two levels can be t using least squares.
The orthogonality of the contrasts allows each coecient to be estimated
independently of the other variables in the model.
kf
This chapter covers 2k factorial designs, 2R fractional factorial designs,
and Plackett Burman PB(n) designs. The entries in the design matrix X are
either 1 or 1. The columns of the design matrix X are orthogonal: cTi cj = 0
for i = j where ci is the ith column of X. Also cTi ci = n, and the absolute
values of the column entries sum to n.
The rst column of X is 1, the vector of ones, but the remaining columns
of X are the coecients of a contrast. Hence the ith column ci has entries
that are 1 or 1, and the entries of the ith column ci sum to 0 for i > 1.

8.1 Factorial Designs

Factorial designs are a special case of the k way Anova designs of Chapter 6,
and these designs use factorial crossing to compare the eects (main eects,
pairwise interactions, . . . , k-fold interaction) of the k factors. If A1 , . . . , Ak are
the factors with li levels for i = 1, . . . , k then there are l1 l2 lk treatments
where each ' treatment uses exactly one level from each factor. The sample
k
size n = m i=1 li m 2k . Hence the sample size grows exponentially fast
with k. Often the number of replications m = 1.

Denition 8.1. An experiment has n runs where a run is used to measure

a response. A run is a treatment = a combination of k levels. So each run
uses exactly one level from each of the k factors.

Often each run is expensive, for example, in industry and medicine. A goal
is to improve the product in terms of higher quality or lower cost. Often the

Springer International Publishing AG 2017 245

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 8
246 8 Orthogonal Designs

subject matter experts can think of many factors that might improve the
product. The number of runs n is minimized by taking li = 2 for i = 1, . . . , k.

Denition 8.2. A 2k factorial design is a k way Anova design where each

factor has two levels: low = 1 and high = 1. The design uses n = m2k runs.
Often the number of replications m = 1. Then the sample size n = 2k .
A 2k factorial design is used to screen potentially useful factors. Usually
at least k = 3 factors are used, and then 23 = 8 runs are needed. Often
the units are time slots, and each time slot is randomly assigned to a run
= treatment. The subject matter experts should choose the two levels. For
example, a quantitative variable such as temperature might be set at 80o F
coded as 1 and 100o F coded as 1, while a qualitative variable such as type
of catalyst might have catalyst A coded as 1 and catalyst B coded as 1.

Improving a process is a sequential, iterative process. Often high values of

the response are desirable (e.g. yield), but often low values of the response
are desirable (e.g. number of defects). Industrial experiments have a budget.
The initial experiment may suggest additional factors that were omitted,
suggest new sets of two levels, and suggest that many initial factors were
not important or that the factor is important, but the level of the factor is
not. (For example, one factor could be a catalyst with chemical yield as the
response. It is possible that both levels of the catalyst produce about the
same yield, but the yield would be 0 if the catalyst was not used. Then the
catalyst is an important factor, but the yield did not depend on the level of
catalyst used in the experiment.)
Suppose k = 5 and A, B, C, D, and E are factors. Assume high response
is desired and high levels of A and C correspond to high response where A
is qualitative (e.g. 2 brands) and C is quantitative but set at two levels (e.g.
temperature at 80 and 100o F ). Then the next stage may use an experiment
with factor A at its high level and at a new level (e.g. a new brand) and
C at the highest level from the previous experiment and at a higher level
determined by subject matter experts (e.g. at 100 and 120o F ).

Rule of thumb 8.1. Do not spend more than 25% of the budget on the
initial experiment. It may be a good idea to plan for four experiments, each
taking 25% of the budget.
p p
Denition 8.3. Recall that a contrast C = i=1 di i where i=1 di =
p
0, and the estimated contrast is C = i=1 di Y i0 where i and Y i0 are
appropriate population and sample means. In a table of contrasts, the
coecients di of the contrast are given where a corresponds to 1 and a +
corresponds to 1. Sometimes a column I corresponding to the overall mean
is given where each entry is a +. The column corresponding to I is not a
contrast.

To make a table of contrasts there is a rule for main eects and a rule for
interactions.
8.1 Factorial Designs 247

a) In a table of contrasts, the column for A starts with a then a + and

the pattern repeats. The column for B starts with 2 s and then 2 +s and
the pattern repeats. The column for C starts with 4 s and then 4 +s and
the pattern repeats. The column for the ith main eects factor starts with
2i1 s and 2i1 +s and the pattern repeats where i = 1, . . . , k.
b) In a table of contrasts, a column for an interaction containing several
factors is obtained by multiplying the columns for each factor where + = 1
and = 1. So the column for ABC is obtained by multiplying the column
for A, the column for B, and the column for C.
A table of contrasts for a 23 design is shown below. The rst column is
for the mean and is not a contrast. The last column corresponds to the cell
means. Note that y 1110 = y111 if m = 1. So y might be replaced by y if m = 1.
Each row corresponds to a run. Only the levels of the main eects A, B, and
C are needed to specify each run. The rst row of the table corresponds to
the low levels of A, B, and C. Note that the divisors are 2k1 except for the
divisor of I which is 2k where k = 3.

I A B C AB AC BC ABC y
+ + + + y 1110
+ + + + y 2110
+ + + + y 1210
+ + + + y 2210
+ + + + y 1120
+ + + + y 2120
+ + + + y 1220
+ + + + + + + + y 2220
divisor 8 4 4 4 4 4 4 4

The table of contrasts for a 24 design is shown on the following page. The
column of ones corresponding to I was omitted. Again rows correspond to
runs and the levels of the main eects A, B, C, and D completely specify the
run. The rst row of the table corresponds to the low levels of A, B, C, and
D. In the second row, the level of A is high while B, C, and D are low. Note
that the interactions are obtained by multiplying the component columns
where + = 1 and = 1. Hence the rst row of the column corresponding
to the ABC entry is ()()() = .
Randomization for a 2k design: The runs are determined by the levels
of the k main eects in the table of contrasts. So a 23 design is determined by
the levels of A, B, and C. Similarly, a 24 design is determined by the levels
of A, B, C, and D. Randomly assign units to the m2k runs. Often the units
are time slots. If possible, perform the m2k runs in random order.
Genuine run replicates need to be used. A common error is to take m
measurements per run, and act as if the m measurements are from m runs.
248 8 Orthogonal Designs

If as a data analyst you encounter this error, average the m measurements

into a single value of the response.
run A B C D AB AC AD BC BD CD ABC ABD ACD BCD ABCD
1 - - - - + + + + + + - - - - +
2 + - - - - - - + + + + + + - -
3 - + - - - + + - - + + + - + -
4 + + - - + - - - - + - - + + +
5 - - + - + - + - + - + - + + -
6 + - + - - + - - + - - + - + +
7 - + + - - - + + - - - + + - +
8 + + + - + + - + - - + - - - -
9 - - - + + + - + - - - + + + -
10 + - - + - - + + - - + - - + +
11 - + - + - + - - + - + - + - +
12 + + - + + - + - + - - + - - -
13 - - + + + - - - - + + + - - +
14 + - + + - + + - - + - - + - -
15 - + + + - - - + + + - - - + -
16 + + + + + + + + + + + + + + +

Denition 8.4. If the response depends on the two levels of the factor,
then the factor is called active. If the response does not depend on the two
levels of the factor, then the factor is called inert.

Active factors appear to change the mean response as the level of the factor
changes from 1 to 1. Inert factors do not appear to change the response as
the level of the factor changes from 1 to 1. An inert factor could be needed
but the level low or high is not important, or the inert factor may not be
needed and so can be omitted from future studies. Often subject matter
experts can tell whether the inert factor is needed or not.
The 2k designs are used for exploratory data analysis: they provide
answers to the following questions.
i) Which combinations of levels are best?
ii) Which factors are active and which are inert? That is, use the 2k design
to screen for factors where the response depends on whether the level is high
or low.
iii) How should the levels be modied to improve the response?

If all 2k runs give roughly the same response, then choose the levels that
are cheapest to increase prot. Also the system tends to be robust to changes
in the factor space so managers do not need to worry about the exact values
of the levels of the factors.
In an experiment, there will be an interaction between management, sub-
ject matter experts (often engineers), and the data analyst (statistician).
8.1 Factorial Designs 249

Remark
# $ 8.1. If m = 1, then there
# $ is one response per run but k main
eects, k2 2 factor interactions, kj j factor interactions, and 1 k way in-
teraction. Then the MSE df = 0 unless at least one high order interaction
is assumed
# $ to be zero. A full model that includes all k main eects and
all k2 two way interactions is a useful starting point for response, residual,
and transformation plots. The higher order interactions can be treated as
potential terms and checked for signicance. As a rule of thumb, signicant
interactions tend to involve signicant main eects.

Denition 8.5. An outlier corresponds to a case that is far from the

bulk of the data.

Rule of thumb 8.2. Mentally add 2 lines parallel to the identity line and
2 lines parallel to the r = 0 line that cover most of the cases. Then a case
is an outlier if it is well beyond these 2 lines. This rule often fails for large
outliers since often the identity line goes through or near a large outlier so
its residual is near zero. Often such outliers are still far from the bulk of
the data, and there will be a gap in the response plot (along the identity
line) separating the bulk of the data from the outliers. Such gaps appear in
Figures 3.7, 3.10b) (in an FF plot), 3.11, and 7.3 where the gap would be
easier to see if the plot was square. A response that is far from the bulk of
the data in the response plot is a large outlier (large in magnitude).
Rule of thumb 8.3. Often an outlier is very good, but more often an
outlier is due to a measurement error and is very bad.
Denition 8.6. A critical mix is a single combination of levels, out of
2k , that gives good results. Hence a critical mix produces good outliers (or a
single outlier if m = 1).

Be able to pick out active and inert factors and good (or the best) combi-
nations of factors (cells or runs) from the table of contrasts = table of runs.
Often the table will only contain the contrasts for the main eects. If high
values of the response are desirable, look for high values of y for m > 1. If
m = 1, then y = y. The following two examples help illustrate the process.

O H C y
5.9
+ 4.0
+ 3.9
+ + 1.2
+ 5.3
+ + 4.8
+ + 6.3
+ + + 0.8
250 8 Orthogonal Designs

Example 8.1. Box et al. (2005, pp. 209210) describes a 23 experiment

with the goal of reducing the wear rate of deep groove bearings. Here m = 1
so n = 8 runs were used. The 23 design employed two levels of osculation
(O), two levels of heat treatment (H), and two dierent cage designs (C).
The response Y is the bearing failure rate and low values of the observed
response y are better than high values.
a) Which two combinations of levels are the best?
b) If two factors are active, which factor is inert?

Solution: a) The two lowest values of y are 0.8 and 1.2 which correspond
to + + + and + + . (Note that if the 1.2 was 4.2, then + + + corresponding
to 0.8 would be a critical mix.)
b) C would be inert since O and H should be at their high + levels.

run R T C D y
1 14
2 + 16
3 + 8
4 + + 22
5 + 19
6 + + 37
7 + + 20
8 + + + 38
9 + 1
10 + + 8
11 + + 4
12 + + + 10
13 + + 12
14 + + + 30
15 + + + 13
16 + + + + 30

Example 8.2. Ledolter and Swersey (2007, p. 80) describes a 24 experi-

ment for a company that manufactures clay plots to hold plants. For one of
the companys newest products, there had been an unacceptably high num-
ber of cracked pots. The production engineers believed that the following
factors are important: R = rate of cooling (slow or fast), T = kiln temper-
ature (2000o F or 2060o F), C = coecient of expansion of the clay (low or
high), and D = type of conveyor belt (metal or rubberized) used to allow
employees to handle the pots. The response y is the percentage of cracked
pots per run (so small y is good).

a) For xed levels of R, T , and C, is the D+ level or D level of D better

(compare run 1 with run 9, 2 with 10, . . . , 8 with 16).
8.1 Factorial Designs 251

b) Fix D at the better level. Is the C or C+ level better?

c) Fix C and D at the levels found in a) and b). Is the R or R+ level
better?
d) Which factor seems to be inert?
Solution: a) D+ since for xed levels of R, T, and C, the number of cracks
is lower if D = + than if D = .
b) C, c) R, d) T .

A 2k design can be t with least squares. In the table of contrasts let a

+ = 1 and a = 1. The design matrix X needs a row for each response:
we cant use the mean response for each xed combination of levels. Let x0
correspond to I, the column of 1s. Let xi correspond to the ith main eect
for i = 1, . . . , k. Let xij correspond to 2 factor interactions, and let xi1 ,...,iG
correspond to G way interactions for G = 2, . . . , k. Let the design matrix X
have columns corresponding to the x. Then X will have n = m2k rows. Let
y be the vector of responses.
The table below relates the quantities in the 23 table of contrasts with the
quantities used in least squares when the design matrix

X = [x0 , x1 , x2 , x3 , x12 , x13 , x23 , x123 ].

Software often does not need the column of ones x0 .

x0 x1 x2 x3 x12 x13 x23 x123 y

I A B C AB AC BC ABC y
The table below relates quantities in the 24 table of contrasts with the
quantities used in least squares. Again the omitted x0 corresponds to I, the
column of ones, while y is the vector of responses.
x1 x2 x3 x4 x12 x13 x14 x23 x24 x34 x123 x124 x134 x234 x1234
A B C D AB AC AD BC BD CD ABC ABD ACD BCD ABCD
Denition 8.7. The least squares model for a 2k design contains a
least squares population coecient for each x in the model. The model
can be written as Y = xT + e with least squares tted values Y = xT .
In matrix form the model is Y = X + e and the vector of tted values is
Y = X . The biggest possible model contains all of the terms. The second
order model contains 0 , all main eects, and all second order interactions,
and is recommended as the initial full model for k 3. The main eects
model removes all interactions. If a model contains an interaction, then the
model should also contain all of the corresponding main eects. Hence if a
model contains x123 , the model should contain x1 , x2 , and x3 .

Denition 8.8. The coecient 0 corresponding to I is equal to the pop-

ulation I eect of x0 , and the (sample) I eect = 0 . For an x other than
x0 , the population eect for x is 2, the change in Y as x changes two
units from 1 to 1, and the (sample) eect is 2. The (sample) coecient
estimates the population coecient .
252 8 Orthogonal Designs

Suppose the model using all of the columns of X is used. If some columns
are removed (e.g. those corresponding to the insignicant eects), then for
2k designs the following quantities remain unchanged for the terms that were
not deleted: the eects, the coecients, and SS(eect) = MS(eect). The
MSE, SE(eect), F and t statistics, pvalues, tted values, and residuals do
change.
The regression equation corresponding to the signicant eects (e.g. found
with a QQ plot of Denition 8.9) can be used to form a reduced model. For
example, suppose the full (least squares) tted model is Yi = 0 + 1 xi1 +
2 xi2 + 3 xi3 + 12 xi12 + 13 xi13 + 23 xi23 + 123 xi123 . Suppose the A, B, and
AB eects are signicant. Then the reduced (least squares) tted model is
Yi = 0 + 1 xi1 + 2 xi2 + 12 xi12 where the coecients (s) for the reduced
model can be taken from the full model since the 2k design is orthogonal.
The coecient 0 corresponding to I is equal to the I eect, but the
coecient of a factor x corresponding to an eect is = 0.5 eect. Consider
signicant eects and assume interactions can be ignored.
i) If a large response Y is desired and > 0, use x = 1. If < 0, use
x = 1.
ii) If a small response Y is desired and > 0, use x = 1. If < 0, use
x = 1.
Rule of thumb 8.4. To predict Y with Y , the number of coecients =
the number of s in the model should be n/2, where the sample size n =
number of runs. Otherwise the model is overtting.
From the regression equation Y = xT , be able to predict Y given x. Be
able to tell whether x = 1 or x = 1 should be used. Given the x values
of the main eects, get the x values of the interactions by multiplying the
columns corresponding to the main eects.
Least squares output in symbols is shown below. Often Estimate is re-
placed by Coef or Coecient. Often Intercept is replaced by Con-
stant. The t statistic and pvalue are for whether the term or eect is sig-
nicant. So t12 and p12 are for testing whether the x12 term or AB eect is
signicant.

Coef or Est. Std.Err t pvalue

Intercept or constant 0 SE(coef) t0 p0
x1 1 SE(coef) t1 p1
x2 2 SE(coef) t2 p2
x3 3 SE(coef) t3 p3
x12 12 SE(coef) t12 p12
x13 13 SE(coef) t13 p13
x23 23 SE(coef) t23 p23
x123 123 SE(coef) t123 p123
8.1 Factorial Designs 253

The least squares coecient = 0.5 (eect). The sum of squares for an x
correspondingto an eect is equal to SS(eect).
 SE(coef) = SE() = 0.5
SE(eect) = M SE/n. Also SE(0 ) = M SE/n.
Example 8.3. a) The biggest possible model for the 23 design is Y =
0 + 1 x1 + 2 x2 + 3 x3 + 12 x12 + 13 x13 + 23 x23 + 123 x123 + e with least
squares tted or predicted values given by Yi = 0 + 1 xi1 + 2 xi2 + 3 xi3 +
12 xi12 + 13 xi13 + 23 xi23 + 123 xi123 .
The second order model is Y = 0 +1 x1 +2 x2 +3 x3 +12 x12 +13 x13 +
23 x23 + e. The main eects model is Y = 0 + 1 x1 + 2 x2 + 3 x3 + e.
b) A typical least squares output for the 23 design with m = 2 is shown
below. Often Estimate is replaced by Coef.
Residual Standard Error=2.8284 = sqrt(MSE)
R-Square=0.9763 F-statistic (df=7, 8)=47.054 pvalue=0

Estimate Std.Err t-value Pr(>|t|)

Intercept 64.25 0.7071 90.8632 0.0000
x1 11.50 0.7071 16.2635 0.0000
x2 -2.50 0.7071 -3.5355 0.0077
x3 0.75 0.7071 1.0607 0.3198
x12 0.75 0.7071 1.0607 0.3198
x13 5.00 0.7071 7.0711 0.0001
x23 0.00 0.7071 0.0000 1.0000
x123 0.25 0.7071 0.3536 0.7328
c) i) The least squares coecient or estimate = eect/2. So in the above
table, the A eect = 2(11.5) = 23. If x corresponds to the least squares
coecient, then the coecient = (xT y)/(xT x).
ii) The sum of squares = mean square corresponding to an x is equal to the
sum of squares = mean square of the corresponding eect. If x corresponds
to the least squares coecient, then the SS = MS = (xT y) 2 T
 /(x x).
iii) Suppose m 2. Then SE(coef) = SE(eect)/2 = 0.5 M SE/(m2k2 ).
Hence in the above table, SE(eect) = 2(0.7071) = 1.412.
iv) The t statistic t0 = coef/SE(coef), and t20 = F0 where t0 tdfe and
F0 F1,dfe where dfe = (m 1)2k is the MSE df. Hence the pvalues for least
squares and the 2k software are the same. For example, the pvalue for testing
the signicance of x1 = pvalue for testing signicance of the A eect = 0.000
in the above table. Also tA = 16.2635 and t2A = FA = 264.501.
v) The MSE, tted values, and residuals are the same for the least squares
output and the 2k software.
Suppose the two levels of the quantitative variable are a < b and x is
2x (a + b)
the actual value used. Then code x as c cx = . Note that
ba
the code gives c = 1 for x = a and c = 1 for x = b. Thus if the 2 levels
are a = 100 and b = 200 but x = 187 is observed, then code x as c =
[2(187) (100 + 200)]/[200 100] = 0.74.
254 8 Orthogonal Designs

There are several advantages to least squares over 2k software. The dis-
advantage of the following four points is that the design will no longer be
orthogonal: the estimated coecients and hence the estimated eects will
depend on the terms in the model. i) If there are several missing values or
outliers, delete the corresponding rows from the design matrix X and the
vector of responses y as long as the number of rows of the design matrix
the number of columns. ii) If the exact quantitative levels are not observed,
replace them by the observed levels cx in the design matrix. iii) If the wrong
levels are used in a run, replace the corresponding row in the design ma-
trix by a row corresponding to the levels actually used. iv) The number of
replications per run i can be mi , that is, we do not need mi m.

Denition 8.9. A normal QQ plot is a plot of the eects versus standard

normal percentiles. There are L = 2k 1 eects for a 2k design.

Rule of thumb 8.5. The nonsignicant eects tend to follow a line

closely in the middle of the QQ plot while the signicant eects do not follow
the line closely. Signicant eects will be the most negative or the most
positive eects.
Know how to nd the eect, the standard error of the eect, the sum
of squares for an eect, and a condence interval for the eect from a table
of contrasts using the following rules.
Let c be a column from the table of contrasts where + = 1 and = 1.
Let y be the column of cell means. Then the eect corresponding to c is
cT y
eect = . (8.1)
2k1

If the number of replications m 2, then the standard error for the eect is

MSE
SE(eect) = . (8.2)
m2k2
Sometimes M SE is replaced by 2 .

M SE
SE(mean) = (8.3)
m2k
where m2k = n, m 2, and sometimes M SE is replaced by 2 .
The sum of squares for an eect is also the mean square for the eect since
df = 1.
M S(eect) = SS(eect) = m2k2 (eect)2 (8.4)
for m 1.
A 95% condence interval (CI) for an eect is

eect tdfe ,0.975 SE(eect) (8.5)

8.1 Factorial Designs 255

where dfe is the MSE degrees of freedom. Use tdfe ,0.975 z0.975 = 1.96 if
dfe > 30. The eect is signicant if the CI does not contain 0, while the eect
is not signicant if the CI contains 0.

Rule of thumb 8.6. Suppose there is no replication so m = 1. Find J

interaction mean squares that are small compared to the bulk of the mean
squares. Add them up to make M SE with dfe = J. So

sum of small MS s

M SE = .
J
This method uses data snooping and M SE tends to underestimate 2 . So
the F test statistics are too large and the pvalues too small. Use this method
for exploratory data analysis, not for inference based on the F distribution.
Rule of thumb 8.7. MS(eect) = SS(eect) 2 21 M SE21 if the
eect is not signicant. M SE 2 2dfe /dfe if the model holds. A rule of
thumb is that an eect is signicant if MS > 5M SE. The rule comes from
the fact that 21,0.975 5.
Below is the ANOVA table for a 23 design. Suppose m = 1. For A, use
H0 : 100 = 200 . For B, use H0 : 010 = 020 . For C, use H0 : 001 = 002 .
For interaction, use H0 : no interaction. If m > 1, the subscripts need an
additional 0, e.g. H0 : 1000 = 2000 .

Source df SS MS F p-value
A 1 SSA MSA FA pA
B 1 SSB MSB FB pB
C 1 SSC MSC FC pC
AB 1 SSAB MSAB FAB pAB
AC 1 SSAC MSAC FAC pAC
BC 1 SSBC MSBC FBC pBC
ABC 1 SSABC MSA FABC pABC
Error (m 1)2k SSE MSE

Following Rule of thumb 8.6, if m = 1, pool J interaction mean squares

that are small compared to the bulk of the data into an MSE with dfe = J.
Such tests are for exploratory purposes only: the MSE underestimates 2 ,
so the F test statistics are too large and the pvalues = P (F1,J > F0 ) are
too small. (Actually the pvalue = pval, an estimated pvalue.) For example
F0 = FA = M SA/M SE. As a convention for using an F table, use the
denominator df closest to dfe = J, but if dfe = J > 30 use denominator df
= .
On the following page is the ANOVA table for a 2k design. For A, use
H0 : 100 = 200 . The other main eects have similar null hypotheses.
For interaction, use H0 : no interaction. If m = 1, use a procedure similar to
Rule of Thumb 8.6 for exploratory purposes.
256 8 Orthogonal Designs

eect
One can use t statistics for eects with t0 = tdfe where dfe
SE(eect)
is the MSE df. Then t20 = M S(ef f ect)/M SE = F0 F1,dfe .

Source df SS MS F p-value
k# main
$ eects 1 e.g. SSA = MSA FA p A
k
2 2 factor interactions 1 e.g. SSAB = MSAB FAB pAB
#k $
3 3 factor interactions 1 e.g. SSABC = MSABC FABC pABC
.. .. .. .. ..
.
# k $ . . . .
k1 k 1 factor interactions
the k factor interaction 1 SSA L = MSA L FAL pAL
Error (m 1)2k SSE MSE

I A B C AB AC BC ABC y
+ + + + 6.333
+ + + + 4.667
+ + + + 9.0
+ + + + 6.667
+ + + + 4.333
+ + + + 2.333
+ + + + 7.333
+ + + + + + + + 4.667
divisor 8 4 4 4 4 4 4 4

Example 8.4. Box et al. (2005, p. 189) describes a 23 experiment designed

to investigate the eects of planting depth (0.5 or 1.4 in.), watering (once or
twice daily), and type of lima bean (baby or large) on yield. The table of
contrasts is shown above. The number of replications m = 3.
a) Find the A eect.
b) Find the AB eect.
c) Find SSA = MSA.
d) Find SSAB = MSAB.
e) If MSE = 0.54, nd SE(eect).
Solution: a) The A eect =
6.333 + 4.667 9 + 6.667 4.333 + 2.333 7.333 + 4.667
= 8.665/4
4
= 2.16625. Note that the appropriate + and signs are obtained from the
A column.
b) The AB eect =
6.333 4.667 9 + 6.667 + 4.333 2.333 7.333 + 4.667
= 1.333/4
4
= 0.33325.
8.1 Factorial Designs 257

Normal QQ plot

2
1
effects
0
1
2

1.0 0.5 0.0 0.5 1.0

nquantiles
Fig. 8.1 QQ plot for Example 8.4

c) SSA = m2k2 (eect)2 = 3(2)(2.16625)2 = 28.1558.

d) SSAB = 6(eect)2 = 6(0.33325)2 = 0.6663.
e) 

MSE 0.54
SE(eect) = = = 0.09 = 0.3.
m2k2 3(2)

The lregpack functions twocub and twofourth can be used to nd the ef-
fects, SE(eect), and QQ plots for 23 and 24 designs. If m = 1, the twofourth
function also makes the response and residual plots based on the second order
model for 24 designs.
For the data in Example 8.4, the output below and on the following page
shows that the A and C eects have values 2.166 and 2.000 while the B
eect is 2.500. These are the three signicant eects shown in the QQ plot
in Figure 8.1. The two commands below produced the output.

z<-c(6.333,4.667,9,6.667,4.333,2.333,7.333,4.667)
twocub(z,m=3,MSE=0.54)

$Aeff
[1] -2.16625
$Beff
[1] 2.50025
258 8 Orthogonal Designs

$Ceff
[1] -2.00025
$ABeff
[1] -0.33325
$ACeff
[1] -0.16675
$BCeff
[1] 0.16675
$ABCeff
[1] 0.00025
$MSA
[1] 28.15583
$MSB
[1] 37.5075
$MSC
[1] 24.006
$MSAB
[1] 0.6663334
$MSAC
[1] 0.1668334
$MSABC
[1] 3.75e-07
$MSE
[1] 0.54
$SEeff
[1] 0.3

8.2 Fractional Factorial Designs

Factorial designs are expensive since n = m2k when there are k factors and m
replications. A fractional factorial design uses n = m2kf where f is dened
below, and so costs much less. Such designs can be useful when the higher
order interactions are not signicant.

kf
Denition 8.10. A 2R fractional factorial design has k factors and
kf
takes m2 runs where the number of replications m is usually 1. The design
is an orthogonal design and each factor has two levels low = 1 and high =
1. R is the resolution of the design.

Denition 8.11. A main eect or q factor interaction is confounded or

aliased with another eect if it is not possible to distinguish between the
two eects.
8.2 Fractional Factorial Designs 259

kf
Remark 8.2. A 2R design has no q factor interaction (or main eect for
q = 1) confounded with any other eect consisting of less than R q factors.
kf
So a 2III design has R = 3 and main eects are confounded with 2 factor
kf
interactions. In a 2IV design, R = 4 and main eects are not confounded
with 2 factor interactions but 2 factor interactions are confounded with other
2 factor interactions. In a 2Vkf design, R = 5 and main eects and 2 factor
interactions are only confounded with 4 and 3 way or higher interactions
respectively. The R = 4 and R = 5 designs are good because the 3 way and
higher interactions are rarely signicant, but these designs are more expensive
than the R = 3 designs.
In a 2Rkf
design, each eect is confounded or aliased with 2f 1 other
eects. Thus the M th main eect is really an estimate of the M th main eect
plus 2f 1 other eects. If R 3 and none of the two factor interactions are
signicant, then the M th main eect is typically a useful estimator of the
population M th main eect.

Rule of thumb 8.8. Main eects tend to be larger than q factor inter-
action eects, and the lower order interaction eects tend to be larger than
the higher order interaction eects. So two way interaction eects tend to be
larger than three way interaction eects.
Rule of thumb 8.9. Signicant interactions tend to have signicant
component main eects. Hence if A, B, C, and D are factors, B and D are
inert and A and C are active, then the AC eect is the two factor interaction
most likely to be active. If only A was active, then the two factor interactions
containing A (AB, AC, and AD) are the ones most likely to be active.
Suppose each run costs $1000 and m = 1. The 2k factorial designs need 2k
runs while fractional factorial designs need 2kf runs. These designs use the
fact that three way and higher interactions tend to be inert for experiments.

Remark 8.3. Let ko = k f . Some good fractional factorial designs for

ko = 3 are shown below. The designs shown use the same table of contrasts
as the 23 design and can be t with 23 software.

23 A B C AB AC BC ABC
241
IV A B C AB+ AC+ BC+ D
252
IIIABC D E BC+ BE+
263
IIIABC D E F AF+
274
IIIABC D E F G

Consider the 241

IV design. It has 4 factors A, B, C, and D. The D main
eect is confounded with the ABC three way interaction, which is likely to be
inert. The D eect is the D eect plus the ABC eect. But if the ABC eect
is not signicant, then the D eect is a good estimator of the population
260 8 Orthogonal Designs

D eect. Confounding = aliasing is the price to pay for using fractional

factorial designs instead of the more expensive factorial designs. The two
factor interactions are followed by a +, e.g. AB+, since these interactions are
confounded with other two factor interactions.
If m = 1, the 241 4
IV design uses 8 runs while a 2 factorial design uses 16
41
runs. The runs for the 2IV are dened by the 4 main eects: use the rst
3 columns and the last column of the table of contrasts for the 23 design to
dene the runs. Randomly assign the units (often time slots) to the runs.

Remark 8.4. Some good fractional factorial designs for ko = k f = 4

are shown below. The designs shown use the same table of contrasts as the
24 design and can be t with 24 software. Here the designs are 1) 24 , and the
fractional factorial designs 2) 251 62 73 84 95
V , 3) 2IV , 4) 2IV , 5) 2IV , 6) 2III , and 7)
1511
2III .
design
1)A B C D AB AC AD BC BD CD ABC ABD ACD BCD ABCD
2)A B C D AB AC AD BC BD CD DE CE BE AE E
3)A B C D AB+ AC+ AD+ BC+ BD+ CD+ E 3int 3int F AF+
4)A B C D AB+ AC+ AD+ BC+ BD+ CD+ E 3int F G AG+
5)A B C D AB+ AC+ AD+ BC+ BD+ CD+ E F G H AH+
6)A B C D AB+ AC+ AD+ BC+ BD+ CD+ E F G H J
7)A B C D E F G H J K L M N O P
kf kf
Remark 8.5. Let ko = k f for a 2R design. The QQ plot for 2R
designs is used in a manner similar to that of 2k designs where k = ko . The
formulas for eects and mean squares are like the formulas for a 2ko design.
Let c be a column from the table of contrasts where + = 1 and = 1.
Let y be the column of cell means. Then M SE = 2 needs to be given or
estimated by setting high order interactions to 0 for m = 1. Typically m = 1
for fractional factorial designs. The following formulas ignore the I eect.
cT y
a) The eect corresponding to c is eect = ko 1 .
2 
MSE
b) The standard error for the eect is SE(eect) = .
m2ko 2

M SE
c) SE(mean) = where m2ko = n.
m2ko
d) The sum of squares and mean square for an eect are
M S(eect) = SS(eect) = m2ko 2 (eect)2 .

Consider the designs given in Remarks 8.3 and 8.4. Least squares estimates
kf
for the 2R designs with ko = 3 use the design matrix corresponding to a 23
design while the designs with ko = 4 use the design matrix corresponding to
the 24 design given in Section 8.1.
Randomly assign units to runs. Do runs in random order if possible. In in-
dustry, units are often time slots (periods of time), so randomization consists
8.2 Fractional Factorial Designs 261

of randomly assigning time slots to units, which is equivalent to doing the

kf
runs in random order. For the above 2R designs, x the main eects using
the corresponding columns in the two tables of contrasts given in Section 8.1
to determine the levels needed in the m2kf runs.
The fractional factorial designs can be t with least squares, and the model
can be written as Y = xT + e with least squares tted values Y = xT .
In matrix form the model is Y = X + e and the vector of tted values is
Y = X .
kf
The biggest possible model for a 2R design where k f = 3 is
Yi = 0 + 1 xi1 + 2 xi2 + 3 xi3 + 12 xi12 + 13 xi13 + 23 xi23 + 123 xi123 + ei
with least squares tted or predicted values given by Yi = 0 + 1 xi1 + 2 xi2 +
3 xi3 + 12 xi12 + 13 xi13 + 23 xi23 + 123 xi123 .
The regression equation corresponding to the signicant eects (e.g. found
with a QQ plot) can be used to form a reduced model. For example, suppose
the full (least squares) tted model is Yi = 0 + 1 xi1 + 2 xi2 + 3 xi3 +
12 xi12 + 13 xi13 + 23 xi23 + 123 xi123 . Suppose the A, B, and AB eects
are signicant. Then the reduced (least squares) tted model is Yi = 0 +
1 xi1 + 2 xi2 + 12 xi12 where the coecients (s) for the reduced model can
be taken from the full model since fractional factorial designs are orthogonal.

For the fractional factorial designs, the coecient 0 corresponding to I

is equal to the I eect, but the coecient of a factor x corresponding to an
eect is = 0.5 eect. Consider signicant eects and assume interactions
can be ignored.
i) If a large response Y is desired and > 0, use x = 1. If < 0, use
x = 1.
ii) If a small response Y is desired and > 0, use x = 1. If < 0, use
x = 1.
From the regression equation Y = xT , be able to predict Y given x. Be
able to tell whether x = 1 or x = 1 should be used. Given the x values of the
main eects, get the x values of the interactions by multiplying the columns
corresponding to the main eects in the interaction. Least squares output
is similar to that in Section 8.1. The least squares coecient = 0.5 (eect).
The sum of squares for an x corresponding
 to an eect is equal to SS(eect).
SE(coef) = SE() = 0.5 SE(eect) = M SE/n. Also SE(0 ) = M SE/n.

Assume none of the interactions are signicant. Then the 274 III fractional
factorial design allows estimation of 7 main eects in 23 = 8 runs. The 21511
III
fractional factorial design allows estimation of 15 main eects in 24 = 16 runs.
The 23126
III fractional factorial design allows estimation of 31 main eects in
25 = 32 runs.
Fractional factorial designs with k f = ko can be t with software meant
for 2ko designs. Hence the lregpack functions twocub and twofourth can
262 8 Orthogonal Designs

be used for the ko = 3 and ko = 4 designs that use the standard table
of contrasts. The response and residual plots given by twofourth are not
appropriate, but the QQ plot and the remaining output are relevant. Some
of the interactions will correspond to main eects for the fractional factorial
design.
For example, if the Example 8.4 data was from a 241 IV design, then the
A, B, and C eects would be the same, but the D eect is the eect labelled
ABC. So the D eect 0.

Normal QQ plot
40
30
effects
20
10
0

1.0 0.5 0.0 0.5 1.0

nquantiles
Fig. 8.2 QQ plot for Example 8.5

Aeff Beff Ceff ABeff ACeff BCeff ABCeff

20.625 38.375 -0.275 28.875 -0.275 -0.625 -2.425

Example 8.5. Montgomery (1984, pp. 344346) gives data from a 274 III
design with the QQ plot shown in Figure 8.2. The goal was to study eye focus
time with factors A = sharpness of vision, B = distance of target from eye,
C = target shape, D = illumination level, E = target size, F = target density,
and G = subject. The lregpack function twocub gave the eects above.
a) What is the D eect?
b) What eects are signicant?

Solution: By the last line in the table given in Remark 8.3, note that for
this design, A, B, C, AB, AC, BC, ABC correspond to A, B, C, D, E, F, G. So
the AB eect from the output is the D eect.
8.3 Plackett Burman Designs 263

a) 28.875, since the D eect is the AB eect.

b) A, B, and D since these are the eects that do not follow the line in
the QQ plot shown in Figure 8.2.

I A B C AB AC BC ABC y
+ + + + 86.8
+ + + + 85.9
+ + + + 79.4
+ + + + 60.0
+ + + + 94.6
+ + + + 85.4
+ + + + 84.5
+ + + + + + + + 80.3

Example 8.6. The above table of 23 contrasts is for 252

III data.
a) Estimate the B eect.
b) Estimate the D eect.
Solution: a)
86.8 85.9 + 79.4 + 60 94.6 85.4 + 84.5 + 80.3
4
= 48.5/4 = 12.125.
b) Use Remark 8.3 to see that the D eect corresponds to the AB column.
So the D eect =
86.8 85.9 79.4 + 60 + 94.6 85.4 84.5 + 80.3
4
= 13.5/4 = 3.375.

8.3 Plackett Burman Designs

Denition 8.12. The Plackett Burman PB(n) designs have k factors

where 2 k n 1. The factors have 2 levels and orthogonal contrasts like
kf
the 2k and 2R designs. The PB(n) designs are resolution 3 designs, but the
confounding of main eects with 2 factor interactions is complex. The PB(n)
designs use n runs where n is a multiple of 4. The values n = 12, 20, 24, 28,
and 36 are especially common.
Fractional factorial designs need at least 2ko runs. Hence if there are 17
main eects, 32 runs are needed for a 21712
III design while a PB(20) design only
needs 20 runs. The price to pay is that the confounding pattern of the main
eects with the two way interactions is complex. Thus the PB(n) designs
are usually used with main eects, and it is assumed that all interactions
are insignicant. So the Plackett Burman designs are main eects designs
264 8 Orthogonal Designs

used to screen k main eects when the number of runs n is small. Often
k = n 4, n 3, n 2, or n 1 is used. We will assume that the number of
replications m = 1.
A contrast matrix for the PB(12) design is shown below. Again the column
of plusses corresponding to I is omitted. If k = 8 then eects A to H are
used but eects J, K, and L are empty. As a convention the mean square
and sum of squares for factor E will be denoted as MSe and SSe while MSE
= 2 .
run A B C D E F G H J K L
1 + - + - - - + + + - +
2 + + - + - - - + + + -
3 - + + - + - - - + + +
4 + - + + - + - - - + +
5 + + - + + - + - - - +
6 + + + - + + - + - - -
7 - + + + - + + - + - -
8 - - + + + - + + - + -
9 - - - + + + - + + - +
10 + - - - + + + - + + -
11 - + - - - + + + - + +
12 - - - - - - - - - - -
The PB(n) designs are k factor 2 level orthogonal designs. So nding
quantities such as eects, MS, SS, least squares estimates, et cetera for PB(n)
kf
designs is similar to nding the corresponding quantities for the 2k and 2R
designs. Randomize units (often time slots) to runs and least squares can be
used.
Remark 8.6. For the PB(n) design, let c be a column from the table of
contrasts where + = 1 and = 1. Let y be the column of responses since
m = 1. If k < n 1, pool the last J = n 1 k empty eects into the
MSE with df = J as the full model. This procedure is done before looking
at the data, so is not data snooping. The MSE can also be given or found
by pooling insignicant MSs into the MSE, but the latter method uses data
snooping. This pooling needs to be done if k = n 1 since then there is no
df for MSE. The following formulas ignore the I eect.
cT y 2cT y
a) The eect corresponding to c is eect = = .
n/2 n
 
MSE 4MSE
b) The standard error for the eect is SE(eect) = = .
n/4 n

M SE
c) SE(mean) = .
n
d) The sum of squares and mean sum of squares for an eect is
n
M S(eect) = SS(eect) = (eect)2 .
4
8.3 Plackett Burman Designs 265

Normal QQ plot for PB12 Design

15
10
effects
5
0
5

1.5 0.5 0.0 0.5 1.0 1.5

nquantiles
Fig. 8.3 QQ Plot for Example 8.7

For the PB(n) design, the least squares coecient = 0.5 (eect). The sum
of squares for an x corresponding
 to an eect is equal to SS(eect). SE(coef)
= SE() = 0.5 SE(eect) = M SE/n. Also SE(0 ) = M SE/n.

Example 8.7. Shown below is least squares output using PB(12) data
from Ledolter and Swersey (2007, pp. 244256). There were k = 10 factors so
the MSE has 1 df and there are too many terms in the model. In this case the
QQ plot shown in Figure 8.3 is more reliable for nding signicant eects.
a) Which eects, if any, appear to be signicant from the QQ plot?
b) Let the reduced model Y = 0 + r1 xr1 + + rj xrj where j is the
number of signicant terms found in a). Write down the reduced model.
c) Want large Y . Using the model in b), choose the x values that will give
large Y , and predict Y .
Estimate Std.Err t-value Pr(>|t|)
Intercept 6.7042 2.2042 3.0416 0.2022
c1 8.5792 2.2042 3.8922 0.1601
c2 -1.7958 2.2042 -0.8147 0.5648
c3 2.3125 2.2042 1.0491 0.4847
c4 4.1208 2.2042 1.8696 0.3127
c5 3.1542 2.2042 1.4310 0.3883
c6 -3.3958 2.2042 -1.5406 0.3665
c7 0.9542 2.2042 0.4329 0.7399
c8 -1.1208 2.2042 -0.5085 0.7005
c9 1.3125 2.2042 0.5955 0.6581
c10 1.7875 2.2042 0.8110 0.5662
266 8 Orthogonal Designs

Solution: a) The most signicant eects are either in the top right or
bottom left corner. Although the points do not all scatter closely about the
line, the point in the bottom left is not signicant. So none of the eects
corresponding to the bottom left of the plot are signicant. A is the signicant
eect with value 2(8.5792) = 17.1584. See the top right point of Figure 8.3.
b) Y = 6.7042 + 8.5792x1 .
c) Y = 6.7042 + 8.5792(1) = 15.2834.

The lregpack function pb12 can be used to nd eects and MS(eect) for
PB(12) data. Least squares output and a QQ plot are also given.

8.4 Summary

1) In a table of contrasts, the contrast for A starts with a then a +

and the pattern repeats. The contrast for B starts with 2 s and then 2
+s and the pattern repeats. The contrast for C starts with 4 s and then
4+s and the pattern repeats. The contrast for the ith main eects factor
starts with 2i1 s and 2i1 +s and the pattern repeats for i = 1, . . . , k.
2) In a table of contrasts, a column for an interaction containing several
factors is obtained by multiplying the columns for each factor where + = 1
and = 1. So the column for ABC is obtained by multiplying the column
for A, the column for B, and the column for C.
3) Let c be a column from the table of contrasts where + = 1 and = 1.
Let y be the column of cell means. Then the eect corresponding to c is
cT y
eect = k1 .
2
4) If the number of replications m 2, then the standard error for the
eect is 
MSE
SE(eect) = .
m2k2
Sometimes M SE is replaced by 2 .
5) 
M SE
SE(mean) =
m2k
where m2k = n, m 2 and sometimes M SE is replaced by 2 .
6) Since df = 1, the sum of squares and mean square for an eect is

M S(eect) = SS(eect) = m2k2 (eect)2

for m 1.
8.4 Summary 267

7) If a single run out of 2k cells gives good values for the response, then
that run is called a critical mix.
8) A factor is active if the response depends on the two levels of the factor,
and is inert, otherwise.
9) Randomization for a 2k design: randomly assign units to the m2k runs.
The runs are determined by the levels of the k main eects in the table of
contrasts. So a 23 design is determined by the levels of A, B, and C. Similarly,
a 24 design is determined by the levels of A, B, C, and D. Perform the m2k
runs in random order if possible.
10) A table of contrasts for a 23 design is shown below. The rst column
is for the mean and is not a contrast. The last column corresponds to the
cell means. Note that y 1110 = y111 if m = 1. So y might be replaced by y if
m = 1.

I A B C AB AC BC ABC y
+ + + + y 1110
+ + + + y 2110
+ + + + y 1210
+ + + + y 2210
+ + + + y 1120
+ + + + y 2120
+ + + + y 1220
+ + + + + + + + y 2220
divisor 8 4 4 4 4 4 4 4

11) Be able to pick out active and inert factors and good (or the best)
combinations of factors (cells or runs) from the table of contrasts = table of
runs.
12) Plotted points far away from the identity line and r = 0 line are
potential outliers, but often the identity line goes through or near an outlier
that is large in magnitude. Then the case has a small residual. Look for gaps
is the response and residual plots.
13) A 95% condence interval (CI) for an eect is

eect tdfe ,0.975 SE(eect)

where dfe is the MSE degrees of freedom. Use tdfe ,0.975 z0.975 = 1.96 if
dfe > 30. The eect is signicant if the CI does not contain 0, while the eect
is not signicant if the CI contains 0.
14) Suppose there is no replication so m = 1. Find J interaction mean
squares that are small compared to the bulk of the mean squares. Add them
up (pool them) to make M SE with dfe = J. So
268 8 Orthogonal Designs

sum of small MS s

M SE = .
J
This method uses data snooping and M SE tends to underestimate 2 . So
the F test statistics are too large and the pvalues = P (F1,J > F0 ) are too
small. For example F0 = FA = M SA/M SE. As a convention for using an
F table, use the denominator df closest to dfe = J, but if dfe = J > 30 use
denominator df = . Use this method for exploratory data analysis, not for
inference based on the F distribution.
15) MS = SS 2 21 M SE21 if the eect is not signicant. M SE
2 2
dfe /dfe if the model holds. A rule of thumb is that an eect is signicant
if MS > 5M SE. The rule comes from the fact that 21,.975 5.
16) The table of contrasts for a 24 design is below. The column of ones
corresponding to I was omitted.
run A B C D AB AC AD BC BD CD ABC ABD ACD BCD ABCD
1 - - - - + + + + + + - - - - +
2 + - - - - - - + + + + + + - -
3 - + - - - + + - - + + + - + -
4 + + - - + - - - - + - - + + +
5 - - + - + - + - + - + - + + -
6 + - + - - + - - + - - + - + +
7 - + + - - - + + - - - + + - +
8 + + + - + + - + - - + - - - -
9 - - - + + + - + - - - + + + -
10 + - - + - - + + - - + - - + +
11 - + - + - + - - + - + - + - +
12 + + - + + - + - + - - + - - -
13 - - + + + - - - - + + + - - +
14 + - + + - + + - - + - - + - -
15 - + + + - - - + + + - - - + -
16 + + + + + + + + + + + + + + +
17) Below is the ANOVA table for a 23 design. Let m = 1. For A, use
H0 : 100 = 200 . For B, use H0 : 010 = 020 . For C, use H0 : 001 = 002 .
For interaction, use H0 : no interaction.

Source df SS MS F p-value
A 1 SSA MSA FA pA
B 1 SSB MSB FB pB
C 1 SSC MSC FC pC
AB 1 SSAB MSAB FAB pAB
AC 1 SSAC MSAC FAC pAC
BC 1 SSBC MSBC FBC pBC
ABC 1 SSABC MSA FABC pABC
Error (m 1)2k SSE MSE
8.4 Summary 269

18) Below is the ANOVA table for a 2k design. For A, use H0 : 100 =
200 . The other main eects have similar null hypotheses. For interaction,
use H0 : no interaction. If m = 1 use a procedure similar to point 14) for
exploratory purposes.

Source df SS MS F p-value
k# main
$ eects 1 e.g. SSA = MSA FA p A
k
#k2$ 2 factor interactions 1 e.g. SSAB = MSAB FAB pAB
3 3 factor interactions 1 e.g. SSABC = MSABC FABC pABC
.. .. .. .. ..
#. k $ . . . .
k1 k 1 factor interactions
the k factor interaction 1 SSA L = MSA L FAL pAL
Error (m 1)2k SSE MSE

19) Genuine run replicates need to be used. A common error is to take m

measurements per run, and act as if the m measurements are from m runs.
If as a data analyst you encounter this error, average the m measurements
into a single value of the response.
eect
20) One can use t statistics for eects with t0 = tdfe where
SE(eect)
dfe is the MSE df. Then t0 = M S(eect)/MSE = F0 F1,dfe .
2

21) A 2k design can be t with least squares. In the table of contrasts

let a + = 1 and a = 1. Then X needs a row for each response:
we cant use the mean response for each xed combination of levels. Let x0
correspond to I, the column of 1s. Let xi correspond to the ith main eect
for i = 1, . . . , k. Let xij correspond to 2 factor interactions, and let xi1 ,...,iG
correspond to G way interactions for G = 2, . . . , k. Let the design matrix X
have columns corresponding to the x. Let y be the vector of responses.
22) The table below relates the quantities in the 23 table of contrasts with
the quantities used in least squares when the design matrix

X = [x0 , x1 , x2 , x3 , x12 , x13 , x23 , x123 ].

Software often does not need the column of ones x0 .

x0 x1 x2 x3 x12 x13 x23 x123 y

I A B C AB AC BC ABC y
23) The table below relates quantities in the 24 table of contrasts with the
quantities used in least squares. Again x0 corresponds to I, the column of
ones, while y is the vector of responses.
x1 x2 x3 x4 x12 x13 x14 x23 x24 x34 x123 x124 x134 x234 x1234
A B C D AB AC AD BC BD CD ABC ABD ACD BCD ABCD
24) A typical least squares output for the 23 design is shown below. Often
Estimate is replaced by Coef.
270 8 Orthogonal Designs

Estimate Std.Err t-value Pr(>|t|)

Intercept 64.25 0.7071 90.8632 0.0000
x1 11.50 0.7071 16.2635 0.0000
x2 -2.50 0.7071 -3.5355 0.0077
x3 0.75 0.7071 1.0607 0.3198
x12 0.75 0.7071 1.0607 0.3198
x13 5.00 0.7071 7.0711 0.0001
x23 0.00 0.7071 0.0000 1.0000
x123 0.25 0.7071 0.3536 0.7328

25) i) The least squares coecient or estimate = eect/2. So in the

above table, the A eect = 2(11.5) = 23. If x corresponds to the least squares
coecient, then the coecient = (xT y)/(xT x).
ii) The sum of squares = means square corresponding to an x is equal to the
sum of squares = mean square of the corresponding eect. If x corresponds
to the least squares coecient, then the SS = MS = (xT y) 2 T
 /(x x).
iii) Suppose m 2. Then SE(coef) = SE(eect)/2 = 0.5 M SE/(m2k2 ).
Hence in the above table, SE(eect) = 2(0.7071) = 1.412.
iv) The t statistic t0 = coef/SE(coef), and t20 = F0 where t0 tdfe and
F0 F1,dfe where dfe = (m 1)2k is the MSE df. Hence the pvalues for least
squares and the 2k software are the same. For example, the pvalue for testing
the signicance of x1 = pvalue for testing signicance of A eect = 0.000 in
the above table. Also tA = 16.2635 and t2A = FA = 264.501.
v) The MSE, tted values, and residuals are the same for the least squares
output and the 2k software.
26) There are several advantages to least squares over 2k software. i) If
there are several missing values or outliers, delete the corresponding rows
from the design matrix X and the vector of responses y as long as the
number of rows of the design matrix the number of columns. ii) If the exact
quantitative levels are not observed, replace them by the observed levels in
the design matrix. See point 27). iii) If the wrong levels are used in a run,
replace the corresponding row in the design matrix by a row corresponding
to the levels actually used.
27) Suppose the two levels of the quantitative variable are a < b and x is
2x (a + b)
the actual value used. Then code x as c = . Note that the code
ba
gives c = 1 for x = a and c = 1 for x = b.
28) A normal QQ plot is a plot of the eects versus standard normal
percentiles. There are L = 2k 1 eects for a 2k design. A rule of thumb
is that nonsignicant eects tend to follow a line closely in the middle of
the plot while the signicant eects do not follow the line closely. Signicant
eects will be the most negative or the most positive eects.
kf
29) A 2R fractional factorial design has k factors and takes m2kf runs
where the number of replications m is usually 1.
8.4 Summary 271

30) Let ko = k f . Some good fractional factorial designs for ko = 3 are

shown below. The designs shown use the same table of contrasts as the 23
design given in point 10), and can be t with 23 software.

23 A B C AB AC BC ABC
241
IV A B C AB+ AC+ BC+ D
252
IIIABC D E BC+ BE+
263
IIIABC D E F AF+
274
IIIABC D E F G

31) Some good fractional factorial designs for ko = k f = 4 are shown

below. The designs shown use the same table of contrasts as the 24 design
given in point 16), and can be t with 24 software. Here the designs are 1)
24 , and the fractional factorial designs 2) 251 62 73 84
V , 3) 2IV , 4) 2IV , 5) 2IV , 6)
95 1511
2III , and 7) 2III .
design
1)A B C D AB AC AD BC BD CD ABC ABD ACD BCD ABCD
2)A B C D AB AC AD BC BD CD DE CE BE AE E
3)A B C D AB+ AC+ AD+ BC+ BD+ CD+ E 3int 3int F AF+
4)A B C D AB+ AC+ AD+ BC+ BD+ CD+ E 3int F G AG+
5)A B C D AB+ AC+ AD+ BC+ BD+ CD+ E F G H AH+
6)A B C D AB+ AC+ AD+ BC+ BD+ CD+ E F G H J
7)A B C D E F G H J K L M N O P
kf
32) Let ko = k f for a 2R design. Then the formulas for eects and
mean squares are like the formulas for a 2ko design. Let c be a column from
the table of contrasts where + = 1 and = 1. Let y be the column of
cell means. Need M SE = 2 to be given or estimated by setting high order
interactions to 0 for m = 1. Typically m = 1 for fractional factorial designs.
cT y
a) The eect corresponding to c is eect = ko 1 .
2 
MSE
b) The standard error for the eect is SE(eect) = .
m2ko 2

M SE
c) SE(mean) = where m2ko = n.
m2ko
d) The mean square and sum of squares for an eect are
M S(eect) = SS(eect) = m2ko 2 (eect)2 .
kf
33) Least squares estimates for the 2R designs in points 30) and 31) are
obtained by using the design matrix corresponding to the table of contrasts
in point 10) for ko = 3 and point 16) for ko = 4.
kf
34) The QQ plot for 2R designs is used in a manner similar to point 28).

35) Randomly assign units to runs. Do runs in random order if possible.

In industry, units are often time slots (periods of time), so randomization
272 8 Orthogonal Designs

consists of randomly assigning time slots to units, which is equivalent to

kf
doing the runs in random order. For the 2R designs in points 30) and 31),
x the main eects using the corresponding columns of contrasts given in
points 10) and 16) to determine the levels needed in the m2kf runs.
36) Active factors appear to change the mean response as the level of
the factor changes from 1 to 1. Inert factors do not appear to change the
response as the level of the factor changes from 1 to 1. An inert factor
could be needed but the level low or high is not important, or the inert factor
may not be needed and so can be omitted from future studies. Often subject
matter experts can tell whether the inert factor is needed or not.
kf
37) A 2R design has no q factor interaction (or main eect for q = 1)
confounded with any other eect consisting of less than R q factors. So
kf
a 2III design has R = 3 and main eects are confounded with 2 factor
kf
interactions. In a 2IV design, R = 4 and main eects are not confounded
with 2 factor interactions but 2 factor interactions are confounded with other
2 factor interactions. In a 2Vkf design, R = 5 and main eects and 2 factor
interactions are only confounded with 4 and 3 way or higher interactions
respectively.
38) In a 2R kf
design, each eect is confounded or aliased with 2f 1 other
eects. Thus the M th main eect is really an estimate of the M th main eect
plus 2f 1 other eects. If R 3 and none of the two factor interactions are
signicant, then the M th main eect is typically a useful estimator of the
population M th main eect.
39) The R = 4 and R = 5 designs are good because the 3 way and higher
interactions are rarely signicant, but these designs are more expensive than
the R = 3 designs.
40) In this text, most of the DOE models can be t with least squares,
and the model can be written as Y = xT + e with least squares tted values
Y = xT . In matrix form the model is Y = X + e and the vector of tted
values is Y = X .
kf
41) The full model for a 23 or 2R design where k f = 3 is
Yi = 0 + 1 xi1 + 2 xi2 + 3 xi3 + 12 xi12 + 13 xi13 + 23 xi23 + 123 xi123 + ei
with least squares tted or predicted values given by
Yi = 0 + 1 xi1 + 2 xi2 + 3 xi3 + 12 xi12 + 13 xi13 + 23 xi23 + 123 xi123 .
42) An interaction such as xi123 satises xi123 = (xi1 )(xi2 )(xi3 ).
kf
43) For orthogonal designs like 2k , 2R , or PB(n) (described in point 51)),
the x value of an eect takes on values 1 or 1. The columns of the design
matrix X are orthogonal: cTi cj = 0 for i = j where ci is the ith column of X.

44) Suppose the full model using all of the columns of X is used. If some
columns are removed (e.g. those corresponding to the insignicant eects),
then for the orthogonal designs in point 43) the following quantities remain
unchanged for the terms that were not deleted: the eects, the coecients,
8.4 Summary 273

and SS(eect) = MS(eect). The MSE, SE(eect), F and t statistics, pvalues,

tted values, and residuals do change.
45) The regression equation corresponding to the signicant eects (e.g.
found with a QQ plot) can be used to form a reduced model. For example,
suppose the full (least squares) tted model is Yi = 0 + 1 xi1 + 2 xi2 +
3 xi3 + 12 xi12 + 13 xi13 + 23 xi23 + 123 xi123 . Suppose the A, B, and AB
eects are signicant. Then the reduced (least squares) tted model is Yi =
0 + 1 xi1 + 2 xi2 + 12 xi12 where the coecients (s) for the reduced model
are taken from the full model.
46) For the designs in 43), the coecient 0 corresponding to I is equal
to the I eect, but the coecient of a factor x corresponding to an eect
is = 0.5 eect. Consider signicant eects and assume interactions can be
ignored.
i) If a large response Y is desired and > 0, use x = 1. If < 0, use x = 1.
ii) If a small response Y is desired and > 0, use x = 1. If < 0, use
x = 1.
47) Rule of thumb: to predict Y with Y , the number of coecients = the
number of s in the model should be n/2, where the sample size n =
number of runs.
48) From the regression equation Y = xT , be able to predict Y given x.
Be able to tell whether x = 1 or x = 1 should be used. Given the x values
of the main eects, get the x values of the interactions using 42).
49) Least squares output for an example and in symbols are shown below
and on the following page for the designs in 43). Often Estimate is replaced
by Coef or Coecient. Often Intercept is replaced by Constant. The
t statistic and pvalue are for whether the term or eect is signicant. So t12
and p12 are for testing whether the x12 term or AB eect is signicant.

Residual Standard Error=2.8284 = sqrt(MSE)

R-Square=0.9763 F-statistic (df=7, 8)=47.054 pvalue=0

Estimate Std.Err t-value Pr(>|t|)

Intercept 64.25 0.7071 90.8632 0.0000
x1 11.50 0.7071 16.2635 0.0000
x2 -2.50 0.7071 -3.5355 0.0077
x3 0.75 0.7071 1.0607 0.3198
x12 0.75 0.7071 1.0607 0.3198
x13 5.00 0.7071 7.0711 0.0001
x23 0.00 0.7071 0.0000 1.0000
x123 0.25 0.7071 0.3536 0.7328
274 8 Orthogonal Designs

Coef or Est. Std.Err t pvalue

Intercept or constant 0 SE(coef) t0 p0

x1 1 SE(coef) t1 p1
x2 2 SE(coef) t2 p2
x3 3 SE(coef) t3 p3
x12 12 SE(coef) t12 p12
x13 13 SE(coef) t13 p13
x23 23 SE(coef) t23 p23
x123 123 SE(coef) t123 p123

50) The least squares coecient = 0.5 (eect). The sum of squares for an
x corresponding
 to an eect is equal to  SS(eect). SE(coef) = SE() = 0.5
SE(eect) = M SE/n. Also SE(0 ) = M SE/n.
51) The Plackett Burman PB(n) designs have k factors where 2 k
kf
n 1. The factors have 2 levels and orthogonal contrasts like the 2k and 2R
designs. The PB(n) designs are resolution 3 designs, but the confounding of
main eects with 2 factor interactions is complex. The PB(n) designs use n
runs where n is a multiple of 4. The values n = 12, 20, 24, 28, and 36 are
especially common.
52) The PB(n) designs are usually used with main eects so assume that all
interactions are insignicant. So they are main eects designs used to screen k
main eects when the number of runs n is small. Often k = n 4, n 3, n 2,
or n 1 is used. We will assume that the number of replications m = 1.
53) If k = n 1 there is no df for MSE. If k < n 1, pool the last
J = n 1 k empty eects into the MSE with df = J as the full model.
This procedure is done before looking at the data, so is not data snooping.
run A B C D E F G H J K L
1 + - + - - - + + + - +
2 + + - + - - - + + + -
3 - + + - + - - - + + +
4 + - + + - + - - - + +
5 + + - + + - + - - - +
6 + + + - + + - + - - -
7 - + + + - + + - + - -
8 - - + + + - + + - + -
9 - - - + + + - + + - +
10 + - - - + + + - + + -
11 - + - - - + + + - + +
12 - - - - - - - - - - -
54) The contrast matrix for the PB(12) design is shown above. Again the
column of plusses corresponding to I is omitted. If k = 8 then eects A to
8.5 Complements 275

H are used but eects J, K, and L are empty. As a convention the mean
square and sum of squares for factor E will be denoted as MSe and SSe while
MSE = 2 .
55) The PB(n) designs are k factor 2 level orthogonal designs. So nding
eects, MS, SS, least squares estimates, et cetera for PB(n) designs is similar
kf
to nding the corresponding quantities for the 2k and 2R designs.
56) For the PB(n) design, let c be a column from the table of contrasts
where + = 1 and = 1. Let y be the column of responses since m = 1.
For k < n 1, MSE can be found for the full model as in 53). MSE can also
be given or found by pooling insignicant MSs into the MSE, but the latter
method uses data snooping.
cT y 2cT y
a) The eect corresponding to c is eect = = .
n/2 n
 
MSE 4MSE
b) The standard error for the eect is SE(eect) = = .
n/4 n

M SE
c) SE(mean) = .
n
d) The sum of squares and mean square for an eect is
n
M S(eect) = SS(eect) = (eect)2 .
4
57) For the PB(n) design, the least squares coecient = 0.5 (eect). The
sum of squares for an x corresponding  to an eect is equal to SS(eect).
SE(coef) = SE() = 0.5 SE(eect) = M SE/n. Also SE(0 ) = M SE/n.

8.5 Complements

Box et al. (2005) and Ledolter and Swersey (2007) are excellent references
for k factor 2 level orthogonal designs.
Suppose it is desired to increase the response Y and that A, B, C, . . . are
the k factors. The main eects for A, B, . . . measure
Y Y
, ,
A B
et cetera. The interaction eect AB measures
Y
.
AB
Hence
Y Y Y
0, 0, and large
A B AB
276 8 Orthogonal Designs

implies that the design is in the neighborhood of a maximum of a response

that looks like a ridge. p
An estimated contrast is C = i=1 di Y i0 , and


  p
d2i
SE(C) = M SE .
i=1 i
n

If di = 1, p = 2k and ni = m, then SE(C) = M SE 2k /m. For a 2k
design, an eect can be written as a contrast with di = 1/2k1 , p = 2k and
ni = m. Thus

 
 
2k
1 1 MSE

SE(eect) = MSE = .
m 22k2 m2k2
i=1

There is an algebra for computing confounding patterns for fractional

factorial designs. Let M be any single letter eect (A, B, C, et cetera), and
let I be the identity element. Then i) IM = M , ii) M M = I and iii) multi-
plication is commutative: LM = M L.
k1
For a 2R design, set one main eect equal to an interaction, e.g. D =
ABC. The equation D = ABC is called a generator. Note that DD = I =
DABC = ABCD. The equation I = ABCD is the generating relationship.
Then M I = M = ABCDM , so M is confounded or aliased with ABCDM .
So A = AI = AABCD = BCD and A is confounded with BCD. Similarly,
BD = BDI = BDABCD = AC, so BD is confounded with AC.
k2
For a 2R design, 2 main eects L and M are set equal to an interaction.
Thus L = I and M 2 = I, but it is also true that L2 M 2 = I. As an illus-
2

tration, consider the 262 IV design with E = ABC and F = BCD. So E =

2
2 2 2
I = ABCE, F = I = BCDF , and F E = I = ABCEBCDF = ADEF.
Hence the generating relationship I = ABCE = BCDF = ADEF has 3
words, and each eect is confounded with 3 other eects. For example,
AI = AABCE = ABCDF = AADEF or A = BCE = ABCDF = DEF .
kf
For a 2R design, f main eects L1 , . . . , Lf are set equal to interactions.
# $ # $
There are f1 equations of the form L2i = I, f2 equations of the form L2i L2j =
# $ # $
I, f3 equations of the form L2i1 L2i2 L2i3 = I, . . . , ff equations of the form
L21 L22 L2f = I. These equations give a generating relationship with 2f 1
words, so each eect is confounded with 2f 1 other eects.
If the generating relationship is I = W1 = W2 = = W2f 1 , then the
resolution R is equal to the length of the smallest word. So I = ABC and
I = ABCE = ABC = ADEF both have R = 3.
The convention is to ignore 3 way or#higher$ order interactions. So the alias
patterns for the k main eects and the k2 2 way interactions with other main
eects and 2 way interactions is of interest.
8.6 Problems 277

8.6 Problems

Problems with an asterisk * are especially important.

Output for 8.1: Residual Standard Error=2.8284

R-Square=0.9763 F-statistic (df=7, 8)=47.054 pvalue=0
Estimate Std.Err t-value Pr(>|t|)
Intercept 64.25 0.7071 90.8632 0.0000
x1 11.50 0.7071 16.2635 0.0000
x2 -2.50 0.7071 -3.5355 0.0077
x3 0.75 0.7071 1.0607 0.3198
x12 0.75 0.7071 1.0607 0.3198
x13 5.00 0.7071 7.0711 0.0001
x23 0.00 0.7071 0.0000 1.0000
x123 0.25 0.7071 0.3536 0.7328

8.1. From the above least squares output, what is the AB eect?

I A B C AB AC BC ABC Y
+ + + + 3.81
+ + + + 4.28
+ + + + 3.74
+ + + + 4.10
+ + + + 3.75
+ + + + 3.66
+ + + + 3.82
+ + + + + + + + 3.68

8.2. Ledolter and Swersey (2007, pp. 108109) describes a 23 experiment

designed to increase subscriptions of the magazine Ladies Home Journal. The
2005 campaign made 8 brochures containing an order card. Each brochure was
mailed to 15042 households, and the response Y was the percentage of orders.
Factor A was front side of order card with (1) highlighting Double our Best
Oer and (+1) highlighting We never had a bigger sale. Factor B was back
side of order card with (1) emphasizing Two extra years free, while (+1)
featured magazine covers of a previous issue. Factor C was brochure cover
with (1) featuring Kelly Ripa and (+1) Dr. Phil. Assume m = 1.
a) Find the A eect.
b) Find the C eect.
c) Find SSC = MSC.
d) If two of the three factors A, B and C are active, which is inactive?
278 8 Orthogonal Designs

I A B C AB AC BC ABC y
+ + + + 86.8
+ + + + 85.9
+ + + + 79.4
+ + + + 60.0
+ + + + 94.6
+ + + + 85.4
+ + + + 84.5
+ + + + + + + + 80.3

8.3. The above table of 23 contrasts is for 252

III data.
a) Estimate the B eect.
b) Estimate the D eect.

8.4. Suppose that for 23 data with m = 2, the MSE = 407.5625. Find
SE(eect).

I A B C AB AC BC ABC y
+ + + + 63.6
+ + + + 76.8
+ + + + 60.3
+ + + + 80.3
+ + + + 67.2
+ + + + 71.3
+ + + + 68.3
+ + + + + + + + 74.3
divisor 8 4 4 4 4 4 4 4

8.5. Ledolter and Swersey (2007, p. 131) describe a 274

III data set shown
with the table of 23 contrasts above. Estimate the D eect.

I A B C AB AC BC ABC y
+ + + + 32
+ + + + 35
+ + + + 28
+ + + + 31
+ + + + 48
+ + + + 39
+ + + + 28
+ + + + + + + + 29
divisor 8 4 4 4 4 4 4 4
8.6 Problems 279

8.6. Kuehl (1994, pp. 361366) describes a 23 experiment designed to

investigate the eects of furnace temperature (1840 or 1880o F), heating time
(23 or 25 sec) and transfer time (10 or 12 sec) on the quality of a leaf spring
used for trucks. (The response Y was a measure of the quality.) The table of
contrasts is shown above.
a) Find the A eect.
b) Find the B eect.
c) Find the AB eect.
d) If m = 1, nd SSA.
e) If m = 1, nd SSB.
f) If m = 1, nd SSAB.
g) If m = 2 and MSE = 9, nd SE(eect).
(The SE is the same regardless of the eect.)
h) Suppose high Y = y is desirable. If two of the factors A, B, and C are
inert and one is active, then which is active and which are inert. (Hint: look
at the 4 highest values of y. Is there a pattern?)
i) If one of the factors has an interaction with the active factor, what is
the interaction (e.g. AB, AC, or BC)?

8.7. Suppose the B eect = 5, SE(ef f ect) = 2, and dfe = 8.
i) Find a 95% condence interval for the B eect.
ii) Is the B eect signicant? Explain briey.
R (along with 1 SAS and 1 Minitab) Problems

Use the command source(G:/lregpack.txt) to download the func-

tions and the command source(G:/lregdata.txt) to download the data.
See Preface or Section 14.1. Typing the name of the R function, e.g.
aov, will display the code for the function. Use the args command, e.g.
args(aov), to display the needed arguments for the function. For some of
the following problems, the R commands can be copied and pasted from
(http://lagrange.math.siu.edu/Olive/lreghw.txt) into R.

8.8. Copy the Box et al. (2005, p. 199) product development data from
(http://lagrange.math.siu.edu/Olive/lregdata.txt) into R.
Then type the following commands.

out <- aov(conversion~K*Te*P*C,devel)

summary(out)

a) Include the output in Word.

b) What are the ve eects with the biggest mean squares?
Note: an AB interaction is denoted by A:B in R.
280 8 Orthogonal Designs

8.9. Get the SAS program for this problem from (http://lagrange.math.siu.
edu/Olive/lreghw.txt). The data is the pilot plant example from Box et al.
(2005, pp. 177186). The response variable is Y = yield, while the three
predictors (T = temp, C = concentration, K = catalyst) are at two levels.
a) Print out the output but do not turn in the rst page.
b) Do the residual and response plots look ok?

8.10. Get the data for this problem. The data is the pilot plant example
from Box et al. (2005, pp. 177186) examined in Problem 8.9. Minitab needs
the levels for the factors and the interactions.
Highlight the data and use the menu commands Edit>Copy. In Minitab,
use the menu command Edit>PasteCells. After a window appears, click on
ok.
Below C1 type A, below C2 type B, below C3 type C and below
C8 type yield.
a) Use the menu command STAT>ANOVA>Balanced Anova put
yield in the responses box and
A|B|C
in the Model box. Click on Storage. When a window appears, click on
Fits and Residuals. Then click on OK. This window will disappear.
Click on OK.
b) Next highlight the bottom 8 lines and use the menu commands
Edit>Delete Cells. Then the data set does not have replication. Use the
menu command STAT>ANOVA>Balanced Anova put yield in the re-
sponses box and
A B C A*C
in the Model box. Click on Storage. When a window appears, click on
Fits and Residuals. Then click on OK. This window will disappear.
Click on OK.
(The model A|B|C would have resulted in an error message, not enough
data.)
c) Print the output by clicking on the top window and then clicking on
the printer icon.
d) Make a response plot with the menu commands Graph>Plot with
yield in the Y box and FIT2 in the X box. Print by clicking on the printer
icon.
e) Make a residual plot with the menu commands Graph>Plot with
RESI2 in the Y box and FIT2 in the X box. Print by clicking on the printer
icon.
f) Do the plots look ok?

8.11. Get the R code and data for this problem from
(http://lagrange.math.siu.edu/Olive/lreghw.txt). The data is the pilot plant
8.6 Problems 281

example from Box et al. (2005, pp. 177186) examined in Problems 8.9 and
8.10.
a) Copy and paste the code into R. Then copy and paste the output into
Notepad. Print out the page of output.
b) The least squares estimate = coecient for x1 is half the A eect. So
what is the A eect?

8.12. a) Obtain and the R program twocub from

(http://lagrange.math.siu.edu/Olive/lregpack.txt). To get the eects, mean
squares, and SE(eect) for the Box et al. (2005, p. 177) pilot plant data, type
the following commands and include the output in Word.

mns <- c(60,72,54,68,52,83,45,80)

twocub(mns,m=2,MSE=8)

b) Which eects appear to be signicant from the QQ plot? (Match the

eects on the plot with the output on the screen.)

8.13. Box et al. (2005, p. 237) describe a 241

IV fractional factorial design.
Assuming that you downloaded the twocub function in the previous problem,
type the following R commands.
mns <- c(20,14,17,10,19,13,14,10)
twocub(mns,m=1)
a) Include the output in Word, print out the output and label the eects
on the output with the corresponding eects from a 241IV fractional factorial
design.
b) Include the QQ plot in Word. Print out the plot. Which eects (from
the fractional factorial design) seem to be signicant?

8.14. a) Download lregpack into R, and type the following commands.

mns <- c(14,16,8,22,19,37,20,38,1,8,4,10,12,30,13,30)

twofourth(mns)

This is the Ledolter and Swersey (2007, p. 80) cracked pots 24 data and
the response and residual plots are from the model without 3 and 4 factor
interactions.
b) Copy the plots into Word and print the plots. Do the response and
residual plots look ok?
282 8 Orthogonal Designs

8.15. Download lregpack into R. The data is the PB(12) example from
Box et al. (2005, p. 287).
a) Type the following commands. Copy and paste the QQ plot into Word
and print the plot.

resp <- c(56,93,67,60,77,65,95,49,44,63,63,61)

pb12(resp,k=5)

b) Copy and paste the output into Notepad and print the output.
c) As a 25 design, the eects B, D, BD, E, and DE were thought to be real.
The PB(12) design works best when none of the interactions is signicant.
From the QQ plot and the output for the PB(12) design, which factors, if
any, appear to be signicant?
d) The output gives the A, B, C, D, and E eects along with the cor-
responding least squares coecients 1 , . . . , 5 . What is the relationship
between the coecients and the eects?
For parts e) to g), act as if the PB(12) design with 5 factors is
appropriate.
e) The full model has Y = 0 + 1 x1 + 2 x2 + 3 x3 + 4 x4 + 5 x5 . The
reduced model is Y = 0 + j xj where xj is the signicant term found in c).
Give the numerical formula for the reduced model.
f) Compute Y using the full model if xi = 1 for i = 1, . . . , 5. Then compute
Y using the reduced model if xj = 1.
g) If the goal of the experiment is to produce large values of Y , should
xj = 1 or xj = 1 in the reduced model? Explain briey.
Chapter 9
More on Experimental Designs

This chapter considers split plot designs briey and reviews the ten designs
considered in Chapter 5 Section 9.1. The one and two way Anova designs,
completely randomized block design, and split plot designs are the building
blocks for more complicated designs. Some split plot designs can be written as
a linear model, Y = xT +e, but the errors are dependent with a complicated
correlation structure.

9.1 Split Plot Designs

Denition 9.1. Split plot designs have two units. The large units are
called whole plots and contain blocks of small units called subplots. The
whole plots get assigned to factor A while the subplots get assigned to factor
B (randomly if the units are experimental but not randomly if the units are
observational). A and B are crossed so the AB interaction can be studied.

The split plot design depends on how whole plots are assigned to A. Three
common methods are described below, and methods a) and b) are described
in more detail in the following subsections. The randomization and split plot
ANOVA table depend on the design used for assigning the whole plots to
factor A.
a) The whole plots are assigned to A completely at random, as in a one
way Anova.
b) The whole plots are assigned to A and to a blocking variable as in a
completely randomized block design (if the whole plots are experimental, but
a complete block design is used if the whole plots are observational).
c) The whole plots are assigned to A, to row blocks, and to column blocks
as in a Latin square.
The key feature of a split plot design is that there are two units of dierent
sizes: one size for each of the 2 factors of interest. The larger units are assigned

Springer International Publishing AG 2017 283

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 9
284 9 More on Experimental Designs

to A. The large units contain blocks of small units assigned to factor B. Also
factors A and B are crossed.

9.1.1 Whole Plots Randomly Assigned to A

Shown below is the split plot ANOVA table when the whole plots are assigned
to factor A as in a one way Anova design. The whole plot error is error(W) and
can be obtained as an A*replication interaction. The subplot error is error(S).
FA = M SA/M SEW, FB = M SB/M SES, and FAB = M SAB/M SES. R
computes the three test statistics and pvalues correctly, but for SAS FA and
the pvalue pA need to be computed using MSA, MSEW, dfA , and dfeW ob-
tained from the ANOVA table. Sometimes error(W) is also denoted as
residuals. There are ma whole plots, and each whole plot contains b sub-
plots. Thus there are mab subplots. As always, the pvalue column actually
gives pval, an estimate of the pvalue.

Source df SS MS F p-value
A a1 SSA MSA FA pA
error(W) or A*repl a(m 1) SSEW MSEW
B b1 SSB MSB FB pB
AB (a 1)(b 1) SSAB MSAB FAB pAB
residuals or error(S) a(m 1)(b 1) SSES MSES

The tests of interest for this split plot design are nearly identical to those of
a two way Anova model. Yijk has i = 1, . . . , a, j = 1, . . . , b, and k = 1, . . . , m.
Keep A and B in the model if there is an AB interaction.
a) The 4 step test for AB interaction is
i) Ho: there is no interaction HA : there is an interaction.
ii) FAB is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that there is an interaction between A
and B, otherwise fail to reject Ho and conclude that there is no interaction
between A and B. (Or there is not enough evidence to conclude that there is
an interaction.)
b) The 4 step test for A main eects is
i) Ho: 100 = = a00 HA : not Ho.
ii) FA is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that the mean response depends on the
level of A, otherwise fail to reject Ho and conclude that the mean response
does not depend on the level of A. (Or there is not enough evidence to
conclude that the response depends on the level of A.)
9.1 Split Plot Designs 285

c) The 4 step test for B main eects is

i) Ho: 010 = = 0b0 HA : not Ho.
ii) FB is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that the mean response depends on the
level of B, otherwise fail to reject Ho and conclude that the mean response
does not depend on the level of B. (Or there is not enough evidence to
conclude that the response depends on the level of B.)

Source df SS MS F p-value
variety 7 763.16 109.02 1.232 0.3421
MSEW 16 1415.83 88.49
treatment 3 30774.3 10258.1 423.44 0.00
variety*treatment 21 2620.1 124.8 5.150 0.00
error(S) 48 1162.8 24.2

Example 9.1. This split plot data is from Chambers and Hastie (1993,
p. 158). There were 8 varieties of guayule (rubber plant) and 4 treatments
were applied to seeds. The response was the rate of germination. The whole
plots were greenhouse ats and the subplots were 4 subplots of the ats. Each
at received seeds of one variety (A). Each subplot contained 100 seeds and
was treated with one of the treatments (B). There were m = 3 replications
so each variety was planted in 3 ats for a total of 24 ats and 4(24) = 96
observations.
Factorial crossing: Variety and treatments (A and B) are crossed since all
combinations of variety and treatment occur. Hence the AB interaction can
be measured.
Blocking: The whole plots are the 24 greenhouse ats. Each at is a block
of 4 subplots. Each of the 4 subplots gets one of the 4 treatments.
Randomization: The 24 ats are assigned to the 8 varieties completely at
random. Use the sample(24) command to generate a random permutation.
The rst 3 numbers of the permutation get variety one, the next 3 get variety
2, . . . , the last 3 get variety 8. Use the sample(4) command 24 times, once
for each at. If 2, 4, 1, 3 was the permutation for the ith at, then the 1st
subplot gets treatment 3, the 2nd gets treatment 1, the 3rd gets treatment
4, and the 4th subplot gets treatment 2.

a) Perform the test corresponding to A.

b) Perform the test corresponding to B.
c) Perform the test corresponding to AB.

Solution: a) Ho: 100 = = 800 Ha: not Ho

FA = 1.232
pval = 0.3421
286 9 More on Experimental Designs

Fail to reject Ho, the mean rate of germination does not depend on va-
riety. (This test would make more sense if there was no variety * treatment
interaction.)
b) Ho: 010 = = 040 Ha: not Ho
FB = 423.44
pval = 0.00
Reject Ho, the mean rate of germination depends on treatment.
c) Ho: no interaction Ha: there is an interaction
FAB = 5.15
pval = 0.00
Reject Ho, there is a variety * treatment interaction.

9.1.2 Whole Plots Assigned to A as in a CRBD

Shown below is the split plot ANOVA table when the whole plots are
assigned to factor A and a blocking variable as in a completely random-
ized block design. The whole plot error is error(W) and can be obtained as
a block*A interaction. The subplot error is error(S). FA = M SA/M SEW,
FB = M SB/M SES, and FAB = M SAB/M SES. Factor A has a levels
and factor B has b levels. There are r blocks of a whole plots. Each whole
plot contains b subplots, and each block contains a whole plots and thus ab
subplots. Hence there are ra whole plots and rab subplots.
SAS computes the last two test statistics and pvalues correctly, and the
last line of SAS output gives FA and the pvalue pA . The initial line of output
for A is not correct. The output for blocks is probably not correct.

Source df SS MS F p-value
blocks r1
A a1 SSA MSA FA pA
error(W) or block*A (r 1)(a 1) SSEW MSEW
B b1 SSB MSB FB pB
AB (a 1)(b 1) SSAB MSAB FAB pAB
error(S) a(r 1)(b 1) SSES MSES

The tests of interest for this split plot design are nearly identical to those
of a two way Anova model. Yijk has i = 1, . . . , a, j = 1, . . . , b and k = 1, . . . , r.
Keep A and B in the model if there is an AB interaction.
a) The 4 step test for AB interaction is
i) Ho: there is no interaction HA : there is an interaction.
ii) FAB is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that there is an interaction between A
and B, otherwise fail to reject Ho and conclude that there is no interaction
9.1 Split Plot Designs 287

between A and B. (Or there is not enough evidence to conclude that there is
an interaction.)
b) The 4 step test for A main eects is
i) Ho: 100 = = a00 HA : not Ho.
ii) FA is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that the mean response depends on the
level of A, otherwise fail to reject Ho and conclude that the mean response
does not depend on the level of A. (Or there is not enough evidence to
conclude that the response depends on the level of A.)
c) The 4 step test for B main eects is
i) Ho: 010 = = 0b0 HA : not Ho.
ii) FB is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that the mean response depends on the
level of B, otherwise fail to reject Ho and conclude that the mean response
does not depend on the level of B. (Or there is not enough evidence to
conclude that the response depends on the level of B.)

Source df SS MS F p-value
Block 5 4.150 0.830
Variety 2 0.178 0.089 0.65 0.5412
Block*Variety 10 1.363 0.136
Date 3 1.962 0.654 23.39 0.00
Variety*Date 6 0.211 0.035 1.25 0.2973
error(S) 45 1.259 0.028

Example 9.2. The ANOVA table above is for the Snedecor and Cochran
(1967, pp. 369372) split plot data where the whole plots are assigned to
factor A and to blocks in a completely randomized block design. Factor A =
variety of alfalfa (ladak, cossack, ranger). Each eld had two cuttings, with
the second cutting on July 7, 1943. Factor B = date of third cutting (none,
Sept. 1, Sept. 20, Oct. 7) in 1943. The response variable was yield (tons per
acre) in 1944. The 6 blocks were elds of land divided into 3 plots of land,
one for each variety. Each of these 3 plots was divided into 4 subplots for
date of third cutting. So each block had 3 whole plots and 12 subplots.
a) Perform the test corresponding to A.
b) Perform the test corresponding to B.
c) Perform the test corresponding to AB.

Solution: a) Ho: 100 = = 300 Ha: not Ho

FA = 0.65
pval = 0.5412
Fail to reject Ho, the mean yield does not depend on variety.
288 9 More on Experimental Designs

b) Ho: 010 = = 040 Ha: not Ho

FB = 23.39
pval = 0.0
Reject Ho, the mean yield depends on cutting date.
c) Ho: no interaction Ha: there is an interaction
FAB = 1.25
pval = 0.2973
Fail to reject Ho, there is no interaction between variety and cutting date.

Warning: Although the split plot model can be written as a linear model,
the errors are not iid and have a complicated correlation structure. It is also
dicult to get tted values and residuals from the software, so the model
cant be easily checked with response and residual plots. These facts make
the split plot model very hard to use for most researchers.

9.2 Review of the DOE Models

The three basic principles of DOE (design of experiments) are

i) use randomization to assign treatments to units.
ii) Use factorial crossing to compare the eects (main eects, pairwise
interactions, . . . , J-fold interaction) of J 2 factors. If A1 , . . . , AJ are the
factors with li levels for i = 1, . . . , J; then there are l1 l2 lJ treatments
where each treatment uses exactly one level from each factor.
iii) Blocking is used to divide units into blocks of similar units where
similar means the units are likely to have similar values of the response
when given the same treatment. Within each block, randomly assign units to
treatments.
Next the 10 designs of Chapter 5 to Section 9.1 are summarized. If the
randomization cannot be done as described, then much stronger assumptions
on the data are needed for inference to be approximately correct. There are
three common ways of assigning units. For inference, i) requires the least
assumptions and iii) the most.
i) Experimental units are randomly assigned.
ii) Observational units are a random sample of units from a population
of units. Each combination of levels determines a population. So a two way
Anova design has ab populations.
iii) Units (such as time slots) can be assigned systematically due to con-
straints (e.g., physical, cost, or time constraints).

I) One way Anova: Factor A has p levels.

a) For a xed eects one way Anova model, the levels are xed.
9.2 Review of the DOE Models 289

b) For a random eects one way Anova model, the levels are a random
sample from a population oflevels.
p
Randomization: Let n = i=1 mi and do the sample(n) command. Assign
the rst m1 units to treatment (level) 1, the next m2 units to treatment 2,
. . . , the last mp units to treatment p.
II) Two way Anova: Factor A has a levels and factor B has b levels. The
two factors are crossed, forming ab cells.
Randomization: Let n = mab and do the sample(n) command. Randomly
assign m units to each of the ab cells. Assign the rst m units to the (A, B) =
(1, 1) cell, the next m units to the (1,2) cell, . . . , the last m units to the (a, b)
cell.
III) k way Anova: There are k factors A1 , . . . , Ak with a1 , . . . , ak levels,
'k
respectively. The k factors are crossed, forming i=1 ai cells.
'k
Randomization: Let n = m i=1 ai and do the sample(n) command. Ran-
domly assign m units to each cell. Each cell is a combination of levels, so the
(1, 1, . . . , 1, 1) cell gets the 1st m units.
IV) Completely randomized block design: Factor A has k levels (treat-
ments), and there are b blocks (a blocking variable has b levels) of k units.
Randomization: Let n = kb and do the sample(k) command b times.
Within each block of k units, randomly assign 1 unit to each treatment.
V) Latin squares: Factor A has a levels (treatments), the row blocking
variable has a blocks of a units, and the column blocking variable has a blocks
of a units. There are a2 units since the row and column blocking variables are
crossed. The treatment factor, row blocking variable, and column blocking
variable are also crossed. A Latin square is such that each of the a treatments
occurs once in each row and once in each column.
Randomization: Pick an a a Latin square. Use the sample(a) command
to assign row levels to numbers 1 to a. Use the sample(a) command to assign
column levels to numbers 1 to a. Use the sample(a) command to assign
treatment levels to the rst a capital letters. If possible, use the sample(a2 )
command to assign units, 1 unit to each cell of the Latin square.
VI) 2k factorial design: There are k factors, each with 2 levels.
Randomization: Let n = m2k and do the sample(n) command. Randomly
assign m units to each cell. Each cell corresponds to a run which is determined
by a string of k +s and s corresponding to the k main eects.
kf
VII) 2R fractional factorial design: There are k factors, each with 2
levels.
Randomization: Let n = 2kf and do the sample(n) command. Randomly
assign 1 unit to each run which is determined by a string of k +s and s
corresponding to the k main eects.
VIII) Plackett Burman P B(n) design: There are k factors, each with 2
levels.
290 9 More on Experimental Designs

Randomization: Let n = 4J for some J. Do the sample(n) command.

Randomly assign 1 unit to each run which is a string of n 1 +s and s.
(Each run corresponds to a row in the design matrix, so we are ignoring the
column of 1s corresponding to I in the design matrix.)
IX) Split plot design where the whole plots are assigned to A as in a one
way Anova design: The whole plot factor A has a levels and each whole plot
is a block of b subplots used to study factor B which has b levels. Split plot
designs have two types of units: the whole plots are the larger units and the
subplots are the smaller units.
Randomization: a) Suppose there are n = ma whole plots. Randomly
assign m whole plots to each level of A with the sample(n) command. Assign
the rst m units (whole plots) to treatment (level) 1, the next m units to
treatment 2, . . . , the last m units to treatment a.
b) Do the sample(b) command ma times, once for each whole plot. Within
each whole plot, randomly assign 1 subplot (unit) to each of the b levels of B.
X) Split plot design where the whole plots are assigned to A and a blocking
variable as in a completely randomized block design: The whole plot factor A
has a levels and each whole plot is a block of b subplots used to study factor
B which has b levels. Split plot designs have two types of units: the whole
plots are the larger units and the subplots are the smaller units. There are
also r blocks of a whole plots. Each whole plot has b subplots. Thus there are
ra whole plots and rab subplots.
Randomization: a) Do the sample(a) command r times, once for each
block. For each block of a whole plots, randomly assign 1 whole plot to each
of the a levels of A.
b) Do the sample(b) command ra times, once for each whole plot. Within
each whole plot, randomly assign 1 subplot to each of the b levels of B.

Try to become familiar with the designs and their randomization so that
you can recognize a design given a story problem.

Example 9.3. Cobb (1998, pp. 200212) describes an experiment on

weight gain for baby pigs. The response Y was the average daily weight
gain in pounds for each piglet (over a period of time). Factor A consisted of
0 mg of an antibiotic or 40 mg of an antibiotic while factor B consisted of 0
mg of vitamin B12 or 5 mg of B12. Hence there were 4 diets (A, B) = (0,0),
(40,0), (0,5) or (40,5). If there were 12 piglets and 3 were randomly assigned
to each diet, what type of experimental design was used?
Solution: A and B are crossed with each combination of (A, B) levels
forming a diet. So the two way Anova (or 22 factorial) design was used.

Example 9.4. In 2008, a PhD student was designing software to analyze

a complex image. 100 portions of the image need to be analyzed correctly,
and the response variable is the proportion of errors. Sixteen test images
9.3 Summary 291

were available and thought to be representative. The goal was to achieve

an average error rate of less than 0.3 if many images were examined. The
student had identied 3 factors to reduce the error rate. Each factor had 2
levels. Thus there were 8 versions of the software that analyze images.
The student selected a single test image and ran a 23 design with 8 time
slots as units. Factor A was active but factors B and C were inert. When
A was at the (+) level the error rate was about 0.27. Briey explain why
this experiment did not give much information about how the software will
behave on many images.
Solution: More images are needed, 1 image is not enough.
(A better design is a completely randomized block design that uses each of
the 16 images as a block and factor A = software version with 8 levels. The
units for the block are 8 time slots so each of the 8 versions of the software
is tested on each test image.)

9.3 Summary

1) The analysis of the response, not that of the residuals, is of primary im-
portance. The response plot can be used to analyze the response in the back-
ground of the tted model. For linear models such as experimental designs,
the estimated mean function is the identity line and should be added as a
visual aid.
2) Assume that the residual degrees of freedom are large enough for testing.
Then the response and residual plots contain much information. Linearity and
constant variance may be reasonable if the plotted points scatter about the
identity line in a (roughly) evenly populated band. Then the residuals should
scatter about the r = 0 line in an evenly populated band. It is easier to check
linearity with the response plot and constant variance with the residual plot.
Curvature is often easier to see in a residual plot, but the response plot can
be used to check whether the curvature is monotone or not. The response plot
is more eective for determining whether the signal to noise ratio is strong
or weak, and for detecting outliers, inuential cases, or a critical mix.
3) The three basic principles of DOE (design of experiments) are
i) use randomization to assign units to treatments.
ii) Use factorial crossing to compare the eects (main eects, pairwise
interactions, . . . , J-fold interaction) for J 2 factors. If A1 , . . . , AJ are the
factors with li levels for i = 1, . . . , J then there are l1 l2 lJ treatments
where each treatment uses exactly one level from each factor.
iii) Blocking is used to divide units into blocks of similar units where
similar means the units are likely to have similar values of the response
when given the same treatment. Within each block randomly assign units to
treatments.
292 9 More on Experimental Designs

4) Split plot designs have two units. The large units are called whole plots
and contain blocks of small units called subplots. The whole plots get assigned
to factor A while the subplots get assigned to factor B (randomly if the units
are experimental but not randomly if the units are observational). A and B
are crossed so the AB interaction can be studied.
5) The split plot design depends on how whole plots are assigned to A.
Three common methods are a) the whole plots are assigned to A completely
at random, as in a one way Anova, b) the whole plots are assigned to A
and to a blocking variable as in a completely randomized block design (if the
whole plots are experimental, a complete block design is used if the whole
plots are observational), c) the whole plots are assigned to A, to row blocks,
and to column blocks as in a Latin square.
6) The split plot ANOVA table when whole plots are assigned to levels of
A as in a one way Anova is shown below. The whole plot error is error(W) and
can be obtained as an A*replication interaction. The subplot error is error(S).
FA = M SA/M SEW, FB = M SB/M SES, and FAB = M SAB/M SES. R
computes the three test statistics and pvalues correctly, but for SAS FA
and the pvalue pA need to be computed using MSA, MSEW, dfA , and dfeW
obtained from the ANOVA table.

Source df SS MS F p-value
A a1 SSA MSA FA pA
error(W) or A*repl a(m 1) SSEW MSEW
B b1 SSB MSB FB pB
AB (a 1)(b 1) SSAB MSAB FAB pAB
residuals or error(S) a(m 1)(b 1) SSES MSES

7) The tests of interest corresponding to 6) are nearly identical to those of

a two way Anova model. Yijk has i = 1, . . . , a, j = 1, . . . , b, and k = 1, . . . , m.
Keep A and B in the model if there is an AB interaction.
a) The 4 step test for AB interaction is
i) Ho: there is no interaction HA : there is an interaction.
ii) FAB is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that there is an interaction between A
and B, otherwise fail to reject Ho and conclude that there is no interaction
between A and B.
b) The 4 step test for A main eects is
i) Ho: 100 = = a00 HA : not Ho.
ii) FA is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that the mean response depends on the
level of A, otherwise fail to reject Ho and conclude that the mean response
does not depend on the level of A.
9.3 Summary 293

c) The 4 step test for B main eects is

i) Ho: 010 = = 0b0 HA : not Ho.
ii) FB is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that the mean response depends on the
level of B, otherwise fail to reject Ho and conclude that the mean response
does not depend on the level of B.
8) The split plot ANOVA table when whole plots are assigned to levels
of A as in a completely randomized block design is shown below. The whole
plot error is error(W) and can be obtained as a block*A interaction. The
subplot error is error(S). FA = M SA/M SEW, FB = M SB/M SES, and
FAB = M SAB/M SES. SAS computes the last two test statistics and pvalues
correctly, and the last line of SAS output gives FA and the pvalue pA . The
initial line of output for A is not correct. The output for blocks is probably
not correct.

Source df SS MS F p-value
blocks r1
A a1 SSA MSA FA pA
error(W) or block*A (r 1)(a 1) SSEW MSEW
B b1 SSB MSB FB pB
AB (a 1)(b 1) SSAB MSAB FAB pAB
error(S) a(r 1)(b 1) SSES MSES

9) The tests of interest corresponding to 8) are nearly identical to those

of a two way Anova model and point 7). Yijk has i = 1, . . . , a, j = 1, . . . , b,
and k = 1, . . . , r. Keep A and B in the model if there is an AB interaction.
a) The 4 step test for AB interaction is
i) Ho: there is no interaction HA : there is an interaction.
ii) FAB is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that there is an interaction between A
and B, otherwise fail to reject Ho and conclude that there is no interaction
between A and B.
b) The 4 step test for A main eects is
i) Ho: 100 = = a00 HA : not Ho.
ii) FA is obtained from output.
iii) The pval is obtained from output.
iv) If pval reject Ho and conclude that the mean response depends on the
level of A, otherwise fail to reject Ho and conclude that the mean response
does not depend on the level of A.
c) The 4 step test for B main eects is
i) Ho: 010 = = 0b0 HA : not Ho.
ii) FB is obtained from output.
iii) The pval is obtained from output.
294 9 More on Experimental Designs

iv) If pval reject Ho and conclude that the mean response depends on the
level of B, otherwise fail to reject Ho and conclude that the mean response
does not depend on the level of B.

9.4 Complements

See Robinson et al. (2009) for a comparison of completely randomized designs,

completely randomized block designs, and split plot designs. Some history of
experimental designs is given by Box (1980, 1984). Also see David (1995,
20067) and Hahn (1982).
The importance of DOE is discussed in Gelman (2005), and a review is
given by Steinberg and Hunter (1984). For experiments done as class projects,
see Hunter (1977).

9.5 Problems

Source df SS MS F p-value
Block 2 77.55 38.78
Method 2 128.39 64.20 7.08 0.0485
Block*Method 4 36.28 9.07
Temp 3 434.08 144.69 41.94 0.00
Method*Temp 6 75.17 12.53 2.96 0.0518
error(S) 12 50.83 4.24

9.1. The ANOVA table above is for the Montgomery (1984, pp. 386389)
split plot data where the whole plots are assigned to factor A and to blocks
in a completely randomized block design. The response variable is tensile
strength of paper. Factor A is (preparation) method with 3 levels (1, 2, 3).
Factor B is temperature with 4 levels (200, 225, 250, 275). The pilot plant
can make 12 runs a day and the experiment is repeated each day, with days
as blocks. A batch of pulp is made by one of the 3 preparation methods. Then
the batch of pulp is divided into 4 samples, and each sample is cooked at one
of the four temperatures.
a) Perform the test corresponding to A.
b) Perform the test corresponding to B.
c) Perform the test corresponding to AB.
9.5 Problems 295

Source df SS MS F p-value
Block 1 0.051 0.051
Nitrogen 3 37.32 12.44 29.62 0.010
Block*Nitrogen 3 1.26 0.42
Thatch 2 3.82 1.91 9.10 0.009
Nitrogen*Thatch 6 4.15 0.69 3.29 0.065
error(S) 12 1.72 0.21

9.2. The ANOVA table above is for the Kuehl (1994, pp. 473481) split
plot data where the whole plots are assigned to factor A and to blocks in
a completely randomized block design. The response variable is the average
chlorophyll content (mg/gm of turf grass clippings). Factor A is nitrogen
fertilizer with 4 levels (1, 2, 3, 4). Factor B is length of time that thatch was
allowed to accumulate with 3 levels (2, 5, or 8 years).
There were 2 blocks of 4 whole plots to which the levels of factor A were
assigned. The 2 blocks formed a golf green which was seeded with turf grass.
The 8 whole plots were plots of golf green. Each whole plot had 3 subplots
to which the levels of factor B were randomly assigned.
a) Perform the test corresponding to A.
b) Perform the test corresponding to B.
c) Perform the test corresponding to AB.

Source df SS MS F p-value
Block 5 4.150 0.830
Variety 2 0.178 0.089 0.65 0.5412
Block*Variety 10 1.363 0.136
Date 3 1.962 0.654 23.39 0.00
Variety*Date 6 0.211 0.035 1.25 0.2973
error(S) 45 1.259 0.028

9.3. The ANOVA table above is for the Snedecor and Cochran (1967, pp.
369372) split plot data where the whole plots are assigned to factor A and to
blocks in a completely randomized block design. Factor A = variety of alfalfa
(ladak, cossack, ranger). Each eld had two cuttings, with the second cutting
on July 7, 1943. Factor B = date of third cutting (none, Sept. 1, Sept. 20,
Oct. 7) in 1943. The response variable was yield (tons per acre) in 1944. The
6 blocks were elds of land divided into 3 plots of land, one for each variety.
Each of these 3 plots was divided into 4 subplots for date of third cutting.
So each block had 3 whole plots and 12 subplots.
a) Perform the test corresponding to A.
b) Perform the test corresponding to B.
c) Perform the test corresponding to AB.
296 9 More on Experimental Designs

9.4. Following Montgomery (1984, pp. 386389), suppose the response

variable is tensile strength of paper. One factor is (preparation) method with
3 levels (1, 2, 3). Another factor is temperature with 4 levels (200, 225,
250, 275).
a) Suppose the pilot plant can make 12 runs a day and the experiment is
repeated each day, with days as blocks. A batch of pulp is made by one of
the 3 preparation methods. Then the batch of pulp is divided into 4 samples,
and each sample is cooked at one of the four temperatures. Which factor,
method, or temperature is assigned to subplots?
b) Suppose the pilot plant could make 36 runs in one day. Suppose that 9
batches of pulp are made, that each batch of pulp is divided into 4 samples,
and each sample is cooked at one of the four temperatures. How should the 9
batches be allocated to the three preparation methods, and how should the
4 samples be allocated to the four temperatures?
c) Suppose the pilot plant can make 36 runs in one day and that the units
are 36 batches of material to be made into pulp. Each of the 12 method tem-
perature combinations is to be replicated 3 times. What type of experimental
design should be used? (Hint: not a split plot.)

R and SAS Problems

Use the command source(G:/lregpack.txt) to download the func-

tions and the command source(G:/lregdata.txt) to download the data.
See Preface or Section 14.1. Typing the name of the R function, e.g. aov,
will display the code for the function. Use the args command, e.g. args(aov),
to display the needed arguments for the function. For some of the following
problems, the R commands and SAS programs can be copied and pasted
from (http://lagrange.math.siu.edu/Olive/lreghw.txt) into R.

9.5. a) Download (http://lagrange.math.siu.edu/Olive/lregdata.txt) into

R, and type the following commands. Then copy and paste the output into
Notepad and print the output.
attach(guay)
out<-aov(plants~variety*treatment + Error(flats),guay)
summary(out)
detach(guay)
This split plot data is from Chambers and Hastie (1993, p. 158). There are
8 varieties of guayule (rubber plant) and 4 treatments were applied to seeds.
The response was the rate of germination. The whole plots were greenhouse
ats and the subplots were subplots of the ats. Each at received seeds of
one variety (A). Each subplot contained 100 seeds and was treated with one
of the treatments (B). There were m = 3 replications so each variety was
planted in 3 ats for a total of 24 ats and 4(24) = 96 observations.
b) Use the output to test whether the response depends on variety.
9.5 Problems 297

9.6. Download (http://lagrange.math.siu.edu/Olive/lregdata.txt) into R,

and type the following commands. Then copy and paste the output into
Notepad and print the output.

attach(steel)
out<-aov(resistance~heat*coating + Error(wplots),steel)
summary(out)
detach(steel)

This split plot steel data is from Box et al. (2005, p. 336). The whole plots
are time slots to use a furnace, which can hold 4 steel bars at one time. Factor
A = heat has 3 levels (360, 370, 380o F). Factor B = coating has 4 levels
(4 types of coating: c1, c2, c3, and c4). The response was corrosion resistance.
a) Perform the test corresponding to A.
b) Perform the test corresponding to B.
c) Perform the test corresponding to AB.

9.7. This is the same data as in Problem 9.6, using SAS. Copy and paste
the SAS program from (http://lagrange.math.siu.edu/Olive/lrsashw.txt)
into SAS, run the program, then print the output. Only include the second
page of output.
To get the correct F statistic for heat, you need to divide MS heat by MS
wplots.

9.8. a) Copy and paste the SAS program from

(http://lagrange.math.siu.edu/Olive/lrsashw.txt) into SAS, run the pro-
gram, then print the output. Only include the second page of output.
This data is from the SAS Institute (1985, pp. 131132). The B and AB
ANOVA table entries are correct, but the correct entry for A is the last line
of output where Block*A is used as the error.
b) Perform the test corresponding to A.
c) Perform the test corresponding to B.
d) Perform the test corresponding to AB.
Chapter 10
Multivariate Models

The multivariate location and dispersion model is a special case of the

multivariate linear regression model when the design matrix is equal to the
vector of ones: X = 1. See Chapter 12. (Similarly, the location model is a
special case of the multiple linear regression model. See Section 2.9.1.) The
multivariate normal and elliptically contoured distributions are important
parametric models for the multivariate location and dispersion model. The
multivariate normal distribution is useful in the large sample theory of the
linear model, covered in Chapter 11, while elliptically contoured distributions
are useful for multivariate linear regression. Section 3.4.1 used prediction re-
gions for iid multivariate data to bootstrap hypothesis tests.

Denition 10.1. An important multivariate location and dispersion

model is a joint distribution with joint pdf

f (z|, )

for a p 1 random vector x that is completely specied by a p 1 population

location vector and a pp symmetric
( positive denite population dispersion
matrix . Thus P (x A) = A f (z)dz for suitable sets A.

The multivariate location and dispersion model is in many ways similar to

the multiple linear regression model. The data are iid vectors from some dis-
tribution such as the multivariate normal (MVN) distribution. The location
parameter of interest may be the mean or the center of symmetry of an
elliptically contoured distribution. Hyperellipsoids will be estimated instead
of hyperplanes, and Mahalanobis distances will be used instead of absolute
residuals to determine if an observation is a potential outlier.
Assume that X 1 , . . . , X n are n iid p 1 random vectors and that the
joint pdf of X 1 is f (z|, ). Also assume that the data X i = xi has been
observed and stored in an n p matrix

Springer International Publishing AG 2017 299

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 10
300 10 Multivariate Models

x1,1 x1,2 . . . x1,p
xT1 x2,1 x2,2
. . . x2,p
) *
W = ... = . .. . . .. = v1 v2 . . . vp
T
.. . . .
xn
xn,1 xn,2 . . . xn,p

where the ith row of W is xTi and the jth column is v j . Each column v j of
W corresponds to a variable. For example, the data may consist of n visitors
to a hospital where the p = 2 variables height and weight of each individual
were measured.
There are some dierences in the notation used in multiple linear regression
and multivariate location dispersion models. Notice that W could be used
as the design matrix in multiple linear regression although usually the rst
column of the regression design matrix is a vector of ones. The n p design
matrix in the multiple linear regression model was denoted by X, and xTi
was the ith row of X. In the multivariate location dispersion model, X and
X i will be used to denote a p 1 random vector with observed value xi ,
and xTi is the ith row of the data matrix W . Johnson and Wichern (1988,
pp. 7, 53) uses X to denote the n p data matrix and an n 1 random
vector, relying on the context to indicate whether X is a random vector or
data matrix. Software tends to use dierent notation. For example, R will
use commands such as
var(x)
to compute the sample covariance matrix of the data. Hence x corresponds
to W , x[,1] is the rst column of x, and x[4, ] is the 4th row of x.

10.1 The Multivariate Normal Distribution

Denition 10.2: Rao (1965, p. 437). A p 1 random vector X has a

pdimensional multivariate normal distribution Np (, ) i tT X has a uni-
variate normal distribution for any p 1 vector t.

If is positive denite, then X has a pdf

1 1
e(1/2)(z ) (z )
T
f (z) = (10.1)
(2)p/2 ||1/2

where ||1/2 is the square root of the determinant of . Note that if p = 1,

then the quadratic form in the exponent is (z )( 2 )1 (z ) and X has
the univariate N (, 2 ) pdf. If is positive semidenite but not positive
denite, then X has a degenerate distribution. For example, the univariate
N (0, 02 ) distribution is degenerate (the point mass at 0).
10.1 The Multivariate Normal Distribution 301

Denition 10.3. The population mean of a random p 1 vector X =

(X1 , . . . , Xp )T is
E(X) = (E(X1 ), . . . , E(Xp ))T
and the p p population covariance matrix

Cov(X) = E(X E(X))(X E(X))T = (ij ).

That is, the ij entry of Cov(X) is Cov(Xi , Xj ) = ij .

The covariance matrix is also called the variancecovariance matrix and

variance matrix. Sometimes the notation Var(X) is used. Note that Cov(X)
is a symmetric positive semidenite matrix. If X and Y are p 1 random
vectors, a a conformable constant vector, and A and B are conformable
constant matrices, then

E(a + X) = a + E(X) and E(X + Y ) = E(X) + E(Y ) (10.2)

and
E(AX) = AE(X) and E(AXB) = AE(X)B. (10.3)
Thus
Cov(a + AX) = Cov(AX) = ACov(X)AT . (10.4)
Some important properties of multivariate normal (MVN) distributions are
given in the following three propositions. These propositions can be proved
using results from Johnson and Wichern (Johnson and Wichern (1988), pp.
127132).

Proposition 10.1. a) If X Np (, ), then E(X) = and

Cov(X) = .

b) If X Np (, ), then any linear combination tT X = t1 X1 + +

tp Xp N1 (tT , tT t). Conversely, if tT X N1 (tT , tT t) for every p 1
vector t, then X Np (, ).
c) The joint distribution of independent normal random variables
is MVN. If X1 , . . . , Xp are independent univariate normal N (i , i2 ) random
vectors, then X = (X1 , . . . , Xp )T is Np (, ) where = (1 , . . . , p )T and
= diag(12 , . . . , p2 ) (so the o diagonal entries ij = 0 while the diagonal
entries of are ii = i2 ).
d) If X Np (, ) and if A is a q p matrix, then AX
Nq (A, AAT ). If a is a p 1 vector of constants and b is a constant, then
a + bX Np (a + b, b2 ). (Note that bX = bI p X with A = bI p .)

It will be useful to partition X, , and . Let X 1 and 1 be q 1 vectors,

let X 2 and 2 be (p q) 1 vectors, let 11 be a q q matrix, let 12
be a q (p q) matrix, let 21 be a (p q) q matrix, and let 22 be a
(p q) (p q) matrix. Then
302 10 Multivariate Models
     
X1 1 11 12
X= , = , and = .
X2 2 21 22

Proposition 10.2. a) All subsets of an MVN are MVN: (Xk1 ,

. . . , Xkq )T Nq (, ) where i = E(Xki ) and ij = Cov(Xki , Xkj ). In
particular, X 1 Nq (1 , 11 ) and X 2 Npq (2 , 22 ).
b) If X 1 and X 2 are independent, then Cov(X 1 , X 2 ) = 12 =
E[(X 1 E(X 1 ))(X 2 E(X 2 ))T ] = 0, a q (p q) matrix of zeroes.
c) If X Np (, ), then X 1 and X 2 are independent i 12 = 0.
d) If X 1 Nq (1 , 11 ) and X 2 Npq (2 , 22 ) are independent, then
     
X1 1 11 0
Np , .
X2 2 0 22

Proposition 10.3. The conditional distribution of an MVN is

MVN. If X Np (, ), then the conditional distribution of X 1 given
that X 2 = x2 is multivariate normal with mean 1 + 12 1
22 (x2 2 ) and
covariance matrix 11 12 1
22 21 . That is,

X 1 |X 2 = x2 Nq (1 + 12 1 1
22 (x2 2 ), 11 12 22 21 ).

Example 10.1. Let p = 2 and let (Y, X)T have a bivariate normal distri-
bution. That is,
     
Y Y Y2 Cov(Y, X)
N2 , 2 .
X X Cov(X, Y ) X

Also recall that the population correlation between X and Y is given by

Cov(X, Y ) X,Y
(X, Y ) =   =
VAR(X) VAR(Y ) X Y
if X > 0 and Y > 0. Then Y |X = x N (E(Y |X = x), VAR(Y |X = x))
where the conditional mean

1 Y2
E(Y |X = x) = Y + Cov(Y, X) 2 (x X ) = Y + (X, Y ) 2 (x X )
X X
and the conditional variance
1
VAR(Y |X = x) = Y2 Cov(X, Y ) 2 Cov(X, Y )
X

 
Y2
= Y2 (X, Y ) 2 (X, Y )
2
X Y2
X
10.2 Elliptically Contoured Distributions 303

= Y2 2 (X, Y )Y2 = Y2 [1 2 (X, Y )].

Also aX + bY is univariate normal with mean aX + bY and variance
a2 X
2
+ b2 Y2 + 2ab Cov(X, Y ).

Remark 10.1. There are several common misconceptions. First, it is not

true that every linear combination tT X of normal random variables
is a normal random variable, and it is not true that all uncorrelated
normal random variables are independent. The key condition in Propo-
sition 10.1b and Proposition 10.2c is that the joint distribution of X is MVN.
It is possible that X1 , X2 , . . . , Xp each has a marginal distribution that is uni-
variate normal, but the joint distribution of X is not MVN. The following
example is from Rohatgi (1976, p. 229). Suppose that the joint pdf of X and
Y is a mixture of two bivariate normal distributions both with EX = EY = 0
and VAR(X) = VAR(Y ) = 1, but Cov(X, Y ) = . Hence f (x, y) =
1 1 1
 exp( (x2 2xy + y 2 )) +
2 2 1 2 2(1 2 )

1 1 1 1 1
 exp( (x2 + 2xy + y 2 )) f1 (x, y) + f2 (x, y)
2 2 1 2 2(1 2 ) 2 2
where x and y are real and 0 < < 1. Since both marginal distributions
of fi (x, y) are N(0,1) for i = 1 and 2 by Proposition
(( 10.2 a), the marginal
distributions of X and Y are N(0,1). Since xyfi (x, y)dxdy = for i = 1
and for i = 2, X and Y are uncorrelated, but X and Y are not independent
since f (x, y) = fX (x)fY (y).

Remark 10.2. In Proposition 10.3, suppose that X = (Y, X2 , . . . , Xp )T .

Let X1 = Y and X 2 = (X2 , . . . , Xp )T . Then E[Y |X 2 ] = 1 + 2 X2 + +
p Xp and VAR[Y |X 2 ] = 2 is a constant that does not depend on X 2 . Hence
Y |X 2 = 1 + 2 X2 + + p Xp + e follows the multiple linear regression
model where e N (0, 2 ). Here 1 = E(Y ) T E(X 2 ), = 1
XX XY , and
2 = Y2 Y X 1
XX XY where X = Cov(X 2 ) and XY = Cov(X 2 , Y ).

10.2 Elliptically Contoured Distributions

Denition 10.4: Johnson (1987, pp. 107108). A p1 random vector X

has an elliptically contoured distribution, also called an elliptically symmetric
distribution, if X has joint pdf

f (z) = kp ||1/2 g[(z )T 1 (z )], (10.5)

and we say X has an elliptically contoured ECp (, , g) distribution.

304 10 Multivariate Models

If X has an elliptically contoured (EC) distribution, then the characteristic

function of X is
X (t) = exp(itT )(tT t) (10.6)
for some function . If the second moments exist, then

E(X) = (10.7)

and
Cov(X) = cX (10.8)
where
cX = 2  (0).

Denition 10.5. The population squared Mahalanobis distance

U D2 = D2 (, ) = (X )T 1 (X ). (10.9)

For elliptically contoured distributions, U has pdf

p/2
h(u) = kp up/21 g(u). (10.10)
(p/2)

For c > 0, an ECp (, cI, g) distribution is spherical about where I is

the p p identity matrix. The multivariate normal distribution Np (, ) has
kp = (2)p/2 , (u) = g(u) = exp(u/2), and h(u) is the 2p density.

The following lemma is useful for proving properties of EC distributions

without using the characteristic function (10.6). See Eaton (1986) and Cook
(1998, pp. 57, 130).

Lemma 10.4. Let X be a p 1 random vector with 1st moments; i.e.

E(X) exists. Let B be any constant full rank p r matrix where 1 r p.
Then X is elliptically contoured i for all such conforming matrices B,

E(X|B T X) = + M B B T (X ) = aB + M B B T X (10.11)

where the p 1 constant vector aB and the p r constant matrix M B both

depend on B.

A useful fact is that aB and M B do not depend on g:

aB = M B B T = (I p M B B T ),

and
M B = B(B T B)1 .
10.2 Elliptically Contoured Distributions 305

See Problem 10.11. Notice that in the formula for M B , can be replaced
by c where c > 0 is a constant. In particular, if the EC distribution has
2nd moments, Cov(X) can be used instead of .

To use Lemma 10.4 to prove interesting properties, partition X, , and

as above Proposition 10.2. Also assume that the (p + 1) 1 vector (Y, X T )T
is ECp+1 (, , g) where Y is a random variable, X is a p 1 vector, and use
     
Y Y Y Y Y X
, = , and = .
X X XY XX

Proposition 10.5. Let X ECp (, , g) and assume that E(X) exists.

a) Any subset of X is EC, in particular X 1 is EC.

b) (Cook 1998 p. 131, Kelker 1970). If Cov(X) is nonsingular,

Cov(X|B T X) = dg (B T X)[ B(B T B)1 B T ]

where the real valued function dg (B T X) is constant i X is MVN.

Proof of a). Let A be an arbitrary full rank q r matrix where 1 r q.

Let  
A
B= .
0
Then B T X = AT X 1 , and
  
X1
E[X|B X] = ET
|A X 1 =
T
X2
     
1 M 1B # T T
$ X 1 1
+ A 0
2 M 2B X 2 2
by Lemma 10.4. Hence E[X 1 |AT X 1 ] = 1 +M 1B AT (X 1 1 ). Since A was
arbitrary, X 1 is EC by Lemma 10.4. Notice that M B = B(B T B)1 =
       1
11 12 A # T T
$ 11 12 A
A 0
21 22 0 21 22 0
 
M 1B
= .
M 2B
Hence
M 1B = 11 A(AT 11 A)1
and X 1 is EC with location and dispersion parameters 1 and 11 . 
306 10 Multivariate Models

Proposition 10.6. Let (Y, X T )T be ECp+1 (, , g) where Y is a random

variable.
a) Assume that E[(Y, X T )T ] exists. Then E(Y |X) = + T X where
= Y T X and
= 1
XX XY .

b) Even if the rst moment does not exist, the conditional median
MED(Y |X) = + T X
where and are given in a).

Proof. a) The trick is to choose B so that Lemma 10.4 applies. Let

 T
0
B= .
Ip

Then B T B = XX and
 
Y X
B = .
XX

Now       
Y Y Y
E |X =E |B T
X X X
 
T 1 T Y Y
= + B(B B) B
X X
by Lemma 10.4. The right-hand side of the last equation is equal to
   
Y X Y Y X 1 1
XX X + Y X XX X
+ 1
XX (X X ) =
XX X

and the result follows since

T = Y X 1
XX .

b) See Croux et al. (2001) for references.

Example 10.2. This example illustrates another application of Lemma

10.4. Suppose that X comes from a mixture of two multivariate normals
with the same mean and proportional covariance matrices. That is, let

X (1 )Np (, ) + Np (, c)

where c > 0 and 0 < < 1. Since the multivariate normal distribution is
elliptically contoured
10.3 Sample Mahalanobis Distances 307

E(X|B T X) = (1 )[ + M 1 B T (X )] + [ + M 2 B T (X )]

= + [(1 )M 1 + M 2 ]B T (X ) + M B T (X ).
Since M B only depends on B and , it follows that M 1 = M 2 = M =
M B . Hence X has an elliptically contoured distribution by Lemma 10.4. See
Problem 10.4 for a related result.

10.3 Sample Mahalanobis Distances

In the multivariate location and dispersion model, sample Mahalanobis dis-

tances play a role similar to that of residuals in multiple linear regression.
The observed data X i = xi for i = 1, . . . , n is collected in an n p matrix W
with n rows xT1 , . . . , xTn . Let the p 1 column vector T (W ) be a multivari-
ate location estimator, and let the p p symmetric positive denite matrix
C(W ) be a dispersion estimator such as the sample covariance matrix.

Denition 10.6. The ith squared Mahalanobis distance is

Di2 = Di2 (T (W ), C(W )) = (X i T (W ))T C 1 (W )(X i T (W )) (10.12)

for each point X i . Notice that Di2 is a random variable (scalar valued).

Notice that the population squared Mahalanobis distance is

2
DX (, ) = (X )T 1 (X ) (10.13)

and that the term 1/2 (X ) is the pdimensional analog to the Z-score
used to transform a univariate N (, 2 ) random variable into  an N (0, 1)
random variable. Hence the sample Mahalanobis distance Di = Di2 is an
analog of the absolute value |Zi | of the sample Z-score Zi = (Xi X)/. Also
notice that the Euclidean distance of xi from the estimate of center T (W )
is Di (T (W ), I p ) where I p is the p p identity matrix.

Example 10.3. The contours of constant density for the Np (, ) distri-

bution are hyperellipsoid boundaries of the form (x )T 1 (x ) = a2 .
An density region R is a set such that P (X R ) = , and for the
Np (, ) distribution, the regions of highest density are sets of the form

{x : (x )T 1 (x ) 2p ()} = {x : Dx
2
(, ) 2p ()}

where P (W 2p ()) = if W 2p . If the X i are n iid random vectors

each with an Np (, ) pdf, then a scatterplot of Xi,k versus Xi,j should
be ellipsoidal for k = j. Similar statements hold if X is ECp (, , g) with
308 10 Multivariate Models

continuous nondecreasing g, but the -density region will use a constant U

obtained from Equation (10.10).

The classical Mahalanobis distance corresponds to the sample mean and

sample covariance matrix

1
n
T (W ) = X = X i,
n i=1

and
1 
n
C(W ) = S = (X i X)(X i X)T
n 1 i=1

and will be denoted by M Di . When T (W ) and

 C(W ) are estimators other
than the sample mean and covariance, Di = Di2 will sometimes be denoted
by RDi . The DD plot was also dened in Denition 3.14.

Denition 10.7: Rousseeuw and Van Driessen (1999). The DD plot

is a plot of the classical Mahalanobis distances MDi versus robust Maha-
lanobis distances RDi .

The DD plot is used as a diagnostic for multivariate normality, elliptical

symmetry, and for outliers. Assume that the data set consists of iid vectors
from an ECp (, , g) distribution with second moments. Then the classi-
cal sample mean and covariance matrix (TM , C M ) = (X, S) is a consistent
estimator for (, cx ) = (E(X), Cov(X)). Assume that an alternative al-
gorithm estimator (TA , C A ) is a consistent estimator for (, aA ) for some
constant aA > 0. By scaling the algorithm estimator, the DD plot can be
constructed to follow the identity line with unit slope and zero intercept if
2
the X i are iid from an Np (, ) distribution. Let(TR , C R ) = (TA , C A / )
denote the scaled algorithm estimator where = 2p,0.5 /med(Di (TA , C A )).
Notice that (TR , C R ) is a valid estimator of location and dispersion. Hence
the robust distances used in the DD plot are given by

RDi = RDi (TR , C R ) = (X i TR (W ))T [C R (W )]1 (X i TR (W ))

= Di (TA , C A ) for i = 1, . . . , n.
The DD plot can be used simultaneously as a diagnostic for whether the
data arise from a multivariate normal distribution or from another EC dis-
tribution with nonsingular covariance matrix. EC data will cluster about a
straight line through the origin; MVN data in particular will cluster about the
identity line. Thus the DD plot can be used to assess the success of numerical
transformations towards elliptical symmetry. This application is important
since many statistical methods assume that the underlying data distribution
10.5 Problems 309

is MVN or EC. If the plotted points do not cluster tightly about some line
through the origin, then the data may not have an EC distribution. Plotted
points that are far from the bulk of the plotted points tend to be outliers. A
DD plot of the continuous predictors is useful for detecting outliers in these
variables. See Example 3.14 and Section 3.6. For regression, the DD plot of
the residuals can be useful. See Chapter 12. The lregpack function ddplot4 will
make the DD plot using the robust RMVN estimator. See Olive and Hawkins
(2010) and Olive (2016c, ch. 5).

10.4 Complements

Johnson and Wichern (1988, 2007), Mardia et al. (1979), and Olive (2016c)
are good references for multivariate statistical analysis. The elliptically con-
toured distributions generalize the multivariate normal distribution and are
discussed in Johnson (1987). Cambanis et al. (1981), Chmielewski (1981),
and Eaton (1986) are also important references. Olive (2002) discussed uses
of the DD plot. Robust estimators are discussed in Olive (2004a, 2016c) and
Zhang et al. (2012).

10.5 Problems

Problems with an asterisk * are especially important.

10.1 . Suppose that

X1 49 3 1 1 0
X2 100 1 6 1 1
.
X3 N4 17 , 1 1 4 0
X4 7 0 1 0 2

a) Find the distribution of X2 .

b) Find the distribution of (X1 , X3 )T .

c) Which pairs of random variables Xi and Xj are independent?

d) Find the correlation (X1 , X3 ).

10.2 . Recall that if X Np (, ), then the conditional distribution of

X 1 given that X 2 = x2 is multivariate normal with mean 1 + 12 122 (x2
2 ) and covariance matrix 11 12 1
22 21 .
Let 12 = Cov(Y, X) and suppose Y and X follow a bivariate normal
distribution
310 10 Multivariate Models
     
Y 49 16 12
N2 , .
X 100 12 25

a) If 12 = 0, nd Y |X. Explain your reasoning.

b) If 12 = 10 nd E(Y |X).
c) If 12 = 10, nd Var(Y |X).

10.3. Let 12 = Cov(Y, X) and suppose Y and X follow a bivariate nor-

mal distribution
     
Y 15 64 12
N2 , .
X 20 12 81

a) If 12 = 10 nd E(Y |X).

b) If 12 = 10, nd Var(Y |X).

c) If 12 = 10, nd (Y, X), the correlation between Y and X.

10.4. Suppose that

X (1 )ECp (, , g1 ) + ECp (, c, g2 )
where c > 0 and 0 < < 1. Following Example 10.2, show that X has
an elliptically contoured distribution assuming that all relevant expectations
exist.

10.5. In Proposition 10.5b, show that if the second moments exist, then
can be replaced by Cov(X).

crancap hdlen hdht Data for 10.6

1485 175 132
1450 191 117
1460 186 122
1425 191 125
1430 178 120
1290 180 117
90 75 51

10.6 . The table (W ) above represents 3 head measurements on 6 people

and one ape. Let X1 = cranial capacity, X2 = head length, and X3 = head
height. Let x = (X1 , X2 , X3 )T . Several multivariate location estimators, in-
cluding the coordinatewise median and sample mean, are found by applying
a univariate location estimator to each random variable and then collecting
the results into a vector. a) Find the coordinatewise median MED(W ).
b) Find the sample mean x.
10.5 Problems 311

10.7. Using the notation in Proposition 10.6, show that if the second
moments exist, then

1
XX XY = [Cov(X)]
1
Cov(X, Y ).

10.8. Using the notation under Lemma 10.4, show that if X is elliptically
contoured, then the conditional distribution of X 1 given that X 2 = x2 is
also elliptically contoured.

10.9 . Suppose Y Nn (X, 2 I). Find the distribution of

(X X)1 X T Y if X is an n p full rank constant matrix and is a p 1
T

constant vector.

10.10. Recall that Cov(X, Y ) = E[(X E(X))(Y E(Y ))T ]. Using the
notation of Proposition 10.6, let (Y, X T )T be ECp+1 (, , g) where Y is a
random variable. Let the covariance matrix of (Y, X T ) be
   
T T Y Y Y X VAR(Y ) Cov(Y, X)
Cov((Y, X ) ) = c =
XY XX Cov(X, Y ) Cov(X)

where c is some positive constant. Show that E(Y |X) = + T X where

= Y T X and

= [Cov(X)]1 Cov(X, Y ).
10.11. (Due to R.D. Cook.) Let X be a p1 random vector with E(X) =
0 and Cov(X) = . Let B be any constant full rank p r matrix where
1 r p. Suppose that for all such conforming matrices B,

E(X|B T X) = M B B T X

where M B a p r constant matrix that depend on B.

Using the fact that B = Cov(X, B T X) = E(XX T B) =
E[E(XX T B|B T X)], compute B and show that M B = B(B T B)1 .
Hint: what acts as a constant in the inner expectation?

R Problems

Use the command source(G:/lregpack.txt) to download the func-

tions and the command source(G:/lregdata.txt) to download the data.
See Preface or Section 14.1. Typing the name of the lregpack function,
e.g. maha, will display the code for the function. Use the args command,
e.g. args(maha), to display the needed arguments for the function. For some
of the following problems, the R commands can be copied and pasted from
(http://lagrange.math.siu.edu/Olive/lreghw.txt) into R.
312 10 Multivariate Models

10.12. a) Download the maha function that creates the classical Maha-
lanobis distances.

b) Enter the following commands and check whether observations 140

look like outliers.

simx2 <- matrix(rnorm(200),nrow=100,ncol=2)

outx2 <- matrix(10 + rnorm(80),nrow=40,ncol=2)
outx2 <- rbind(outx2,simx2)
maha(outx2)

10.13 . a) Assuming that you have done the two source commands above
Problem 10.12 (and the library(MASS) command), type the command
ddcomp(buxx). This will make 4 DD plots based on the DGK, FCH, FMCD,
and median ball estimators. The DGK and median ball estimators are the
two attractors used by the FCH estimator. With the leftmost mouse button,
move the cursor to each outlier and click. This data is the Buxton (1920)
data and cases with numbers 61, 62, 63, 64, and 65 were the outliers with
head lengths near 5 feet. After identifying the outliers in each plot, hold the
rightmost mouse button down and click on Stop to advance to the next plot.
When done, hold down the Ctrl and c keys to make a copy of the plot. Then
paste the plot in Word.
b) Repeat a) but use the command ddcomp(cbrainx). This data is the
Gladstone (1905) data and some infants are multivariate outliers.
c) Repeat a) but use the command ddcomp(museum[,-1]). This data is the
Schaahausen (1878) skull measurements and cases 4860 were apes while
the rst 47 cases were humans.

Note: the DD plot is useful for detecting xoutliers: outliers in

the predictors. The median ball estimator likely has the most outlier re-
sistance but is not a consistent estimator of (E(x), c Cov(x)). The RFCH
or RMVN estimators are consistent estimators of (E(x), c Cov(x)) for some
constant c > 0 that depends on the distribution of x, if the nontrivial pre-
dictors xi are iid from a large class of elliptically contoured distributions. If
x contains some predictors that are not continuous random variables, such
as the constant 1 or gender, then let the continuous random variables be
collected into w, and make the DD plot using the w.
Chapter 11
Theory for Linear Models

Theory for linear models is used to show that linear models have good
statistical properties. Linear model theory previously proved in the text in-
cludes Propositions 2.1, 2.2, 2.3, 2.10, 3.1, 3.2, 4.1, 4.2, and Theorem 3.3.
Some matrix manipulations are illustrated in Example 4.1. Unproved results
include Propositions 2.4, 2.5, 2.6, 2.11, Theorems 2.6, 2.7, and 2.8.
Warning: This chapter is much harder than the previous chapters. Often
a linear model theory course is taught at the Masters level.

11.1 Projection Matrices and the Column Space

Vector spaces, subspaces, and column spaces should be familiar from linear
algebra, but are reviewed below.

Denition 11.1. A set V Rk is a vector space if for any vectors

x, y, z V, and scalars a and b, the operations of vector addition and scalar
multiplication are dened as follows:
1) (x + y) + z = x + (y + z).
2) x + y = y + x.
3) There exists 0 V such that x + 0 = x = 0 + x.
4) For any x V, there exists y = x such that x + y = y + x = 0.
5) a(x + y) = ax + ay.
6) (a + b)x = ax + by.
7) (ab) x = a(b x).
8) 1 x = x.

Hence for a vector space, addition is associative and commutative, there

is an additive identity vector 0, there is an additive inverse x for each
x V, scalar multiplication is distributive and associative, and 1 is the
scalar identity element.

Springer International Publishing AG 2017 313

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 11
314 11 Theory for Linear Models

Two important vector spaces are Rk and V = {0}. Showing that a set M
is a subspace is a common method to show that M is a vector space.

Denition 11.2. Let M be a nonempty subset of a vector space V. If i)

ax M x M and for any scalar a, and ii) x + y M x, y M, then
M is a vector space known as a subspace.

Denition 11.3. The set of all linear combinations of x 1 , . . . , xn is the

n
vector space known as span(x1 , . . . , xn ) = {y Rk : y = i=1 ai xi for some
constants a1 , . . . , an }.

k Let x1 , . . . , xk V. If scalars 1 , . . . , k not all

Denition 11.4.
zero such that i=1 i xi = 0, then x1 , . . . , xk are linearly dependent. If
k
i=1 i x i = 0 only if i = 0 i = 1, . . . , k, then x1 , . . . , xk are lin-
early independent. Suppose {x1 , . . . , xk } is a linearly independent set and
V = span(x1 , . . . , xk ). Then {x1 , . . . , xk } is a linearly independent spanning
set for V, known as a basis.

Denition 11.5. Let A = [a1 a2 . . . am ] be an n m matrix. The space

spanned by the columns of A = column space of A = C(A). Then C(A) =
{y Rn : y = Aw for some w Rm } = {y : y = w1 a1 + w2 a2 + + wm am
for some scalars w1 , . . . ., wm } = span(a1 , . . . , am ).

The space spanned by the rows of A is the row space of A. The row space
of A is the column space C(AT ) of AT . Note that

w1

m

Aw = [a1 a2 . . . am ] ... = wi a i .
wm i=1

With the design matrix X, dierent notation is used to denote the columns
of X since both the columns and rows X are important. Let
T
x1
..
X = [v 1 v 2 . . . v p ] = .
xTn
be an n p matrix. Note that C(X) = {y Rn : y = Xb for some b Rp }.
Hence Xb is a typical element of C(X) and Aw is a typical element of C(A).
Note that
T T
x1 x1 b b1
.. 
p
.. ..
Xb = . b = . = [v 1 v 2 . . . v p ] . = bi v i .
T T i=1
xn xn b bp
11.1 Projection Matrices and the Column Space 315

If the function X f (b) = Xb where the f indicates that the operation

X f : Rp Rn is being treated as a function, then C(X) is the range of X f .
Hence some authors call the column space of A the range of A.
Let B be n k, and let A be n m. One way to show C(A) = C(B)
is to show that i) x Rm , y Rk such that Ax = By C(B) so
C(A) C(B), and ii) y Rk , x Rm such that By = Ax C(A) so
C(B) C(A). Another way to show C(A) = C(B) is to show that a basis
for C(A) is also a basis for C(B).

Denition 11.6. The dimension of a vector space V = dim(V) = the

number of vectors in a basis of V. The rank of a matrix A = rank(A) =
dim(C(A)), the dimension of the column space of A. Let A be n m. Then
rank(A) = rank(AT ) min(m, n). If rank(A) = min(m, n), then A has full
rank, or A is a full rank matrix.

Denition 11.7. The null space of A = N (A) = {x : Ax = 0} = kernel

of A. The nullity of A = dim[N (A)]. The subspace V = {y Rk : y V}
is the orthogonal complement of V, where y V means y T x = 0 x V.
N (AT ) = [C(A)] , so N (A) = [C(AT )] .

Theorem 11.1: Rank Nullity Theorem. Let A be n m. Then

rank(A) + dim(N (A)) = m.

Generalized inverses are useful for the non-full rank linear model and for
dening projection matrices.

Denition 11.8. A generalized inverse of an n m matrix A is any

m n matrix A satisfying AA A = A.

Other names are conditional inverse, pseudo inverse, g-inverse, and p-

inverse. Usually a generalized inverse is not unique, but if A1 exists, then
A = A1 is unique.

Notation: G := A means G is a generalized inverse of A.

Recall that if A is idempotent, then A2 = A. A matrix A is tripotent if

A = A. For both these cases, A := A since AAA = A. It will turn out
3

that symmetric idempotent matrices are projection matrices.

Denition 11.9. Let V be a subspace of Rn . Then every y Rn can

be expressed uniquely as y = w + z where w V and z V . Let X =
[v 1 v 2 . . . v p ] be np, and let V = C(X) = span(v 1 , . . . , v p ). Then the nn
matrix P V = P X is a projection matrix on C(X) if P X y = w y Rn .
(Here y = w + z = wy + z y , so w depends on y.)

Note: Some authors call a projection matrix an orthogonal projection

matrix, and call an idempotent matrix a projection matrix.
316 11 Theory for Linear Models

Theorem 11.2. Projection Matrix Theorem. a) P X is unique.

b) P X = X(X T X) X T where (X T X) is any generalized inverse
of X T X.
c) A is a projection matrix on C(A) i A is symmetric and idempotent.
Hence P X is a projection matrix on C(P X ) = C(X), and P X is symmetric
and idempotent. Also, each column pi of P X satises P X pi = pi C(X).
d) I n P X is the projection matrix on [C(X)] .
e) A = P X i i) y C(X) implies Ay = y and ii) y C(X) implies
Ay = 0.
f) P X X = X, and P X W = W if each column of W C(X).
g) P X v i = v i .
h) If C(X R ) is a subspace of C(X), then P X P X R = P X R P X = P X R .
i) The eigenvalues of P X are 0 or 1.
j) Let tr(A) = trace(A). Then rank(P X ) = tr(P X ).
k) P X is singular unless X is a nonsingular nn matrix, and then P X = I n .
l) Let X = [Z X r ] where rank(X) = rank(X r ) = r so the columns of X r
form a basis for C(X). Then
 
0 0
0 (X Tr X r )1

is a generalized inverse of X T X, and P X = X r (X Tr X r )1 X Tr .

Some results from linear algebra are needed to prove parts of the above
theorem. Unless told otherwise, matrices in this text are real. Then the
eigenvalues of a symmetric matrix A are real. If A is symmetric, then
rank(A) = number of nonzero eigenvalues of A. Recall that if AB is
a square matrix, then tr(AB) = tr(BA). Similarly, if A1 is m1 m2 ,
A2 is m2 m3 , . . . , Ak1 is mk1 mk , and Ak is mk m1 , then
tr(A1 A2 Ak ) = tr(Ak A1 A2 Ak1 ) = tr(Ak1 Ak A1 A2 Ak2 ) =
= tr(A2 A3 Ak A1 ). Also note that a scalar is a 1 1 matrix, so
tr(a) = a.
For part d), note that if y = w + z, then (I n P X )y = z [C(X)] .
Hence the result follows from the denition of a projection matrix by inter-
changing the roles of w and z. Part e) follows from the denition of a pro-
jection matrix since if y C(X), then y = y + 0 where y = w and 0 = z. If
y C(X) then y = 0 + y where 0 = w and y = z. Part g) is a special case
of f). In k), P X is singular unless p = n since rank(X) = r p < n unless
p = n, and P X is an nn matrix. Need rank(P X ) = n for P X to be nonsin-
gular. For h), P X P X R = P X R by f) since each column of P X r C(P X ).
Taking transposes and using symmetry shows P X R P X = P X R . For i), if
is an eigenvalue of P X , then for some x = 0, x = P X x = P 2X x = 2 x
since P X is idempotent by c). Hence = 2 is real since P X is symmetric,
so = 0 or = 1. Then j) follows from i) since rank(P X ) = number of
nonzero eigenvalues of P X = tr(P X ).
11.2 Quadratic Forms 317

11.2 Quadratic Forms

Denition 11.10. Let A benan n n matrix and let x Rn . Then a

T n
quadratic form x Ax = i=1 j=1 aij xi xj , and a linear form is Ax.
Suppose A is a symmetric matrix. Then A is positive denite (A > 0)
if xT Ax > 0 x = 0, and A is positive semidenite (A 0) if
xT Ax 0 x.

Notation: The matrix A in a quadratic form xT Ax will be symmetric

unless told otherwise. Suppose B is not symmetric. Since the quadratic form
is a scalar, xT Bx = (xT Bx)T = xT B T x = xT (B+B T )x/2, and the matrix
A = (B + B T )/2 is symmetric. If A 0, then the eigenvalues i of A are
real and nonnegative. If A 0, let 1 2 n 0. If A > 0, then
n > 0. Some authors say symmetric A is nonnegative denite if A 0, and
that A is positive semidenite if A 0 and there exists a nonzero x such
that xT Ax = 0. Then A is singular.

The spectral decomposition theorem is very useful. One application for

linear models is dening the square root matrix. See Chapter 4.

Theorem 11.3: Spectral Decomposition Theorem. Let A be an n

n symmetric matrix with eigenvector eigenvalue pairs (1 , t1 ), (2 , t2 ), . . . ,
(n , tn ) where tTi ti = 1 and tTi tj = 0 if i = j for i = 1, . . . , n. Hence
Ati = i ti . Then the spectral decomposition of A is

n
A= i ti tTi = 1 t1 tT1 + + n tn tTn .
i=1

Let T = [t1 t2 tn ] be the n n orthogonal matrix with ith col-

umn ti . Then  T T T = T T T = I. Let = diag(1 , . . . , n ) and let 1/2 =
diag( 1 , . . . , n ). Then A = T T T .

a positive denite n n symmetric matrix with

Denition 11.11. If A is
n
spectral decomposition A = i=1 i ti tTi , then A = T T T and

n
1 T
A1 = T 1 T T = ti ti .

i=1 i

The square root matrix A1/2 = T 1/2 T T is a positive denite symmetric

matrix such that A1/2 A1/2 = A.

The following theorem is often useful. Both the expected value and trace
are linear operators. Hence tr(A + B) = tr(A) + tr(B), and E[tr(X)] =
tr(E[X]) when the expected value of the random matrix X exists.
318 11 Theory for Linear Models

Theorem 11.4: expected value of a quadratic form. Let x be a ran-

dom vector with E(x) = and Cov(x) = . Then

E(xT Ax) = tr(A) + T A.

Proof. Two proofs are given. i) Searle (1971, p. 55): Note that E(xxT ) =
+ T . Since the quadratic form is a scalar and the trace is a linear
operator, E[xT Ax] = E[tr(xT Ax)] = E[tr(AxxT )] = tr(E[AxxT ]) =
tr(A + AT ) = tr(A) + tr(AT ) = tr(A) + T A.
nii) Graybill (1976, p. 140):
n  Using E(xi xj ) = ij + i j , E[xT Ax] =
n n
j=1 aij (ij + i j ) = tr(A) + A. 
T
i=1 j=1 aij E(xi xj ) = i=1

Much of the theoretical results for quadratic forms assume that the ei
are iid N (0, 2 ). These exact results are often special cases of large sample
theory that holds for a large class of iid zero mean error distributions that
have V (ei ) 2 . For linear models, Y is typically an n 1 random vector.
The following theorem from statistical inference will be useful:

Theorem 11.5. Suppose x y, g(x) is a function of x alone, and h(y)

is a function of y alone. Then g(x) h(y).

The following theorem shows that independence of linear forms implies

independence of quadratic forms.

Theorem 11.6. If A and B are symmetric matrices and AY BY , then

Y T AY Y T BY .

Proof. Let g(AY ) = Y T AT A AY = Y T AA AY = Y T AY , and

let h(BY ) = Y T B T B BY = Y T BB BY = Y T BY . Then the result
follows by Theorem 11.5. 

Theorem 11.7. Let Y Nn (, ). Let u = AY and w = BY . Then

AY BY i Cov(u, w) = AB T = 0 i BAT = 0. Note that if
= I n , then AY BY i AB T = 0 i BAT = 0.
2

Proof. Note that

     
u AY A
= = Y
w BY B

has a multivariate normal distribution. Hence AY BY i Cov(u, w) = 0.

Taking transposes shows Cov(u, w) = AB T = 0 i BAT = 0. 

One of the most useful theorems for proving that Y T AY Y T BY is

Craigs Theorem. Taking transposes shows AB = 0 i BA = 0. Note
that if AB = 0, then () holds. Note AB = 0 is a sucient condition
for Y T AY Y T BY if 0, but necessary and sucient if > 0.
11.2 Quadratic Forms 319

Theorem 11.8: Craigs Theorem. Let Y Nn (, ).

a) If > 0, then Y T AY Y T BY i AB = 0 i BA = 0.
b) If 0, then Y T AY Y T BY if AB = 0 (or if BA = 0).
c) If 0, then Y T AY Y T BY i
() AB = 0, AB = 0, BA = 0, and T AB = 0.

Proof. For a) and b), AB = 0 implies Y T AY Y T BY by c) or

by Theorems 11.5, 11.6, and 11.7. See Reid and Driscoll (1988) for why
Y T AY Y T BY implies AB = 0 in a).
c) See Driscoll and Krasnicka (1995).

For the following theorem, note that if A = AT = A2 , then A is a

projection matrix since A is symmetric and idempotent. An n n projection
matrix A is not a full rank matrix unless A = I n . See Theorem 11.2 j) and
k). Often results are given for Y Nn (0, I), and then the Y Nn (0, 2 I)
case is handled as in b) below, since Y / Nn (0, I).

Theorem 11.9. Let A = AT be symmetric.

a) Let Y Nn (0, I). Then Y T AY 2r i A is idempotent of rank
r = tr(A) since then A is a projection matrix.
b) Let Y Nn (0, 2 I). Then

Y T AY
2r or Y T AY 2 2r
2
i A is idempotent of rank r.
c) If Y Nn (0, ) where > 0, then Y T AY 2r i A is idempotent
of rank r = rank(A).

Note that a) is a special case of c). To see that b) holds, note Z = Y /

Nn (0, I). Hence by a)

Y T AY
= Z T AZ 2r
2
i A is idempotent of rank r.
The following theorem is a corollary of Craigs Theorem.

Theorem 11.10. Let Y Nn (0, I n ), with A and B symmetric. If

Y T AY 2r and Y T BY 2d , then Y T AY Y T BY i AB = 0.

Theorem 11.11. If Y Nn (, ) with > 0, then the population

squared Mahalanobis distance (Y )T 1 (Y ) 2n .

Proof. Let Z = 1/2 (Y ) Nn (0, I). Then Z = (Z1 , .

. . , Zn )T where
1 n
the Zi are iid N (0, 1). Hence (Y ) (Y ) = Z Z = i=1 Zi2 2n .
T T


320 11 Theory for Linear Models

For large sample theory, the noncentral 2 distributionis important.

n
If Z1 , . . . , Zn are independent N (0, 1) random variables, then i=1 Zi2 2n .
2 n 2
The noncentral (n, ) distribution is the distribution of i=1 Yi where
Y1 , . . . , Yn are independent N (i , 1) random variables. Note that if Y
N (, 1), then Y 2 2 (n = 1, = 2 /2), and if Y N ( 2, 1), then
Y 2 2 (n = 1, ).
Denition 11.12. Suppose Y1 , . . . , Yn are independent N (i , 1) random
n
variables so that Y = (Y1 , . . . , Yn )T Nn (, I n ). Then Y T Y = i=1 Yi2
2 T 2
(n, = /2), a noncentral (n, ) distribution, n with n degrees of free-
dom and noncentrality parameter = T /2 = 12 i=1 2i 0. The noncen-
trality parameter = T = 2 is also used. If W 2n , then W 2 (n, 0)
so = 0. The 2n distribution is also called the central 2 distribution.
Some of the proof ideas for the following theorem came from Marden
(2012, pp. 48, 9697). Recall that if Y1 , . . . , Yk are independent with moment
k
generating functions (mgfs) mYi (t), then the mgf of i=1 Yi is mk Yi (t) =
i=1
+k
mYi (t). If Y 2 (n, ), then the probability density function (pdf) of Y
i=1
is rather hard to use, but is given by


e j y 2 +j1 ey/2
n

f (y) = n = p (j)fn+2j (y)

j=0
j! 2 2 +j ( n2 + j) j=0

where p (j) = P (W = j) is the probability mass function of a Poisson()

random variable W , and fn+2j (y) is the pdf of a 2n+2j random variable. If
= 0, dene 0 = 1 in the rst sum, and p0 (0) = 1 with p0 (j) = 0 for
j > 0 in the second sum. For computing moments and the moment gen-
erating function,( the integration
and summation
( operations
 can be inter-

changed. Hence 0 f (y)dy = j=0 p (j) 0 fn+2j (y)dy = j=0 p (j) = 1.
Similarly, if mn+2j (t) = (1 2t)(n+2j)/2 is the mgf of a 2n+2j ran-
(
dom variable, then the mgf of Y is mY (t) = E(etY ) = 0 ety f (y)dy =
 ( ty 
j=0 p (j) 0 e fn+2j (y)dy = j=0 p (j)mn+2j (t).

Theorem 11.12. a) If Y 2 (n, ), then the moment generating func-

tion of Y is mY (t) = (1 2t)n/2 exp([1 (1 2t)1 ]) =
(1 2t)n/2 exp[2t/(1 2t)] for t < 0.5.
b) If Yi %2 (ni , i ) are independent
& for i = 1, . . . , k, then
k k k
i=1 Yi
2
i=1 ni , i=1 i .
c) If Y 2 (n, ), then E(Y ) = n + 2 and V (Y ) = 2n + 8.

Proof. Two proofs are given. a) i) From the above remarks, and
 
xj e j
using ex = , mY (t) = (1 2t)(n+2j)/2 =
j=0
j! j=0
j!
% &j
e
 12t
(1 2t)n/2 =
j=0
j!
11.2 Quadratic Forms 321
   
2t
(1 2t)n/2 exp + = (1 2t)n/2 exp .
1 2t 1 2t

ii) Let W N ( , 1) where = 2. Then W 2 2 (1, /2) = 2 (1, ).
Let W X where X 2n1 2 (n 1, 0), and let Y = W 2 + X 2 (n, )
by b). Then mW 2 (t) =
,  
2 2 1 1
E(etW ) = etw exp (w )2 dw =
2 2
,  
1 2 1
exp tw2 (w2 2 w + ) dw =
2 2 2
,  
1 1 2
exp (w 2tw2 2 w + ) dw =
2 2
,   ,  
1 1 2 1 1
exp (w (1 2t) 2 w + ) dw = exp A dw
2 2 2 2

where A = [ 1 2t (w b)]2 + c with

2t
b= and c =
1 2t 1 2t
after algebra. Hence m2W (t) =
 ,   

c/2 1 1 1 1 1 c/2 1
e  exp (w b) dw = e
2
1 2t 2 1 1
2 12t 1 2t
12t

(
since the integral = 1 = f (w)dw where f (w) is the N (b, 1/(1 2t)) pdf.

Thus  
1 t
mW 2 (t) = exp .
1 2t 1 2t
So mY (t) = mW 2 +X (t) = mW 2 (t)mX (t) =
  (n1)/2  
1 t 1 1 t
exp = exp =
1 2t 1 2t 1 2t (1 2t)n/2 1 2t
 
n/2 2t
(1 2t) exp .
1 2t
b) i) By a), mk Yi (t) =
i=1

+
k +
k
mYi (t) = (1 2t)ni /2 exp(i [1 (1 2t)1 ]) =
i=1 i=1
322 11 Theory for Linear Models
 
k 
k
1
(1 2t) i=1 ni /2 exp i [1 (1 2t) ] ,
i=1
 

k 
k
2
the ni , i mgf.
i=1 i=1
ii) Let Yi = where the Z i Nni (i , I ni ) are independent. Let
Z Ti Z i

Z1 1
Z2 2

Z = . Nk ni . , I k ni Nk ni (Z , I k ni ).
.. i=1 .. i=1 i=1 i=1

Zk k
 

k 
k 
k
Then Z Z = T
Z Ti Z i = Yi 2 ni , Z where
i=1 i=1 i=1

TZ Z 
k
T 
k
i i
Z = = = i .
2 i=1
2 i=1

c) i) Let W 2 (1, ) X 2n1 2 (n 1, 0). Then by b) Y =

W +X 2 (n, ). Let Z N (0, 1)
and = 2. Then +Z N ( , 1), and
W = ( +Z)2 . Thus E(W ) = E[( +Z)2 ] = +2 E(Z)+E(Z 2 ) = +1.
Using the binomial theorem
n  
n n i ni
(x + y) = xy
i=0
i

with x = , y = Z, and n = 4, E(W 2 ) = E[( + Z)4 ] =

E[ 2 + 4 3/2 Z + 6Z 2 + 4 Z 3 + Z 4 ] = 2 + 6 + 3

since E(Z) = E(Z 3 ) = 0, and E(Z 4 ) = 3 by Problem 11.8. Hence V (W ) =

E(W 2 ) [E(W )]2 = 2 + 6 + 3 ( + 1)2 = 2 + 6 + 3 2 2 1 = 4 + 2.
Thus E(Y ) = E(W ) + E(X) = + 1 + n 1 = n + = n + 2, and
V (Y ) = V (W ) + V (X) = 4 + 2 + 2(n 1) = 8 + 2n.
ii) Let Zi N (i , 1) so E(Zi2 ) = 2 + 2i = 1 + 2i . By Problem 11.8,
E(Zi3 ) = 3i +  3i , and E(Zi4 ) = 4i + 62i + 3. Hence Y  2 (n, ) where
n n
i=1 Zi where Z Nn (, I). So
T 2 2
Y = Z Z =
 E(Y ) = i=1 E(Zi ) =
n 2 T n 2
i=1 (1 + i ) = n + = n + 2, and V (Y ) = i=1 V (Zi ) =


n 
n 
n
[E(Zi4 ) (E[Zi2 ])2 ] = [4i + 62i + 3 4i 22i 1] = [42i + 2]
i=1 i=1 i=1
11.3 Least Squares Theory 323

= 2n + 4T = 2n + 8. 

For the following theorem, see Searle (1971, p. 57). Most of the results in
Theorem 11.14 are corollaries of Theorem 11.13. Recall that the matrix in a
quadratic form is symmetric, unless told otherwise.

Theorem 11.13. If Y Nn (, ) where > 0, then Y T AY

(rank(A), T A/2) i A is idempotent.
2

Theorem 11.14. a) If Y Nn (0, ) where is a projection matrix,

then Y T Y 2 (rank()) where rank() = tr().
b) If Y Nn (0, I), then Y T AY 2r i A is idempotent with rank(A) =
tr(A) = r.
c) If Y Nn (0, ) where > 0, then Y T AY 2r i A is idempotent
and rank(A) = r = rank(A).
 
Y TY T
d) If Y Nn (0, 2 I), then 2
n, .
2 2 2
e) If Y Nn (, I), then Y T AY 2 (r, T A/2) i A is idempotent
with rank(A) = tr(A) = r.
 
Y T AY T A
f) If Y Nn (, 2 I), then 2
r, i A is idempotent
2 2 2
with rank(A) = tr(A) = r.

The following theorem is useful for constructing ANOVA tables. See Searle
(1971, pp. 6061).

Theorem 11.15. Generalized Cochrans Theorem. Let Y

Nn (, ). Let Ai = ATi have rank ri for i = 1, . . . , k, and let
k
A = i=1 Ai = A
T
have rank r. Then Y T Ai Y 2 (ri , T Ai /2),
and the Y T Ai Y are independent, and Y T AY 2 (r, T A/2), i
I) any 2 of a) Ai are idempotent i,
b) Ai Aj = 0 i < j,
c) A is idempotent
k
are true; or II) c) is true and d) r = i=1 ri ;
or III) c) is true and e) A1 , .., Ak1 are idempotent and Ak 0 is
singular.

11.3 Least Squares Theory

Denition 11.13. Estimating equations are used to nd estimators of

unknown parameters. The least squares criterion and log likelihood for max-
imum likelihood estimators are important examples.
324 11 Theory for Linear Models

Estimating equations are often used with a model, like Y = X + e,

and often have a variable that is used in the equations to nd the es-
timator of the vector of parameters in the model. For example, the log
likelihood log(L(, 2 )) has and 2 as variables for a parametric statistical
model where and 2 are xed unknown parameters, and maximizing the
log likelihood with respect to these variables gives the maximum likelihood
estimators of the parameters and 2 . So the term is both a variable in
the estimating equations, which could be replaced by another variable such
as , and a vector of parameters in the model. In the theorem below, we
could replace by where is a vector of parameters in the linear model
and a variable in the least squares criterion which is an estimating equation.

Theorem 11.16. Let = X C(X) where Yi = xTi + ri () and the

residual ri () depends on . The least squares estimator is the value
n R
p
of that minimizes the least squares criterion
r
i=1 i
2
() = Y X2 .

Proof. Following Seber and Lee (2003, pp. 3636), let Y = = P X Y

C(X), r = (I P X )Y [C(X)] , and C(X). Then (Y )T (
) = (Y P X Y )T (P X Y P X ) = Y T (I P X )P X (Y ) = 0 since
P X = . Thus Y 2 = (Y + )T (Y + ) =

Y 2 +  2 + 2(Y )T ( ) Y 2

with equality i  2 = 0 i = = X. Since = X the result

follows. 

Denition 11.14. The normal equations are

X T X = X T Y .

To see that the normal equations hold, note that r = Y Y C(X)

since X(Y Y ) = 0. Thus r [C(X)] = N (X T ), and X T (Y Y ) = 0.
Hence X T Y = X T X = X T Y .

The maximum likelihood estimator uses the log likelihood as an estimating

equation. Note that it is crucial to observe that the likelihood function is a
function of (and that y1 , . . . , yn act as xed constants). Also, if the MLE
exists, then , the parameter space.

Denition 11.15. Let f (y|) be the joint pdf of Y1 , . . . , Yn . If Y = y is

observed, then the likelihood function L() = f (y|). For each sample
point y = (y1 , . . . , yn ), let (y) be a parameter value at which L(|y) attains
its maximum as a function of with y held xed. Then a maximum likelihood
estimator (MLE) of the parameter based on the sample Y is (Y ).
11.3 Least Squares Theory 325

Denition 11.16. Let the log likelihood of 1 and 2 be log[L( 1 , 2 )].

If 2 is the MLE of 2 , then the log prole likelihood is log[Lp ( 1 )] =
log[L( 1 , 2 )].

We can often x and then show is the MLE by direct maximization.

Then the MLE or 2 can be found by maximizing the log prole likelihood
function log[Lp ()] or log[Lp ( 2 )] where Lp () = L(, = ).

Remark 11.1. a) Know how to nd the max and min of a function h that
is continuous on an interval [a,b] and dierentiable on (a, b). Solve h (x) 0
and nd the places where h (x) does not exist. These values are the critical
points. Evaluate h at a, b, and the critical points. One of these values will
be the min and one the max.
b) Assume h is continuous. Then a critical point o is a local max of h()
if h is increasing for < o in a neighborhood of o and if h is decreasing for
> o in a neighborhood of o . The rst derivative test is often used.
d2
c) If h is strictly concave h() < 0 for all , then any local max
d2
of h is a global max.
d2
d) Suppose h (o ) = 0. The 2nd derivative test states that if 2 h(o ) < 0,
d
then o is a local max.
e) If h() is a continuous function on an interval with endpoints a < b
(not necessarily nite), and dierentiable on (a, b) and if the critical point
is unique, then the critical point is a global maximum if it is a local
maximum (because otherwise there would be a local minimum and the critical
point would not be unique). To show that is the MLE (the global maximizer
of h() = log L()), show that log L() is dierentiable on (a, b). Then show
d
that is the unique solution to the equation log L() = 0 and that the
d
2
d
2nd derivative evaluated at is negative: 2 log L()| < 0. Similar remarks
d
hold for nding 2 using the prole likelihood.

Theorem 11.17. Let Y = X + e = Y + r where X is full rank, and

Y Nn (X, 2 I). Then the MLE of is the least squares estimator and
the MLE of 2 is RSS/n = (n p)M SE/n.

Proof. The likelihood function is

 
1
L(, 2 ) = (2 2 )n/2 exp y X2
.
2 2

For xed 2 , maximizing the likelihood is equivalent to maximizing

 
1
exp y X ,2
2 2
326 11 Theory for Linear Models

which is equivalent to minimizing y X2 . But the least squares estimator

minimizes y X2 by Theorem 11.16. Hence is the MLE of .
Let Q = y X 2 . Then the MLE of 2 can be found by maximizing
the log prole likelihood log(Lp ( 2 )) where
 
2 1 1
Lp ( ) = exp Q .
(2 2 )n/2 2 2

Let = 2 . Then
n 1
log(Lp ( 2 )) = c log( 2 ) 2 Q,
2 2
and
n 1
log(Lp ( )) = c log( ) Q.
2 2
Hence
d log(Lp ( )) n Q set
= + 2 = 0
d 2 2
or n + Q = 0 or n = Q or
n 2
Q r np
= = 2 = i=1 i = M SE,
n n n
which is a unique solution.
Now
-
d2 log(Lp ( )) n 2Q -- n 2n n
2
= 2
3- = 2 3 = 2 < 0.
d 2 2 = 2 2 2

Thus by Remark 11.1, 2 is the MLE of 2 . 

There are two ways to compute . Use = (X T X)1 X T Y , and use
sample covariance matrices. The population OLS coecients are dened be-
low. Let xTi = (1 uTi ) where ui is the vector of nontrivial predictors. Let
1
n
Xjk = X ok = uok for k = 2, . . . , p. The subscript ok means sum over
n j=1
the rst subscript j. Let u = (uo,2 , . . . , uo,p )T be the sample mean of the
ui . Note that regressing on u is equivalent to regressing on x if there is an
intercept 1 in the model.
Denition 11.17. Using the above notation, let xTi = (1 uTi ), and
let T = (1 TS ) where 1 is the intercept and the slopes vector S =
(2 , . . . , p )T . Let the population covariance matrices

Cov(u) = E[(u E(u))(u E(u))T ] = u , and

Cov(u, Y ) = E[(u E(u))(Y E(Y ))] = uY .

11.3 Least Squares Theory 327

Then the population coecients from an OLS regression of Y on u (even if

a linear model does not hold) are

1 = E(Y ) TS E(u) and S = 1

u uY .
Denition 11.18. Let the sample covariance matrices be

1  1 
n n
u = (ui u)(ui u)T and uY = (ui u)(Yi Y ).
n 1 i=1 n 1 i=1

Let the method of moments or maximum likelihood estimators be u =

1 1 1
n n n
(ui u)(ui u)T and uY = (ui u)(Yi Y ) = ui Yi u Y .
n i=1 n i=1 n i=1

P
Denition 11.19. The notation as n means that is a
consistent estimator of , or, equivalently, that converges in probability
to .

Lemma 11.18:
 Seber
 and
 Lee (2003,  p. 106). Let X = (1 X 1 ).
nY nY n nuT
T
Then X Y = = n , XT X = ,
T
X1 Y i=1 ui Yi nu X T1 X 1
 
1
+ uT D 1 u uT D 1
and (X T X)1 = n
D 1 u D 1
1
where the (p 1) (p 1) matrix D 1 = [(n 1) u ]1 = u /(n 1).

Theorem 11.19. Second way to compute : Suppose that wi =

(Yi , uTi )T are iid random vectors such that Y2 , 1
u , and uY exist. Then
T P
1 = Y S u 1 and
n 1 1 1 P
S = u uY = u uY = u uY S as n .
n1
The above theorem can be proved using Lemma 11.18 and using the fact
that the sample covariance matrices are consistent estimators of the popula-
tion covariance matrices. It is important to note that this result is for iid wi
with second moments: a linear model Y = X + e does not need to hold.
Also, X is a random matrix, and the least squares regression is conditional
on X. Some properties of the least squares estimators and related quantities
are given below, where X is a constant matrix.

Theorem 11.20. Let Y = X + e = Y + r where X is full rank, E(e) =

0, and Cov(e) = 2 I. Let P = P X be the projection matrix on C(X) so
Y = P X, r = Y Y = (I P )Y , and P X = X so X T P = X T .
i) The predictor variables and residuals are orthogonal. Hence the columns
328 11 Theory for Linear Models

of X and the residual vector are orthogonal: X T r = 0.

ii) E(Y ) = X.
iii) Cov(Y ) = Cov(e) = 2 I.
iv) The tted values and residuals are uncorrelated: Cov(r, Y ) = 0.
v) The least squares estimator is an unbiased estimator of : E() = .
vi) Cov() = 2 (X T X)1 .
Proof. i) X T r = X T (I P )Y = 0Y = 0, while ii) and iii) are immedi-
ate.
iv) Cov(r, Y ) = E([r E(r)][Y E(Y )]T ) =

E([(I P )Y (I P )E(Y )][P Y P E(Y )]T ) =

E[(I P )[Y E(Y )][Y E(Y )]T P ] = (I P ) 2 IP = 2 (I P )P = 0.

v) E() = E[(X T X)1 X T Y ] = (X T X)1 X T E[Y ] = (X T X)1 X T X
= .
vi) Cov() = Cov[(X T X)1 X T Y ] = Cov(AY ) = ACov(Y )AT =

2 (X T X)1 X T IX(X T X)1 = 2 (X T X)1 . 

Denition 11.20. Let a, b, and c be n 1 constant vectors. A linear

estimator aT Y of cT is the best linear unbiased estimator (BLUE) of cT
if E(aT Y ) = cT , and for any other unbiased linear estimator bT Y of cT ,
V ar(aT Y ) V ar(bT Y ).
The following theorem is useful for nding the BLUE when X has full
rank. Note that if Z is a random variable, then the covariance matrix of Z
is Cov(Z) = Cov(Z, Z) = V (Z). Note that the theorem shows that cT X =
aT is the BLUE of cT X = aT where aT = cT X and = X.
Theorem 11.21. Gauss Markov Theorem. Let Y = X + e where
X is full rank, E(e) = 0, and Cov(e) = 2 I. Then aT is the BLUE of aT
for every constant p 1 vector a.
Proof. (Guttman (1982, pp. 141142):) Note that aT (X T X)1 X T Y =
a = dT Y is linear, and by Theorem 11.20 vi), V (aT ) = V (dT Y ) =
T

Cov(aT ) = aT Cov()a = 2 aT (X T X)1 a. (11.1)

Let cT Y be any other linear unbiased estimator of aT . Then E(cT Y ) =

aT = cT E(Y ) = cT X for any Rp , the parameter space of . Hence
aT = cT X. Recall that

V (Z X) = Cov(Z X) = V (Z) + V (X) 2Cov(Z, X). (11.2)

Now Cov(cT Y , aT ) = Cov(cT Y , dT Y ) = cT Cov(Y )d = 2 cT d =

2 cT X(X T X)1 a = 2 aT (X T X)1 a = V (aT ) by (11.1). Hence
11.3 Least Squares Theory 329

Cov(cT Y , dT Y ) = V (aT ) = V (dT Y ). (11.3)

By (11.2), 0 V (cT Y aT ) = V (cT Y dT Y ) = V (cT Y ) + V (dT Y )

2Cov(cT Y , dT Y ) = V (cT Y ) + V (dT Y ) 2V (dT Y ) = V (cT Y ) V (dT Y )
by (11.3). Thus V (cT Y ) V (dT Y ) = V (aT ). 

The following theorem gives some properties of the least squares estimators
and MSE under the normal least squares model. Similar properties will be
developed without the normality assumption.

Theorem 11.22. Suppose Y = X + e where X is full rank, e

Nn (0, 2 I), and Y Nn (X, 2 I).
a) Np (, 2 (X T X)1 ).
( )T X T X( )
b) 2p .
2
c) M SE.
RSS (n p)M SE
d) = 2np .
2 2

Proof. Let P = P X .
a) Since A = (X T X)1 X T is a constant matrix,

= AY Np (AE(Y ), ACov(Y )AT )

Np ((X T X)1 X T X, 2 (X T X)1 X T IX(X T X)1 )

Np (, 2 (X T X)1 ).
b) The population Mahalanobis distance of is

( )T X T X( )
= ( )T [Cov()]1 ( ) 2p
2
by Theorem 11.11.
c) Since Cov(, r) = Cov((X T X)1 X T Y , (I P )Y ) =
2 (X T X)1 X T I(I P ) = 0, r. Thus RSS = r2 , and M SE.
d) Since P X = X and X T P = X T , it follows that X T (I P ) = 0 and
(I P )X = 0. Thus RSS = r T r = Y T (I P )Y =

(Y X)T (I P )(Y X) = eT (I P )e.

Since e Nn (0, 2 I), then by Theorem 11.9 b), eT (I P )e/ 2 2np

where n p = rank(I P ) = tr(I P ). 
330 11 Theory for Linear Models

11.3.1 Hypothesis Testing

Suppose Y = X + e where rank(X) = p, E(e) = 0 and Cov(e) = 2 I. Let

L be an r p constant matrix with rank(L) = r, let c be an r 1 constant
vector, and consider testing H0 : L = c. First theory will be given for when
e Nn (0, 2 I). The large sample theory will be given for when the iid zero
mean ei have V (ei ) = 2 . Note that the normal model will satisfy the large
sample theory conditions.
The partial F test and its special cases the ANOVA F test and the Wald
t test use c = 0. Let the full model use Y , x1 1, x2 , . . . , xp , and let
the reduced model use Y , x1 = xj1 1, xj2 , . . . , xjk where {j1 , . . . , jk }
{1, . . . , p} and j1 = 1. Here 1 k < p, and if k = 1, then the model is
Yi = 1 + ei . Hence the full model is Yi = 1 + 2 xi,2 + + p xi,p + ei , while
the reduced model is Yi = 1 +j2 xi,j2 + +jk xi,jk +ei . In matrix form, the
full model is Y = X + e and the reduced model is Y = X R R + eR where
the columns of X R are a proper subset of the columns of X. i) The partial
F test has H0 : jk+1 = = jp = 0, or H0 : the reduced model is good, or
H0 : L = 0 where L is a (p k) p matrix where the ith row of L has a
1 in the jk+i th position and zeroes elsewhere. In particular, if 1 , . . . , k are
the only i in the reduced model, then L = [0 I pk ] and 0 is a (p k) k
matrix. Hence r = p k = number of predictors in the full model but not in
the reduced model. ii) The ANOVA F test is the special case of the partial
F test where the reduced model is Yi = 1 +i . Hence H0 : 2 = = p = 0,
or H0 : none of the nontrivial predictors x2 , . . . , xp are needed in the linear
model, or H0 : L = 0 where L = [0 I p1 ] and 0 is a (p 1) 1 vector.
Hence r = p 1. iii) The Wald t test uses the reduced model that deletes
the jth predictor from the full model. Hence H0 : j = 0, or H0 : the jth
predictor xj is not needed in the linear model given that the other predictors
are in the model, or H0 : Lj = 0 where Lj = [0, . . . , 0, 1, 0, . . . , 0] is a 1 p
row vector with a 1 in the jth position for j = 1, . . . , p. Hence r = 1.
A way to get the test statistic FR for the partial F test is to t the
full model and the reduced model. Let RSS be the RSS of the full model,
and let RSS(R) be the RSS of the reduced model. Similarly, let M SE and
M SE(R) be the MSE of the full and reduced models. Let dfR = n k and
dfF = n p be the degrees of freedom for the reduced and full models. Then
RSS(R) RSS
FR = where r = dfR dfF = p k = number of predictors
rM SE
in the full model but not in the reduced model.
If Np (, 2 (X T X)1 ), then

L c Nr (L c, 2 L(X T X)1 LT ).

If H0 is true, then L c Nr (0, 2 L(X T X)1 LT ), and by Theorem 11.11

1
rF1 = (L c)T [L(X T X)1 LT ]1 (L c) 2r .
2
11.3 Least Squares Theory 331

D
Let rFR = 2 rF1 /M SE. If H0 is true, rFR 2r for a large class of zero
mean error distributions. See Theorem 11.25 c).
D
Denition 11.21. If Z n Z as n , then Z n converges in distri-
bution to the random vector Z, and Z is the limiting distribution of Z n
means that the distribution of Z is the limiting distribution of Z n . The
D
notation Z n Nk (, ) means Z Nk (, ).

Remark 11.2. a) Z is the limiting distribution of Z n , and does not de-

pend on the sample size n (since Z is found by taking the limit as n ).
D
b) When Z n Z, the distribution of Z can be used to approximate
probabilities P (Z n c) P (Z c) at continuity points c of the cdf FZ (z).
Often the limiting distribution is a continuous distribution, so all points c
are continuity points.
D
c) Often the two quantities Z n Nk (, ) and Z n Nk (, ) behave
similarly. A big dierence is that the distribution on the RHS (right-hand
D D
side) can depend on n for but not for . In particular, if Z n Nk (, ),
D
then AZ n + b Nm (A + b, AAT ), provided the RHS does not depend
on n, where A is an m k constant matrix and b is an m 1 constant vector.
d) We often want a normal approximation where the RHS can depend on n.
Write Z n ANk (, ) for an approximate multivariate normal distribution
where the RHS may depend on n. For normal linear model, if e Nn (0, 2 I),
then Np (, 2 (X T X)1 ). If the ei are iid with E(ei ) = 0 and V (ei ) =
2 , use the multivariate normal approximation ANp (, 2 (X T X)1 ) or
ANp (, M SE(X T X)1 ). The RHS depends on n since the number of
rows of X is n.

Theorem 11.23. Suppose n and are positive denite and symmetric.

D P 1/2 D
If W n Nk (, ) and n , then Z n = n (W n ) Nk (0, I),
1 D
and Z Tn Z n = (W n )T n (W n ) 2k .

Theorem 11.24. If Wn Fr,dn where the positive integer dn as

D
n , then rWn 2r .

Proof. If X1 2d1 X2 2d2 , then

X1 /d1
Fd1 ,d2 .
X2 /d2
k
If Ui 21 are iid, then i=1 Ui 2k . Let d1 = r and k = d2 = dn . Hence if
X2 2dn , then
d n
X2 Ui P
= i=1 = U E(Ui ) = 1
dn dn
332 11 Theory for Linear Models

D
by the law of large numbers. Hence if W Fr,dn , then rWn 2r . 

By the LS CLT (Theorem 2.8),

D
n( ) Np (0, 2 W ) (11.4)

where X T X/n W 1 . Equivalently,

D
(X T X)1/2 ( ) Np (0, 2 I p ). (11.5)

If = 2 W , then n = nM SE(X T X)1 . Hence

ANp (, M SE(X T X)1 ), and

1
(L c)T [L(X T X)1 LT ]1 (L c) 2r
D
rFR = (11.6)
M SE
as n .

Denition 11.22. A test with test statistic Tn is a large sample right tail
test if the test rejects H0 if Tn > an and P (Tn > an ) = n as n
when H0 is true.

Typically we want 0.1, and the values = 0.05 or = 0.01 are

common. (An analogy is a large sample 100(1 )% condence interval or
prediction interval.)

Remark 11.3. Suppose P (W 2q (1 )) = 1 and P (W > 2q (1

)) = where W 2q . Suppose P (W Fq,dn (1 )) = 1 when W
Fq,dn . Also write 2q (1 ) = 2q,1 and Fq,dn (1 ) = Fq,dn ,1 . Suppose
P (W > z1 ) = when W N (0, 1), and P (W > tdn ,1 ) = when
W td n .
i) Theorem 11.24 is important because it can often be shown that a statistic
D
Tn = rWn 2r when H0 is true. Then tests that reject H0 when Tn >
2r (1 ) or when Tn /r = Wn > Fr,dn (1 ) are both large sample right
tail tests if the positive integer dn as n . Large sample F tests
and intervals are used instead of 2 tests and intervals since the F tests and
intervals are more accurate for moderate n.
D
ii) An analogy is that if test statistic Tn N (0, 1) when H0 is true, then
tests that reject H0 if Tn > z1 or if Tn > tdn ,1 are both large sample
right tail tests if the positive integer dn as n . Large sample t
tests and intervals are used instead of Z tests and intervals since the t tests
and intervals are more accurate for moderate n.
iii) Often n 10p starts to give good results for the OLS output for error
distributions not too far from N (0, 1). Larger values of n tend to be needed
if the zero mean iid errors have a distribution that is far from a normal
distribution. Also see Proposition 2.5.
11.3 Least Squares Theory 333

Theorem 11.25, Partial F Test Theorem. Suppose H0 : L = 0 is

true for the partial F test. Under the OLS full rank model, a)
1
FR = (L)T [L(X T X)1 LT ]1 (L).
rM SE
b) If e Nn (0, 2 I), then FR Fr,np .
D
c) For a large class of zero mean error distributions rFR 2r .
d) The partial F test that rejects H0 : L = 0 if FR > Fr,np (1 ) is a
large sample right tail test for the OLS model for a large class of zero mean
error distributions.

Proof sketch. a), b) Let Y Nn (X, 2 I n ) where X is an np matrix

of rank p. Let X = [X 1 X 2 ] and = ( T1 T2 )T where X 1 is an n k
matrix and r = p k. Consider testing H0 : 2 = 0. (The columns of X can
be rearranged so that H0 corresponds to the partial F test.) Let P be the
projection matrix on C(X). Then r T r = Y T (I P )Y = eT (I P )e =
(Y X)T (I P )(Y X) since P X = X and X T P = X T imply that
X T (I P ) = 0 and (I P )X = 0.
Suppose that H0 : 2 = 0 is true so that Y Nn (X 1 1 , 2 I n ). Let
P 1 be the projection matrix on C(X 1 ). By the above argument, r TR r R =
Y T (I P 1 )Y = (Y X 1 1 )T (I P 1 )(Y X 1 1 ) = eTR (I P 1 )eR where
eR Nn (0, 2 I n ) when H0 is true. Or use RHS = Y T (I P 1 )Y

T1 X T1 (I P 1 )Y + T1 X T1 (I P 1 )X 1 1 Y T (I P 1 )X 1 1 ,

and the last three terms equal 0 since X T1 (I P 1 ) = 0 and (I P 1 )X 1 = 0.

Hence
Y T (I P )Y Y T (P P 1 )Y
2
np 2r
2 2
by Theorem 11.9 b) using e and eR instead of Y , and Craigs Theorem 11.8
b) since np = rank(I P ) = tr(I P ), r = rank(P P 1 ) = tr(P P 1 ) =
p k, and (I P )(P P 1 ) = 0.
If X1 2d1 X2 2d2 , then

X1 /d1
Fd1 ,d2 .
X2 /d2

Hence
Y T (P P 1 )Y /r Y T (P P 1 )Y
= Fr,np
Y (I P )Y /(n p)
T rM SE
when H0 is true. Since RSS = Y T (I P )Y and RSS(R) = Y T (I P 1 )Y ,
RSS(R) RSS = Y T (I P 1 [I P ])Y = Y T (P P 1 )Y , and thus

Y T (P P 1 )Y
FR = Fr,np .
rM SE
334 11 Theory for Linear Models

Seber and Lee (2003, p. 100) show that

RSS(R) RSS = (L)T [L(X T X)1 LT ]1 (L).

D
c) By (11.6), rFR 2r as n where c = 0.
D
d) By Theorem 11.24, if Wn Fr,dn then rWn 2r as n and
dn . Hence the result follows by c). 

Let X tnp . Then X 2 F1,np . The two tail Wald t test for H0 :
j = 0 versus H1 : j = 0 is equivalent to the corresponding right tailed F
test since rejecting H0 if |X| > tnp (1 ) is equivalent to rejecting H0 if
X 2 > F1,np (1 ).

Denition 11.23. The pvalue of a test is the probability, assuming H0 is

true, of observing a test statistic as extreme as the test statistic Tn actually
observed. For a right tail test, pvalue = PH0 (of observing a test statistic
Tn ).

Under the OLS model where FR Fq,np when H0 is true (so the ei are
iid N (0, 2 )), the pvalue = P (W > FR ) where W Fq,np . In general, we
can only estimate the pvalue. Let pval be the estimated pvalue. Then pval
P
= P (W > FR ) where W Fq,np , and pval pvalue an n for the
large sample partial F test. The pvalues in output are usually actually pvals
(estimated pvalues).

Denition 11.24. Let Y F (d1 , d2 ) F (d1 , d2 , 0). Let X1

X1 /d1
2 (d1 , ) X2 2 (d2 , 0). Then W = F (d1 , d2 , ), a noncen-
X2 /d2
tral F distribution with d1 and d2 numerator and denominator degrees of
freedom, and noncentrality parameter .

Theorem 11.26, distribution of FR under normality when H0 may

not hold. Assume Y = X + e where e Nn (0, 2 I). Let X = [X 1 X 2 ]
be full rank, and let the reduced model Y = X 1 1 + eR . Then
 
Y T (P P 1 )Y /r T X T (P P 1 )X
FR = T F r, n p, .
Y (I P )Y /(n p) 2 2

If H0 : 2 = 0 is true, then = 0.

Proof. Note that the denominator is the M SE, and (n p)M SE/ 2
2np by the proof of Theorem 11.25. By Theorem 11.14 f),
 
T X T (P P 1 )X
Y (P P 1 )Y /
T 2 2
r,
2 2
11.4 Nonfull Rank Linear Models 335

where r = rank(P P 1 ) = tr(P P 1 ) = p k since P P 1 is a projection

matrix (symmetric and idempotent). 

11.4 Nonfull Rank Linear Models

Denition 11.25. The nonfull rank linear model is Y = X + e where

X has rank r < p n, and X is an n p matrix.
Nonfull rank models are often used in experimental design. Much of the
nonfull rank model theory is similar to that of the full rank model, but there
are some dierences. Now the generalized inverse (X T X) is not unique.
Similarly, is a solution to the normal equations, but depends on the gener-
alized inverse and is not unique. Some properties of the least squares estima-
tors are summarized below. Let P = P X be the projection matrix on C(X).
Recall that projection matrices are symmetric and idempotent but singular
unless P = I. Also recall that P X = X, so X T P = X T .
Theorem 11.27. i) P = X(X T X) X T is the unique projection matrix
on C(X) and does not depend on the generalized inverse (X T X) .
ii) = (X T X) X T Y does depend on (X T X) and is not unique.
iii) Y = X = P Y , r = Y Y = Y X = (I P )Y and RSS = r T r
are unique and so do not depend on (X T X) .
iv) is a solution to the normal equations: X T X = X T Y .
v) Rank(P ) = r and rank(I P ) = n r.
vi) Let = X and = X. Suppose there exists a constant vector c
such that E(cT ) = cT . Then among the class of linear unbiased estimators
of cT , the least squares estimator cT is BLUE.
RSS rT r
vii) If Cov(Y ) = Cov(e) = 2 I, then M SE = = is an
nr nr
2
unbiased estimator of .
viii) Let the columns of X 1 form a basis for C(X). For example, take r lin-
early independent columns of X to form X 1 . Then P = X 1 (X T1 X 1 )1 X T1 .

Denition 11.26. Let a and b be constant vectors. Then aT is es-

timable if there exists a linear unbiased estimator bT Y so E(bT Y ) = aT .

The term estimable is misleading since there are nonestimable quantities

aT that can be estimated with biased estimators. For full rank models, aT
is estimable for any p 1 constant vector a since aT is a linear unbiased
estimator of aT . See the Gauss Markov Theorem 11.21. Estimable quantities
tend to go with the nonfull rank linear model. We can avoid nonestimable
functions by using a full rank model instead of a nonfull rank model (delete
columns of X until it is full rank).
336 11 Theory for Linear Models

Theorem 11.28. a) The quantity aT is estimable i aT = bT X i

a = X T b (for some constant vector b) i a C(X T ).
b) If aT is estimable and a least squares estimator is any solution to
the normal equations X T X = X T Y , then aT is unique and aT is the
BLUE of aT .

Remark 11.4. There are several ways to show whether aT is estimable

or nonestimable. i) For the full rank model, aT is estimable: use the BLUE
aT .
Now consider the nonfull rank model. ii) If aT is estimable: use the
BLUE aT .
iii) There are two more ways to check whether aT is estimable.
a) If there is a constant vector b such that E(bT Y ) = aT , then aT is
estimable.
b) If aT = bT X or a = X T b or aT C(X T ), then aT is estimable.

11.5 Summary

1) The set of all linear combinations of x1 , . . . , xn is the vector space known as

n
span(x1 , . . . , xn ) = {y Rk : y = i=1 ai xi for some constants a1 , . . . , an }.
2) Let A = [a1 a2 . . . am ] be an n m matrix. The space spanned by
the columns of A = column space of A = C(A). Then C(A) = {y Rn :
y = Aw for some w Rm } = {y : y = w1 a1 + w2 a2 + + wm am for some
scalars w1 , . . . ., wm } = span(a1 , . . . , am ).
3) A generalized inverse of an n m matrix A is any m n matrix A
satisfying AA A = A.
4) The projection matrix P = P X onto the column space of X is
unique, symmetric, and idempotent. P X = X, and P W = W if each
column of W C(X). The eigenvalues of P X are 0 or 1. Rank(P ) = tr(P ).
Hence P is singular unless X is a nonsingular nn matrix, and then P = I n .
If C(X R ) is a subspace of C(X), then P X P X R = P X R P X = P X R .
5) I n P is the projection matrix on [C(X)] .
6) Let A be a positive denite symmetric matrix. The square root matrix
1/2
A is a positive denite symmetric matrix such that A1/2 A1/2 = A.
7) The matrix A in a quadratic form xT Ax will be symmetric unless
told otherwise.
8) Theorem 11.4. Let x be a random vector with E(x) = and
Cov(x) = . Then E(xT Ax) = tr(A) + T A.
9) Theorem 11.6. If A and B are symmetric matrices and AY BY ,
then Y T AY Y T BY .
10) The important part of Craigs Theorem is that if Y Nn (, ),
then Y T AY Y T BY if AB = 0.
11.5 Summary 337

11) Theorem 11.9. Let A = AT be symmetric. a) If Y Nn (0, I), then

Y AY 2r i A is idempotent of rank r. b) If Y Nn (0, 2 I), then
T

Y T AY 2 2r i A is idempotent of rank r.
12) Often theorems are given for when Y Nn (0, I). If Y Nn (0, 2 I),
then apply the theorem using Z = Y / Nn (0, I).
13) Suppose Y1 , . . . , Yn are independent N (i , 1) random
n variables so that
Y = (Y1 , . . . , Yn )T Nn (, I n ). Then Y T Y = i=1 i (n, =
Y 2 2
T 2
/2), a noncentral (n, ) distribution,with n degrees of freedom and
n
noncentrality parameter = T /2 = 12 i=1 2i 0. The noncentrality
T
parameter = = 2 is also used.
14) Theorem 11.16. Let = X C(X) where Yi = xTi + ri () and
the residual ri () depends on . The least squares estimator is the
n of2 R that minimizes
p
value the least squares criterion
r
i=1 i () = Y X 2
.
15) Let xTi = (1 uTi ), and let T = (1 TS ) where 1 is the intercept and
the slopes vector S = (2 , . . . , p )T . Let the population covariance matrices
Cov(u) = u , and Cov(u, Y ) = uY . If the (Yi , uTi )T are iid, then the
population coecients from an OLS regression of Y on u are

1 = E(Y ) TS E(u) and S = 1

u uY .

16) Theorem 11.19: Second way to compute : Suppose that wi =

(Yi , uTi )T are iid random vectors such that Y2 , 1
u , and uY exist. Then
T P
1 = Y S u 1 and
n 1 1 1 P
S = uY = u uY = u uY S as n .
n1 u

17) Theorem 11.20. Let Y = X + e = Y + r where X is full rank,

E(e) = 0, and Cov(e) = 2 I. Let P = P X be the projection matrix on
C(X) so Y = P X, r = Y Y = (I P )Y , and P X = X so X T P = X T .
i) The predictor variables and residuals are orthogonal. Hence the columns
of X and the residual vector are orthogonal: X T r = 0.
ii) E(Y ) = X.
iii) Cov(Y ) = Cov(e) = 2 I.
iv) The tted values and residuals are uncorrelated: Cov(r, Y ) = 0.
v) The least squares estimator is an unbiased estimator of : E() = .
vi) Cov() = 2 (X T X)1 .
18) Theorem 11.25, Partial F Test Theorem. Suppose H0 : L = 0
is true for the partial F test. Under the OLS full rank model, a)
1
FR = (L)T [L(X T X)1 LT ]1 (L).
rM SE
338 11 Theory for Linear Models

b) If e Nn (0, 2 I), then FR Fr,np .

D
c) For a large class of zero mean error distributions rFR 2r .
d) The partial F test that rejects H0 : L = 0 if FR > Fr,np (1 ) is a
large sample right tail test for the OLS model for a large class of zero mean
error distributions.

11.6 Complements

Three good books on linear model theory, in increasing order of diculty,

are Myers and Milton (1990), Seber and Lee (2003), and Christensen (2013).
Other texts include Freedman (2005), Graybill (1976), Guttman (1982),
Hocking (2013), Rao (1973), Ravishanker and Dey (2002), Rencher and
Schaalje (2008), Schee (1959), and Searle (1971).
A good reference for quadratic forms and the noncentral 2 , t, and F
distributions is Johnson and Kotz (1970, ch. 2831).
Least squares theory can be extended in at least two ways. The rst ex-
tension follows Chang and Olive (2010) closely. Suppose = ( TU )T
where the intercept is and the vector of slopes is U . Then for the linear
model, = 1 and S = U = (2 , . . . , p )T . Let x = (1 uT )T , and sup-
pose Y u|uT U , e.g. Yi = + uTi U + ei , or Yi = m(uTi U ) + ei , or a
GLM (generalized linear model). These models are 1D regression models. If
the ui are iid from an elliptically contoured distribution, then often the OLS
P
estimator S c U for some constant c = 0.
Let = ( TU ) and suppose the full model is Y
T
u|( + uT U ).
Consider testing A U = 0. Let the full model be Y u|( + uTR R + uTO O ),
and let the reduced model be Y u|( + uTR R ) where uT = (uTR uTO )
and uO denotes the terms outside of the reduced model. Notice that OLS
ANOVA F test corresponds to Ho: U = 0 and uses A = I p1 . The tests for
H0 : i = 0 use A = (0, . . . , 0, 1, 0, . . . , 0) where the 1 is in the ith position
and are equivalent to the OLS t tests. The test Ho: O = 0 uses A = [0 I j ]
if O is a j 1 vector. For simplicity, let U = (1 , . . . , p1 )T (start the
numbering at 1 instead of 2 and use for the intercept). For the linear model,
we could use L = (a A) where a is a known r 1 vector and r = k if A is
k (p 1).
Assume Y u|( + TU u), which is equivalent to Y u| TU u. Let the
population OLS residual
v = Y TS u
with
2 = E[(Y TS u)2 ] = E(v 2 ),
and let the OLS residual be
T
r = Y S u. (11.7)
11.6 Complements 339

Then under regularity conditions, results i) iv) below hold.

i) Li and Duan (1989): The OLS slopes estimator S = c U for some
constant c.
ii) Li and Duan (1989) and Chen and Li (1998):
D
n( S c U ) Np1 (0, C OLS )

where

C OLS = 1 T 2 T 1
u E[(Y S u) (u E(u))(u E(u)) ] u .
iii) Chen and Li (1998): Let A be a known full rank constant k (p 1)
matrix. If the null hypothesis Ho: A U = 0 is true, then
D
n(A S cA U ) = nA S Nk (0, AC OLS AT )

and
AC OLS AT = 2 A 1 T
uA .
To create test statistics, the estimator

1  2 1 
n n
T
2 = MSE = ri = (Yi S ui )2
np np
i=1 i=1

will be useful. The estimator C OLS =

 n 
1 1  T 1
u [(Yi S ui )2 (ui u)(ui u)T ] u
n i=1

can also be useful. Notice that for general 1D regression models, the OLS
MSE estimates 2 rather than the error variance 2 .
iv) Result iii) suggests that a test statistic for Ho : A U = 0 is
T 1
WOLS = n S AT [A u AT ]1 L S / 2 2k .
D

Under regularity conditions, if Ho : A U = 0 is true, then the test statistic

n1 D
FR = WOLS 2k /k
kn
as n . This result means that the OLS partial F tests are large sample
tests for a large class of nonlinear models where Y u|uT U .
The second extension of least squares theory is to an autoregressive AR(p)
time series model: Yt = 0 + 1 Yt1 + + p Ytp + et . In matrix form, this
model is Y = X + e =
340 11 Theory for Linear Models

Yp+1 1 Yp Yp1 . . . Y1 0 ep+1
Yp+2 1 Yp+1 Yp . . . Y2
1 ep+2
.. = .. .. .. .. .. .. + .. .
. . . . . . . .
Yn 1 Yn1 Yn2 . . . Ynp p en

If the AR(p) model is stationary, then under regularity conditions, OLS

partial F tests are large sample tests for this model. See Anderson (1971, pp.
210217).

11.7 Problems

11.1. Suppose Yi = xTi + i where the errors are independent N (0, 2 ).

Then the likelihood function is
 
2 n/2 1
2
L(, ) = (2 ) exp y X .
2
2 2

a) Since the least squares estimator minimizes y X2 , show that

is the MLE of .
b) Then nd the MLE 2 of 2 .

11.2. Suppose Yi = xTi + ei where the errors are iid double exponential
(0, ) where > 0. Then the likelihood function is
 
1 
n
1 1
L(, ) = n n exp |Yi xTi | .
2 i=1
n
Suppose that is a minimizer of i=1 |Yi xTi |.
a) By direct maximization, show that is an MLE of regardless of the
value of .
b) Find an MLE of by maximizing
 
1 
n
1 1
L() L(, ) = n n exp |Yi xTi | .
2 i=1

11.3. Suppose Yi = xTi + i where the errors are independent

N (0, 2 /wi ) where wi > 0 are known constants. Then the likelihood func-
tion is
 n  n  
+ 1 1 1 
n
2
L(, ) = wi exp wi (yi xi ) .
T 2

i=1
2 n 2 2 i=1
11.7 Problems 341
n
a) Suppose that W minimizes i=1 wi (yi xTi )2 . Show that W is the
MLE of .
b) Then nd the MLE 2 of 2 .

11.4. Suppose Y Nn (X, 2 V ) for known positive denite n n ma-

trix V . Then the likelihood function is
 n  
1 1 1 1 1
L(, 2 ) = exp (y X) T
V (y X) .
2 |V |1/2 n 2 2

a) Suppose that G minimizes (y X)T V 1 (y X). Show that G

is the MLE of .
b) Find the MLE 2 of 2 .

11.5. Find the vector a such that aT Y is an unbiased estimator for E(Yi )
if the usual linear model holds.

11.6. Write the following quantities as bT Y or Y T AY or AY .

 
a) Y , b) i (Yi Yi )2 , c) i (Yi )2 , d) , e) Y

11.7. Show that I H = I X(X T X)1 X T is idempotent, that is,

show that (I H)(I H) = (I H)2 = I H.

11.8. Let Y N (, 2 ) so that E(Y ) = and Var(Y ) = 2 = E(Y 2 )

[E(Y )]2 . If k 2 is an integer, then

E(Y k ) = (k 1) 2 E(Y k2 ) + E(Y k1 ).

Let Z = (Y )/ N (0, 1). Hence k = E(Y )k = k E(Z k ). Use this

fact and the above recursion relationship E(Z k ) = (k 1)E(Z k2 ) to nd
a) 3 and b) 4 .

11.9. Let A and B be matrices with the same number of rows. If C is

another matrix such that A = BC, is it true that rank(A) = rank(B)?
Prove or give a counterexample.

11.10. Let x be an n 1 vector and let B be an n n matrix. Show that

xT Bx = xT B T x.
(The point of this problem is that if B is not a symmetric n n matrix,
B + BT
then xT Bx = xT Ax where A = is a symmetric n n matrix.)
2
11.11. Consider the model Yi = 1 +2 Xi,2 + +p Xi,p +ei = xTi +ei .
The least squares estimator minimizes

n
QOLS () = (Yi xTi )2
i=1
342 11 Theory for Linear Models

and the weighted least squares estimator minimizes


n
QW LS () = wi (Yi xTi )2
i=1

where the wi , Yi and xi are known quantities. Show that


n 
n
wi (Yi xTi )2 = (Yi xTi )2
i=1 i=1

by identifying Yi , and xi . (Hence the WLS estimator is obtained from the

least squares regression of Yi on xi without an intercept.)

11.12. Suppose that X is an n p matrix but the rank of X < p < n.

Then the normal equations X T X = X T Y have innitely many solutions.
Let be a solution to the normal equations. So X T X = X T Y . Let G =
(X T X) be a generalized inverse of (X T X). Assume that E(Y ) = X and
Cov(Y ) = 2 I. It can be shown that all solutions to the normal equations
have the form bz given below.
a) Show that bz = GX T Y + (GX T X I)z is a solution to the normal
equations where the p 1 vector z is arbitrary.
b) Show that E(bz ) = .
(Hence some authors suggest that bz should be called a solution to the
normal equations but not an estimator of .)
c) Show that Cov(bz ) = 2 GX T XGT .
d) Although G is not unique, the projection matrix P = XGX T onto
C(X) is unique. Use this fact to show that Y = Xbz does not depend on G
or z.
e) There are two ways to show that aT is an estimable function. Either
show that there exists a vector c such that E(cT Y ) = aT , or show that
a C(X T ). Suppose that a = X T w for some xed vector w. Show that
E(aT bz ) = aT .
(Hence aT is estimable by aT bz where bz is any solution of the normal
equations.)
f) Suppose that a = X T w for some xed vector w. Show that
V ar(aT bz ) = 2 wT P w.
Chapter 12
Multivariate Linear Regression

This chapter will show that multivariate linear regression with m 2 re-
sponse variables is nearly as easy to use, at least if m is small, as multiple
linear regression which has m = 1 response variable. Plots for checking the
model are given, and prediction regions that are robust to nonnormality are
developed. For hypothesis testing, it is shown that the Wilks lambda statis-
tic, Hotelling Lawley trace statistic, and Pillais trace statistic are robust to
nonnormality.
Some of the proofs in this chapter are at a higher level than the rest of
the book.

12.1 Introduction

Denition 12.1. The response variables are the variables that you want
to predict. The predictor variables are the variables used to predict the
response variables.

Denition 12.2. The multivariate linear regression model

y i = B T xi + i
for i = 1, . . . , n has m 2 response variables Y1 , . . . , Ym and p predictor
variables x1 , x2 , . . . , xp where x1 1 is the trivial predictor. The ith case
is (xTi , y Ti ) = (1, xi2 , . . . , xip , Yi1 , . . . , Yim ) where the 1 could be omitted.
The model is written in matrix form as Z = XB + E where the matrices
are dened below. The model has E(k ) = 0 and Cov(k )) =  = (ij *)
for k = 1, . . . , n. Then the p m coecient matrix B = 1 2 . . . m
and the m m covariance matrix  are to be estimated, and E(Z) = XB
while E(Yij ) = xTi j . The i are assumed to be iid. Multiple linear regression
corresponds to m = 1 response variable, and is written in matrix form as Y =

Springer International Publishing AG 2017 343

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 12
344 12 Multivariate Linear Regression

X + e. Subscripts are needed for the m multiple linear regression models

Y j = X j + ej for j = 1, . . . , m where E(ej ) = 0. For the multivariate
linear regression model, Cov(ei , ej ) = ij I n for i, j = 1, . . . , m where I n is
the n n identity matrix.

Notation. The multiple linear regression model uses m = 1. The

multivariate linear model y i = B T xi + i for i = 1, . . . , n has m 2,
and multivariate linear regression and MANOVA models are special cases.
This chapter will use x1 1 for the multivariate linear regression model. The
multivariate location and dispersion model is the special case where
X = 1 and p = 1. See Chapter 10.

The data matrix W = [X Z] except usually the rst column 1 of X is

omitted for software. The n m matrix

Y1,1 Y1,2 . . . Y1,m T
Y2,1 Y2,2 . . . Y2,m ) y
* .1
Z= . .. . . . = Y 1 Y 2 . . . Y m = .. .
.. . . ..
y Tn
Yn,1 Yn,2 . . . Yn,m

The n p design matrix of predictor variables is

x1,1 x1,2 . . . x1,p T
x2,1 x2,2 x
. . . x2,p
) * .1
X= . .. . . .. = v 1 v 2 . . . v p = ..
.. . . .
xTn
xn,1 xn,2 . . . xn,p

where v 1 = 1.
The p m matrix

1,1 1,2 . . . 1,m
2,1 2,2 . . . 2,m
) *
B= . .. .. .. = 1 2 . . . m .
.. . . .
p,1 p,2 . . . p,m

The n m matrix

1,1 1,2 . . . 1,m T
2,1 2,2 
. . . 2,m ) * .1
E= . .. .. . = e1 e2 . . . em = .. .
.. . . ..
Tn
n,1 n,2 . . . n,m

Considering the ith row of Z, X, and E shows that y Ti = xTi B + Ti .

Denition 12.3. In the multiple linear regression model, m = 1 and

12.1 Introduction 345

Yi = xi,1 1 + xi,2 2 + + xi,p p + ei = xTi + ei (12.1)

for i = 1, . . . , n. In matrix notation, these n equations become

Y = X + e, (12.2)

where Y is an n 1 vector of response variables, X is an n p matrix of

predictors, is a p1 vector of unknown coecients, and e is an n1 vector
of unknown errors. Equivalently,

Y1 x1,1 x1,2 . . . x1,p 1 e1
Y2 x2,1 x2,2 . . . x2,p 2 e2

.. = .. .. . . .. .. + .. . (12.3)
. . . . . . .
Yn xn,1 xn,2 . . . xn,p p en

The ei are iid with zero mean and variance 2 , and multiple linear regression
is used to estimate the unknown parameters and 2 .

Each response variable in a multivariate linear regression model follows a

multiple linear regression model Y j = X j + ej for j = 1, . . . , m where it
is assumed that E(ej ) = 0 and Cov(ej ) = jj I n . Hence the errors corre-
sponding to the jth response are uncorrelated with variance j2 = jj . Notice
that the same design matrix X of predictors is used for each of the m
models, but the jth response variable vector Y j , coecient vector j , and
error vector ej change and thus depend on j.
Now consider the ith case (xTi , y Ti ) which corresponds to the ith row of Z
and the ith row of X. Then

Yi1 = 11 xi1 + + p1 xip + i1 = xTi 1 + i1
Yi2 = 12 xi1 + + p2 xip + i2 = xTi 2 + i2

..
.
Yim = 1m xi1 + + pm xip + im = xTi m + im

or y i = xi + i = E(y i ) + i where

xTi 1
xTi 2

E(y i ) = xi = B T xi = .. .
.
xTi m

The notation y i |xi and E(y i |xi ) is more accurate, but usually the condi-
tioning is suppressed. Taking xi to be a constant (or condition on xi if the
predictor variables are random variables), y i and i have the same covariance
matrix. In the multivariate regression model, this covariance matrix  does
not depend on i. Observations from dierent cases are uncorrelated (often
346 12 Multivariate Linear Regression

independent), but the m errors for the m dierent response variables for the
same case are correlated. If X is a random matrix, then assume X and E
are independent and that expectations are conditional on X.

Example 12.1. Suppose it is desired to predict the response variables

Y1 = height and Y2 = height at shoulder of a person from partial skeletal
remains. A model for prediction can be built from nearly complete skeletons
or from living humans, depending on the population of interest (e.g., ancient
Egyptians or modern US citizens). The predictor variables might be x1 1,
x2 = femur length, and x3 = ulna length. The two heights of individuals with
x2 = 200mm and x3 = 140mm should be shorter on average than the two
heights of individuals with x2 = 500mm and x3 = 350mm. In this example
Y1 , Y2 , x2 , and x3 are quantitative variables. If x4 = gender is a predictor
variable, then gender (coded as male = 1 and female = 0) is qualitative.

Denition 12.4. Least squares is the classical method for tting multi-
variate linear regression. The least squares estimators are
) *
B = (X T X)1 X T Z = 1 2 . . . m .

The predicted values or tted values

Y1,1 Y1,2 . . . Y1,m
)
* Y2,1 Y2,2 . . . Y2,m

Z = X B = Y 1 Y 2 . . . Y m = . .. .. .. .
.. . . .
Yn,1 Yn,2 . . . Yn,m

The residuals E = Z Z = Z X B =
T
1 1,1 1,2 . . . 1,m
T ) * 2,1 2,2 . . . 2,m
2
. = r 1 r 2 . . . r m = .. .. .. . .
.. . . . ..
Tn n,1 n,2 . . . n,m

These quantities can be found from the m multiple linear regressions of Y j

on the predictors: j = (X T X)1 X T Y j , Y j = X j , and r j = Y j Y j for
j = 1, . . . , m. Hence i,j = Yi,j Yi,j where Y j = (Y1,j , . . . , Yn,j )T . Finally,
,d =

1  T
T n
(Z Z)T (Z Z) (Z X B)T (Z X B) E E
= = = i i .
nd nd nd n d i=1
12.1 Introduction 347

The choices d = 0 and d = p are common. If d = 1, then ,d=1 = S r , the

sample covariance matrix of the residual vectors i , since the sample mean
of the i is 0. Let  = ,p be the unbiased estimator of  . Also,

,d = (n d)1 Z T [I X(X T X)1 X]Z,

and
E = [I X(X T X)1 X]Z.

The following two theorems show that the least squares estimators are
fairly good. Also see Theorem 12.7 in Section 12.4. Theorem 12.2 can also be
n1
used for ,d = Sr .
nd

Theorem 12.1, Johnson and Wichern (1988, p. 304): Suppose X

has full rank p < n and the covariance structure of Denition 12.2 holds. Then
E(B) = B so E( j ) = j , Cov( j , k ) = jk (X T X)1 for j, k = 1, . . . , p.
Also E and B are uncorrelated, E(E) = 0, and
 T 
E E
E(  ) = E = .
np

Below, Wn = OP (1), if for every  > 0 there exist positive constants D

and N such that
P (|Wn | D ) 1 
for all n N . Also Wn = OP (Xn ) if |Wn /Xn | = OP (1). A k r matrix
A = An = OP (n1/2 ) if A = [ai,j (n)] and each ai,j (n) = OP (n1/2 ). If

= + OP (n1/2 ), then is a n consistent estimator of .

n
Theorem 12.2. S r =  + OP (n1/2 ) and n1 i=1 i Ti =  +
OP(n1/2 ) if the following three
n conditions hold: B B = OP (n1/2 ),
1 n T 1 T 1/2
n i=1 i xi = OP (1), and n i=1 xi xi = OP (n ).

T
Proof. Note that y i = B T xi + i = B xi +i . Hence i = (B B)T xi +
i . Thus

n 
n 
n
i Ti = (i i +i )(i i +i )T = [i Ti +i (i i )T +(i i )Ti ]
i=1 i=1 i=1


n 
n 
n
= i Ti + ( i xTi )(B B) + (B B)T ( xi Ti )+
i=1 i=1 i=1


n
(B B)T ( xi xTi )(B B).
i=1
348 12 Multivariate Linear Regression
n n
Thus 1
n i Ti
i=1 = 1
n i=1 i Ti +

OP (1)OP (n1/2 ) + OP (n1/2 )OP (1) + OP (n1/2 )OP (n1/2 )OP (n1/2 ),
n
and the result follows since n1 i=1 i Ti =  + OP (n1/2 ) and

n 1 T
n
Sr = i i . 
n 1 n i=1

S r and  are also n consistent estimators of  by Su and Cook (2012,
p. 692). See Theorem 12.7.

12.2 Plots for the Multivariate Linear Regression Model

This section suggests using residual plots, response plots, and the DD plot
to examine the multivariate linear model. The residual plots are often used
to check for lack of t of the multivariate linear model. The response plots
are used to check linearity and to detect inuential cases and outliers. The
response and residual plots are used exactly as in the m = 1 case correspond-
ing to multiple linear regression and experimental design models. See earlier
chapters of this book, Olive et al. (2015), Olive and Hawkins (2005), and
Cook and Weisberg (1999a, p. 432; 1999b).

Notation. Plots will be used to simplify the regression analysis, and in

this text a plot of W versus Z uses W on the horizontal axis and Z on the
vertical axis.

Denition 12.5. A response plot for the jth response variable is a plot
of the tted values Yij versus the response Yij . The identity line with slope
one and zero intercept is added to the plot as a visual aid. A residual plot
corresponding to the jth response variable is a plot of Yij versus rij .

Remark 12.1. Make the m response and residual plots for any multi-
variate linear regression. In a response plot, the vertical deviations from the
identity line are the residuals rij = Yij Yij . Suppose the model is good, the
jth error distribution is unimodal and not highly skewed for j = 1, . . . , m,
and n 10p. Then the plotted points should cluster about the identity line
in each of the m response plots. If outliers are present or if the plot is not
linear, then the current model or data need to be transformed or corrected.
If the model is good, then each of the m residual plots should be ellipsoidal
with no trend and should be centered about the r = 0 line. There should not
be any pattern in the residual plot: as a narrow vertical strip is moved from
left to right, the behavior of the residuals within the strip should show little
change. Outliers and patterns such as curvature or a fan shaped plot are bad.
12.2 Plots for the Multivariate Linear Regression Model 349

Rule of thumb 12.1. Use multivariate linear regression if

n max((m + p)2 , mp + 30, 10p))
provided that the m response and residual plots all look good. Make the
DD plot of the i . See Denition 10.7. If a residual plot would look good
after several points have been deleted, and if these deleted points were not
gross outliers (points far from the point cloud formed by the bulk of the
data), then the residual plot is probably good. Beginners often nd too many
things wrong with a good model. For practice, use the computer to generate
several multivariate linear regression data sets, and make the m response and
residual plots for these data sets. This exercise will help show that the plots
can have considerable variability even when the multivariate linear regression
model is good. The lregpack function MLRsim simulates response and residual
plots for various distributions when m = 1.

Rule of thumb 12.2. If the plotted points in the residual plot look like
a left or right opening megaphone, the rst model violation to check is the
assumption of nonconstant variance. (This is a rule of thumb because it is
possible that such a residual plot results from another model violation such
as nonlinearity, but nonconstant variance is much more common.)

Remark 12.2. Residual plots magnify departures from the model while
the response plots emphasize how well the multivariate linear regression model
ts the data.

Denition 12.6. An RR plot is a scatterplot matrix of the m sets of

residuals r 1 , . . . , r m .

Denition 12.7. An FF plot is a scatterplot matrix of the m sets of

tted values of response variables Y 1 , . . . , Y m . The m response variables
Y 1 , . . . , Y m can be added to the plot.

Remark 12.3. Some applications for multivariate linear regression need

the m errors to be linearly related, and larger sample sizes may be needed if
the errors are not linearly related. For example, the asymptotic optimality of
the prediction regions of Section 12.3 needs the error vectors to be iid from
an elliptically contoured distribution. Make the RR plot and a DD plot of
the residual vectors i to check that the errors are linearly related. Make a
DD plot of the continuous predictor variables to check for x-outliers. Make
a DD plot of Y1 , . . . ., Ym to check for outliers, especially if it is assumed that
the response variables come from an elliptically contoured distribution.

The RMVN DD plot of the residual vectors i is used to check the error
distribution, to detect outliers, and to display the nonparametric prediction
region developed in Section 12.3. The DD plot suggests that the error dis-
tribution is elliptically contoured if the plotted points cluster tightly about
a line through the origin as n . The plot suggests that the error distri-
bution is multivariate normal if the line is the identity line. If n is large and
350 12 Multivariate Linear Regression

the plotted points do not cluster tightly about a line through the origin, then
the error distribution may not be elliptically contoured. These applications
of the DD plot for iid multivariate data are discussed in Olive (2002, 2008,
2013a) and Section 10.3. The RMVN estimator has not yet been proven to be
a consistent estimator when computed from residual vectors, but simulations
suggest that the RMVN DD plot of the residual vectors is a useful diagnostic
plot. The lregpack function mregddsim can be used to simulate the DD plots
of the residual vectors for various distributions.
Predictor transformations for the continuous predictors can be made ex-
actly as in Section 3.1, while response transformations can be made as in
Section 3.2 for each of the m response variables.

Warning: The Rule of thumb 3.2 does not always work. For example, the
log rule may fail. If the relationships in the scatterplot matrix are already
linear or if taking the transformation does not increase the linearity, then
no transformation may be better than taking a transformation. For the Arc
data set evaporat.lsp with m = 1, the log rule suggests transforming the
response variable Evap, but no transformation works better.

12.3 Asymptotically Optimal Prediction Regions

In this section, we will consider a more general multivariate regression model,

and then consider the multivariate linear model as a special case. Given n
cases of training or past data (x1 , y 1 ), . . . , (xn , y n ) and a vector of predictors
xf , suppose it is desired to predict a future test vector y f .

Denition 12.8. A large sample 100(1 )% prediction region is a set An

such that P (y f An ) 1 as n , and is asymptotically optimal if the
volume of the region converges in probability to the volume of the population
minimum volume covering region.

The classical large sample 100(1 )% prediction region for a future value
xf given iid data x1 , . . . , , xn is {x : Dx 2
(x, S) 2p,1 }, while for multi-
variate linear regression, the classical large sample 100(1 )% prediction
region for a future value y f given xf and past data (x1 , y i ), . . . , (xn , y n ) is
{y : Dy2
(y f ,  ) 2m,1 }. See Johnson and Wichern (1988, pp. 134, 151,
312). By equation (10.10), these regions may work for multivariate normal
xi or i , but otherwise tend to have undercoverage. Olive (2013a) replaced
2p,1 by the order statistic D(U 2
n)
where Un decreases to n(1 ). This
section will use a similar technique from Olive (2016b) to develop possibly
the rst practical large sample prediction region for the multivariate linear
model with unknown error distribution. The following technical theorem will
be needed to prove Theorem 12.4.
12.3 Asymptotically Optimal Prediction Regions 351

Theorem 12.3. Let a > 0 and assume that (n , n ) is a consistent esti-

mator of (, a).
a) Dx2
(n , n ) a1 Dx
2
(, ) = oP (1).
1
b) Let 0 < 0.5. If (n , n ) (, a) = Op (n ) and a n 1 =
OP (n ), then
1 2
2
Dx (n , n ) Dx (, ) = OP (n ).
a

Proof. Let Bn denote the subset of the sample space on which n has
an inverse. Then P (Bn ) 1 as n . Now
1
2
Dx (n , n ) = (x n )T n (x n ) =
 
1 1 1
(x n )T + n (x n ) =
a a
   1 
1 1
(x n )T + n (x n ) + (x n )T (x n ) =
a a
1 1
(x n )T ( 1 + a n )(x n ) +
a
 1 

(x + n ) T
(x + n )
a
1 2
= (x )T 1 (x ) + (x )T 1 ( n )+
a a
1 1 1
( n )T 1 ( n ) + (x n )T [a n 1 ](x n )
a a
on Bn , and the last three terms are oP (1) under a) and OP (n ) under b).


Now suppose a prediction region for an m 1 random vector y f given

a vector of predictors xf is desired for the multivariate linear model. If we
had many cases z i = B T xf + i , then we could use a multivariate prediction
region computed from the z i . Instead, Theorem 12.4 will use a multivariate
T
prediction region on the pseudodata z i = B xf +i = y f +i for i = 1, . . . , n.
This takes the data cloud of the n residual vectors i and centers the cloud at
y f . Note that z i = (BB+B)T xf +(i i +i ) = z i +(BB)T xf +i i =
z i + (B B)T xf (B B)T xi = z i + OP (n1/2 ). Hence the distances
based on the z i and the distances based on the z i have the same quantiles,
asymptotically (for quantiles that are continuity points of the distribution
of z i ).
If the i are iid from an ECm (0, , g) distribution with continuous
decreasing g and nonsingular covariance matrix  = c for some
352 12 Multivariate Linear Regression

constant c > 0, then the population asymptotically optimal prediction region

is {y : Dy (B T xf ,  ) D1 } where P (Dy (B T xf ,  ) D1 ) = 1 .

For example, if the iid i Nm (0,  ), then D1 = 2m,1 . If the error
distribution is not elliptically contoured, then the above region still has
100(1 )% coverage, but prediction regions with smaller volume may exist.
A natural way to make a large sample prediction region is to estimate the
target population minimum volume covering region, but for moderate sam-
ples and many error distributions, the natural estimator that covers n(1)
of the cases tends to have undercoverage as high as min(0.05, /2). This em-
pirical result is not too surprising since it is well known that the performance
of a prediction region on the training data is superior to the performance on
future test data, due in part to the unknown variability of the estimator. To
compensate for the undercoverage, let qn be as in Theorem 12.4.

Theorem 12.4. Suppose y i = E(y i |xi ) + i = y i + i where Cov(i ) =

 > 0, and where the zero mean f and the i are iid for i = 1, . . . , n.
Given xf , suppose the tted model produces y f and nonsingular  . Let
z i = y f + i and
1
Di2 Di2 (y f ,  ) = (z i y f )T  (z i y f )

for i = 1, . . . , n. Let qn = min(1 + 0.05, 1 + m/n) for > 0.1 and

qn = min(1 /2, 1 + 10m/n), otherwise.

If qn < 1 + 0.001, set qn = 1 . Let 0 < < 1 and h = D(Un ) where

D(Un ) is the 100 qn th sample quantile of the Mahalanobis distances Di . Let
the nominal 100(1 )% prediction region for y f be given by
1
{z : (z y f )T  (z y f ) D(U
2
n)
}=

{z : Dz
2
(y f ,  ) D(U
2
n)
} = {z : Dz (y f ,  ) D(Un ) }. (12.4)
a) Consider the n prediction regions for the data where (y f,i , xf,i ) =
(y i , xi ) for i = 1, . . . , n. If the order statistic D(Un ) is unique, then Un of the
n prediction regions contain y i where Un /n 1 as n .
b) If (y f ,  ) is a consistent estimator of (E(y f ),  ), then (12.4) is a
large sample 100(1 )% prediction region for y f .
c) If (y f ,  ) is a consistent estimator of (E(y f ),  ), and the i come
from an elliptically contoured distribution such that the highest density re-
gion is {z : Dz (0,  ) D1 }, then the prediction region (12.4) is asymp-
totically optimal.

Proof. a) Suppose (xf , y f ) = (xi , y i ). Then

12.3 Asymptotically Optimal Prediction Regions 353

1 1
2
Dy i
(y i ,  ) = (y i y i )T  (y i y i ) = Ti  i = D2 i (0,  ).

Hence y i is in the ith prediction region {z : Dz (y i ,  ) D(Un ) (y i ,  )}

i i is in prediction region {z : Dz (0,  ) D(Un ) (0,  )}, but exactly Un
of the i are in the latter region by construction, if D(Un ) is unique. Since
D(Un ) is the 100(1 )th percentile of the Di asymptotically, Un /n 1 .
b) Let P [Dz (E(y f ),  ) D1 (E(y f ),  )] = 1 . Since  > 0,
P D
Theorem 12.3 shows that if (y f ,  ) (E(y f ),  ) then D(y f ,  )
Dz (E(y f ),  ). Hence the percentiles of the distances converge in distribu-
tion, and the probability that y f is in {z : Dz (y f ,  ) D1 (y f ,  )}
converges to 1 = the probability that y f is in {z : Dz (E(y f ),  )
D1 (E(y f ),  )} at continuity points D1 of the distribution of D(E(y f ),
 ).
c) The asymptotically optimal prediction region is the region with the
smallest volume (hence highest density) such that the coverage is 1 , as
n . This region is {z : Dz (E(y f ),  ) D1 (E(y f ),  )} if the
asymptotically optimal region for the i is {z : Dz (0,  ) D1 (0,  )}.
Hence the result follows by b). 

1
Notice that if  exists, then 100qn % of the n training data y i are in their
corresponding prediction region with xf = xi , and qn 1 even if (y i ,  )
is not a good estimator or if the regression model is misspecied. Hence the
coverage qn of the training data is robust to model assumptions. Of course the
volume of the prediction region could be large if a poor estimator (y i ,  ) is
used or if the i do not come from an elliptically contoured distribution. The
response, residual, and DD plots can be used to check model assumptions.
If the plotted points in the RMVN DD plot cluster tightly about some line
through the origin and if n max[3(m + p)2 , mp + 30], we expect the volume
of the prediction region to be fairly low for the least squares estimators.
If n is too small, then multivariate data is sparse and the covering ellipsoid
for the training data may be far too small for future data, resulting in severe
undercoverage. Also notice that qn = 1 /2 or qn = 1 + 0.05 for n 20p.
At the training data, the coverage qn 1 , and qn converges to the
nominal coverage 1 as n . Suppose n 20p. Then the nominal 95%
prediction region uses qn = 0.975 while the nominal 50% prediction region
uses qn = 0.55. Prediction distributions depend both on the error distribution
and on the variability of the estimator (y f ,  ). This variability is typically
unknown but converges to 0 as n . Also, residuals tend to underestimate
errors for small n. For moderate n, ignoring estimator variability and using
qn = 1 resulted in undercoverage as high as min(0.05, /2). Letting the
coverage qn decrease to the nominal coverage 1 inates the volume of
the prediction region for small n, compensating for the unknown variability
of (y f ,  ).
354 12 Multivariate Linear Regression

Theorem 12.5 will show that the prediction region (12.4) can also be found
by applying the nonparametric prediction region, described below, on the z i .
Olive (2013a, 2016c: ch. 5) derived prediction regions for a future observation
xf given n iid p 1 random vectors xi . These regions are reviewed below
and then similar regions are used for multivariate linear regression. Suppose
(T, C) is an estimator of multivariate location and dispersion (, ) such as
the classical estimator (x, S). For h > 0, consider the hyperellipsoid
{z : (z T )T C 1 (z T ) h2 } = {z : Dz
2
h2 } = {z : Dz h}. (12.5)
A future observation xf is in region (12.5) if Dxf h. If (T, C) is a consistent
estimator of (, d), then (12.5) is a large sample (1 )100% prediction
region if h = D(Un ) where D(Un ) is the qn th sample quantile of the Di with
p replacing m. The classical parametric multivariate
 normal large sample
2
prediction region uses (T, C) = (x, S) and h = p,1 . The nonparametric
region uses the classical estimator (T, C) = (x, S) and h = D(Un ) . The
semiparametric region uses (T, C) = (TRM V N , C RM V N ) and h = D(Un ) . The
parametric MVN region uses (T, C) = (TRM V N , C RM V N ) and h2 = 2p,qn
where P (W 2p,qn ) = qn if W 2p .
Consider the multivariate linear regression model. The semiparametric and
parametric regions are only conjectured to be large sample prediction re-
gions, but are useful as diagnostics. Let  = ,d=p , z i = y f + i , and
Di2 (y f , S r ) = (z i y f )T S 1
r (z i y f ) for i = 1, . . . , n. Then the large sample
nonparametric 100(1 )% prediction region is
{z : Dz
2
(y f , S r ) D(U
2
n)
} = {z : Dz (y f , S r ) D(Un ) }, (12.6)
while the (Johnson and Wichern 1988: p. 312) classical large sample 100(1
)% prediction region is

{z : Dz2
(y f ,  ) 2m,1 } = {z : Dz (y f ,  ) 2m,1 }. (12.7)

Theorem 12.5 relates prediction region (12.4) to the above nonparametric

prediction region (12.6) originally created for iid multivariate data. Recall
that S r dened in Denition 12.4 is the sample covariance matrix of the
residual vectors i . For the multivariate linear regression model, if D1 is a
continuity point of the distribution of D, Assumption D1 above Theorem 12.7
holds, and the i have a nonsingular covariance matrix, then (12.6) is a large
sample 100(1 )% prediction region for y f .

Theorem 12.5. For multivariate linear regression, when least squares is

used to compute y f , S r , and the pseudodata z i , prediction region (12.6) is
the Olive (2013a) nonparametric prediction region applied to the z i .

Proof. Multivariate linear regression with least squares satises Theo-

rem 12.4 by Su and Cook (2012). (See Theorem 12.7.) Let (T, C) be the
12.3 Asymptotically Optimal Prediction Regions 355

sample mean and sample covariance matrix (dened above Denition 10.7)
applied to the z i . The sample mean and sample covariance matrix of the resid-
ual vectors is (0, S r ) since least squares was used. Hence the z i = y f +i have
sample covariance matrix S r , and sample mean y f . Hence (T, C) = (y f , S r ),
and the Di (y f , S r ) are used to compute D(Un ) . 

The RMVN DD plot of the residuals will be used to display the prediction
regions for multivariate linear regression. See Example 12.3. The nonparamet-
ric prediction region for multivariate linear regression of Theorem 12.5 uses
(T, C) = (y f , S r ) in (12.4), and has simple geometry. Let Rr be the non-
parametric prediction region (12.6) applied to the residuals i with y f = 0.
Then Rr is a hyperellipsoid with center 0, and the nonparametric prediction
region is the hyperellipsoid Rr translated to have center y f . Hence in a DD
plot, all points to the left of the line M D = D(Un ) correspond to y i that are
in their prediction region, while points to the right of the line are not in their
prediction region.
The nonparametric prediction region has some interesting properties. This
prediction region is asymptotically optimal if the i are iid for a large class
of elliptically contoured ECm (0, , g) distributions. Also, if there are 100
dierent values (xjf , y jf ) to be predicted, we only need to update y jf for
j = 1, . . . , 100, we do not need to update the covariance matrix S r .
It is common practice to examine how well the prediction regions work on
the training data. That is, for i = 1, . . . , n, set xf = xi and see if y i is in
the region with probability near to 1 with a simulation study. Note that
y f = y i if xf = xi . Simulation is not needed for the nonparametric prediction
region (12.6) for the data since the prediction region (12.6) centered at y i
contains y i i Rr , the prediction region centered at 0, contains i since i =
y i y i . Thus 100qn % of prediction regions corresponding to the data (y i , xi )
contain y i , and 100qn % 100(1 )%. Hence the prediction regions work
well on the training data and should work well on (xf , y f ) similar to the
training data. Of course simulation should be done for (xf , y f ) that are not
equal to training data cases. See Section 12.5.
This training data result holds provided that the multivariate linear regres-
sion using least squares is such that the sample covariance matrix S r of the
residual vectors is nonsingular, the multivariate regression model need
not be correct. Hence the coverage at the n training data cases (xi , y i )
is robust to model misspecication. Of course, the prediction regions may
be very large if the model is severely misspecied, but severity of misspec-
ication can be checked with the response and residual plots. Coverage for
a future value y f can also be arbitrarily bad if there is extrapolation or if
(xf , y f ) comes from a dierent population than that of the data.
356 12 Multivariate Linear Regression

12.4 Testing Hypotheses

This section considers testing a linear hypothesis H0 : LB = 0 versus

H1 : LB = 0 where L is a full rank r p matrix.

Denition 12.9. Assume rank(X) = p. The total corrected (for the

mean) sum of squares and cross products matrix is
 
1
T = R + W e = Z T I n 11T Z.
n

Note that T /(n 1) is the usual sample covariance matrix y if all n of the
y i are iid, e.g. if B = 0. The regression sum of squares and cross products
matrix is
 
1 1 T 1
R = Z X(X X) X 11 Z = Z T X B Z T 11T Z.
T T T
n n
T
Let H = B LT [L(X T X)1 LT ]1 LB. The error or residual sum of squares
and cross products matrix is

W e = (Z Z)T (Z Z) = Z T Z Z T X B = Z T [I n X(X T X)1 X T ]Z.

T
Note that W e = E E and W e /(n p) =  .

Warning: SAS output uses E instead of W e .

The MANOVA table is shown below.

Summary MANOVA Table

Source matrix df
Regression or Treatment R p 1
Error or Residual We n p
Total (corrected) T n1

Denition 12.10. Let 1 2 m be the ordered eigenvalues of

W 1
e H. Then there are four commonly used test statistics.
The Roys maximum root statistic is max (L) = 1 .
The Wilks statistic is (L) = |(H + W e )1 W e | = |W 1
e H + I|
1
=
+
m
(1 + i )1 .
i=1

m
i
The Pillais trace statistic is V (L) = tr[(H + W e )1 H] = .
i=1
1 + i
12.4 Testing Hypotheses 357


m
The Hotelling-Lawley trace statistic is U (L) = tr[W 1
e H] = i .
i=1

Typically some function of one of the four above statistics is used to

get pval, the estimated pvalue. Output often gives the pvals for all four test
statistics. Be cautious about inference if the last three test statistics do not
lead to the same conclusions (Roys test may not be trustworthy for r > 1).
Theory and simulations developed below for the four statistics will provide
more information about the sample sizes needed to use the four test statistics.

Theorem 12.6. The Hotelling-Lawley trace statistic

1 1
U (L) = [vec(LB)]T [  (L(X T X)1 LT )1 ][vec(LB)]. (12.8)
np

Proof. Using the Searle (1982, p. 333) identity

tr(AGT DGC) = [vec(G)]T [CA D T ][vec(G)], it follows that
1 T
(n p)U (L) = tr[  B LT [L(X T X)1 LT ]1 LB]
1 1
= [vec(LB)]T [  (L(X T X)1 LT )1 ][vec(LB)] = T where A =  ,
G = LB, D = [L(X T X)1 LT ]1 , and C = I. Hence (12.8) holds. 

D
Some notation is useful to show (12.8) and to show that (np)U (L) 2rm
under mild conditions if H0 is true. Following Henderson and Searle (1979),
let matrix A = [a1 a2 . . . ap ]. Then the vec operator stacks the columns
of A on top of one another so

a1
a2

vec(A) = . .
..
ap

Let A = (aij ) be an m n matrix and B a p q matrix. Then the

Kronecker product of A and B is the mp nq matrix

a11 B a12 B a1n B
a21 B a22 B a2n B

AB = . .. .. .
.. . .
am1 B am2 B amn B

An important fact is that if A and B are nonsingular square matrices, then

[A B]1 = A1 B 1 . The following assumption is important.
358 12 Multivariate Linear Regression

Assumption D1: Let hi be the ith diagonal element of X(X T X)1 X T .

P
Assume max1in hi 0 as n , assume that the zero mean iid error
1
vectors have nite fourth moments, and assume that X T X W 1 .
P
n
Su and Cook (2012) prove a central limit type theorem for  and B for
the partial envelopes estimator, and the least squares estimator is a special
case. These results prove the following theorem. Their theorem also shows
that for multiple linear regression (m = 1), 2 = M SE is a n consistent
estimator of 2 .

Theorem 12.7: Multivariate Least Squares Central Limit Theo-

rem (MLS CLT). For the least squares estimator, if assumption D1 holds,

then  is a n consistent estimator of  , and
D
n vec(B B) Npm (0,  W ).

Theorem 12.8. If assumption D1 holds and if H0 is true, then

D
(n p)U (L) 2rm .
D
Proof. By Theorem 12.7, n vec(B B) Npm (0,  W ). Then

under H0 , n vec(LB) Nrm (0,  LW LT ), and n [vec(LB)]T [ 1
D

T 1 D
(LW L ) ][vec(LB)] rm . This result also holds if W and  are re-
2

placed by W = n(X T X)1 and  . Hence under H0 and using the proof of
Theorem 12.6,
1
T = (np)U (L) = [vec(LB)]T [  (L(X T X)1 LT )1 ][vec(LB)] 2rm .
D

Some more details on the above results may be useful. Consider testing a
linear hypothesis H0 : LB = 0 versus H1 : LB = 0 where L is a full rank
r p matrix. For now assume the error distribution is multivariate normal
Nm (0,  ). Then

1 1
2 2

vec(B B) = .. Npm (0,  (X T X)1 )
.
m m

where
12.4 Testing Hypotheses 359

11 (X T X)1 12 (X T X)1 1m (X T X)1
21 (X T X)1 22 (X T X)1 2m (X T X)1

C =  (X T X)1 = .. .. .. .
. . .
m1 (X T X)1 m2 (X T X)1 mm (X T X)1

Now let A be an rm pm block diagonal matrix: A = diag(L, . . . , L).

Then A vec(B B) = vec(L(B B)) =

L( 1 1 )
L( 2 2 )

.. Nrm (0,  L(X T X)1 LT )
.
L( m m )

where D =  L(X T X)1 LT = ACAT =

11 L(X T X)1 LT 12 L(X T X)1 LT 1m L(X T X)1 LT
21 L(X T X)1 LT 22 L(X T X)1 LT 2m L(X T X)1 LT

.. .. .. .
. . .
m1 L(X T X)1 LT m2 L(X T X)1 LT mm L(X T X)1 LT

Under H0 , vec(LB) = A vec(B) = 0, and

L 1
L 2

vec(LB) = .. Nrm (0,  L(X T X)1 LT ).
.
L m

Hence under H0 ,

[vec(LB)]T [ 1 1 T 1
 (L(X X) L ) ][vec(LB)] rm ,
T 2

and
1
T = [vec(LB)]T [  (L(X T X)1 LT )1 ][vec(LB)] 2rm .
D
(12.9)

A large sample level test will reject H0 if pval where

 
T
pval = P < Frm,nmp . (12.10)
rm

Since least squares estimators are asymptotically normal, if the i are iid
for a large class of distributions,
360 12 Multivariate Linear Regression

1 1
2 2
D
n vec(B B) = n .. Npm (0,  W )
.
m m

where
XT X P
W 1 .
n
Then under H0 ,

L 1
L 2
D
n vec(LB) = n .. Nrm (0,  LW LT ),
.
L m
and
n [vec(LB)]T [ 1 T 1 D
 (LW L ) ][vec(LB)] rm .
2

Hence (12.9) holds, and (12.10) gives a large sample level test if the least
squares estimators are asymptotically normal.
Kakizawa (2009) shows, under stronger assumptions than Theorem 12.8,
that for a large class of iid error distributions, the following test statistics
have the same 2rm limiting distribution when H0 is true, and the same non-
central 2rm ( 2 ) limiting distribution with noncentrality parameter 2 when
H0 is false under a local alternative. Hence the three tests are robust to the
assumption of normality. The limiting null distribution is well known when
the zero mean errors are iid from a multivariate normal distribution. See
D D
Khattree and Naik (1999, p. 68): (n p)U (L) 2rm , (n p)V (L) 2rm ,
D
and [n p 0.5(m r + 3)] log((L)) 2rm . Results from Kshirsagar
(1972, p. 301) suggest that the third chi-square approximation is very good
if n 3(m + p)2 for multivariate normal errors.
Theorems 12.6 and 12.8 are useful for relating multivariate tests with
the partial F test for multiple linear regression that tests whether a reduced
model that omits some of the predictors can be used instead of the full model
that uses all p predictors. The partial F test statistic is
 
SSE(R) SSE(F )
FR = /M SE(F )
dfR dfF
where the residual sums of squares SSE(F ) and SSE(R) and degrees
of freedom dfF and dfr are for the full and reduced model while the
mean square error M SE(F ) is for the full model. Let the null hypothe-
sis for the partial F test be H0 : L = 0 where L sets the coecients
of the predictors in the full model but not in the reduced model to 0.
Seber and Lee (2003, p. 100) show that
12.4 Testing Hypotheses 361

[L]T (L(X T X)1 LT )1 [L]

FR =
r 2
is distributed as Fr,np if H0 is true and the errors are iid N (0, 2 ). Note
that for multiple linear regression with m = 1, FR = (n p)U (L)/r since
1
= 1/ 2 . Hence the scaled Hotelling Lawley test statistic is the partial
F test statistic extended to m > 1 predictor variables by Theorem 12.6.
D
By Theorem 12.8, for example, rFR 2r for a large class of nonnormal
D
error distributions. If Zn Fk,dn , then Zn 2k /k as dn . Hence using
the Fr,np approximation gives a large sample test with correct asymptotic
level, and the partial F test is robust to nonnormality.
Similarly, using an Frm,npm approximation for the following test statistics
gives large sample tests with correct asymptotic level by Kakizawa (2009) and
similar power for large n. The large sample test will have correct asymptotic
level as long as the denominator degrees of freedom dn as n , and
dn = n pm reduces to the partial F test if m = 1 and U (L) is used. Then
the three test statistics are
[n p 0.5(m r + 3)] np np
log((L)), V (L), and U(L).
rm rm rm
By Berndt and Savin (1977) and Anderson (1984, pp. 333, 371),

V (L) log((L)) U (L).

Hence the Hotelling Lawley test will have the most power and Pillais test
will have the least power.
Following Khattree and Naik (1999, pp. 6768), there are several ap-
proximations used by the SAS software. For the Roys largest root test, if
h = max(r, m), use

nph+r
max (L) F (h, n p h + r).
h
The simulations in Section 12.5 suggest that this approximation is good for
r = 1 but poor for r > 1. Anderson (1984, p. 333) states that Roys largest
root test has the greatest power if r = 1 but is
an inferior test
for r > 1. Let
g = np(mr+1)/2, u = (rm2)/4 and t = r2 m2 4/ m2 + r2 5 for
P P
m2 +r2 5 > 0 and t = 1, otherwise. Assume H0 is true. Thus U 0, V 0,
P
and 1 as n . Then
gt 2u 1 1/t 1 1/t
1/t
F (rm, gt 2u) or (n p)t 2rm .
rm 1/t
362 12 Multivariate Linear Regression

For large n and t > 0, log() = t log(1/t ) = t log(1 + 1/t 1)

t(1 1/t ) t(1 1/t )/1/t . If it cannot be shown that
P
(n p)[ log() t(1 1/t )/1/t ] 0 as n ,

then it is possible that the approximate 2rm distribution may be the limiting
distribution for only a small class of iid error distributions. When the i are
iid Nm (0,  ), there are some exact results. For r = 1,

npm+1 1
F (m, n p m + 1).
m
For r = 2,

2(n p m + 1) 1 1/2
F (2m, 2(n p m + 1)).
2m 1/2
For m = 2,
2(n p) 1 1/2
F (2r, 2(n p)).
2r 1/2
Let s = min(r, m), m1 = (|r m| 1)/2 and m2 = (n p m 1)/2. Note
that s(|r m| + s) = min(r, m) max(r, m) = rm. Then

np V np V 2m2 + s + 1 V
=
rm 1 V /s s(|r m| + s) 1 V /s 2m1 + s + 1 s V

F (s(2m1 +s+1), s(2m2 +s+1)) F (s(|rm|+s), s(np)) = F (rm, s(np)).

This approximation is asymptotically correct by Slutskys theorem since
P np
1 V /s 1. Finally, U=
rm
np 2(sm2 + 1)
U 2 U F (s(2m1 + s + 1), 2(sm2 + 1))
s(|r m| + s) s (2m1 + s + 1)

F (s(|r m| + s), s(n p)) = F (rm, s(n p)).

This approximation is asymptotically correct for a wide range of iid error
distributions.
Multivariate analogs of tests for multiple linear regression can be derived
with appropriate choice of L. Assume a constant x1 = 1 is in the model. As
a textbook convention, use = 0.05 if is not given.
The four step MANOVA test of linear hypotheses is useful.
i) State the hypotheses H0 : LB = 0 and H1 : LB = 0.
ii) Get test statistic from output.
iii) Get pval from output.
iv) State whether you reject H0 or fail to reject H0 . If pval , reject H0
12.4 Testing Hypotheses 363

and conclude that LB = 0. If pval > , fail to reject H0 and conclude that
LB = 0 or that there is not enough evidence to conclude that LB = 0.
The MANOVA test of H0 : B = 0 versus H1 : B = 0 is the special case
T T
corresponding to L = I and H = B X T X B = Z Z, but is usually not a
test of interest.

The analog of the ANOVA F test for multiple linear regression is the
MANOVA F test that uses L = [0 I p1 ] to test whether the nontrivial
predictors are needed in the model. This test should reject H0 if the response
and residual plots look good, n is large enough, and at least one response
plot does not look like the corresponding residual plot. A response plot for
Yj will look like a residual plot if the identity line appears almost horizontal,
hence the range of Yj is small. Response and residual plots are often useful
for n 10p.
The 4 step MANOVA F test of hypotheses uses L = [0 I p1 ].
i) State the hypotheses H0 : the nontrivial predictors are not needed in the
mreg model H1 : at least one of the nontrivial predictors is needed.
ii) Find the test statistic F0 from output.
iii) Find the pval from output.
iv) If pval , reject H0 . If pval > , fail to reject H0 . If H0 is rejected,
conclude that there is a mreg relationship between the response variables
Y1 , . . . , Ym and the predictors x2 , . . . , xp . If you fail to reject H0 , conclude
that there is a not a mreg relationship between Y1 , . . . , Ym and the predictors
x2 , . . . , xp . (Or there is not enough evidence to conclude that there is a
mreg relationship between the response variables and the predictors. Get the
variable names from the story problem.)

The Fj test of hypotheses uses Lj = [0, . . . , 0, 1, 0, . . . , 0], where the 1 is in

the jth position, to test whether the jth predictor xj is needed in the model
given that the other p 1 predictors are in the model. This test is an analog
of the t tests for multiple linear regression. Note that xj is not needed in the
model corresponds to H0 : B j = 0 while xj needed in the model corresponds
to H1 : B j = 0 where B Tj is the jth row of B.
The 4 step Fj test of hypotheses uses Lj = [0, . . . , 0, 1, 0, . . . , 0] where the
1 is in the jth position.
i) State the hypotheses H0 : xj is not needed in the model
H1 : xj is needed.
ii) Find the test statistic Fj from output.
iii) Find pval from output.
iv) If pval , reject H0 . If pval > , fail to reject H0 . Give a nontechnical
sentence restating your conclusion in terms of the story problem. If H0 is
rejected, then conclude that xj is needed in the mreg model for Y1 , . . . , Ym
given that the other predictors are in the model. If you fail to reject H0 , then
conclude that xj is not needed in the mreg model for Y1 , . . . , Ym given that
the other predictors are in the model. (Or there is not enough evidence to
364 12 Multivariate Linear Regression

conclude that xj is needed in the model. Get the variable names from the
story problem.)
The Hotelling Lawley statistic

j1

1 j2
1 T 1 1
Fj = B j  B j = (j1 , j2 , . . . , jm )  ..
dj dj .
jm
T
where B j is the jth row of B and dj = (X T X)1
jj , the jth diagonal entry of
T 1
(X X) . The statistic Fj could be used for forward selection and backward
elimination in variable selection.

The 4 step MANOVA partial F test of hypotheses has a full model using
all of the variables and a reduced model where r of the variables are deleted.
The ith row of L has a 1 in the position corresponding to the ith variable
to be deleted. Omitting the jth variable corresponds to the Fj test while
omitting variables x2 , . . . , xp corresponds to the MANOVA F test. Using
L = [0 I k ] tests whether the last k predictors are needed in the multivariate
linear regression model given that the remaining predictors are in the model.
i) State the hypotheses H0 : the reduced model is good H1 : use the full
model.
ii) Find the test statistic FR from output.
iii) Find the pval from output.
iv) If pval , reject H0 and conclude that the full model should be used.
If pval > , fail to reject H0 and conclude that the reduced model is good.

The lregpack function mltreg produces the m response and residual plots,
gives B,  , the MANOVA partial F test statistic and pval corresponding
to the reduced model that leaves out the variables given by indices (so x2
and x4 in the output below with F = 0.77 and pval = 0.614), Fj and the
pval for the Fj test for variables 1, 2, . . . , p (where p = 4 in the output below
so F2 = 1.51 with pval = 0.284), and F0 and pval for the MANOVA F test
(in the output below F0 = 3.15 and pval= 0.06). Right click Stop on the plots
m times to advance the plots and to get the cursor back on the command
line in R.
The command out <- mltreg(x,y,indices=c(2)) would produce a
MANOVA partial F test corresponding to the F2 test while the command
out <- mltreg(x,y,indices=c(2,3,4)) would produce a MANOVA par-
tial F test corresponding to the MANOVA F test for a data set with p = 4
predictor variables. The Hotelling Lawley trace statistic is used in the tests.

out <- mltreg(x,y,indices=c(2,4))

$Bhat
[,1] [,2] [,3]
12.4 Testing Hypotheses 365

[1,] 47.96841291 623.2817463 179.8867890

[2,] 0.07884384 0.7276600 -0.5378649
[3,] -1.45584256 -17.3872206 0.2337900
[4,] -0.01895002 0.1393189 -0.3885967
$Covhat
[,1] [,2] [,3]
[1,] 21.91591 123.2557 132.339
[2,] 123.25566 2619.4996 2145.780
[3,] 132.33902 2145.7797 2954.082
$partial
partialF Pval
[1,] 0.7703294 0.6141573

$Ftable
Fj pvals
[1,] 6.30355375 0.01677169
[2,] 1.51013090 0.28449166
[3,] 5.61329324 0.02279833
[4,] 0.06482555 0.97701447

$MANOVA
MANOVAF pval
[1,] 3.150118 0.06038742

#Output for Example 12.2

y<-marry[,c(2,3)]; x<-marry[,-c(2,3)];
mltreg(x,y,indices=c(3,4))
$partial
partialF Pval
[1,] 0.2001622 0.9349877

$Ftable
Fj pvals
[1,] 4.35326807 0.02870083
[2,] 600.57002201 0.00000000
[3,] 0.08819810 0.91597268
[4,] 0.06531531 0.93699302
$MANOVA
MANOVAF pval
[1,] 295.071 1.110223e-16
366 12 Multivariate Linear Regression

Example 12.2. The above output is for the Hebbler (1847) data from
the 1843 Prussia census. Sometimes if the wife or husband was not at the
household, then s/he would not be counted. Y1 = number of married civilian
men in the district, Y2 = number of women married to civilians in the district,
x2 = population of the district in 1843, x3 = number of married military men
in the district, and x4 = number of women married to military men in the
district. The reduced model deletes x3 and x4 . The constant uses x1 = 1.

a) Do the MANOVA F test.

b) Do the F2 test.
c) Do the F4 test.
d) Do an appropriate 4 step test for the reduced model that deletes x3
and x4 .
e) The output for the reduced model that deletes x1 and x2 is shown below.
Do an appropriate 4 step test.
$partial
partialF Pval
[1,] 569.6429 0

Solution:
a) i) H0 : the nontrivial predictors are not needed in the mreg model
H1 : at least one of the nontrivial predictors is needed
ii) F0 = 295.071
iii) pval = 0
iv) Reject H0 , the nontrivial predictors are needed in the mreg model.
b) i) H0 : x2 is not needed in the model H1 : x2 is needed
ii) F2 = 600.57
iii) pval = 0
iv) Reject H0 , population of the district is needed in the model.
c) i) H0 : x4 is not needed in the model H1 : x4 is needed
ii) F4 = 0.065
iii) pval = 0.937
iv) Fail to reject H0 , number of women married to military men is not
needed in the model given that the other predictors are in the model.
d) i) H0 : the reduced model is good H1 : use the full model.
ii) FR = 0.200
iii) pval = 0.935
iv) Fail to reject H0 , so the reduced model is good.
e) i) H0 : the reduced model is good H1 : use the full model.
ii) FR = 569.6
iii) pval = 0.00
iv) Reject H0 , so use the full model.
12.5 An Example and Simulations 367

12.5 An Example and Simulations

The semiparametric prediction region and parametric MVN prediction region

(described above Theorem 12.5) applied to the z i are only conjectured to be
large sample prediction regions, but are added to the DD plot as visual aids.
Cases below the horizontal line that crosses the identity line correspond to
the semiparametric region while cases below the horizontal line that ends
at the identity line correspond to the parametric MVN region. A vertical
line dropped down from this point of intersection does correspond to a large
sample prediction region for multivariate normal error vectors. Note that
z i = y f + i , and adding a constant y f to all of the residual vectors does
not change the Mahalanobis distances, so the DD plot of the residuals can
be used to display the prediction regions.

3.0 3.6 4.2 2.5 4.5

300
L

150
3.0 3.6 4.2

log(W)

140

H
80
4.5

log(S)
2.5

log(M)
2
0

150 300 80 140 0 2 4

Fig. 12.1 Scatterplot Matrix of the Mussels Data.

Example 12.3. Cook and Weisberg (1999a, pp. 351, 433, 447) give a data
set on 82 mussels sampled o the coast of New Zealand. Let Y1 = log(S)
and Y2 = log(M ) where S is the shell mass and M is the muscle mass.
The predictors are X2 = L, X3 = log(W ), and X4 = H: the shell length,
log(width), and height.
a) First use the multivariate location and dispersion model for this data.
Figure 12.1 shows a scatterplot matrix of the data and Figure 12.2 shows a
368 12 Multivariate Linear Regression

48

10
8
8
78
16
37
6
RD

79
4
2

1 2 3 4 5
MD
Fig. 12.2 DD Plot of the Mussels Data, MLD Model.

Response Plot
5.5

37
4.5
Y

79
3.5
2.5

2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0

FIT

Residual Plot
0.4

37
0.2
0.0
RES

48 11
0.4

79

2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0

FIT

Fig. 12.3 Plots for Y1 = log(S).

12.5 An Example and Simulations 369

Response Plot

4
3
2
Y

25
8
1

48
0

1.5 2.0 2.5 3.0 3.5 4.0

FIT

Residual Plot
0.5
0.0
RES

0.5

25

8
1.0

48

1.5 2.0 2.5 3.0 3.5 4.0

FIT

Fig. 12.4 Plots for Y2 = log(M ).

7

48
6

8
5

79
4
RD
3
2
1
0

0 1 2 3 4 5
MD

Fig. 12.5 DD Plot of the Residual Vectors for the Mussel Data.
370 12 Multivariate Linear Regression

DD plot of the data with multivariate prediction regions added. These plots
suggest that the data may come from an elliptically contoured distribution
that is not multivariate normal. The semiparametric and nonparametric 90%
prediction regions consist of the cases below the RD = 5.86 line and to the
left of the M D = 4.41 line. These two lines intersect on a line through the
origin that is followed by the plotted points. The parametric MVN prediction
region is given by the points below the RD = 3.33 line and does not contain
enough cases.
b) Now consider the multivariate linear regression model. Figures 12.3
and 12.4 give the response and residual plots for Y1 and Y2 . The response
plots show strong linear relationships. For Y1 , case 79 sticks out while for Y2 ,
cases 8, 25, and 48 are not t well. Highlighted cases had Cooks distance
> min(0.5, 2p/n). See Cook (1977). A residual vector  = ( ) +  is a
combination of  and a discrepancy   that tends to have an approximate
multivariate normal distribution. The   term can dominate for small to
moderate n when  is not multivariate normal, incorrectly suggesting that
the distribution of the error  is closer to a multivariate normal distribution
than is actually the case. Figure 12.5 shows the DD plot of the residual vec-
tors. The plotted points are highly correlated but do not cover the identity
line, suggesting an elliptically contoured error distribution that is not mul-
tivariate normal. The nonparametric 90% prediction region for the residuals
consists of the points to the left of the vertical line M D = 2.60. Comparing
Figures 12.2 and 12.5, the residual distribution is closer to a multivariate
normal distribution. Cases 8, 48, and 79 have especially large distances. The
four Hotelling Lawley Fj statistics were greater than 5.77 with pvalues less
than 0.005, and the MANOVA F statistic was 337.8 with pvalue 0.
The response, residual, and DD plots are eective for nding inuential
cases, for checking linearity, for checking whether the error distribution is
multivariate normal or some other elliptically contoured distribution, and
for displaying the nonparametric prediction region. Note that cases to the
right of the vertical line correspond to cases with y i that are not in their
prediction region. These are the cases corresponding to residual vectors with
large Mahalanobis distances. Adding a constant does not change the distance,
so the DD plot for the residuals is the same as the DD plot for the z i .
c) Now suppose the same model is used except Y2 = M . Then the response
and residual plots for Y1 remain the same, but the plots shown in Figure 12.6
show curvature about the identity and r = 0 lines. Hence the linearity con-
dition is violated. Figure 12.7 shows that the plotted points in the DD plot
have correlation well less than one, suggesting that the error distribution is
no longer elliptically contoured. The nonparametric 90% prediction region
for the residual vectors consists of the points to the left of the vertical line
M D = 2.52, and contains 95% of the training data. Note that the plots can
be used to quickly assess whether power transformations have resulted in a
linear model, and whether inuential cases are present. R code for producing
the seven gures is shown below.
12.5 An Example and Simulations 371

Response Plot

50
40
30
Y
20
10
0

0 10 20 30 40
FIT

Residual Plot
15
10
5
RES
0
10 5

0 10 20 30 40
FIT

Fig. 12.6 Plots for Y2 = M .

6
5
4
RD

3
2
1
0

0.5 1.0 1.5 2.0 2.5 3.0 3.5

MD

Fig. 12.7 DD Plot When Y2 = M .

372 12 Multivariate Linear Regression

y <- log(mussels)[,4:5]
x <- mussels[,1:3]
x[,2] <- log(x[,2])
z<-cbind(x,y)
pairs(z, labels=c("L","log(W)","H","log(S)","log(M)"))
ddplot4(z) #right click Stop
out <- mltreg(x,y) #right click Stop 4 times
ddplot4(out$res) #right click Stop
y[,2] <- mussels[,5]
tem <- mltreg(x,y) #right click Stop 4 times
ddplot4(tem$res) #right click Stop

12.5.1 Simulations for Testing

A small simulation was used to study the Wilks test, the Pillais trace
test, the Hotelling Lawley trace test, and the Roys largest root test for the
Fj tests and the MANOVA F test for multivariate linear regression. The rst
row of B was always 1T and the last row of B was always 0T . When the null
hypothesis for the MANOVA F test is true, all but the rst row corresponding
to the constant are equal to 0T . When p 3 and the null hypothesis for the
MANOVA F test is false, then the second to last row of B is (1, 0, . . . , 0),
the third to last row is (1, 1, 0, . . . , 0) et cetera as long as the rst row is
not changed from 1T . First m 1 error vectors wi were generated such that
the m errors are iid with variance 2 . Let the m m matrix A = (aij ) with
aii = 1 and aij = where 0 < 1 for i = j. Then i = Awi so that
 = 2 AAT = (ij ) where the diagonal entries ii = 2 [1 + (m 1) 2 ] and
the o diagonal entries ij = 2 [2 + (m 2) 2 ] where = 0.10. Hence the
correlations are (2+(m2) 2 )/(1+(m1) 2 ). As gets close to 1, the error
vectors cluster about the line in the direction of (1, . . . , 1)T . See Maronna and
Zamar (2002). We used wi Nm (0, I), wi (1 )Nm (0, I) + Nm (0, 25I)
with 0 < < 1 and = 0.25 in the simulation, wi multivariate td with
d = 7 degrees of freedom, or wi lognormal - E(lognormal): where the m
components of wi were iid with distribution ez E(ez ) where z N (0, 1).
Only the lognormal distribution is not elliptically contoured.
The simulation used 5000 runs, and H0 was rejected if the F statistic
was greater than Fd1 ,d2 (0.95) where P (Fd1 ,d2 < Fd1 ,d2 (0.95)) = 0.95 with
d1 = rm and d2 = n mp for the test statistics

[n p 0.5(m r + 3)] np np
log((L)), V (L), and U(L),
rm rm rm
while d1 = h = max(r, m) and d2 = n p h + r for the test statistic
12.5 An Example and Simulations 373

Table 12.1 Test Coverages: MANOVA F H0 is True.

w dist n test F1 F2 Fp1 Fp FM
MVN 300 W 1 0.043 0.042 0.041 0.018
MVN 300 P 1 0.040 0.038 0.038 0.007
MVN 300 HL 1 0.059 0.058 0.057 0.045
MVN 300 R 1 0.051 0.049 0.048 0.993
MVN 600 W 1 0.048 0.043 0.043 0.034
MVN 600 P 1 0.046 0.042 0.041 0.026
MVN 600 HL 1 0.055 0.052 0.050 0.052
MVN 600 R 1 0.052 0.048 0.047 0.994
MIX 300 W 1 0.042 0.043 0.044 0.017
MIX 300 P 1 0.039 0.040 0.042 0.008
MIX 300 HL 1 0.057 0.059 0.058 0.039
MIX 300 R 1 0.050 0.050 0.051 0.993
MVT(7) 300 W 1 0.048 0.036 0.045 0.020
MVT(7) 300 P 1 0.046 0.032 0.042 0.011
MVT(7) 300 HL 1 0.064 0.049 0.058 0.045
MVT(7) 300 R 1 0.055 0.043 0.051 0.993
LN 300 W 1 0.043 0.047 0.040 0.020
LN 300 P 1 0.039 0.045 0.037 0.009
LN 300 HL 1 0.057 0.061 0.058 0.041
LN 300 R 1 0.049 0.055 0.050 0.994

Table 12.2 Test Coverages: MANOVA F H0 is False.

n m = p test F1 F2 Fp1 Fp FM
30 5 W 0.012 0.222 0.058 0.000 0.006
30 5 P 0.000 0.000 0.000 0.000 0.000
30 5 HL 0.382 0.694 0.322 0.007 0.579
30 5 R 0.799 0.871 0.549 0.047 0.997
50 5 W 0.984 0.955 0.644 0.017 0.963
50 5 P 0.971 0.940 0.598 0.012 0.871
50 5 HL 0.997 0.979 0.756 0.053 0.991
50 5 R 0.996 0.978 0.744 0.049 1
105 10 W 0.650 0.970 0.191 0.000 0.633
105 10 P 0.109 0.812 0.050 0.000 0.000
105 10 HL 0.964 0.997 0.428 0.000 1
105 10 R 1 1 0.892 0.052 1
150 10 W 1 1 0.948 0.032 1
150 10 P 1 1 0.941 0.025 1
150 10 HL 1 1 0.966 0.060 1
150 10 R 1 1 0.965 0.057 1
450 20 W 1 1 0.999 0.020 1
450 20 P 1 1 0.999 0.016 1
450 20 HL 1 1 0.999 0.035 1
450 20 R 1 1 0.999 0.056 1
374 12 Multivariate Linear Regression

nph+r
max (L).
h
Denote these statistics by W , P , HL, and R. Let the coverage be the propor-
tion of times that H0 is rejected. We want coverage near 0.05 when H0 is true
and coverage close to 1 for good power when H0 is false. With 5000 runs,
coverage outside of (0.04,0.06) suggests that the true coverage is not 0.05.
Coverages are tabled for the F1 , F2 , Fp1 , and Fp test and for the MANOVA
F test denoted by FM . The null hypothesis H0 was always true for the Fp
test and always false for the F1 test. When the MANOVA F test was true,
H0 was true for the Fj tests with j = 1. When the MANOVA F test was
false, H0 was false for the Fj tests with j = p, but the Fp1 test should be
hardest to reject for j = p by construction of B and the error vectors.
When the null hypothesis H0 was true, simulated values started to get close
to nominal levels for n 0.8(m+p)2 , and were fairly good for n 1.5(m+p)2 .
The exception was Roys test which rejects H0 far too often if r > 1. See
Table 12.1 where we want values for the F1 test to be close to 1 since H0 is
false for the F1 test, and we want values close to 0.05, otherwise. Roys test
was very good for the Fj tests but very poor for the MANOVA F test. Results
are shown for m = p = 10. As expected from Berndt and Savin (1977), Pillais
test rejected H0 less often than Wilks test which rejected H0 less often than
the Hotelling Lawley test. Based on a much larger simulation study Pelawa
Watagoda (2013, pp. 111112), using the four types of error distributions
and m = p, the tests had approximately correct level if n 0.83(m + p)2 for
the Hotelling Lawley test, if n 2.80(m + p)2 for the Wilks test (agreeing
with Kshirsagar (1972) n 3(m + p)2 for multivariate normal data), and if
n 4.2(m + p)2 for Pillais test.
In Table 12.2, H0 is only true for the Fp test where p = m, and we want
values in the Fp column near 0.05. We want values near 1 for high power
otherwise. If H0 is false, often H0 will be rejected for small n. For example,
if n 10p, then the m residual plots should start to look good, and the
MANOVA F test should be rejected. For the simulated data, the test had
fair power for n not much larger than mp. Results are shown for the lognormal
distribution.
Some R output for reproducing the simulation is shown below. The lregpack
function is mregsim and etype = 1 uses data from an MVN distribution. The
fcov line computed the Hotelling Lawley statistic using equation (12.8) while
the hotlawcov line used Denition 12.10. The mnull=T part of the command
means we want the rst value near 1 for high power and the next three
numbers near the nominal level 0.05 except for mancv where we want all
of the MANOVA F test statistics to be near the nominal level of 0.05. The
mnull=F part of the command means we want all values near 1 for high power
except for the last column (for the terms other than mancv) corresponding to
the Fp test where H0 is true so we want values near the nominal level of 0.05.
The coverage is the proportion of times that H0 is rejected, so coverage
is short for power and level: we want the coverage near 1 for high power
12.5 An Example and Simulations 375

when H0 is false, and we want the coverage near the nominal level 0.05 when
H0 is true. Also see Problem 12.10.

mregsim(nruns=5000,etype=1,mnull=T)
$wilkcov
[1] 1.0000 0.0450 0.0462 0.0430
$pilcov
[1] 1.0000 0.0414 0.0432 0.0400
$hotlawcov
[1] 1.0000 0.0522 0.0516 0.0490
$roycov
[1] 1.0000 0.0512 0.0500 0.0480
$fcov
[1] 1.0000 0.0522 0.0516 0.0490
$mancv
wcv pcv hlcv rcv fcv
[1,] 0.0406 0.0332 0.049 0.1526 0.049

mregsim(nruns=5000,etype=2,mnull=F)

$wilkcov
[1] 0.9834 0.9814 0.9104 0.0408
$pilcov
[1] 0.9824 0.9804 0.9064 0.0372
$hotlawcov
[1] 0.9856 0.9838 0.9162 0.0480
$roycov
[1] 0.9848 0.9834 0.9156 0.0462
$fcov
[1] 0.9856 0.9838 0.9162 0.0480
$mancv
wcv pcv hlcv rcv fcv
[1,] 0.993 0.9918 0.9942 0.9978 0.9942

12.5.2 Simulations for Prediction Regions

The same type of data and 5000 runs were used to simulate the prediction
regions for y f given xf for multivariate regression. With n=100, m=2, and
p=4, the nominal coverage of the prediction region is 90%, and 92% of the
training data is covered. Following Olive (2013a), consider the prediction
region {z : (z T )T C 1 (z T ) h2 } = {z : Dz
2
h2 } = {z : Dz h}.
Then the ratio of the prediction region volumes
376 12 Multivariate Linear Regression

hm det(C i )
i

m
h2 det(C 2 )

was recorded where i = 1 was the nonparametric region, i = 2 was the

semiparametric region, and i = 3 was the parametric MVNregion. Here h1
and h2 were the cuto D(Un ) (Ti , C i ) for i = 1, 2, and h3 = 2m,qn .

Table 12.3 Coverages for 90% Prediction Regions.

w dist n m = p ncov scov mcov voln volm
MVN 48 2 0.901 0.905 0.888 0.941 0.964
MVN 300 5 0.889 0.887 0.890 1.006 1.015
MVN 1200 10 0.899 0.896 0.896 1.004 1.001
MIX 48 2 0.912 0.927 0.710 0.872 0.097
MIX 300 5 0.906 0.911 0.680 0.882 0.001
MIX 1200 10 0.904 0.911 0.673 0.889 0+
MVT(7) 48 2 0.903 0.910 0.825 0.914 0.646
MVT(7) 300 5 0.899 0.909 0.778 0.916 0.295
MVT(7) 1200 10 0.906 0.911 0.726 0.919 0.061
LN 48 2 0.912 0.926 0.651 0.729 0.090
LN 300 5 0.915 0.917 0.593 0.696 0.009
LN 1200 10 0.912 0.916 0.593 0.679 0+

If, as conjectured, the RMVN estimator is a consistent estimator when

applied to the residual vectors instead of iid data, then the volume ratios
converge in probability to 1 if the iid zero mean errors Nm (0,  ), and
the volume ratio converges to 1 for i = 1 for a large class of elliptically
contoured distributions. These volume ratios were denoted by voln and volm
for the nonparametric and parametric MVN regions. The coverage was the
proportion of times the prediction region contained y f where ncov, scov,
and mcov are for the nonparametric, semiparametric, and parametric MVN
regions.
In the simulations, we took n = 3(m + p)2 and m = p. Table 12.3 shows
that the coverage of the nonparametric region was close to 0.9 in all cases.
The volume ratio voln was fairly close to 1 for the three elliptically contoured
distributions. Since the volume of the prediction region is proportional to hm ,
the volume can be very small if h is too small and m is large. Parametric
prediction regions usually give poor estimates of h when the parametric dis-
tribution is misspecied. Hence the parametric MVN region only performed
well for multivariate normal data.
Some R output for reproducing the simulation is shown below. The lreg-
pack function is mpredsim and etype = 1 uses data from an MVN distribution.
The term ncvr is ncov in Table 12.3. Since up = 0.94, 94% of the train-
ing data is covered by the nominal 90% nominal prediction region. Also see
Problem 12.11.
12.6 Summary 377

mpredsim(nruns=5000,etype=1)
$ncvr
[1] 0.9162
$scvr
[1] 0.916
$mcvr
[1] 0.9138
$voln
[1] 0.9892485
$vols
[1] 1
$volm
[1] 1.004964
$up
[1] 0.94

12.6 Summary

1) The multivariate linear regression model is a special case of the multivari-

ate linear model where at least one predictor variable xj is continuous.
2) The multivariate linear regression model y i = B T xi + i for
i = 1, . . . , n has m 2 response variables Y1 , . . . , Ym and p predictor variables
x1 , x2 , . . . , xp . The ith case is (xTi , y Ti ) = (xi1 , xi2 , . . . , xip , Yi1 , . . . , Yim ). The
constant xi1 = 1 is in the model, and is often omitted from the case and
the data matrix. The model is written in matrix form as Z = XB + E.
The model has E(k ) = 0 and Cov(k ) =  = (ij ) for k = 1, . . . , n. Also
E(ei ) = 0 while Cov(ei , ej ) = ij I n for i, j = 1, . . . , m. Then B and  are
unknown matrices of parameters to be estimated, and E(Z) = XB while
E(Yij ) = xTi j .
3) Each response variable in a multivariate linear regression model follows
a multiple linear regression model Y j = X j + ej for j = 1, . . . , m where it
is assumed that E(ej ) = 0 and Cov(ej ) = jj I n .
4) For each variable Yk make a response plot of Yik versus Yik and a residual
plot of Yik versus rik = Yik Yik . If the multivariate linear regression model
is appropriate, then the plotted points should cluster about the identity line
in each of the m response plots. If outliers are present or if the plot is not
linear, then the current model or data need to be transformed or corrected.
If the model is good, then each of the m residual plots should be ellipsoidal
with no trend and should be centered about the r = 0 line. There should not
be any pattern in the residual plot: as a narrow vertical strip is moved from
left to right, the behavior of the residuals within the strip should show little
change. Outliers and patterns such as curvature or a fan shaped plot are bad.
378 12 Multivariate Linear Regression

5) Make a scatterplot matrix of Y1 , . . . , Ym and of the continuous predic-

tors. Use power transformations to remove strong nonlinearities.
6) Consider testing LB = 0 where L is an r p full rank matrix. Let
T T
W e = E E and W e /(n p) =  . Let H = B LT [L(X T X)1 LT ]1 LB.
Let 1 2 m be the ordered eigenvalues of W 1e H. Then there
are four commonly used test statistics.
The Wilks statistic is (L) = |(H + W e )1 W e | = |W 1
e H + I|
1
=
+m
(1 + i )1 .
i=1

m
i
The Pillais trace statistic is V (L) = tr[(H + W e )1 H] = .
i=1
1 + i
m
The Hotelling-Lawley trace statistic is U (L) = tr[W 1
e H] = i .
i=1
The Roys maximum root statistic is max (L) = 1 .
7) Theorem: The Hotelling-Lawley trace statistic
1 1
U (L) = [vec(LB)]T [  (L(X T X)1 LT )1 ][vec(LB)].
np

8) Assumption D1: Let hi be the ith diagonal element of X(X T X)1

P
X T . Assume max(h1 , . . . , hn ) 0 as n , assume that the zero mean iid
1
error vectors have nite fourth moments, and assume that X T X W 1 .
P
n
9) Multivariate Least Squares Central Limit Theorem (MLS
CLT): For the least squares estimator, if assumption D1 holds, then  is
D
a n consistent estimator of  , and n vec(B B) Npm (0,  W ).
10) Theorem: If assumption D1 holds and if H0 is true, then
D
(n p)U (L) 2rm .
D
11) Under regularity conditions, [n p + 1 0.5(m r + 3)] log((L))
D D
2rm , (n p)V (L) 2rm , and (n p)U (L) 2rm .
These statistics are robust against nonnormality.
12) Forthe Wilks Lambda test, 
[n p + 1 0.5(m r + 3)]
pval = P log((L)) < Frm,nrm .
rm  
np
For the Pillais trace test, pval = P V (L) < Frm,nrm .
rm  
np
For the Hotelling Lawley trace test, pval=P U (L) < Frm,nrm .
rm
The above three tests are large sample tests, P(reject H0 |H0 is true)
as n , under regularity conditions.
13) The 4 step MANOVA F test of hypotheses uses L = [0 I p1 ].
i) State the hypotheses H0 : the nontrivial predictors are not needed in the
12.6 Summary 379

mreg model H1 : at least one of the nontrivial predictors is needed.

ii) Find the test statistic Fo from output.
iii) Find the pval from output.
iv) If pval , reject H0 . If pval > , fail to reject H0 . If H0 is rejected,
conclude that there is a mreg relationship between the response variables
Y1 , . . . , Ym and the predictors x2 , . . . , xp . If you fail to reject H0 , conclude
that there is a not a mreg relationship between Y1 , . . . , Ym and the predictors
x2 , . . . , xp . (Get the variable names from the story problem.)
14) The 4 step Fj test of hypotheses uses Lj = [0, . . . , 0, 1, 0, . . . , 0] where
the 1 is in the jth position. Let B Tj be the jth row of B. The hypotheses are
equivalent to H0 : B Tj = 0 H1 : B Tj = 0. i) State the hypotheses
H0 : xj is not needed in the model H1 : xj is needed in the model.
ii) Find the test statistic Fj from output.
iii) Find pval from output.
iv) If pval , reject H0 . If pval > , fail to reject H0 . Give a nontechnical
sentence restating your conclusion in terms of the story problem. If H0 is
rejected, then conclude that xj is needed in the mreg model for Y1 , . . . , Ym .
If you fail to reject H0 , then conclude that xj is not needed in the mreg model
for Y1 , . . . , Ym given that the other predictors are in the model.
15) The 4 step MANOVA partial F test of hypotheses has a full model
using all of the variables and a reduced model where r of the variables are
deleted. The ith row of L has a 1 in the position corresponding to the ith
variable to be deleted. Omitting the jth variable corresponds to the Fj test
while omitting variables x2 , . . . , xp corresponds to the MANOVA F test.
i) State the hypotheses H0 : the reduced model is good
H1 : use the full model.
ii) Find the test statistic FR from output.
iii) Find the pval from output.
iv) If pval , reject H0 and conclude that the full model should be used.
If pval > , fail to reject H0 and conclude that the reduced model is good.
16) The 4 step MANOVA F test should reject H0 if the response and
residual plots look good, n is large enough, and at least one response plot
does not look like the corresponding residual plot. A response plot for Yj will
look like a residual plot if the identity line appears almost horizontal, hence
the range of Yj is small.
17) The lregpack function mltreg produces the m response and residual
plots, gives B,  , the MANOVA partial F test statistic and pval corre-
sponding to the reduced model that leaves out the variables given by indices
(so x2 and x4 in the output below with F = 0.77 and pval = 0.614), Fj
and the pval for the Fj test for variables 1, 2, . . . , p (where p = 4 in the
output below so F2 = 1.51 with pval = 0.284), and F0 and pval for the
MANOVA F test (in the output below F0 = 3.15 and pval= 0.06). The
command out <- mltreg(x,y,indices=c(2)) would produce a MANOVA
partial F test corresponding to the F2 test while the command out <-
mltreg(x,y,indices=c(2,3,4)) would produce a MANOVA partial F test
380 12 Multivariate Linear Regression

corresponding to the MANOVA F test for a data set with p = 4 predictor

variables. The Hotelling Lawley trace statistic is used in the tests.

out <- mltreg(x,y,indices=c(2,4))

$Bhat
[,1] [,2] [,3]
[1,] 47.96841291 623.2817463 179.8867890
[2,] 0.07884384 0.7276600 -0.5378649
[3,] -1.45584256 -17.3872206 0.2337900
[4,] -0.01895002 0.1393189 -0.3885967

$Covhat
[,1] [,2] [,3]
[1,] 21.91591 123.2557 132.339
[2,] 123.25566 2619.4996 2145.780
[3,] 132.33902 2145.7797 2954.082

$partial
partialF Pval
[1,] 0.7703294 0.6141573

$Ftable
Fj pvals
[1,] 6.30355375 0.01677169
[2,] 1.51013090 0.28449166
[3,] 5.61329324 0.02279833
[4,] 0.06482555 0.97701447

$MANOVA
MANOVAF pval
[1,] 3.150118 0.06038742

18) Given B = [ 1 2 m ] and xf , nd y f = (y1 , . . . , ym )T where

T
yi = i xf .

1  T
T n
E E
19)  = = i i while the sample covariance matrix of
np n p i=1
T
np E E
the residuals is S r =  = . Both  and S r are n consistent
n1 n1
estimators of  for a large class of error distributions for i .
20) The 100(1 )% nonparametric prediction region could be applied to
T
the residual vectors or to z i = y f + i = B xf + i for i = 1, . . . , n. This
takes the data cloud of the n residual vectors i and centers the cloud at y f .
12.7 Complements 381

Let
Di2 (y f , S r ) = (z i y f )T S 1
r (z i y f )

for i = 1, . . . , n. Let qn = min(1 + 0.05, 1 + m/n) for > 0.1 and

qn = min(1 /2, 1 + 10m/n), otherwise.

If qn < 1 + 0.001, set qn = 1 . Let 0 < < 1 and h = D(Un ) where

D(Un ) is the qn th sample quantile of the Di . The 100(1 )% nonparametric
prediction region for y f is

{y : (y y f )T S 1
r (y y f ) D(Un ) } = {y : Dy (y f , S r ) D(Un ) }.
2

a) Consider the n prediction regions for the data where (y f,i , xf,i ) =
(y i , xi ) for i = 1, . . . , n. If the order statistic D(Un ) is unique, then Un of the
n prediction regions contain y i where Un /n 1 as n .
b) If (y f , S r ) is a consistent estimator of (E(y f ),  ) then the nonpara-
metric prediction region is a large sample 100(1 )% prediction region for
yf .
c) If (y f , S r ) is a consistent estimator of (E(y f ),  ), and the i come
from an elliptically contoured distribution such that the highest density re-
gion is {y : Dy (0,  ) D1 }, then the nonparametric prediction region
is asymptotically optimal.
21) On the DD plot for the residuals, the cases to the left of the vertical
line correspond to cases that would have y f = y i in the nonparametric
prediction region if xf = xi , while the cases to the right of the line would
not have y f = y i in the nonparametric prediction region.

22) The DD plot for the residuals is interpreted almost exactly as a DD

plot for iid multivariate data is interpreted. Plotted points clustering about
the identity line suggests that the i may be iid from a multivariate normal
distribution, while plotted points that cluster about a line through the origin
with slope greater than 1 suggests that the i may be iid from an elliptically
contoured distribution that is not MVN. The semiparametric and paramet-
ric MVN prediction regions correspond to horizontal lines on the DD plot.
Robust distances have not been shown to be consistent estimators of the
population distances, but are useful for a graphical diagnostic.

12.7 Complements

Multivariate linear regression is a semiparametric method that is nearly as

easy to use as multiple linear regression if m is small. The material on plots
and testing followed Olive et al. (2015) closely. Section 12.3 followed Olive
(2016b) closely. The m response and residual plots should be made as well
as the DD plot, and the response and residual plots are very useful for the
382 12 Multivariate Linear Regression

m = 1 case of multiple linear regression and experimental design. These

plots speed up the model building process for multivariate linear models
since the success of power transformations achieving linearity can be quickly
assessed, and inuential cases can be quickly detected. Work is needed on
variable selection and on determining the sample sizes for when the tests
and prediction regions start to work well. Response and residual plots can
look good for n 10p, but for testing and prediction regions, we may need
n a(m + p)2 where 0.8 a 5 even for well behaved elliptically contoured
error distributions. Cook and Setodji (2003) use the FF plot.
Often observations (Y1 , . . . , Ym , x2 , . . . , xp ) are collected on the same per-
son or thing and hence are correlated. If transformations can be found such
that the m response plots and residual plots look good, and n is large
(n max[(m + p)2 , mp + 30)] starts to give good results), then multivariate
linear regression can be used to eciently analyze the data. Examining m
multiple linear regressions is an incorrect method for analyzing the data.
Important competing methods for multivariate linear regression include
envelope estimators and partial least squares. See recent work by Cook such
as Cook and Su (2013), Cook et al. (2013), and Su and Cook (2012). Olive
(2016c, ch. 12) provides robust multivariate linear regression estimators. Pre-
diction regions for competing methods could be made following Section 12.3.
The R software was used to make plots and software. See R Core Team
(2016). The function mpredsim was used to simulate the prediction regions,
mregsim was used to simulate the tests of hypotheses, and mregddsim sim-
ulated the DD plots for various distributions. The function mltreg makes
the response and residual plots and computes the Fj , MANOVA F , and
MANOVA partial F test pvalues, while the function ddplot4 makes the DD
plots.
Variable selection, also called subset or model selection, is the search for
a subset of predictor variables that can be deleted without important loss of
information. There is the full model x = (xTI , xO )T where xI is a candidate
submodel. It is crucial to verify that a multivariate regression model is appro-
priate for the full model. For each of the m response variables, use the
response plot and the residual plot for the full model to check this
assumption. Variable selection for multivariate linear regression is discussed
in Fujikoshi et al. (2014). R programs are needed to make variable selection
easy. Forward selection would be especially useful.
To do crude variable selection, t the model, leave out the variable with
the largest Fj test pvalue > 0.1, and t the model, and repeat. The statistic
Cp (I) = (p k)(FI 1) + k may also be useful. Here p is the number of
variables in the full model, k is the number of variables in the candidate
model I, and FI is the MANOVA partial F statistic for testing whether the
p k variables xO (in the full model but not in the candidate model I) can
be deleted. Models that have Cp (I) k are certainly interesting. Check the
nal submodel xI for multivariate linear regression with the FF, RR plots,
and the response and residual plots for the full model and for the candidate
12.8 Problems 383

model for each of the m response variables Y1 , . . . , Ym . The submodels use

YIj for j = 1, . . . , m.
If n < 10p, do forward selection until there are J n/10 predictors. Check
that the model with J predictors is reasonable. Then compute Cp (I) for each
model considered in the forward selection.
The theory for multivariate linear regression assumes that the model is
known before gathering data. If variable selection and response transforma-
tions are performed to build a model, then the estimators are biased and
results for inference fail to hold in that pvalues and coverage of condence
and prediction regions will be wrong. When m = 1, see, for example, Berk
(1978), Copas (1983), Miller (1984), and Rencher and Pun (1980). Hence it
is a good idea to do a pilot study to suggest which transformations and vari-
ables to use. Then do a larger study (without using variable selection) using
variables suggested by the pilot study.
There is little competition for the nonparametric prediction region for
multivariate regression. The parametric MVN region works if the errors i
come from an MVN distribution, but this region tends to have volume that
is too small if the error distribution is not MVN. For m not much larger than
2, compute the Hyndman (1996) prediction region or the Lei et al. (2013)
100 qn % prediction region R on the pseudodata z i = y f + i . For m = 1, a
similar procedure is done by Lei and Wasserman (2014).
Khattree and Naik (1999, pp. 9198) discuss testing H0 : LBM = 0
versus H1 : LBM = 0 where M = I gives a linear test of hypotheses.

12.8 Problems

PROBLEMS WITH AN ASTERISK * ARE ESPECIALLY USE-

FUL.

12.1 . Consider the Hotelling Lawley test statistic. Let

1
T (W ) = n [vec(LB)]T [  (LW LT )1 ][vec(LB)].

Let
XT X 1
= W .
n
1
Show T (W ) = [vec(LB)]T [  (L(X T X)1 LT )1 ][vec(LB)].

12.2. Consider the Hotelling Lawley test statistic. Let T =

1
[vec(LB)]T [  (L(X T X)1 LT )1 ][vec(LB)].
384 12 Multivariate Linear Regression

T
Let L = Lj = [0, . . . , 0, 1, 0, . . . , 0] have a 1 in the jth position. Let bj = LB
be the jth row of B. Let dj = Lj (X T X)1 LTj = (X T X)1 jj , the jth diagonal
1 T 1
entry of (X T X)1 . Then Tj = dj bj  bj . The Hotelling Lawley statistic

T
U = tr([(n p)  ]1 B LT [L(X T X)1 LT ]1 LB]).

1 1 T
Hence if L = Lj , then Uj = dj (np) tr(  bj bj ).
Using tr(ABC) = tr(CAB) and tr(a) = a for scalar a, show that
(n p)Uj = Tj .

12.3. Consider the Hotelling Lawley test statistic. Using the Searle (1982,
p. 333) identity

tr(AGT DGC) = [vec(G)]T [CA D T ][vec(G)],

1 T
show (n p)U(L) = tr[  B LT [L(X T X)1 LT ]1 LB]
1
= [vec(LB)]T [  (L(X T X)1 LT )1 ][vec(LB)] by identifying A, G, D,
and C.

$Ftable
Fj pvals
[1,] 82.147221 0.000000e+00
[2,] 58.448961 0.000000e+00
[3,] 15.700326 4.258563e-09
[4,] 9.072358 1.281220e-05
[5,] 45.364862 0.000000e+00

$MANOVA
MANOVAF pval
[1,] 67.80145 0

12.4. The above output is for the R Seatbelts data set where Y1 =
drivers = number of drivers killed or seriously injured, Y2 = f ront = number
of front seat passengers killed or seriously injured, and Y3 = back = num-
ber of back seat passengers killed or seriously injured. The predictors were
x2 = kms = distance driven, x3 = price = petrol price, x4 = van = number
of van drivers killed, and x5 = law = 0 if the law was in eect that month
and 1 otherwise. The data consists of 192 monthly totals in Great Britain
from January 1969 to December 1984, and the compulsory wearing of seat
belts law was introduced in February 1983.
a) Do the MANOVA F test.
b) Do the F4 test.
12.8 Problems 385

12.5. Sketch a DD plot of the residual vectors i for the multivariate

linear regression model if the error vectors i are iid from a multivariate
normal distribution.

y<-USJudgeRatings[,c(9,10,12)]
x<-USJudgeRatings[,-c(9,10,12)]
mltreg(x,y,indices=c(2,5,6,7,8))
$partial
partialF Pval
[1,] 1.649415 0.1855314

$MANOVA
MANOVAF pval
[1,] 340.1018 1.121325e-14

12.6. The above output is for the R judge ratings data set consisting of
lawyer ratings for n = 43 judges. Y1 = oral = sound oral rulings, Y2 = writ =
sound written rulings, and Y3 = rten = worthy of retention. The predictors
were x2 = cont = number of contacts of lawyer with judge, x3 = intg =
judicial integrity, x4 = dmnr = demeanor, x5 = dilg = diligence, x6 =
cf mg = case ow managing, x7 = deci = prompt decisions, x8 = prep =
preparation for trial, x9 = f ami = familiarity with law, and x10 = phys =
physical ability.
a) Do the MANOVA F test.
b) Do the MANOVA partial F test for the reduced model that deletes
x2 , x5 , x6 , x7 , and x8 .

12.7. Let i be p 1 and suppose

     
1 1 0 (X T X)1 12 (X T X)1
N2p , 11 T .
2 2 0 21 (X X)1 22 (X T X)1

Find the distribution of

 
1 1
[L 0] = L 1
2 2

where L 1 = 0 and L is r p with r p. Simplify.

R Problems
Warning: Use the command source(G:/lregpack.txt) to download
the programs. See Preface or Section 14.1. Typing the name of the
lregpack function, e.g. ddplot4, will display the code for the function. Use the
args command, e.g. args(ddplot4), to display the needed arguments for the
function. For some of the following problems, the R commands can be copied
and pasted from (http://lagrange.math.siu.edu/Olive/lrsashw.txt) into R.
386 12 Multivariate Linear Regression

12.8. This problem examines multivariate linear regression on the Cook

and Weisberg (1999a) mussels data with Y1 = log(S) and Y2 = log(M ) where
S is the shell mass and M is the muscle mass. The predictors are X2 = L,
X3 = log(W ), and X4 = H: the shell length, log(width), and height.
a) The R command for this part makes the response and residual plots
for each of the two response variables. Click the rightmost mouse button and
highlight Stop to advance the plot. When you have the response and residual
plots for one variable on the screen, copy and paste the two plots into Word.
Do this two times, once for each response variable. The plotted points fall in
roughly evenly populated bands about the identity or r = 0 line.
b) Copy and paste the output produced from the R command for this part
from $partial on. This gives the output needed to do the MANOVA F test,
MANOVA partial F test, and the Fj tests.
c) The R command for this plot makes a DD plot of the residuals and
adds the lines corresponding to the three prediction regions of Section 12.3.
The robust cuto is larger than the semiparametric cuto. Place the plot in
Word. Do the residuals appear to follow a multivariate normal distribution?
d) Do the MANOVA partial F test where the reduced model deletes X3
and X4 .
e) Do the F2 test.
f) Do the MANOVA F test.

12.9. This problem examines multivariate linear regression on the SAS

Institute (1985, p. 146) Fitness Club Data data with Y1 = chinups, Y2 =
situps, and Y3 = jumps. The predictors are X2 = weight, X3 = waist, and
X4 = pulse.
a) The R command for this part makes the response and residual plots for
each of the three variables. Click the rightmost mouse button and highlight
Stop to advance the plot. When you have the response and residual plots for
one variable on the screen, copy and paste the two plots into Word. Do this
three times, once for each response variable. Are there any outliers?
b) The R command for this part makes a DD plot of the residuals and
adds the lines corresponding to the three prediction regions of Section 12.3.
The robust cuto is larger than the semiparametric cuto. Place the plot in
Word. Are there any outliers? (Right click Stop once.)

12.10. This problem uses the lregpack function mregsim to simulate the
Wilks test, Pillais trace test, Hotelling Lawley trace test, and Roys largest
root test for the Fj tests and the MANOVA F test for multivariate linear
regression. When mnull = T the rst row of B is 1T while the remaining
rows are equal to 0. Hence the null hypothesis for the MANOVA F test is
true. When mnull = F the null hypothesis is true for p = 2, but false for
p > 2. Now the rst row of B is 1T and the last row of B is 0. If p > 2,
then the second to last row of B is (1, 0, . . . , 0), the third to last row is
(1, 1, 0, . . . , 0) et cetera as long as the rst row is not changed from 1T . First
m iid errors z i are generated such that the m errors are iid with variance
12.8 Problems 387

2 . Then i = Az i so that  = 2 AAT = (ij ) where the diagonal entries

ii = 2 [1 + (m 1) 2 ] and the o diagonal entries ij = 2 [2 + (m 2) 2 ]
where = 0.10. Terms like Wilkcov give the percentage of times the Wilks
test rejected the F1 , F2 , . . . , Fp tests. The $mancv wcv pcv hlcv rcv fcv output
gives the percentage of times the 4 test statistics reject the MANOVA F test.
Here hlcov and fcov both correspond to the Hotelling Lawley test using the
formulas in Problem 12.3.
5000 runs will be used so the simulation may take several minutes. Sample
sizes n = (m + p)2 , n = 3(m + p)2 , and n = 4(m + p)2 were interesting. We
want coverage near 0.05 when H0 is true and coverage close to 1 for good
power when H0 is false. Multivariate normal errors were used in a) and b)
below.
a) Copy the coverage parts of the output produced by the R commands
for this part where n = 20, m = 2, and p = 4. Here H0 is true except for
the F1 test. Wilks and Pillais tests had low coverage < 0.05 when H0 was
false. Roys test was good for the Fj tests, but why was Roys test bad for
the MANOVA F test?
b) Copy the coverage parts of the output produced by the R commands
for this part where n = 20, m = 2, and p = 4. Here H0 is false except for the
F4 test. Which two tests seem to be the best for this part?

12.11. This problem uses the lregpack function mpredsim to simulate the
prediction regions for y f given xf for multivariate regression. With 5000 runs
this simulation may take several minutes. The R command for this problem
generates iid lognormal errors then subtracts the mean, producing z i . Then
the i = Az i are generated as in Problem 12.10 with n=100, m=2, and p=4.
The nominal coverage of the prediction region is 90%, and 92% of the training
data is covered. The ncvr output gives the coverage of the nonparametric
region. What was ncvr?
Chapter 13
GLMs and GAMs

This chapter contains some extensions of the multiple linear regression model.
See Denition 1.1 for the 1D regression model, sucient predictor (SP =
h(x)), estimated sucient predictor (ESP = h(x)), generalized linear model
(GLM), and the generalized additive model (GAM). When using a GAM
to check a GLM, the notation ESP may be used for the GLM, and EAP
(estimated additive predictor) may be used for the ESP of the GAM. De-
nition 1.2 denes the response plot of ESP versus Y .
Suppose the sucient predictor SP = h(x). Often SP = T x if 1 cor-
responds to the constant and x1 1. If x only contains the nontrivial pre-
dictors, then SP = + T x is often used. Much of Chapter 1 examines this
special case, including response plots, variable selection, interactions, factors,
and the interpretation of the parameters i .

13.1 Introduction

First we describe some regression models in the following three denitions.

The most general model uses SP = h(x) as dened in Denition 1.1. The
GAM with SP = AP will be useful for checking the model (often a GLM)
with SP = + T x. Thus the additive error regression model with SP = AP
is useful for checking the multiple linear regression model. The model with
SP = + T x tends to have the most theory for inference and variable
selection. For the models below, the model estimated mean function and
often a nonparametric estimator of the mean function, such as lowess, will
be added to the response plot as a visual aid. For all of the models in the
following three denitions, Y1 , ..., Yn are independent, but often the subscripts
are suppressed. For example, Y = SP + e is used instead of Yi = Yi |xi =
Yi |SPi = SPi + ei = h(xi ) + ei for i = 1, ..., n.

Springer International Publishing AG 2017 389

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 13
390 13 GLMs and GAMs

Denition 13.1. i) The additive error regression model Y = SP +

e has conditional mean function E(Y |SP ) = SP and conditional variance
function V (Y |SP ) = 2 = V (e). See Section 13.2. The response plot of ESP
versus Y and the residual plot of ESP versus r = Y Y are used just
as for multiple linear regression. The estimated model (conditional) mean
function is the identity line Y = ESP . The response transformation model
is Y = t(Z) = SP + e where the response transformation t(Z) can be found
using a graphical method similar to Section 3.2.
 
eSP
ii) The binary regression model is Y binomial 1, = .
1 + eSP
This model has E(Y |SP ) = = (SP ) and V (Y |SP ) = (SP )(1 (SP )).
eESP
Then = is the estimated mean function. See Section 13.3.
1 + eESP
 
eSP
iii) The binomial regression model is Yi binomial mi , = .
1 + eSP
Then E(Yi |SPi ) = mi (SPi ) and V (Yi |SPi ) = mi (SPi )(1 (SPi )), and
mi eESP
E(Yi |xi ) = mi = is the estimated mean function. See Sec-
1 + eESP
tion 13.3.
# $
iv) The Poisson regression (PR) model Y Poisson eSP has
E(Y |SP ) = V (Y |SP ) = exp(SP ). The estimated mean and variance func-
tions are E(Y |x) = eESP . See Section 13.4.
v) Suppose Y has a gamma G(, ) distribution so that E(Y ) = and
V (Y ) = 2 . The Gamma regression model Y G (, = (SP )/)
has E(Y |SP ) = (SP ) and V (Y |SP ) = [(SP )]2 /. The estimated mean
function is E(Y |x) = (ESP ). The choices (SP ) = SP , (SP ) = exp(SP )
and (SP ) = 1/SP are common. Since (SP ) > 0, Gamma regression mod-
els that use the identity or reciprocal link run into problems if (ESP ) is
negative for some of the cases.

Alternatives to the binomial and Poisson regression models are needed

because often the mean function for the model is good, but the variance
function is not: there is overdispersion. See Section 13.8.

A useful alternative to the binomial regression model is a betabinomial

regression (BBR) model. Following Simono (2003, pp. 9394) and Agresti
(2002, pp. 554555), let = / and = (1 )/, so = /( + ) and
() ()
= 1/(+). Let B(, ) = . If Y has a betabinomial distribution,
( + )
  BB(m, , ), then the probability mass function of Y is P (Y = y) =
Y
m B( + y, + m y)
for y = 0, 1, 2, ..., m where 0 < < 1 and > 0.
y B(, )
Hence > 0 and > 0. Then E(Y ) = m/( + ) = m and V(Y ) =
m(1 )[1 + (m 1)/(1 + )]. If Y | binomial(m, ) and beta(, ),
13.1 Introduction 391

then Y BB(m, , ). As 0, it can be shown that V () 0, and the

betabinomial distribution converges to the binomial distribution.

Denition 13.2. The BBR model states that Y1 , ..., Yn are independent
random variables where Yi |SPi BB(mi , (SPi ), ). Hence E(Yi |SPi ) =
mi (SPi ) and

V (Yi |SPi ) = mi (SPi )(1 (SPi ))[1 + (mi 1)/(1 + )].

The BBR model has the same mean function as the binomial regression
model, but allows for overdispersion. As 0, it can be shown that the
BBR model converges to the binomial regression model.

A useful alternative to the PR model is a negative binomial regression

(NBR) model. If Y has a (generalized) negative binomial distribution, Y
N B(, ), then the probability mass function of Y is
   y
(y + )
P (Y = y) = 1
() (y + 1) + +

for y = 0, 1, 2, ... where > 0 and > 0. Then E(Y ) = and V(Y ) =
+ 2 /. (This distribution is a generalization of the negative binomial (, )
distribution where = /( + ) and > 0 is an unknown real parameter
rather than a known integer.)

Denition 13.3. The negative binomial regression (NBR) model

is Y |SP NB(exp(SP), ). Thus E(Y |SP ) = exp(SP ) and
 
exp(SP )
V (Y |SP ) = exp(SP ) 1 + = exp(SP ) + exp(2 SP ).

The NBR model has the same mean function as the PR model but allows
for overdispersion. Following Agresti (2002, p. 560), as 1/ 0, it can
be shown that the NBR model converges to the PR model.

Generalized linear models are an important class of parametric 1D regres-

sion models that include multiple linear regression, logistic regression, and
Poisson regression. Assume that there is a response variable Y and a k 1
vector of nontrivial predictors x. Before dening a generalized linear model,
the denition of a one parameter exponential family is needed. Let f (y) be
a probability density function (pdf) if Y is a continuous random variable,
and let f (y) be a probability mass function (pmf) if Y is a discrete random
variable. Assume that the support of the distribution of Y is Y and that the
parameter space of is .

Denition 13.4. A family of pdfs or pmfs {f (y|) : } is a 1-

parameter exponential family if
392 13 GLMs and GAMs

f (y|) = k()h(y) exp[w()t(y)] (13.1)

where k() 0 and h(y) 0. The functions h, k, t, and w are real valued
functions.

In the denition, it is crucial that k and w do not depend on y and that

h and t do not depend on . The parameterization is not unique since, for
example, w could be multiplied by a nonzero constant m if t is divided by m.
Many other parameterizations are possible. If h(y) = g(y)IY (y), then usually
k() and g(y) are positive, so another parameterization is

f (y|) = exp[w()t(y) + d() + S(y)]IY (y) (13.2)

where S(y) = log(g(y)), d() = log(k()), and the support Y does not depend
on . Here the indicator function IY (y) = 1 if y Y and IY (y) = 0, otherwise.
Denition 13.5. Assume that the data is (Yi , xi ) for i = 1, ..., n. An im-
portant type of generalized linear model (GLM) for the data states that
the Y1 , ..., Yn are independent random variables from a 1-parameter exponen-
tial family with pdf or pmf
 
c((xi ))
f (yi |(xi )) = k((xi ))h(yi ) exp yi . (13.3)
a()

Here is a known constant (often a dispersion parameter), a() is a known

function, and (xi ) = ( + T xi ). Let E(Yi ) E(Yi |xi ) = (xi ). The
GLM also states that g((xi )) = + T xi where the link function g is
a dierentiable monotone function. Then the canonical link function is
g((xi )) = c((xi )) = + T xi , and the quantity + T x is called the
linear predictor.
The GLM parameterization (13.3) can be written in several ways. By
Equation (13.2), f (yi |(xi )) = exp[w((xi ))yi + d((xi )) + S(y)]IY (y) =
 
c((xi )) b(c((xi ))
exp yi + S(y) IY (y)
a() a()
 
i b(i )
= exp yi + S(y) IY (y)
a() a()
where i = c((xi )) is called the natural parameter, and b() is some known
function.
Notice that a GLM is a parametric model determined by the 1-parameter
exponential family, the link function, and the linear predictor. Since the link
function is monotone, the inverse link function g 1 () exists and satises

(xi ) = g 1 ( + T xi ). (13.4)
13.2 Additive Error Regression 393

Also notice that the Yi follow a 1-parameter exponential family where

c()
t(yi ) = yi and w() = ,
a()

and notice that the value of the parameter (xi ) = ( + T xi ) depends

on the value of xi . Since the model depends on x only through the linear
predictor + T x, a GLM is a 1D regression model. Thus the linear predictor
is also a sucient predictor.

The end of Section 1.1 discusses several things to check and consider after
selecting a 1D regression model. The following three sections illustrate three
of the most important generalized linear models. Inference and variable selec-
tion for these GLMs are discussed in Sections 13.5 and 13.6. Their generalized
additive model analogs are discussed in Section 13.7.

13.2 Additive Error Regression

The linear regression model Y = SP + e = + T x + e includes multiple

linear regression and many experimental design models as special cases. These
models were covered in Chapters 29.
If Y is quantitative, a useful extension is the additive error regression
model Y = SP + e where SP = h(x). See Denition 13.1 i). If e N (0, 2 ),
then Y N (SP, 2 ). If SP = + T x, then the resulting linear regression
model is also a GLM and an additive error regression model. The normality
assumption is too restrictive since the error distribution is rarely normal. If
m is a smooth function, the additive error single index model, where SP =
m( + T x), is an important special case.
Response plots, residual plots, and response transformations for the addi-
tive error regression model are very similar to those for the linear regression
models. See Olive (2004b). Prediction intervals are given in Olive (2013a).
The GAM additive error regression model is useful for checking the mul-
T
tiple linear regression (MLR) model. Let ESP = + x be the ESP for
p
MLR where x = (x1 , ..., xp )T . Let ESP = EAP = + j=1 Sj (xj ) be the
ESP for the GAM additive error regression model.
After making the usual checks on the MLR model, there are two useful
plots that use the GAM. If the plotted points of the EE plot of EAP versus
ESP cluster tightly about the identity line, then the MLR and the GAM
produce similar tted values. Ceres plots and partial residual plots, see De-
nition 3.15, can be useful for visualizing whether a predictor transformation
tj (xj ) is needed for the jth predictor xj . A plot of xj versus Sj (xj ) is also
useful. If the plot is linear, then no transformation may be needed. If the plot
is nonlinear, the shape of the plot, along with ceres plots and the graphical
394 13 GLMs and GAMs

methods of Section 3.1, may be useful for suggesting the transformation tj .

The additive error regression GAM can be t with all p of the Sj unspecied,
or t p GAMs where Si is linear except for unspecied Sj where j = 1, ..., p.
Some of these applications for checking GLMs with GAMs will be discussed
in Section 13.7.
Suppose SP = m( + T x). Olive (2008: ch. 12, 2010: ch. 15), Olive and
Hawkins (2005), and Chang and Olive (2010) show that variable selection
methods using Cp and the partial F test, originally meant for multiple linear
regression, can be used for the additive error single index model.

13.3 Binary, Binomial, and Logistic Regression

Multiple linear regression is used when the response variable is quantitative,

but for many data sets the response variable is categorical and takes on two
values: 0 or 1. The occurrence of the category that is counted is labelled as a
1 or a success, while the nonoccurrence of the category that is counted is
labelled as a 0 or a failure. For example, a success = occurrence could
be a person who contracted lung cancer and died within 5 years of detection.
Often the labelling is arbitrary, e.g., if the response variable is gender taking
on the two categories female and male. If males are counted, then Y = 1 if the
subject is male and Y = 0 if the subject is female. If females are counted, then
this labelling is reversed. For a binary response variable, a binary regression
model is often appropriate.
Denition 13.6. The binomial regression model states that Y1 , ..., Yn
are independent random variables with Yi binomial(mi , (xi )). The binary
regression model is the special case where mi 1 for i = 1, ..., n while the
logistic regression (LR) model is the special case of binomial regression
where
exp( + T xi )
P (success|xi ) = (xi ) = . (13.5)
1 + exp( + T xi )
If the sucient predictor SP = + T x, then the most used binomial
regression models are such that Y1 , ..., Yn are independent random variables
with Yi binomial(mi , ( + T xi )), or

Yi |SPi binomial(mi , (SPi )). (13.6)

Note that the conditional mean function E(Yi |SPi ) = mi (SPi ) and the
conditional variance function V (Yi |SPi ) = mi (SPi )(1 (SPi )).
Thus the binary logistic regression model says that

Y |SP binomial(1, (SP))

13.3 Binary, Binomial, and Logistic Regression 395

where
exp(SP )
(SP ) =
1 + exp(SP )
for the LR model. Note that the conditional mean function E(Y |SP ) =
(SP ) and the conditional variance function V (Y |SP ) = (SP )(1 (SP )).
For the LR model, the Y are independent and
 
exp(ESP)
Y |x binomial 1, ,
1 + exp(ESP)

or Y |SP Y |ESP binomial(1, (ESP)).

To see that the binary logistic regression model is a GLM, assume that Y is
a binomial(1, ) random variable. For a one parameter family, take a() 1.
Then the pmf of Y is
   
1 y 1
f (y) = P (Y = y) = (1 )1y = (1 ) exp[log( ) y].
y y . /0 1 1
./01 k()0 . /0 1
h(y)0 c()

Hence this family is a 1-parameter

 exponential
 family with = = E(Y )

and canonical link c() = log . This link is known as the logit link,
1
and if g((x)) = g((x)) = c((x)) = + T x then the inverse link satises

exp( + T x)
g 1 ( + T x) = = (x) = (x).
1 + exp( + T x)

Hence the GLM corresponding to the binomial(1, ) distribution with canon-

ical link is the binary logistic regression model.
Although the logistic regression model is the most important model for
binary regression, several other models are also used. Notice that (x) =
P (S|x) is the population probability of success S given x, while 1 (x) =
P (F |x) is the probability of failure F given x. In particular, for binary re-
gression, (x) = P (Y = 1|x) = 1 P (Y = 0|x). If this population pro-
portion = ( + T x), then the model is a 1D regression model. The
model is a GLM if the link function g is dierentiable and monotone so
that g(( + T x)) = + T x and g 1 ( + T x) = ( + T x). Usually
the inverse link function corresponds to the cumulative distribution func-
tion of a location scale family. For example, for logistic regression, g 1 (x) =
exp(x)/(1 + exp(x)) which is the cdf of the logistic L(0, 1) distribution. For
probit regression, g 1 (x) = (x) which is the cdf of the normal N (0, 1) dis-
tribution. For the complementary log-log link, g 1 (x) = 1 exp[ exp(x)]
which is the cdf for the smallest extreme value distribution. For this model,
g((x)) = log[ log(1 (x))] = + T x.
396 13 GLMs and GAMs

Another important binary regression model is the discriminant function

model. See Hosmer and Lemeshow (2000, pp. 4344) . Assume that j =
P (Y = j) and that x|Y = j Nk (j , ) for j = 0, 1. That is, the conditional
distribution of x given Y = j follows a multivariate normal distribution with
mean vector j and covariance matrix which does not depend on j. Notice
that = Cov(x|Y ) = Cov(x). Then as for the binary logistic regression
model,
exp( + T x)
P (Y = 1|x) = (x) = .
1 + exp( + T x)
Denition 13.7. Under the conditions above, the discriminant func-
tion parameters are given by

= 1 (1 0 ) (13.7)
 
1
and = log 0.5(1 0 )T 1 (1 + 0 ).
0
The logistic regression (maximum likelihood) estimator also tends to per-
form well for this type of data. An exception is when the Y = 0 cases and
Y = 1 cases can be perfectly or nearly perfectly classied by the ESP. Let the
T
logistic regression ESP = + x. Consider the response plot of the ESP
versus Y . If the Y = 0 values can be separated from the Y = 1 values by
the vertical line ESP = 0, then there is perfect classication. See Figure 13.1
b). In this case the maximum likelihood estimator for the logistic regression
parameters (, ) does not exist because the logistic curve cannot approxi-
mate a step function perfectly. See Atkinson and Riani (2000, pp. 251254).
If only a few cases need to be deleted in order for the data set to have per-
fect classication, then the amount of overlap is small and there is nearly
perfect classication.
Ordinary least squares (OLS) can also be useful for logistic regression. The
ANOVA F test, partial F test, and OLS t tests are often asymptotically valid
when the conditions in Denition 13.7 are met, and the OLS ESP and LR
ESP are often highly correlated. See Haggstrom (1983) and Theorem 13.1
below. Assume that Cov(x) x and that Cov(x, Y ) = x,Y . Let j =
E(x|Y = j) for j = 0, 1. Let Ni be the number of Ys that are equal to i for
i = 0, 1. Then
1 
i = xj
Ni
j:Yj =i

for i = 0, 1 while i = Ni /n and 1 = 1 0 . Notice that Theorem 13.1 holds

as long as Cov(x) is nonsingular and Y is binary with values 0 and 1. The
LR and discriminant function models need not be appropriate.
Theorem 13.1. Assume that Y is binary and that Cov(x) = x is non-
singular. Let (OLS , OLS ) be the OLS estimator found from regressing Y
13.3 Binary, Binomial, and Logistic Regression 397

on a constant and x (using software originally meant for multiple linear re-
gression). Then
1 1
OLS = x xY = 0 1 x (1 0 )

OLS = 0 1 1
P
x (1 0 ) as n .
Proof. From Theorem 11.19,
1 P
OLS = x xY OLS as n

1
n
and xY = xi Yi x Y .
n i=1

1  
Thus xY = xj (1) + xj (0) x 1 =
n
j:Yj =1 j:Yj =0

1 1
(N1 1 ) (N1 1 + N0 0 )1 = 1 1 12 1 1 0 0 =
n n
1 (1 1 )1 1 0 0 = 1 0 (1 0 )
and the result follows. 

The discriminant function estimators D and D are found by replacing

the population quantities 1 , 0 , 1 , 0 , and by sample quantities. Also

n2 1
D = x OLS .
N0 N1
Now when the conditions of Denition 13.7 are met and if 1 0 is small
enough so that there is not perfect classication, then LR = 1 (1 0 ).
Empirically, the OLS ESP and LR ESP are highly correlated for many LR
data sets where the conditions are not met, e.g. when some of the predictors
are factors. This suggests that LR d 1 x (1 0 ) for many LR data
sets where d is some constant depending on the data.

Using Denition 13.7 makes simulation of logistic regression data straight-

forward. Set 0 = 1 = 0.5, = I, and 0 = 0. Then = 0.5T1 1
and = 1 . For binary data the Yi only take two values, 0 and 1, and the
residuals do not behave very well. Hence the response plot will be used both
as a goodness of t plot and as a lack of t plot.

Denition 13.8. For binary logistic regression, the response plot or esti-
T
mated sucient summary plot is the plot of the ESP = + xi versus Yi
with the estimated mean function
398 13 GLMs and GAMs

exp(ESP )
(ESP ) =
1 + exp(ESP )

added as a visual aid.

A scatterplot smoother such as lowess is also added as a visual aid. Alter-

natively, divide the ESP into J slices with approximately the same number
of cases in each slice. Then
 compute
 the sample mean = sample proportion
in slice s: s = Y s = s Yi / s mi where mi 1 and the sum is over the
cases in slice s. Then plot the resulting step function. 
Suppose that x is a k 1 vector of predictors, N1 = Yi = the num-
ber of 1s and N0 = n N1 = the number of 0s. Also assume that k
min(N0 , N1 )/5. Then if the parametric estimated mean function (ESP )
looks like a smoothed version of the step function, then the LR model is likely
to be useful. In other words, the observed slice proportions should scatter
fairly closely about the logistic curve (ESP ) = exp(ESP )/[1 + exp(ESP )].
The response plot is a powerful method for assessing the adequacy of the
binary LR regression model. Suppose that both the number of 0s and the
number of 1s is large compared to the number of predictors k, that the ESP
takes on many values and that the binary LR model is a good approximation
to the data. Then Y |ESP binomial(1, (ESP ). Unlike the response plot
for multiple linear regression where the mean function is always the identity
line, the mean function in the response plot for LR can take a variety of
shapes depending on the range of the ESP. For LR, the (estimated) mean
function is
exp(ESP )
(ESP ) = .
1 + exp(ESP )

If the ESP = 0, then Y |SP binomial(1,0.5). If the ESP = 5, then Y |SP

binomial(1, 0.007) while if the ESP = 5, then Y |SP binomial(1,
0.993). Hence if the range of the ESP is in the interval (, 5), then the
mean function is at and (ESP ) 0. If the range of the ESP is in the
interval (5, ), then the mean function is again at but (ESP ) 1. If
5 < ESP < 0, then the mean function looks like a slide. If 1 < ESP <
1 then the mean function looks linear. If 0 < ESP < 5, then the mean
function rst increases rapidly and then less and less rapidly. Finally, if 5 <
ESP < 5 then the mean function has the characteristic ESS shape shown
in Figure 13.1 c).
This plot is very useful as a goodness of t diagnostic. Divide the ESP into
J slices each containing approximately n/J cases. Compute the sample
mean = sample proportion of the Y s in each slice and add the resulting step
function to the response plot. This is done in Figure 13.1 c) with J = 4
slices. This step function is a simple nonparametric estimator of the mean
function (SP ). If the step function follows the estimated LR mean function
(the logistic curve) closely, then the LR model ts the data well. The plot
13.3 Binary, Binomial, and Logistic Regression 399

of these two curves is a graphical approximation of the goodness of t tests

described in Hosmer and Lemeshow (2000, pp. 147156).
The deviance test described in Section 13.5 is used to test whether = 0,
and is the analog of the ANOVA F test for multiple linear regression. If the
binary LR model is a good approximation to the data but = 0, then the
predictors x are not needed in the model and (xi ) = Y (the usual
univariate estimator of the success proportion) should be used instead of the
LR estimator
T
exp( + xi )
(xi ) = T
.
1 + exp( + xi )
If the logistic curve clearly ts the step function better than the line Y = Y ,
then H0 will be rejected, but if the line Y = Y ts the step function about
as well as the logistic curve (which should only happen if the logistic curve
is linear with a small slope), then Y may be independent of the predictors.
See Figure 13.1 a).

For binomial logistic regression, the response plot needs to be modied

and a check for overdispersion is needed.
Denition 13.9. Let Zi = Yi /mi . Then the conditional distribution Zi |xi
of the LR binomial regression model can be visualized with a response plot
T
of the ESP = + xi versus Zi with the estimated mean function

exp(ESP )
(ESP ) =
1 + exp(ESP )

added as a visual aid. Divide the ESP into J slices with

approximately
 the
same number of cases in each slice. Then compute s = s Yi / s mi where
the sum is over the cases in slice s. Then plot the resulting step function
or the lowess curve. For binary data the step function is simply the sample
proportion in each slice.

Both the lowess curve and step function are simple nonparametric estima-
tors of the mean function (SP ). If the lowess curve or step function tracks
the logistic curve (the estimated mean) closely, then the LR mean function
is a reasonable approximation to the data.
Checking the LR model in the nonbinary case is more dicult because
the binomial distribution is not the only distribution appropriate for data
that takes on values 0, 1, ..., m if m 2. Hence both the mean and variance
functions need to be checked. Often the LR mean function is a good approx-
imation to the data, the LR MLE is a consistent estimator of , but the
LR model is not appropriate. The problem is that for many data sets where
E(Yi |xi ) = mi (SPi ), it turns out that V (Yi |xi ) > mi (SPi )(1 (SPi )).
This phenomenon is called overdispersion. The BBR model of Denition 13.2
is a useful alternative to LR.
400 13 GLMs and GAMs

For both the LR and BBR models, the conditional distribution of Y |x can
still be visualized with a response plot of the ESP versus Zi = Yi /mi with the
estimated mean function E(Zi |xi ) = (SP ) = (ESP ) and a step function
or lowess curve added as visual aids.
Since the binomial regression model is simpler than the BBR model, graph-
ical diagnostics for the goodness of t of the LR model would be useful. The
following plot was suggested by Olive (2013b) to check for overdispersion.

Denition 13.10. To check for overdispersion, use the OD plot of the

estimated model variance VM V (Y |SP ) versus the squared residuals V =
[Y E(Y |SP )]2 . For the LR model, V (Yi |SP ) = mi (ESPi )(1 (ESPi ))
and E(Yi |SP ) = mi (ESPi ).

Numerical summaries are also available. The deviance G2 is a statistic

used to assess the goodness of t of the logistic regression model much as R2
is used for multiple linear regression. When the mi are small, G2 may not
be reliable but the response plot is still useful. If the Yi are not too close
to 0 or mi , if the response and OD plots look good, and the deviance G2
satises G2 /(n k 1) 1, then the LR model is likely useful. If G >
2

(n k 1) + 3 n k + 1, then a more complicated count model may be

needed.
Combining the response plot with the OD plot is a powerful method for
assessing the adequacy of the LR model. To motivate the OD plot, recall that
if a count Y is not too close to 0 or m, then a normal approximation is good
for the binomial distribution. Notice that if Yi = E(Y |SP ) + 2 V (Y |SP ),
then [Yi E(Y |SP )]2 = 4V (Y |SP ). Hence if both the estimated mean and
estimated variance functions are good approximations, and if the counts are
not too close to 0 or mi , then the plotted points in the OD plot will scatter
about a wedge formed by the V = 0 line and the line through the origin
with slope 4: V = 4V (Y |SP ). Only about 5% of the plotted points should
be above this line.
When the counts are small, the OD plot is not wedge shaped, but if the LR
model is correct, the least squares (OLS) line should be close to the identity
line through the origin with unit slope. If the data are binary, the response
plot is enough to check the binomial regression assumption.
Suppose the bulk of the plotted points in the OD plot fall in a wedge.
Then the identity line, slope 4 line, and OLS line will be added to the plot
as visual aids. It is easier to use the OD plot to check the variance function
than the response plot since judging the variance function with the straight
lines of the OD plot is simpler than judging the variability about the logistic
curve. Also outliers are often easier to spot with the OD plot. For the LR
model, V (Yi |SP ) = mi (ESPi )(1 (ESPi )) and E(Yi |SP ) = mi (ESPi ).
The evidence of overdispersion increases from slight to high as the scale of the
vertical axis increases from 4 to 10 times that of the horizontal axis. There is
considerable evidence of overdispersion if the scale of the vertical axis is more
13.3 Binary, Binomial, and Logistic Regression 401

than 10 times that of the horizontal, or if the percentage of points above the
slope 4 line through the origin is much larger than 5%.
If the binomial LR OD plot is used but the data follows a betabinomial re-
gression model, then Vmod = V (Yi |SP ) mi (ESP )(1(ESP )) while V =
[Yi mi (ESP )]2 (Yi E(Yi ))2 . Hence E(V ) V (Yi ) mi (ESP )(1
(ESP ))[1 + (mi 1)/(1 + )], so the plotted points with mi = m should
1 + m
scatter about a line with slope 1 + (m 1) = .
1+ 1+

a) b)
0.6

0.6
Y

Y
0.0

1.6 1.2 0.8 0.0 50 0 50

ESP ESP

c) ESSP d) OD Plot
0.0 0.4 0.8
0.6

Vhat
Y
0.0

10 0 5 10 0.00 0.10 0.20

ESP Vmodhat
Fig. 13.1 Response Plots for Museum Data

The rst example is for binary data. For binary data, G2 is not approxi-
mately 2 and some plots of residuals have a pattern whether the model is
correct or not. For binary data the OD plot is not needed, and the plotted
points follow a curve rather than falling in a wedge. The response plot is
very useful if the logistic curve and step function of observed proportions are
added as visual aids. The logistic curve gives the estimated LR probability
of success. For example, when ESP = 0, the estimated probability is 0.5.

Example 13.1. Schaahausen (1878) gives data on skulls at a museum.

The 1st 47 skulls are humans while the remaining 13 are apes. The response
variable ape is 1 for an ape skull. The response plot in Figure 13.1 a) uses
the predictor face length. The model ts very poorly since the probability
of a 1 decreases then increases. The response plot in Figure 13.1b) uses the
402 13 GLMs and GAMs

a) ESSP b) OD Plot

1.2
0.8

Vhat
0.6
Z
0.4

0.0
0.0

4 0 4 0.5 2.0
ESP Vmodhat
Fig. 13.2 Visualizing the Death Penalty Data

predictor head height and perfectly classies the data since the ape skulls
can be separated from the human skulls with a vertical line at ESP = 0.
Christmann and Rousseeuw (2001) also used the response plot to visualize
overlap. The response plot in Figure 13.1c uses predictors lower jaw length,
face length, and upper jaw length. None of the predictors is good individually,
but together provide a good LR model since the observed proportions (the
step function) track the model proportions (logistic curve) closely. The OD
plot in Figure 13.1d) is curved and is not needed for a binary response.

Example 13.2. Abraham and Ledolter (2006, pp. 360364) describe

death penalty sentencing in Georgia. The predictors are aggravation level
from 1 to 6 (treated as a continuous variable) and race of victim coded as
1 for white and 0 for black. There were 362 jury decisions and 12 level race
combinations. The response variable was the number of death sentences in
each combination. The response plot (ESSP) in Figure 13.2a shows that the
Yi /mi are close to the estimated LR mean function (the logistic curve). The
step function based on 5 slices also tracks the logistic curve well. The OD
plot is shown in Figure 13.2b with the identity, slope 4, and OLS lines added
as visual aids. The vertical scale is less than the horizontal scale, and there
is no evidence of overdispersion.

Example 13.3. Collett (1999, pp. 216219) describes a data set where
the response variable is the number of rotifers that remain in suspension in
a tube. A rotifer is a microscopic invertebrate. The two predictors were the
density of a stock solution of Ficolli and the species of rotifer coded as 1
13.4 Poisson Regression 403

a) ESSP b) OD Plot

0.8

1000
Vhat
Z
0.4

0
0.0

3 0 2 0 20 40
ESP Vmodhat
Fig. 13.3 Plots for Rotifer Data

for polyarthra major and 0 for keratella cochlearis. Figure 13.3a shows the
response plot (ESSP). Both the observed proportions and the step function
track the logistic curve well, suggesting that the LR mean function is a good
approximation to the data. The OD plot suggests that there is overdispersion
since the vertical scale is about 30 times the horizontal scale. The OLS line
has slope much larger than 4 and two outliers seem to be present.

13.4 Poisson Regression

If the response variable Y is a count, then the Poisson regression model is

often useful. For example, counts often occur in wildlife studies where a region
is divided into subregions and Yi is the number of a specied type of animal
found in the subregion.
Denition 13.11. The Poisson regression (PR) model states that
Y1 , ..., Yn are independent random variables with Yi Poisson((xi )) where
(xi ) = exp( + T xi ). Thus Y |SP Poisson(exp(SP)). Notice that
Y |SP = 0 Poisson(1). Note that the conditional mean and variance func-
tions are equal: E(Y |SP ) = V (Y |SP ) = exp(SP ).
To see that the PR model is a GLM, assume that Y is a Poisson() random
variable. For a one parameter family, take a() 1. Then the pmf of Y is

e y 1
f (y) = P (Y = y) = e
= ./01 exp[log() y]
y! y! . /0 1
k()0 ./01 c()
h(y)0

for y = 0, 1, . . . , where > 0. Hence this family is a 1-parameter exponential

family with = = E(Y ), and the canonical link is the log link c() =
log(). Since g((x)) = c((x)) = + T x, the inverse link satises
404 13 GLMs and GAMs

g 1 ( + T x) = exp( + T x) = (x).

Hence the GLM corresponding to the Poisson() distribution with canonical

link is the Poisson regression model.

In the response plot for Poisson regression, the shape of the estimated
mean function (ESP ) = exp(ESP ) depends strongly on the range of the
ESP. The variety of shapes occurs because the plotting software attempts
to ll the vertical axis. Hence if the range of the ESP is narrow, then the
exponential function will be rather at. If the range of the ESP is wide, then
the exponential curve will look at in the left of the plot but will increase
sharply in the right of the plot.

Denition 13.12. The estimated sucient summary plot (ESSP) or re-

T
sponse plot is a plot of the ESP = + xi versus Yi with the estimated
mean function
(ESP ) = exp(ESP )
added as a visual aid. A scatterplot smoother such as lowess is also added as
a visual aid.
This plot is very useful as a goodness of t diagnostic. The lowess curve
is a nonparametric estimator of the mean function and is represented as a
jagged curve to distinguish it from the estimated PR mean function (the
exponential curve). See Figure 13.4a). If the number of predictors k < n/10,
if there is no overdispersion, and if the lowess curve follows the exponential
curve closely (except possibly for the largest values of the ESP), then the PR
mean function may be a useful approximation for E(Y |x). A useful lack
of t plot is a plot of the ESP versus the deviance residuals that are often
available from the software.

The deviance test described in Section 13.5 is used to test whether = 0,

and is the analog of the ANOVA F test for multiple linear regression. If the
PR model is a good approximation to the data but = 0, then the predictors
x are not needed in the model and (xi ) = Y (the sample mean) should
be used instead of the PR estimator
T
(xi ) = exp( + xi ).

If the exponential curve clearly ts the lowess curve better than the line
Y = Y , then H0 should be rejected, but if the line Y = Y ts the lowess
curve about as well as the exponential curve (which should only happen if
the exponential curve is approximately linear with a small slope), then Y
may be independent of the predictors. See Figure 13.6a).

Warning: For many count data sets where the PR mean function is good,
the PR model is not appropriate but the PR MLE is still a consistent esti-
13.4 Poisson Regression 405

mator of . The problem is that for many data sets where E(Y |x) = (x) =
exp(SP ), it turns out that V (Y |x) > exp(SP ). This phenomenon is called
overdispersion. Adding parametric and nonparametric estimators of the
standard deviation function to the response plot can be useful. See and Cook
and Weisberg (1999a, pp. 401403). The NBR model of Denition 13.3 is a
useful alternative to PR.

Since the Poisson regression model is simpler than the NBR model, graph-
ical diagnostics for the goodness of t of the PR model would be useful. The
following plot was suggested by Winkelmann (2000, p. 110).

Denition 13.13. To check for overdispersion, use the OD plot of the

estimated model variance VM V (Y |SP ) versus the squared residuals V =
[Y E(Y |SP )]2 . For the PR model, V (Y |SP ) = exp(ESP ) = E(Y |SP ) and
V = [Y exp(ESP )]2 .

Numerical summaries are also available. The deviance G2 , described in

Section 13.5, is a statistic used to assess the goodness of t of the Poisson
regression model much as R2 is used for multiple linear regression. For Poisson
regression, G2 is approximately chi-square with n p 1 degrees of freedom.
Since a 2d random variable has mean d and standard deviation
2d, the
98th
percentile of the 2d distribution is approximately d + 3 d d + 2.121 2d.
If the response and OD plots look good, and G2 /(n k 1) 1, then the
PR model is likely useful. If G2 > (n k 1) + 3 n k 1, then a more
complicated count model than PR may be needed. A good discussion of such
count models is in Simono (2003).
For PR, Winkelmann (2000, p. 110) suggested that the plotted points in
the OD plot should scatter about the identity line through the origin with unit
slope and that the OLS line should be approximately equal to the identity
line if the PR model is appropriate. But in simulations, it was found that the
following two observations make the OD plot much easier to use for Poisson
regression.
First, recall that a normal approximation is good for both the Poisson
and negative binomial distributions
 if the count Y is not too small. Notice
that if Y = E(Y |SP ) + 2 V (Y |SP ), then [Y E(Y |SP )]2 = 4V (Y |SP ).
Hence if both the estimated mean and estimated variance functions are good
approximations, the plotted points in the OD plot for Poisson regression will
scatter about a wedge formed by the V = 0 line and the line through the
origin with slope 4: V = 4V (Y |SP ). If the normal approximation is good,
only about 5% of the plotted points should be above this line.
Second, the evidence of overdispersion increases from slight to high as the
scale of the vertical axis increases from 4 to 10 times that of the horizontal
axis. (The scale of the vertical axis tends to depend on the few cases with
the largest V (Y |SP ), and P [(Y E(Y |SP ))2 > 10V (Y |SP )] can be ap-
proximated with a normal approximation or Chebyshevs inequality.) There
is considerable evidence of overdispersion if the scale of the vertical axis is
406 13 GLMs and GAMs

more than 10 times that of the horizontal, or if the percentage of points above
the slope 4 line through the origin is much larger than 5%. Hence the identity
line and slope 4 line are added to the OD plot as visual aids, and one should
check whether the scale of the vertical axis is more than 10 times that of the
horizontal.
Combining the response plot with the OD plot is a powerful method for
assessing the adequacy of the Poisson regression model. It is easier to use the
OD plot to check the variance function than the response plot since judging
the variance function with the straight lines of the OD plot is simpler than
judging two curves. Also outliers are often easier to spot with the OD plot.
For Poisson regression, judging the mean function from the response plot
may be rather dicult for large counts since the mean function is curved
and lowess does not track the exponential function very well for large counts.
Simple diagnostic plots for the Poisson regression model can be made using
weighted least squares (WLS). To see this, assume that all n of the counts
T
Yi are large. Then log((xi )) = log((x  i ))+ log(Y
 i ) log(Yi )=+ xi , or
Yi
log(Yi ) = + T xi + ei where ei = log . The error ei does not have
(xi )
Yi (xi )
zero mean or constant variance, but if (xi ) is large  N (0, 1)
(xi )
by the central limit theorem. Recall that log(1 + x) x for |x| < 0.1. Then,
heuristically,
 
(xi ) + Yi (xi ) Yi (xi )
ei = log =
(xi ) (xi )
 
1 Yi (xi ) 1
  N 0, .
(xi ) (xi ) (xi )
This suggests that for large (xi ), the errors ei are approximately 0 mean
with variance 1/(xi ). If the (xi ) were known, and all of the Yi were large,
then a weighted least squares of log(Yi ) on xi with weights wi = (xi ) should
produce good estimates of (, ). Since the (xi ) are unknown, the estimated
weights wi = Yi could be used. Since P (Yi = 0) > 0, the estimators given in
the following denition are used. Let Zi = Yi if Yi > 0, and let Zi = 0.5 if
Yi = 0.

Denition 13.14. The minimum chisquare estimator of the param-

eters (, ) in a Poisson regression model are (M , M ), and are found from
the weighted least squares regression of log(Zi ) on xi with weights wi = Zi .
Equivalently,
use the ordinary
least squares (OLS) regression (without inter-
cept) of Zi log(Zi ) on Zi (1, xTi )T .

The minimum chisquare estimator tends to be consistent if n is xed

and all n counts Yi increase to , while the Poisson regression maximum
likelihood estimator (, ) tends to be consistent if the sample size n .
13.4 Poisson Regression 407

See Agresti (2002, pp. 611612). However, the two estimators are often close
for many data sets.
The basic idea of the following two plots for Poisson regression is to trans-
form the data towards a linear model, then make the response plot of W
versus W and residual plot of the residuals W W for the transformed
response variable W . The mean function is the identity line and the verti-
cal deviations from the identity line are the WLS residuals. The plots are
based on weighted least squares (WLS) regression. Use the
equivalent OLS
regression (without intercept) of W = Zi log(Zi ) on Zi (1, xTi )T . Then
T
the
plot of the tted values W = Zi (M + M xi ) versus the response
Zi log(Zi ) should have points that scatter about the identity line. These
results and the equivalence of the minimum chisquare estimator to an OLS
estimator suggest the following diagnostic plots.

Denition 13.15. For a Poisson regression model, a weighted t re-

T
sponse plot is a plot of Zi ESP = Zi ( + xi ) versus Zi log(Zi ).
T
The weighted residual plot is a plot of Zi ( + xi ) versus the WLS
T
residuals rW i = Zi log(Zi ) Zi ( + xi ).

If the Poisson regression model is appropriate and the PR estimator is

good, then the plotted points in the weighted t response plot should follow
the identity line. When the counts Yi are small, the WLS residuals can-
not be expected to be approximately normal. Often the larger counts are t
better than the smaller counts and hence the residual plots have a left open-
ing megaphone shape. This fact makes residual plots for Poisson regression
rather hard to use, but cases with large WLS residuals may not be t very
well by the model. Both the weighted t response and residual plots perform
better for simulated PR data with many large counts than for data where all
of the counts are less than 10.

Example 13.4. For the Ceriodaphnia data of Myers et al. (2002, pp.
136139), the response variable Y is the number of Ceriodaphnia organisms
counted in a container. The sample size was n = 70 and seven concentra-
tions of jet fuel (x1 ) and an indicator for two strains of organism (x2 ) were
used as predictors. The jet fuel was believed to impair reproduction so high
concentrations should have smaller counts. Figure 13.4 shows 4 plots for this
data. In the response plot of Figure 13.4a, the lowess curve is represented as
a jagged curve to distinguish it from the estimated PR mean function (the
exponential curve). The horizontal line corresponds to the sample mean Y .
The OD plot in Figure 13.4b suggests that there is little evidence of overdis-
persion. These two plots as well as Figures 13.4c and 13.4d suggest that the
Poisson regression model is a useful approximation to the data.
408 13 GLMs and GAMs

a) ESSP b) OD Plot

100

300
Vhat
Y

40

0
0 1.5 2.5 3.5 4.5 20 40 60 80
ESP Ehat

c) WFRP Based on MLE d) WRP Based on MLE

sqrt(Z) * log(Z)

2
20 40

MWRES
0
2
0

0 10 30 0 10 30
MWFIT MWFIT

Fig. 13.4 Plots for Ceriodaphnia Data

Example 13.5. For the crab data, the response Y is the number of satel-
lites (male crabs) near a female crab. The sample size n = 173 and the pre-
dictor variables were the color, spine condition, caparice width, and weight
of the female crab. Agresti (2002, pp. 126131) rst uses Poisson regression,
and then uses the NBR model with = 0.98 1. Figure 13.5a suggests that
there is one case with an unusually large value of the ESP. The lowess curve
does not track the exponential curve all that well. Figure 13.5b suggests that
overdispersion is present since the vertical scale is about 10 times that of
the horizontal scale and too many of the plotted points are large and greater
than the slope 4 line. Figure 13.5c also suggests that the Poisson regression
mean function is a rather poor t since the plotted points fail to cover the
identity line. Although the exponential mean function ts the lowess curve
better than the line Y = Y , an alternative model to the NBR model may t
the data better. In later chapters, Agresti uses binomial regression models
for this data.

Example 13.6. For the popcorn data of Myers et al. (2002, p. 154), the
response variable Y is the number of inedible popcorn kernels. The sample
size was n = 15 and the predictor variables were temperature (coded as 5,
6, or 7), amount of oil (coded as 2, 3, or 4), and popping time (75, 90, or
105). One batch of popcorn had more than twice as many inedible kernels
as any other batch and is an outlier. Ignoring the outlier in Figure 13.6a
suggests that the line Y = Y will t the data and lowess curve better than
the exponential curve. Hence Y seems to be independent of the predictors.
Notice that the outlier sticks out in Figure 13.6b and that the vertical scale is
well over 10 times that of the horizontal scale. If the outlier was not detected,
13.4 Poisson Regression 409

a) ESSP b) OD Plot

15

60 120
Vhat
Y
5
0

0
0.5 1.5 2.5 2 4 6 8 12
ESP Ehat

c) WFRP Based on MLE d) WRP Based on MLE

sqrt(Z) * log(Z)

MWRES
8

4
4

0
0

0 2 4 6 0 2 4 6
MWFIT MWFIT

Fig. 13.5 Plots for Crab Data

a) ESSP b) OD Plot
80

Vhat
0 1500
Y
20

3.0 3.5 4.0 20 40 60

ESP Ehat

c) WFRP Based on MLE d) WRP Based on MLE

sqrt(Z) * log(Z)
10 30 50

2 2 6
MWRES

10 20 30 40 10 20 30 40
MWFIT MWFIT

Fig. 13.6 Plots for Popcorn Data

then the Poisson regression model would suggest that temperature and time
are important predictors, and overdispersion diagnostics such as the deviance
would be greatly inated. However, we probably need to delete the high
temperature, low oil, and long popping time combination, to conclude that
the response is independent of the predictors.
410 13 GLMs and GAMs

13.5 Inference

This section gives a very brief discussion of inference for the logistic regression
(LR) and Poisson regression (PR) models. Inference for these two models is
very similar to inference for the multiple linear regression (MLR) model. For
all three of these models, Y is independent of the k 1 vector of predictors
x = (x1 , ..., xk )T given the sucient predictor + T x: Y x|( + T x).
To perform inference for LR and PR, computer output is needed. Shown
below is output using symbols and Arc output from a real data set with
k = 2 nontrivial predictors. This data set is the banknote data set described
in Cook and Weisberg (1999a, p. 524). There were 200 Swiss bank notes of
which 100 were genuine (Y = 0) and 100 counterfeit (Y = 1). The goal of the
analysis was to determine whether a selected bill was genuine or counterfeit
from physical measurements of the bill.

Label Estimate Std. Error Est/SE p-value

Constant se() zo,0 for Ho: = 0
x1 1 se(1 ) zo,1 = 1 /se(1 ) for Ho: 1 = 0
.. .. .. .. ..
. . . . .
xk k se(k ) zo,k = k /se(k ) for Ho: k = 0
Number of cases: n
Degrees of freedom: n - k - 1
Pearson X2:
Deviance: D = G^2

Binomial Regression
Kernel mean function = Logistic
Response = Status
Terms = (Bottom Left)
Trials = Ones

Coefficient Estimates
Label Estimate Std. Error Est/SE p-value
Constant -389.806 104.224 -3.740 0.0002
Bottom 2.26423 0.333233 6.795 0.0000
Left 2.83356 0.795601 3.562 0.0004

Scale factor: 1.
Number of cases: 200
Degrees of freedom: 197
Pearson X2: 179.809
Deviance: 99.169
13.5 Inference 411

Point estimators for the mean function are important. Given values of
x = (x1 , ..., xk )T , a major goal of binary logistic regression is to estimate the
success probability P (Y = 1|x) = (x) with the estimator
T
exp( + x)
(x) = T
. (13.8)
1 + exp( + x)

Similarly, a major goal of Poisson regression is to estimate the mean

E(Y |x) = (x) with the estimator
T
(x) = exp( + x). (13.9)

For tests, pval, the estimated pvalue, is an important quantity. Again

what output labels as pvalue is typically pval. Recall that Ho is rejected if
the pval . A pval between 0.07 and 1.0 provides little evidence that Ho
should be rejected, a pval between 0.01 and 0.07 provides moderate evidence
and a pval less than 0.01 provides strong statistical evidence that Ho should
be rejected. Statistical evidence is not necessarily practical evidence, and
reporting the pval along with a statement of the strength of the evidence is
more informative than stating that the pval is less than some chosen value
such as = 0.05. Nevertheless, as a homework convention, use = 0.05 if
is not given.

Investigators also sometimes test whether a predictor Xj is needed in the

model given that the other k 1 nontrivial predictors are in the model with
the following 4 step Wald test of hypotheses.
i) State the hypotheses Ho: j = 0 Ha: j = 0.
ii) Find the test statistic zo,j = j /se(j ) or obtain it from output.
iii) The pval = 2P (Z < |zoj |) = 2P (Z > |zoj |). Find the pval from output
or use the standard normal table.
iv) State whether you reject Ho or fail to reject Ho and give a nontechnical
sentence restating your conclusion in terms of the story problem.

If Ho is rejected, then conclude that Xj is needed in the GLM model for

Y given that the other k 1 predictors are in the model. If you fail to reject
Ho, then conclude that Xj is not needed in the GLM model for Y given that
the other k 1 predictors are in the model. (Or there is not enough evidence
to conclude that Xj is needed in the model.) Note that Xj could be a very
useful GLM predictor, but may not be needed if other predictors are added
to the model.

The Wald condence interval (CI) for j can also be obtained using the
output: the large sample 100 (1 ) % CI for j is j z1/2 se(j ).
412 13 GLMs and GAMs

The Wald test and CI tend to give good results if the sample size n is large.
Here 1 refers to the coverage of the CI. A 90% CI uses z1/2 = 1.645, a
95% CI uses z1/2 = 1.96, and a 99% CI uses z1/2 = 2.576.

For a GLM, often 3 models are of interest: the full model that uses all k of
the predictors xT = (xTR , xTO ), the reduced model that uses the r predictors
xR , and the saturated model that uses n parameters 1 , ..., n where n is
the sample size. For the full model the k + 1 parameters , 1 , ..., k are
estimated while the reduced model has r + 1 parameters. Let lSAT (1 , ..., n )
be the likelihood function for the saturated model and let lF U LL (, ) be the
likelihood function for the full model. Let LSAT = log lSAT (1 , ..., n ) be the
log likelihood function for the saturated model evaluated at the maximum
likelihood estimator (MLE) (1 , ..., n ) and let LF U LL = log lF U LL (, ) be
the log likelihood function for the full model evaluated at the MLE (, ).
Then the deviance D = G2 = 2(LF U LL LSAT ). The degrees of freedom
for the deviance = dfF U LL = n k 1 where n is the number of parameters
for the saturated model and k + 1 is the number of parameters for the full
model.
The saturated model for logistic regression states that for i = 1, ..., n, the
Yi |xi are independent binomial(mi , i ) random variables where i = Yi /mi .
The saturated model is usually not very good for binary data (all mi = 1)
or if the mi are small. The saturated model can be good if all of the mi are
large or if i is very close to 0 or 1 whenever mi is not large.
The saturated model for Poisson regression states that for i = 1, ..., n,
the Yi |xi are independent Poisson(i ) random variables where i = Yi . The
saturated model is usually not very good for Poisson data, but the saturated
model may be good if n is xed and all of the counts Yi are large.
If X 2d , then E(X) = d and VAR(X) = 2d. An observed value of
X > d + 3 d is unusually large and an observed value of X < d 3 d is
unusually small.
When the saturated model is good, a rule of thumb is that the logistic
or Poisson
regression model is ok if G2 n k 1 (or if G2 n k
1 + 3 n k 1). For binary LR, the 2nk+1 approximation for G2 is rarely
good even for large sample sizes n. For LR, the response plot is often a much
better diagnostic for goodness of t, especially when ESP = + T xi takes
on many values andwhen k + 1 << n. For PR, both the response plot and
G2 n k 1 + 3 n k 1 should be checked.
Response = Y
Terms = (X1 , ..., Xk )
13.5 Inference 413

Sequential Analysis of Deviance

Total Change
Predictor df Deviance df Deviance
Ones n 1 = dfo G2o
X1 n2 1
X2 n3 1
.. .. .. ..
. . . .
Xk n k 1 = dfF U LL G2F U LL 1
-----------------------------------------
Data set = cbrain, Name of Fit = B1
Response = sex
Terms = (cephalic size log[size])
Sequential Analysis of Deviance
Total Change
Predictor df Deviance | df Deviance
Ones 266 363.820 |
cephalic 265 363.605 | 1 0.214643
size 264 315.793 | 1 47.8121
log[size] 263 305.045 | 1 10.7484
The above Arc output, shown in symbols and for a real data set, is used
for the deviance test described below. Assume that the response plot has
been made and that the logistic or Poisson regression model ts the data
well in that the nonparametric step or lowess estimated mean function fol-
lows the estimated model mean function closely and there is no evidence of
overdispersion. The deviance test is used to test whether = 0. If this is the
case, then the predictors are not needed in the GLM model. If Ho : = 0
is not rejected, then for Poisson regression the estimator = Y should be

n 
n
used while for logistic regression = Yi / mi should be used. Note that
i=1 i=1
= Y for binary logistic regression since mi 1 for i = 1, ..., n.

The 4 step deviance test is

i) H0 : = 0 HA : = 0,
ii) test statistic G2 (o|F ) = G2o G2F U LL .
iii) The pval = P (2 > G2 (o|F )) where 2 2k has a chisquare distri-
bution with k degrees of freedom. Note that k = k + 1 1 = dfo dfF U LL =
n 1 (n k 1).
iv) Reject H0 if the pval and conclude that there is a GLM relationship
between Y and the predictors X1 , ..., Xk . If pval > , then fail to reject H0 and
conclude that there is not a GLM relationship between Y and the predictors
X1 , ..., Xk . (Or there is not enough evidence to conclude that there is a GLM
relationship between Y and the predictors.)
414 13 GLMs and GAMs

This test can be performed in R by obtaining output from the full and
null model.
outf <- glm(Y~x1 + x2 + ... + xk, family = binomial)
outn <- glm(Y~1,family = binomial)
anova(outn,outf,test="Chi")
Resid. Df Resid. Dev Df Deviance P(>|Chi|)
1 *** ****
2 *** **** k G^2(0|F) pvalue
The output below, shown both in symbols and for a real data set, can be
used to perform the change in deviance test. If the reduced model leaves out
a single variable Xi , then the change in deviance test becomes H0 : i = 0
versus HA : i = 0. This test is a competitor of the Wald test. This change in
deviance test is usually better than the Wald test if the sample size n is not
large, but the Wald test is often easier for software to produce. For large n
the test statistics from the two tests tend to be very similar (asymptotically
equivalent tests).
T
If the reduced model is good, then the EE plot of ESP (R) = R + R xRi
T
versus ESP = + xi should be highly correlated with the identity line
with unit slope and zero intercept.

Response = Y Terms = (X1 , ..., Xk ) (Full Model)

Label Estimate Std. Error Est/SE p-value

Constant se() zo,0 for Ho: = 0
x1 1 se(1 ) zo,1 = 1 /se(1 ) for Ho: 1 = 0
.. .. .. .. ..
. . . . .
xk k se(k ) zo,k = k /se(k ) for Ho: k = 0
Degrees of freedom: n - k - 1 = dfF U LL
Deviance: D = G2F U LL

Response = Y Terms = (X1 , ..., Xr ) (Reduced Model)

Label Estimate Std. Error Est/SE p-value

Constant se() zo,0 for Ho: = 0
x1 1 se(1 ) zo,1 = 1 /se(1 ) for Ho: 1 = 0
.. .. .. .. ..
. . . . .
xr r se(r ) zo,r = k /se(r ) for Ho: r = 0
Degrees of freedom: n - r - 1 = dfRED
Deviance: D = G2RED
(Full Model) Response = Status,
Terms = (Diagonal Bottom Top)
Label Estimate Std. Error Est/SE p-value
13.5 Inference 415

Constant 2360.49 5064.42 0.466 0.6411

Diagonal -19.8874 37.2830 -0.533 0.5937
Bottom 23.6950 45.5271 0.520 0.6027
Top 19.6464 60.6512 0.324 0.7460

Degrees of freedom: 196

Deviance: 0.009

(Reduced Model) Response = Status, Terms = (Diagonal)

Label Estimate Std. Error Est/SE p-value
Constant 989.545 219.032 4.518 0.0000
Diagonal -7.04376 1.55940 -4.517 0.0000

Degrees of freedom: 198

Deviance: 21.109

After obtaining an acceptable full model where

SP = + 1 x1 + + k xk = + T x = + TR xR + TO xO

try to obtain a reduced model

SP (red) = + R1 xR1 + + Rr xRr = R + TR xR

where the reduced model uses r of the predictors used by the full model and
xO denotes the vector of k r predictors that are in the full model but not
the reduced model. For logistic regression, the reduced model is Yi |xRi
independent Binomial(mi , (xRi )) while for Poisson regression the reduced
model is Yi |xRi independent Poisson((xRi )) for i = 1, ..., n.
Assume that the response plot looks good. Then we want to test H0 : the
reduced model is good (can be used instead of the full model) versus HA :
use the full model (the full model is signicantly better than the reduced
model). Fit the full model and the reduced model to get the deviances G2F U LL
and G2RED .

The 4 step change in deviance test is

i) H0 : the reduced model is good HA : use the full model,
ii) test statistic G2 (R|F ) = G2RED G2F U LL .
iii) The pval = P (2 > G2 (R|F )) where 2 2kr has a chisquare
distribution with k r degrees of freedom. Note that k is the number of
nontrivial predictors in the full model while r is the number of nontrivial
predictors in the reduced model. Also notice that k r = (k + 1) (r + 1) =
dfRED dfF U LL = n r 1 (n k 1).
iv) Reject H0 if the pval and conclude that the full model should be
used. If pval > , then fail to reject H0 and conclude that the reduced model
is good.
416 13 GLMs and GAMs

This test can be performed in R by obtaining output from the full and
reduced model.
outf <- glm(Y~x1 + x2 + ... + xk, family = binomial)
outr <- glm(Y~ x3 + x5 + x7,family = binomial)
anova(outr,outf,test="Chi")
Resid. Df Resid. Dev Df Deviance P(>|Chi|)
1 *** ****
2 *** **** k-r G^2(R|F) pvalue
Interpretation of coecients: if x1 , ..., xi1 , xi+1 , ..., xk can be held xed,
then increasing xi by 1 unit increases the sucient predictor SP by i units.
As a special case, consider logistic regression. Let (x) = P (success|x) = 1
P(failure|x) where a success is what is counted and a failure is what is not
counted (so if the Yi are binary, (x) = P (Yi = 1|x)). Then the estimated
(x) T
odds of success is (x) = = exp( + x). In logistic regression,
1 (x)
increasing a predictor xi by 1 unit (while holding all other predictors xed)
multiplies the estimated odds of success by a factor of exp(i ).

Output for Full Model, Response = gender, Terms =

(age log[age] breadth circum headht
height length size log[size])
Number of cases: 267, Degrees of freedom: 257,
Deviance: 234.792

Logistic Regression Output for Reduced Model,

Response = gender, Terms = (height size)
Label Estimate Std. Error Est/SE p-value
Constant -6.26111 1.34466 -4.656 0.0000
height -0.0536078 0.0239044 -2.243 0.0249
size 0.0028215 0.000507935 5.555 0.0000

Number of cases: 267, Degrees of freedom: 264

Deviance: 313.457

Example 13.7. Let the response variable Y = gender = 0 for F and 1

for M. Let x1 = height (in inches) and x2 = size of head (in mm3 ). Logistic
regression is used, and data is from Gladstone (1905). There is output above.

a) Predict (x) if height = x1 = 65 and size = x2 = 3500.

b) The full model uses the predictors listed above to the right of Terms.
Perform a 4 step change in deviance test to see if the reduced model can be
used. Both models contain a constant.
Solution: a) ESP = + 1 x1 + 2 x2 = 6.26111 0.0536078(65) +
0.0028215(3500) = 0.1296. So
13.5 Inference 417

eESP 1.1384
(x) = = = 0.5324.
1 + eESP 1 + 1.1384
b) i) H0 : the reduced model is good HA : use the full model
ii) G2 (R|F ) = 313.457 234.792 = 78.665
iii) Now df = 264 257 = 7, and comparing 78.665 with 27,0.999 = 24.32
shows that the pval = 0 < 1 0.999 = 0.001.
iv) Reject H0 , use the full model.

Example 13.8. Suppose that Y is a 1 or 0 depending on whether the

person is or is not credit worthy. Let x1 through x6 be the predictors and
use the following output to perform a 4 step deviance test. The credit data is
available from the texts website as le credit.lsp, and is from Fahrmeir and
Tutz (2001).

Response = y
Sequential Analysis of Deviance
All fits include an intercept.
Total Change
Predictor df Deviance | df Deviance
Ones 999 1221.73 |
x1 998 1177.11 | 1 44.6148
x2 997 1176.55 | 1 0.561629
x3 996 1168.33 | 1 8.21723
x4 995 1168.20 | 1 0.137583
x5 994 1163.44 | 1 4.75625
x6 993 1158.22 | 1 5.21846

Solution: i) Ho: 1 = = 6 Ha: not Ho

ii) G2 (0|F ) = 1221.73 1158.22 = 63.51
iii) Now df = 999 993 = 6, and comparing 63.51 with 26,0.999 = 22.46
shows that the pval = 0 < 1 0.999 = 0.001.
iv) Reject Ho, there is an LR relationship between Y = credit worthiness
and the predictors x1 , ..., x6 .

Coefficient Estimates
Label Estimate Std. Error Est/SE p-value
Constant -5.84211 1.74259 -3.353 0.0008
jaw ht 0.103606 0.0383650 ? ??

Example 13.9. A museum has 60 skulls, some of which are human and
some of which are from apes. Consider trying to estimate whether the skull
type is human or ape from the height of the lower jaw. Use the above logistic
regression output to answer the following problems. The museum data is
available from the texts website as le museum.lsp, and is from Schaahausen
(1878).
418 13 GLMs and GAMs

a) Predict (x) if x = 40.0.

b) Find a 95% CI for .
c) Perform the 4 step Wald test for Ho : = 0.
Solution: a) exp[ESP ] = exp[+ (40)] = exp[5.84211+0.103606(40)] =
exp[1.69787] = 0.1830731. So

eESP 0.1830731
(x) = = = 0.1547.
1 + eESP 1 + 0.1830731

b) 1.96SE() = 0.103606 1.96(0.03865) = 0.103606 0.0751954 =

[0.02841, 0.1788].
c) i) Ho: = 0 Ha : = 0
0.103606
ii) Z0 = = = 2.7005.
SE() 0.038365
iii) Using a standard normal table, pval = 2P (Z < 2.70) = 2(0.0035) =
0.0070.
iv) Reject Ho, jaw height is a useful LR predictor for whether the skull is
human or ape (so is needed in the LR model).

Label Estimate Std. Error Est/SE p-value

Constant -0.406023 0.877382 -0.463 0.6435
bombload 0.165425 0.0675296 2.450 0.0143
exper -0.0135223 0.00827920 -1.633 0.1024
type 0.568773 0.504297 1.128 0.2594

Example 13.10. Use the above output to perform inference on the num-
ber of locations where aircraft was damaged. The output is from a Poisson
regression. The variable exper = total months of aircrew experience while
type of aircraft was coded as 0 or 1. There were n = 30 cases. Data is from
Montgomery et al. (2001).
a) Predict (x) if bombload = x1 = 7.0, exper = x2 = 80.2, and type
= x3 = 1.0.
b) Perform the 4 step Wald test for Ho : 2 = 0.
c) Find a 95% condence interval for 3 .
Solution: a) ESP = + 1 x1 + 2 x2 + 3 x3 = 0.406023 + 0.165426(7)
0.0135223(80.2) + 0.568773(1) = 0.2362. So (x) = exp(ESP ) =
exp(0.2360) = 1.2665.
b) i) Ho: 2 = 0 Ha: 2 = 0
ii) t02 = 1.633.
iii) pval = 0.1024
iv) Fail to reject Ho, exper in not needed in the PR model for number of
locations given that bombload and type are in the model.
c) 3 1.96SE(3 ) = 0.568773 1.96(0.504297) = 0.568773 0.9884 =
[0.4196, 1.5572].
13.6 Variable Selection 419

13.6 Variable Selection

This section gives some rules of thumb for variable selection for logistic and
Poisson regression when SP = + T x. Before performing variable selection,
a useful full model needs to be found. The process of nding a useful full
model is an iterative process. Given a predictor x, sometimes x is not used
by itself in the full model. Suppose that Y is binary. Then to decide what
functions of x should be in the model, look at the conditional distribution of
x|Y = i for i = 0, 1. The rules shown in Table 13.1 are used if x is an indicator
variable or if x is a continuous variable. Replace normality by symmetric
with similar spreads and symmetric with dierent spreads in the second
and third lines of the table. See and Cook and Weisberg (1999a, p. 501) and
and Kay and Little (1987).

The full model will often contain factors and interactions. If w is a nominal
variable with J levels, make w into a factor by using J 1 (indicator or)
dummy variables x1,w , ..., xJ1,w in the full model. For example, let xi,w = 1
if w is at its ith level, and let xi,w = 0, otherwise. An interaction is a product
of two or more predictor variables. Interactions are dicult to interpret.
Often interactions are included in the full model, and then the reduced model
without any interactions is tested. The investigator is often hoping that the
interactions are not needed.

As in Chapter 3, a scatterplot matrix is used to examine the marginal

relationships of the predictors and response. Place Y on the top or bottom
of the scatterplot matrix. Variables with outliers, missing values, or strong
nonlinearities may be so bad that they should not be included in the full

Table 13.1 Building the Full Logistic Regression Model

distribution of x|y = i variables to include in the model

x|y = i is an indicator x
x|y = i N (i , 2 ) x
x|y = i N (i , i2 ) x and x2
x|y = i has a skewed distribution x and log(x)
x|y = i has support on (0,1) log(x) and log(1 x)

model. Suppose that all values of the variable x are positive. The log rule
says add log(x) to the full model if max(xi )/ min(xi ) > 10. For the binary
logistic regression model, it is often useful to mark the plotted points by a 0
if Y = 0 and by a + if Y = 1.

To make a full model, use the above discussion and then make a response
plot to check that the full model is good. The number of predictors in the
full model should be much smaller than the number of data cases n. Suppose
420 13 GLMs and GAMs

that the Yi are binary for i = 1, ..., n. Let N1 = Yi = the number of 1s and
N0 = nN1 = the number of 0s. A rough rule of thumb is that the full model
should use no more than min(N0 , N1 )/5 predictors and the nal submodel
should have r predictor variables where r is small with r min(N0 , N1 )/10.
For Poisson regression, a rough rule of thumb is that the full model should
use no more than n/5 predictors and the nal submodel should use no more
than n/10 predictors.

Variable selection, also called subset or model selection, is the search for
a subset of predictor variables that can be deleted without important loss of
information. A model for variable selection for a GLM can be described by

SP = + T x = + TS xS + TE xE = + TS xS (13.10)

where x = (xTS , xTE )T is a k 1 vector of nontrivial predictors, xS is an

rS 1 vector and xE is a (k rS ) 1 vector. Given that xS is in the model,
E = 0 and E denotes the subset of terms that can be eliminated given that
the subset S is in the model.

Since S is unknown, candidate subsets will be examined. Let xI be the

vector of r terms from a candidate subset indexed by I, and let xO be the
vector of the remaining terms (out of the candidate submodel). Then

SP = + TI xI + TO xO . (13.11)

Denition 13.16. The model with SP = + T x that uses all of the

predictors is called the full model. A model with SP = + TI xI that only
uses the constant and a subset xI of the nontrivial predictors is called a
submodel. The full model is a submodel.

Suppose that S is a subset of I and that model (13.10) holds. Then

SP = + TS xS = + TS xS + T(I/S) xI/S + 0T xO = + TI xI (13.12)

where xI/S denotes the predictors in I that are not in S. Since this is true
regardless of the values of the predictors, O = 0 if the set of predictors S is
a subset of I. Let (, ) and (I , I ) be the estimates of (, ) and (, I )
obtained from tting the full model and the submodel, respectively. Denote
T
the ESP from the full model by ESP = + xi and denote the ESP from
the submodel by ESP (I) = I + I xIi .
13.6 Variable Selection 421

Denition 13.17. An EE plot is a plot of ESP (I) versus ESP .

Variable selection is closely related to the change in deviance test for

a reduced model. You are seeking a subset I of the variables to keep in the
model. The AIC(I) statistic is used as an aid in backward elimination and
forward selection. The full model and the model Imin found with the smallest
AIC are always of interest. Burnham and Anderson (2004) suggest that if
(I) = AIC(I) AIC(Imin ), then models with (I) 2 are good, models
with 4 (I) 7 are borderline, and models with (I) > 10 should not be
used as the nal submodel. Create a full model. The full model has a deviance
at least as small as that of any submodel. The nal submodel should have an
EE plot that clusters tightly about the identity line. As a rough rule of thumb,
a good submodel I has corr(ESP (I), ESP ) 0.95. Find the submodel II
with the smallest number of predictors such that (II ) 2. Then submodel
II is the initial submodel to examine. Also examine submodels I with fewer
predictors than II with (I) 7.

Backward elimination starts with the full model with k nontrivial vari-
ables, and the predictor that optimizes some criterion is deleted. Then there
are k 1 variables left, and the predictor that optimizes some criterion is
deleted. This process continues for models with k 2, k 3, ..., 2, and 1 pre-
dictors.
Forward selection starts with the model with 0 variables, and the pre-
dictor that optimizes some criterion is added. Then there is 1 variable in the
model, and the predictor that optimizes some criterion is added. This process
continues for models with 2, 3, ..., k 2, and k 1 predictors. Both forward
selection and backward elimination result in a sequence, often dierent, of k
models {x1 }, {x1 , x2 }, ..., {x1 , x2 , ..., xk1 }, {x1 , x2 , ..., xk } = full model.

All subsets variable selection can be performed with the following pro-
cedure. Compute the ESP of the GLM and compute the OLS ESP found by
the OLS regression of Y on x. Check that |corr(ESP, OLS ESP)| 0.95.This
high correlation will exist for many data sets. Then perform multiple linear
regression and the corresponding all subsets OLS variable selection with the
Cp (I) criterion. If the sample size n is large and Cp (I) 2(r + 1) where the
subset I has r + 1 variables including a constant, then corr(OLS ESP, OLS
ESP(I)) will be high by the proof of Proposition 3.1c, and hence corr(ESP,
ESP(I)) will be high. In other words, if the OLS ESP and GLM ESP are
highly correlated, then performing multiple linear regression and the corre-
sponding MLR variable selection (e.g., forward selection, backward elimina-
tion, or all subsets selection) based on the Cp (I) criterion may provide many
interesting submodels.
422 13 GLMs and GAMs

Know how to nd good models from output. The following rules of thumb
(roughly in order of decreasing importance) may be useful. It is often not
possible to have all 12 rules of thumb to hold simultaneously. Let submodel I
have rI + 1 predictors, including a constant. Do not use more predictors than
submodel II , which has no more predictors than the minimum AIC model.
It is possible that II = Imin = If ull . Assume the response plot for the full
model is good. Then the submodel I is good if
i) the response plot for the submodel looks like the response plot for the full
model.
ii) corr(ESP,ESP(I)) 0.95.
iii) The plotted points in the EE plot cluster tightly about the identity line.
iv) Want the pval 0.01 for the change in deviance test that uses I as the
reduced model.
v) For binary LR want rI +1 min(N1 , N0 )/10. For PR, want rI +1 n/10.
vi) Fit OLS to the full and reduced models. The plotted points in the plot of
the OLS residuals from the submodel versus the OLS residuals from the full
model should cluster tightly about the identity line.
vii) Want the deviance G2 (I) G2 (f ull) but close. (G2 (I) G2 (f ull) since
adding predictors to I does not increase the deviance.)
viii) Want AIC(I) AIC(Imin ) + 7 where Imin is the minimum AIC model
found by the variable selection procedure.
ix) Want hardly any predictors with pvals > 0.05.
x) Want few predictors with pvals between 0.01 and 0.05.
xi) Want G2 (I) n rI 1 + 3 n rI 1.
xii) The OD plot should look good.

Heuristically, backward elimination tries to delete the variable that will

increase the deviance the least. An increase in deviance greater than 4 (if the
predictor has 1 degree of freedom) may be troubling in that a good predic-
tor may have been deleted. In practice, the backward elimination program
may delete the variable such that the submodel I with j predictors has a)
the smallest AIC(I), b) the smallest deviance G2 (I), or c) the biggest pval
(preferably from a change in deviance test but possibly from a Wald test)
in the test Ho: i = 0 versus Ha: i = 0 where the model with j + 1 terms
from the previous step (using the j predictors in I and the variable xj+1 ) is
treated as the full model.

Heuristically, forward selection tries to add the variable that will decrease
the deviance the most. A decrease in deviance less than 4 (if the predictor
has 1 degree of freedom) may be troubling in that a bad predictor may have
been added. In practice, the forward selection program may add the variable
13.6 Variable Selection 423

such that the submodel I with j nontrivial predictors has a) the smallest
AIC(I), b) the smallest deviance G2 (I), or c) the smallest pval (preferably
from a change in deviance test but possibly from a Wald test) in the test
Ho: i = 0 versus Ha: i = 0 where the current model with j terms plus the
predictor xi is treated as the full model (for all variables xi not yet in the
model).

Suppose that the full model is good and is stored in M1. Let M2, M3,
M4, and M5 be candidate submodels found after forward selection, backward
elimination, etc. Make a scatterplot matrix of the ESPs for M2, M3, M4,
M5, and M1. Good candidates should have estimated sucient predictors
that are highly correlated with the full model estimated sucient predictor
(the correlation should be at least 0.9 and preferably greater than 0.95). For
binary logistic regression, mark the symbols (0 and +) using the response
variable Y .

The nal submodel should have few predictors, few variables with large
Wald pvals (0.01 to 0.05 is borderline), a good response plot, and an EE plot
that clusters tightly about the identity line. If a factor has I 1 dummy
variables, either keep all I 1 dummy variables or delete all I 1 dummy
variables, do not delete some of the dummy variables.

Some logistic regression output can be unreliable if (x) = 1 or (x) = 0

exactly. Then ESP = or ESP = , respectively. Some binary logistic
regression output can also be unreliable if there is perfect classication of 0s
and 1s so that the 0s are to the left and the 1s to the right of ESP = 0 in
the response plot. Then the logistic regression MLE LR does not exist, and
variable selection rules of thumb may fail. Note that when there is perfect
classication, the logistic regression model is very useful, but the logistic
curve cannot approximate a step function rising from 0 to 1 at ESP = 0,
arbitrarily closely.

Example 13.11. The following output is for forward selection and back-
ward elimination. All models use a constant. For forward selection, the min
AIC model uses {F}LOC, TYP, AGE, CAN, SYS, PCO, and PH. Model II
uses {F}LOC, TYP, AGE, CAN, and SYS. Let model I use {F}LOC, TYP,
AGE, and CAN. This model may be good, so for forward selection, models
II and I are the rst models to examine.
424 13 GLMs and GAMs

Forward Selection comment

Base terms: ({F}LOC TYP)

Deviance Pearson X2 | k AIC > min AIC + 7
Add:AGE 141.873 187.84 | 5 151.873

Base terms: ({F}LOC TYP AGE)

Deviance Pearson X2| k AIC < min AIC + 7
Add:CAN 134.595 170.367 | 6 146.595
({F}LOC TYP AGE CAN) could be a good model

Base terms: ({F}LOC TYP AGE CAN)

Deviance Pearson X2 | k AIC < min AIC + 2
Add:SYS 128.441 179.753 | 7 142.441
({F}LOC TYP AGE CAN SYS) could be a good model

Base terms: ({F}LOC TYP AGE CAN SYS)

Deviance Pearson X2 | k AIC < min AIC + 2
Add:PCO 126.572 186.71 | 8 142.572
PCO not important since AIC < min AIC + 2

Base terms: ({F}LOC TYP AGE CAN SYS PCO)

Deviance Pearson X2 | k AIC
Add:PH 123.285 191.264 | 9 141.285 min AIC
PH not important since AIC < min AIC + 2
Backward Elimination

Current terms: (AGE CAN {F}LOC PCO PH PRE SYS TYP)

Deviance Pearson X2|k AIC min AIC model
Delete:PRE 123.285 191.264 |9 141.285

Current terms: (AGE CAN {F}LOC PCO PH SYS TYP)

Deviance Pearson X2|k AIC < min AIC + 2
Delete:PH 126.572 186.71 |8 142.572
PH not important

Current terms: (AGE CAN {F}LOC PCO SYS TYP)

Deviance Pearson X2|k AIC < min AIC + 2
Delete:PCO 128.441 179.753 |7 142.441
PCO not important
(AGE CAN {F}LOC SYS TYP) could be good model

Current terms: (AGE CAN {F}LOC SYS TYP)

Deviance Pearson X2|k AIC < min AIC + 7
Delete:SYS 134.595 170.367 |6 146.595
13.6 Variable Selection 425

SYS may not be important

(AGE CAN {F}LOC TYP) could be good model

Current terms: (AGE CAN {F}LOC TYP)

Deviance Pearson X2|k AIC > min AIC + 7
Delete:CAN 141.873 187.84 |5 151.873 AIC
B1 B2 B3 B4
df 255 258 259 263
# of predictors 11 8 7 3
# with 0.01 Wald p-value 0.05 2 1 0 0
# with Wald p-value > 0.05 4 0 0 0
G2 233.765 237.212 243.482 278.787
AIC 257.765 255.212 259.482 286.787
corr(B1:ETAU,Bi:ETAU) 1.0 0.99 0.97 0.80
p-value for change in deviance test 1.0 0.328 0.045 0.000
Example 13.12. The above table gives summary statistics for 4 models
considered as nal submodels after performing variable selection. One pre-
dictor was a factor, and a factor was considered to have a bad Wald p-value
> 0.05 if all of the dummy variables corresponding to the factor had p-values
> 0.05. Similarly the factor was considered to have a borderline p-value with
0.01 p-value 0.05 if none of the dummy variables corresponding to the
factor had a p-value < 0.01 but at least one dummy variable had a p-value
between 0.01 and 0.05. The response was binary and logistic regression was
used. The response plot for the full model B1 was good. Model B2 was the
minimum AIC model found. There were 267 cases: for the response, 113 were
0s and 154 were 1s.
Which two models are the best candidates for the nal submodel? Explain
briey why each of the other 2 submodels should not be used.
Solution: B2 and B3 are best. B1 has too many predictors with rather
large p-values. For B4, the AIC is too high and the corr and p-value are too
low.

Example 13.13. The ICU data is available from the texts website and
from STATLIB (http://lib.stat.cmu.edu/DASL/Datafiles/ICU.html).
Also see Hosmer and Lemeshow (2000, pp. 2325). The survival of 200 pa-
tients following admission to an intensive care unit was studied with logistic
regression. The response variable was STA (0 = Lived, 1 = Died). Predictors
were AGE, SEX (0 = Male, 1 = Female), RACE (1 = White, 2 = Black, 3 =
Other), SER= Service at ICU admission (0 = Medical, 1 = Surgical), CAN=
Is cancer part of the present problem? (0 = No, 1 = Yes), CRN= History
of chronic renal failure (0 = No, 1 = Yes), INF= Infection probable at ICU
admission (0 = No, 1 = Yes), CPR= CPR prior to ICU admission (0 = No, 1
= Yes), SYS= Systolic blood pressure at ICU admission (in mm Hg), HRA=
Heart rate at ICU admission (beats/min), PRE= Previous admission to an
426 13 GLMs and GAMs

Response Plot

1.0
0.8
0.6
Y
0.4
0.2
0.0

20 10 0 10 20 30 40
ESP
Fig. 13.7 Visualizing the ICU Data

EE PLOT for Model without Race

40
30
20
ESP
10
0
20 10

5 0 5 10 15 20
ESPS
Fig. 13.8 EE Plot Suggests Race is an Important Predictor
13.6 Variable Selection 427

ICU within 6 months (0 = No, 1 = Yes), TYP= Type of admission (0 =

Elective, 1 = Emergency), FRA= Long bone, multiple, neck, single area, or
hip fracture (0 = No, 1 = Yes), PO2= PO2 from initial blood gases (0 if >60,
1 if 60), PH= PH from initial blood gases (0 if 7.25, 1 if <7.25), PCO=
PCO2 from initial blood gases (0 if 45, 1 if >45), Bic= Bicarbonate from
initial blood gases (0 if 18, 1 if <18), CRE= Creatinine from initial blood
gases (0 if 2.0, 1 if >2.0), and LOC= Level of consciousness at admission
(0 = no coma or stupor, 1= deep stupor, 2 = coma).

EE PLOT for Model with Race

40
30
20
ESP
10
0
20 10

20 10 0 10 20 30
ESPS
Fig. 13.9 EE Plot Suggests Race is an Important Predictor

Factors LOC and RACE had two indicator variables to model the three
levels. The response plot in Figure 13.7 shows that the logistic regression
model using the 19 predictors is useful for predicting survival, although the
output has (x) = 1 or (x) = 0 exactly for some cases. Note that the
step function of slice proportions tracks the model logistic curve fairly well.
Variable selection, using forward selection and backward elimination with
the AIC criterion, suggested the submodel using AGE, CAN, SYS, TYP, and
LOC. The EE plot of ESP(sub) versus ESP(full) is shown in Figure 13.8.
The plotted points in the EE plot should cluster tightly about the identity
line if the full model and the submodel are good. Since this clustering did
not occur, the submodel seems to be poor. The lowest cluster of points and
the case on the right nearest to the identity line correspond to black patients.
The main cluster and upper right cluster correspond to patients who are not
black.
428 13 GLMs and GAMs

Figure 13.9 shows the EE plot when RACE is added to the submodel.
Then all of the points cluster about the identity line. Although numerical
variable selection did not suggest that RACE is important, perhaps since
output had (x) = 1 or (x) = 0 exactly for some cases, the two EE plots
suggest that RACE is important. Also the RACE variable could be replaced
by an indicator for black. This example illustrates how the plots can be
used to quickly improve and check the models obtained by following logistic
regression with variable selection even if the MLE LR does not exist.

P1 P2 P3 P4
df 144 147 148 149
# of predictors 6 3 2 1
# with 0.01 Wald p-value 0.05 1 0 0 0
# with Wald p-value > 0.05 3 0 1 0
G2 127.506 131.644 147.151 149.861
AIC 141.506 139.604 153.151 153.861
corr(P1:ETAU,Pi:ETAU) 1.0 0.954 0.810 0.792
p-value for change in deviance test 1.0 0.247 0.0006 0.0

Example 13.14. The above table gives summary statistics for 4 models
considered as nal submodels after performing variable selection. Poisson
regression was used. The response plot for the full model P1 was good. Model
P2 was the minimum AIC model found.
Which model is the best candidate for the nal submodel? Explain briey
why each of the other 3 submodels should not be used.
Solution: P2 is best. P1 has too many predictors with large pvalues and
more predictors than the minimum AIC model. P3 and P4 have corr and
pvalue too low and AIC too high.

Variable selection for GLMs is very similar to that for multiple linear
regression. Finding a model II from variable selection, and using GLM output
for model II does not give valid tests and condence intervals. If there is a
good full model that was found before examining the response, and if II is
the minimum AIC model, then the Olive (2016a,b,c) bootstrap tests may
be useful. These tests are similar to those for the minimum Cp model for
multiple linear regression described in Section 3.4.1.

13.7 Generalized Additive Models

There are many alternatives to the binomial and Poisson regression GLMs.
Alternatives to the binomial GLM of Denition 13.6 include the discriminant
function model of Denition 13.7, the quasi-binomial model, the binomial
generalized additive model (GAM), and the beta-binomial model of Deni-
tion 13.2.
13.7 Generalized Additive Models 429

Alternatives to the Poisson GLM of Denition 13.11 include the quasi-

Poisson model, the Poisson GAM, and the negative binomial regression model
of Denition 13.3. Other alternatives include the zero truncated Poisson
model, the zero truncated negative binomial model, the hurdle or zero in-
ated Poisson model, the hurdle or zero inated negative binomial model,
the hurdle or zero inated additive Poisson model, and the hurdle or zero
inated additive negative binomial model. See Zuur et al. (2009), Simono
(2003), and Hilbe (2011).

Many of these models can be visualized with response plots. An interesting

research project would be to make response plots for these models, adding
the conditional mean function and lowess to the plot. Also make OD plots to
check whether the model handled overdispersion. This section will examine
several of the above models, especially GAMs.

Denition 13.18. In a 1D regression, Y is independent of x given the

sucient predictor SP = h(x) where SP = + T x for a GLM. In a gener-
Y is independent of x = (x1 , ..., xp )T given the additive
alized additive model,
p
predictor AP = + j=1 Sj (xj ) for some (usually unknown) functions Sj .
T
The estimated sucient predictor ESP = h(x) and  ESP = + x for a
p
GLM. The estimated additive predictor EAP = + j=1 Sj (xj ). An ESP
response plot is a plot of ESP versus Y while an EAPresponse plot is a plot
of EAP versus Y .

Note that a GLM is a special case of the GAM using Sj (xj ) = j xj for
j = 1, ..., p. A GLM with SP = + 1 x1 + 2 x2 + 3 x1 x2 is a special case of a
GAM with x3 x1 x2 . A GLM with SP = + 1 x1 + 2 x21 + 3 x2 is a special
case of a GAM with S1 (x1 ) = 1 x1 + 2 x21 and S2 (x2 ) = 3 x2 . A GLM with
p terms may be equivalent to a GAM with k terms w1 , ..., wk where k < p.
The plotted points in the EE plot dened below should scatter tightly
about the identity line if the GLM is appropriate and if the sample size is
large enough so that the ESP is a good estimator of the SP and the EAP is a
good estimator of the AP. If the clustering is not tight but the GAM gives a
reasonable approximation to the data, as judged by the EAPresponse plot,
then examine the Sj of the GAM to see if some simple terms such as x2i can
be added to the GLM so that the modied GLM has a good ESPresponse
plot. (This technique is easiest if the GLM and GAM have the same p terms
x1 , ..., xp . The technique is more dicult, for example, if the GLM has terms
x1 , x21 , and x2 while the GAM has terms x1 and x2 .)

Denition 13.19. An EE plot is a plot of EAP versus ESP.

Denition 13.20. Recall the binomial GLM

 
exp(SPi )
Yi |SPi binomial mi , .
1 + exp(SPi )
430 13 GLMs and GAMs

Let (w) = exp(w)/[1 + exp(w)].  

exp(APi )
i) The binomial GAM is Yi |APi binomial mi , . The
1 + exp(APi )
EAPresponse plot adds the estimated mean function (EAP ) and a step
function to the plot as done for the ESPresponse plot of Section 13.3.
ii) The quasi-binomial model is a 1D regression model with E(Yi |xi ) =
mi (SPi ) and V (Yi |xi ) = mi (SPi )(1 (SPi )) where the dispersion
parameter > 0. Note that this model and the binomial GLM have the
same conditional mean function, and the conditional variance functions are
the same if = 1.

Denition 13.21. Recall the Poisson GLM Y |SP P oisson(exp(SP )).

i) The Poisson GAM is Y |AP P oisson(exp(AP )). The EAPresponse
plot adds the estimated mean function exp(EAP ) and lowess to the plot as
done for the ESPresponse plot of Section 13.4.
ii) The quasi-Poisson model is a 1D regression model with E(Y |x) =
exp(SP ) and V (Y |x) = exp(SP ) where the dispersion parameter > 0.
Note that this model and the Poisson GLM have the same conditional mean
function, and the conditional variance functions are the same if = 1.

For the quasi-binomial model, the conditional mean and variance functions
are similar to those of the binomial distribution, but it is not assumed that
Y |SP has a binomial distribution. Similarly, it is not assumed that Y |SP
has a Poisson distribution for the quasi-Poisson model.

Next, some notation is needed to derive the zero truncated Poisson re-
gression model. Y has a zero truncated Poisson distribution, Y ZT P (),
e y
if the probability mass function (pmf) of Y is f (y) = for
(1 e ) y!
y = 1, 2, 3, ... where > 0. The ZTP pmf is obtained from a Poisson distri-
bution where y = 0 values are truncated, so not allowed.
 If W P oisson()
with pmf fW (y), then P (W = 0) = e , so f (y) = 1 e =
  y=1 W

y=0 fW (y) y=1 fW (y). So the ZTP pmf f (y) = fW (y)/(1 e ) for

y = 0.   
Now E(Y ) = y=1 yf (y) = y=0 yf (y) = y=0 yfW (y)/(1 e ) =

E(W )/(1 e ) = /(1 e ).  

Similarly, E(Y 2 ) = y=1 y 2 f (y) = y=0 y 2 f (y) = y=0 y 2 fW (y)/(1
e ) = E(W 2 )/(1 e ) = [2 + ]/(1 e ). So
 2
2 +
V (Y ) = E(Y ) (E(Y )) =
2 2
.
1 e 1 e

Denition 13.22. The zero truncated Poisson regression model has

Y |SP ZT P (exp(SP )). Hence the parameter (SP ) = exp(SP ),
13.7 Generalized Additive Models 431

exp(SP )
E(Y |x) = and
1 exp( exp(SP ))

 2
[exp(SP )]2 + exp(SP ) exp(SP )
V (Y |SP ) = .
1 exp( exp(SP )) 1 exp( exp(SP ))

The quasi-binomial, quasi-Poisson, and zero truncated Poisson regression

models have GAM analogs that replace SP by AP. Denitions 13.1, 13.2,
and 13.3 give important GAM models where SP = AP. Several of these models
are GAM analogs of models discussed in Sections 13.2, 13.3, and 13.4.

13.7.1 Response Plots

It is well known that the residual plot of ESP or EAP versus the residuals
(on the vertical axis) is useful for checking the model, but there are several
other plots using the ESP that can be generalized to a GAM by replacing the
ESP by the EAP . The response plots are used to visualize the 1D regression
model or GAM in the background of the data. For 1D regression, a response
plot is the plot of the ESP versus the response Y with the estimated model
conditional mean function and a scatterplot smoother often added as visual
aids. Note that the response plot is used to visualize Y |SP while for the
additive error regression model, a residual plot of the ESP versus the residual
is used to visualize e|SP . For a GAM, these two plots replace the ESP by
the EAP . Assume that the ESP or EAP takes on many values.
Suppose the zero mean constant variance errors e1 , ..., en are iid from a
unimodal distribution that is not highly skewed. For additive error regression,
see Denition 13.1i), the estimated mean function is the identity line with
unit slope and zero intercept. If the sample size n is large, then the plotted
points should scatter about the identity line and the residual = 0 line in
an evenly populated band for the response and residual plots, with no other
pattern. To avoid overtting, assume n 10d where d is the model degrees
of freedom. Hence d = p for multiple linear regression with OLS.
If Zi = Yi /mi , then the conditional distribution Zi |xi of the binomial
GAM can be visualized with a response plot of the EAP versus Zi with
exp(EAP )
the estimated mean function of the Zi , E(Z|AP ) = , and a
1 + exp(EAP )
scatterplot smoother added to the plot as a visual aids. Instead of adding a
lowess curve to the plot, consider the following alternative. Divide the EAP
into J slices with
 approximately
 the same number of cases in each slice. Then
compute s = s Yi / s mi where the sum is over the cases in slice s. Then
plot the resulting step function. For binary data the step function is simply
432 13 GLMs and GAMs

the sample proportion in each slice. The response plot for the beta-binomial
GAM is similar.
The lowess curve and step function are simple nonparametric estimators
of the mean function (AP ) or (SP ). If the lowess curve or step function
tracks the logistic curve (the estimated conditional mean function) closely,
then the logistic conditional mean function is a reasonable approximation to
the data.
The Poisson GAM response plot is a plot of EAP versus Y with
E(Y |AP ) = exp(EAP ) and lowess added as visual aids. For both the
GAM and the GLM response plots, the lowess curve should be close to the
exponential curve, except possibly for the largest values of the ESP or EAP
in the upper right corner of the plot. Here, lowess often underestimates the
exponential curve because lowess downweights the largest Y values too much.
Similar plots can be made for a negative binomial regression or GAM.
Following the discussion
above Denition 13.15, the weighted forward re-
sponse plot is a plot of Zi EAP versus Zi log(Zi ). The weighted residual
plot
is a plot of Z i EAP versus the WLS residuals rW i = Z i log(Z i)
Zi EAP . These plots can also be used for the negative binomial GAM. If the
counts Yi are large and E(Y |AP ) = exp(EAP ) is a good approximation to
the conditional mean function E(Y |AP ) = exp(AP ), then the plotted points
in the weighted forward response plot and weighted residual plot should scat-
ter about the identity line and r = 0 lines in roughly evenly populated bands.
See Examples 13.4, 13.5, and 13.6.

13.7.2 The EE Plot for Variable Selection

Variable selection is the search for a subset of variables that can be deleted
without important loss of information. Olive and Hawkins (2005) make an
EE plot of ESP (I) versus ESP where ESP (I) is for a submodel I and ESP
is for the full model. This plot can also be used to complement the hypothesis
test that the reduced model I (which is selected before gathering data) can
be used instead of the full model. The obvious extension to GAMs is to make
the EE plot of EAP (I) versus EAP . If the tted full model and submodel
I are good, then the plotted points should follow the identity line with high
correlation (use correlation 0.95 as a benchmark).
To justify this claim, assume that there exists a subset S of predictor
variables such that if xS is in the model, then none of the other predictors
is needed in the model. Write E for these (extraneous) variables not in S,
partitioning x = (xTS , xTE )T . Then


p   
AP = + Sj (xj ) = + Sj (xj )+ Sk (xk ) = + Sj (xj ). (13.13)
j=1 jS kE jS
13.7 Generalized Additive Models 433

The extraneous terms that can be eliminated given that the subset S is in
the model have Sk (xk ) = 0 for k E.
Now suppose that I is a candidate subset of predictors and that S I.
Then

p  
AP = + Sj (xj ) = + Sj (xj ) = + Sk (xk ) = AP (I),
j=1 jS kI

(if I includes predictors from E, these will have Sk (xk ) = 0). For any subset
I that includes all relevant predictors, the correlation corr(AP, AP(I)) = 1.
Hence if the full model and submodel are reasonable and if EAP and EAP(I)
are good estimators of AP and AP(I), then the plotted points in the EE plot
of EAP(I) versus EAP will follow the identity line with high correlation.

13.7.3 An EE Plot for Checking the GLM

One useful application of a GAM is for checking whether the corresponding

GLM has the correct form of the predictors xj in the model. Suppose a GLM
and the corresponding GAM are both t with the same link function where
at least one general Sj (xj ) was used. Since the GLM is a special case of the
GAM, the plotted points in the EE plot of EAP versus ESP should follow
the identity line with very high correlation if the tted GLM and GAM are
roughly equivalent. If the correlation is not very high and the GAM has some
nonlinear Sj (xj ), update the GLM, and remake the EE plot. For example,
update the GLM by adding terms such as x2j and possibly x3j , or add log(xj )
if xj is highly skewed. Then remake the EAP versus ESP plot.

13.7.4 Examples

For the binary logistic GAM, the EAP will not be a consistent estimator
of the AP if the estimated probability (AP ) = (EAP ) is exactly zero or
one. The following example will show that GAM output and plots can still
be used for exploratory data analysis. The example also illustrates that EE
plots are useful for detecting cases with high leverage and clusters of cases.
Numerical diagnostics, such as analogs of Cooks distances (Cook 1977), tend
to fail if there is a cluster of two or more inuential cases.

Example 13.15. For the ICU data of Example 13.13, a binary general-
ized additive model was t with unspecied functions for AGE, SYS, and
HRA, and linear functions for the remaining 16 variables. Output suggested
that functions for SYS and HRA are linear but the function for AGE may
434 13 GLMs and GAMs

1.0
0.8
0.6
Y

0.4
0.2
0.0

20 0 20 40 60
EAP
Fig. 13.10 Visualizing the ICU GAM
40
30
20
ESP
10
0
20 10

20 0 20 40 60
EAP
Fig. 13.11 GAM and GLM give Similar Success Probabilities
13.7 Generalized Additive Models 435

be slightly curved. Several cases had (AP ) equal to zero or one, but the
response plot in Figure 13.10 suggests that the full model is useful for pre-
dicting survival. Note that the ten slice step function closely tracks the logistic
curve. To visualize the model with the response plot, use Y |x binomial[1,
(EAP ) = eEAP /(1+eEAP )]. When x is such that EAP < 5, (EAP ) 0.
If EAP > 5, (EAP ) 1, and if EAP = 0, then (EAP ) = 0.5. The logistic
curve gives (EAP ) P (Y = 1|x) = (AP ). The dierent estimated bi-
nomial distributions have (AP ) = (EAP ) that increases according to the
logistic curve as EAP increases. If the step function tracks the logistic curve
closely, the binary GAM gives useful smoothed estimates of (AP ) provided
that the number of 0s and 1s are both much larger than the model degrees
of freedom so that the GAM is not overtting.
A binary logistic regression was also t, and Figure 13.11 shows the plot of
EAP versus ESP. The plot shows that the near zero and near one probabilities
are handled dierently by the GAM and GLM, but the estimated success
probabilities for the two models are similar: (ESP ) (EAP ). Hence we
used the GLM and perform variable selection as in Example 13.13.

Example 13.16. For binary data, Kay and Little (1987) suggest exam-
ining the two distributions x|Y = 0 and x|Y = 1. Use predictor x if the two
distributions are roughly symmetric with similar spread. Use x and x2 if the
distributions are roughly symmetric with dierent spread. Use x and log(x)
if one or both of the distributions are skewed. The log rule says add log(x)
to the model if min(x) > 0 and max(x)/ min(x) > 10. The Gladstone (1905)
data is useful for illustrating these suggestions. The response was gender with
Y = 1 for male and Y = 0 for female. The predictors were age, height, and
the head measurements circumference, length, and size. When the GAM was
t without log(age) or log(size), the Sj for age, height, and circumf erence
were nonlinear. The log rule suggested adding log(age), and log(size) was
added because size is skewed. The GAM for this model had plots of Sj (xj )
that were fairly linear. The response plot is not shown but was similar to
Figure 13.10, and the step function tracked the logistic curve closely. When
EAP = 0, the estimated probability of Y = 1 (male) is 0.5. When EAP > 5
the estimated probability is near 1, but near 0 for EAP < 5. The response
plot for the binomial GLM, not shown, is similar. See Problem 13.14 for
another analysis of this data set.

Example 13.17. Wood (2006, pp. 8286) describes heart attack data
where the response Y is the number of heart attacks for mi patients suspected
of suering a heart attack. The enzyme ck (creatine kinase) was measured for
the patients and it was determined whether the patient had a heart attack
or not. A binomial GLM with predictors x1 = ck, x2 = [ck]2 , and x3 = [ck]3
was t and had AIC = 33.66. The binomial GAM with predictor x1 was t
in R, and Figure 13.12 shows that the EE plot for the GLM was not too
good. The log rule suggests using ck and log(ck), but ck was not signicant.
436 13 GLMs and GAMs

8
6
4
ESPp
2
0
2
4

2 0 2 4
EAP
Fig. 13.12 EE plot for cubic GLM for Heart Attack Data
4
2
ESPl
0
2

2 0 2 4
EAP
Fig. 13.13 EE plot with log(ck) in the GLM
13.8 Overdispersion 437

1.0
0.8
0.6
Z
0.4
0.2
0.0

2 0 2 4
ESPl
Fig. 13.14 Response Plot for Heart Attack Data

Hence a GLM with the single predictor log(ck) was t. Figure 13.13 shows
the EE plot, and Figure 13.14 shows the response plot where the Zi = Yi /mi
track the logistic curve closely. There was no evidence of overdispersion and
the model had AIC = 33.45. The GAM using log(ck) had a linear S, and
the correlation of the plotted points in the EE plot, not shown, was one. See
Problem 13.22.

13.8 Overdispersion

Denition 13.23. Overdispersion occurs when the actual conditional

variance function V (Y |x) is larger than the model conditional variance
function VM (Y |x).

Overdispersion can occur if the model is missing factors, if the response

variables are correlated, if the population follows a mixture distribution, or
if outliers are present. Typically it is assumed that the model is correct so
V (Y |x) = VM (Y |x). Hence the subscript M is usually suppressed. A GAM
has conditional mean and variance functions EM (Y |AP ) and VM (Y |AP )
where the subscript M indicates that the function depends on the model.
Then overdispersion occurs if V (Y |x) > VM (Y |AP ) where E(Y |x) and
V (Y |x) denote the actual conditional mean and variance functions. Then
438 13 GLMs and GAMs

the assumptions that E(Y |x) = EM (Y |x) m(AP ) and V (Y |x) =

VM (Y |AP ) v(AP ) need to be checked.
First check that the assumption E(Y |x) = m(SP ) is a reasonable approx-
imation to the data using the response plot with lowess and the estimated
conditional mean function EM (Y |x) = m(SP ) added as visual aids. Overdis-
persion can occur even if the model conditional mean function E(Y |SP )
is a good approximation to the data. For example, for many data sets
where E(Yi |xi ) = mi (SPi ), the binomial regression model is inappropriate
since V (Yi |xi ) > mi (SPi )(1 (SPi )). Similarly, for many data sets where
E(Y |x) = (x) = exp(SP ), the Poisson regression model is inappropriate
since V (Y |x) > exp(SP ). If the conditional mean function is adequate, then
we suggest checking for overdispersion using the OD plot.

Denition 13.24. For 1D regression, the OD plot is a plot of the esti-

mated model variance VM (Y |SP ) versus the squared residuals
V = [Y EM (Y |SP )]2 . Replace SP by AP for a GAM.

The OD plot has been used by Winkelmann (2000, p. 110) for the Poisson
regression model where VM (Y |SP ) = EM (Y |SP ) = exp(ESP ). For binomial
and Poisson regression, the OD plot can be used to complement tests and
diagnostics for overdispersion such as those given in Cameron and Trivedi
(2013), Collett (1999, ch. 6), and Winkelmann (2000). See discussion below
Denitions 13.10 and 13.13 for how to interpret the OD plot with the identity
line, OLS line, and slope 4 line added as visual aids, and for discussion of the
numerical summaries G2 and X 2 for GLMs.
Denition 13.1, with SP = AP, gives EM (Y |AP ) = m(AP ) and VM (Y |AP )
= v(AP ) for several models. Often m(AP ) = m(EAP ) and v(AP ) =
v(EAP ), but additional parameters sometimes need to be estimated.
Hence v(AP ) = mi (EAPi )(1 (EAPi ))[1 + (mi 1)/(1 + )],
v(AP ) = exp(EAP ) + exp(2 EAP ), and v(AP ) = [m(EAP )]2 / for
the beta-binomial, negative binomial, and gamma GAMs, respectively. The
beta-binomial regression model is often used if the binomial regression is
inadequate because of overdispersion, and the negative binomial GAM is
often used if the Poisson GAM is inadequate.
Since the Poisson regression (PR) model is simpler than the negative bi-
nomial regression (NBR) model, and the binomial logistic regression (LR)
model is simpler beta-binomial regression (BBR) model, the graphical di-
agnostics for the goodness of t of the PR and LR models are very useful.
Combining the response plot with the OD plot is a powerful method for
assessing the adequacy of the Poisson and logistic regression models. NBR
and BBR models should also be checked with response and OD plots. See
Examples 13.213.6.

Example 13.18. The species data is from Cook and Weisberg (1999a,
pp. 285286) and Johnson and Raven (1973). The response variable is the
total number of species recorded on each of 29 islands in the Galapagos
13.8 Overdispersion 439

Archipelago. Predictors include area of island, areanear = the area of the

closest island, the distance to the closest island, the elevation, and endem =
the number of endemic species (those that were not introduced from else-
where). A scatterplot matrix of the predictors suggested that log transfor-
mations should be taken. Poisson regression suggested that log(endem) and
log(areanear) were the important predictors, but the deviance and Pear-
son X 2 statistics suggested overdispersion was present since both statistics
were near 71.4 with 26 degrees of freedom. The residual plot also suggested
increasing variance with increasing tted value. A negative binomial regres-
sion suggested that only log(endem) was needed in the model, and had a
deviance of 26.12 on 27 degrees of freedom. The residual plot for this model
was roughly ellipsoidal. The negative binomial GAM with log(endem) had
an S that was linear and the plotted points in the EE plot had correlation
near 1.
400
300
Y

200
100
0

0 1 2 3 4 5 6

EAP
Fig. 13.15 Response Plot for Negative Binomial GAM

The response plot with the exponential and lowess curves added as visual
aids is shown in Figure 13.15. The interpretation is that Y |x negative
binomial with E(Y |x) exp(EAP ). Hence if EAP = 0, E(Y |x) 1. The
negative binomial and Poisson GAM have the same conditional mean func-
tion. If the plot was for a Poisson GAM, the interpretation would be that
Y |x Poisson(exp(EAP )). Hence if EAP = 0, Y |x Poisson(1).
440 13 GLMs and GAMs

5000
3000
Vhat
1000
0

0 1000 2000 3000 4000

ModVar
Fig. 13.16 OD Plot for Negative Binomial GAM

Figure 13.16 shows the OD plot for the negative binomial GAM with
the identity line and slope 4 line through the origin added as visual aids.
The plotted points fall within the slope 4 wedge, suggesting that the neg-
ative binomial regression model has successfully dealt with overdispersion.
Here E(Y |AP ) = exp(EAP ) and V (Y |AP ) = exp(EAP ) + exp(2EAP )
where = 1/37.

13.9 Complements

GLMs were introduced by Nelder and Wedderburn (1972) . Also see McCul-
lagh and Nelder (1989), Myers et al. (2002), Olive (2010), Andersen and
Skovgaard (2010), Agresti (2013, 2015), and Cook and Weisberg (1999a, ch.
2123). Collett (2003) and Hosmer and Lemeshow (2000) are excellent texts
on logistic regression while Cameron and Trivedi (2013) and Winkelmann
(2008) cover Poisson regression. Alternatives to Poisson regression mentioned
in Section 13.7 are covered by Zuur et al. (2009), Simono (2003), and Hilbe
(2011). See Hillis and Davis (1994) Davis for a widely used algorithm to com-
pute the GLM. Cook and Zhang (2015) show that envelope methods have the
potential to signicantly improve GLMs.
13.9 Complements 441

Following Cook and Weisberg (1999a, p. 396), a residual plot is a plot of

a function of the predictors versus the residuals, while a model checking plot
is a plot of a function of the predictors versus the response. Hence response
plots are a special case of model checking plots. See Cook and Weisberg
(1997, 1999a, pp. 397, 514, and 541). Cook and Weisberg (1999a, p. 515) add
a lowess curve to the response plot. The scatterplot smoother lowess is due
to Cleveland (1979).
In a 1D regression model, Y x|h(x) where the real valued function
h : Rp R. Then a plot of h(x) versus Y is a response plot. For this
model, Y |x can be replaced by Y |h(x), and the response plot is also called
an estimated sucient summary plot. Note that h(x) = SP or AP and
h(x) = ESP or EAP for 1D regression and the generalized additive model,
respectively. The response plot is essential for understanding the model and
for checking goodness and lack of t if the estimated sucient predictor
T
+ x takes on many values. See Olive (2013b).
For Binomial regression and BBR, and for Poisson regression and NBR, the
OD plot can be used to complement tests and diagnostics for overdispersion
such as those given in Cameron and Trivedi (2013), Collett (1999, ch. 6),
Hilbe (2011), Winkelmann (2000), and Zuur et al. (2009).
Olive and Hawkins (2005) give a simple variable selection procedure (all
subsets, forward selection, backwards elimination) that can be applied to lo-
gistic regression and Poisson regression using readily available OLS software.
Variable selection using the AIC criterion is discussed in Burnham and
Anderson (2004), Cook and Weisberg (1999a), and Hastie (1987). Agresti
(2013) incorporates some of the ideas from Section 13.6. It is conjectured
that the bootstrap can be used much as in Section 3.4.1 for inference after
variable selection using AIC or the Olive and Hawkins (2005) technique using
OLS and Cp . See Olive (2016a,b,c).
The existence of the logistic regression MLE is discussed in Albert and
Andersen (1984) and Santer and Duy (1986) .
Results from Cameron and Trivedi (1998, p. 89) suggest that if a Poisson
regression model is t using OLS software for MLR, then a rough approxi-
mation is P R OLS /Y . So a rough approximation is PR ESP (OLS
ESP)/Y . Results from Haggstrom (1983) suggest that if a binary regression
model is t using OLS software for MLR, then a rough approximation is
LR OLS /M SE.
Useful references for generalized additive models include Hastie and Tib-
shirani (1990), Wood (2006), and Zuur et al. (2009). Large sample theory for
the GAM is given by Wang et al. (2011). Olive (2013b) suggested plots for
GAMs given in Sections 13.7 and 13.7. Section 3.2 of this book and Olive
(2013a, 2017) suggested a graphical method for response transformations
and prediction intervals that can be adapted to the additive error regression
model.
Many survival regression models are 1D regression models, but interest is
in the conditional survival function rather than the conditional mean func-
442 13 GLMs and GAMs

tion. Olive (2010, ch. 16) discusses plots useful for visualizing the Cox pro-
portional hazards regression model, Weibull proportional hazards regression
model, and accelerated failure time models. It is also shown that inference
and variable selection for these models is very similar to that of GLMs. Again
the Olive (2016a,b,c) bootstrap tests may be useful after variable selection
with AIC.
Plots were made in R and Splus, see R Core Team (2016). The Wood
(2006) library mgcv was used for tting a GAM, and the Venables and Rip-
ley (2010) library MASS was used for the negative binomial family. The Lesno
and Lancelot (2010) R package aod has function betabin for beta binomial
regression and is also useful for tting negative binomial regression. SAS
has proc genmod, proc gam, and proc countreg which are useful for t-
ting GLMs such as Poisson regression, GAMs such as the Poisson GAM,
and overdispersed count regression models. The lregpack R functions include
lrplot which makes response and OD plots for binomial regression; lrplot2
which makes the response plot for binary regression; prplot which makes
the response, weighted forward response, weighted residual, and OD plots
for Poisson regression; and prsim which makes the last 4 plots for simulated
Poisson or negative binomial regression models.

13.10 Problems

PROBLEMS WITH AN ASTERISK * ARE USEFUL.

Output for problem 13.1: Response = sex
Coefficient Estimates
Label Estimate Std. Error Est/SE p-value
Constant -18.3500 3.42582 -5.356 0.0000
circum 0.0345827 0.00633521 5.459 0.0000
13.1. Consider trying to estimate the proportion of males from a popu-
lation of males and females by measuring the circumference of the head. Use
the above logistic regression output to answer the following problems.

a) Predict (x) if x = 550.0.

b) Find a 95% CI for .

c) Perform the 4 step Wald test for Ho : = 0.

Output for Problem 13.2 Response = sex
Coefficient Estimates
Label Estimate Std. Error Est/SE p-value
Constant -19.7762 3.73243 -5.298 0.0000
circum 0.0244688 0.0111243 2.200 0.0278
length 0.0371472 0.0340610 1.091 0.2754
13.10 Problems 443

13.2 . Now the data is as in Problem 13.1, but try to estimate the pro-
portion of males by measuring the circumference and the length of the head.
Use the above logistic regression output to answer the following problems.

a) Predict (x) if circumference = x1 = 550.0 and length = x2 = 200.0.

b) Perform the 4 step Wald test for Ho : 1 = 0.

c) Perform the 4 step Wald test for Ho : 2 = 0.

Output for problem 13.3

Response = ape
Terms = (lower jaw, upper jaw, face length)
Trials = Ones
Sequential Analysis of Deviance
All fits include an intercept.
Total Change
Predictor df Deviance | df Deviance
Ones 59 62.7188 |
lower jaw 58 51.9017 | 1 10.8171
upper jaw 57 17.1855 | 1 34.7163
face length 56 13.5325 | 1 3.65299

13.3 . A museum has 60 skulls of apes and humans. Lengths of the lower
jaw, upper jaw, and face are the explanatory variables. The response variable
is ape (= 1 if ape, 0 if human). Using the output above, perform the four step
deviance test for whether there is an LR relationship between the response
variable and the predictors.

Output for Problem 13.4.

Full Model
Response = ape
Coefficient Estimates
Label Estimate Std. Error Est/SE p-value
Constant 11.5092 5.46270 2.107 0.0351
lower jaw -0.360127 0.132925 -2.709 0.0067
upper jaw 0.779162 0.382219 2.039 0.0415
face length -0.374648 0.238406 -1.571 0.1161

Number of cases: 60
Degrees of freedom: 56
Pearson X2: 16.782
Deviance: 13.532

Reduced Model
Response = ape
444 13 GLMs and GAMs

Coefficient Estimates
Label Estimate Std. Error Est/SE p-value
Constant 8.71977 4.09466 2.130 0.0332
lower jaw -0.376256 0.115757 -3.250 0.0012
upper jaw 0.295507 0.0950855 3.108 0.0019

Number of cases: 60
Degrees of freedom: 57
Pearson X2: 28.049
Deviance: 17.185

13.4 . Suppose the full model is as in Problem 13.3, but the reduced
model omits the predictor face length. Perform the 4 step change in deviance
test to examine whether the reduced model can be used.

The following three problems use the possums data from Cook and Weis-
berg (1999a).

Output for Problem 13.5

Data set = Possums, Response = possums
Terms = (Habitat Stags)
Coefficient Estimates
Label Estimate Std. Error Est/SE p-value
Constant -0.652653 0.195148 -3.344 0.0008
Habitat 0.114756 0.0303273 3.784 0.0002
Stags 0.0327213 0.00935883 3.496 0.0005

Number of cases: 151 Degrees of freedom: 148

Pearson X2: 110.187
Deviance: 138.685

13.5 . Use the above output to perform inference on the number of pos-
sums in a given tract of land. The output is from a Poisson regression.
a) Predict (x) if habitat = x1 = 5.8 and stags = x2 = 8.2.

b) Perform the 4 step Wald test for Ho : 1 = 0.

c) Find a 95% condence interval for 2 .

Output for Problem 13.6

Response = possums Terms = (Habitat Stags)
Total Change
Predictor df Deviance | df Deviance
Ones 150 187.490 |
Habitat 149 149.861 | 1 37.6289
Stags 148 138.685 | 1 11.1759
13.10 Problems 445

13.6 . Perform the 4 step deviance test for the same model as in Prob-
lem 13.5 using the output above.
Output for Problem 13.7
Terms = (Acacia Bark Habitat Shrubs Stags Stumps)
Label Estimate Std. Error Est/SE p-value
Constant -1.04276 0.247944 -4.206 0.0000
Acacia 0.0165563 0.0102718 1.612 0.1070
Bark 0.0361153 0.0140043 2.579 0.0099
Habitat 0.0761735 0.0374931 2.032 0.0422
Shrubs 0.0145090 0.0205302 0.707 0.4797
Stags 0.0325441 0.0102957 3.161 0.0016
Stumps -0.390753 0.286565 -1.364 0.1727
Number of cases: 151
Degrees of freedom: 144
Deviance: 127.506
13.7 . Let the reduced model be as in Problem 13.5 and use the output
for the full model be shown above. Perform a 4 step change in deviance test.
B1 B2 B3 B4
df 945 956 968 974
# of predictors 54 43 31 25
# with 0.01 Wald p-value 0.05 5 3 2 1
# with Wald p-value > 0.05 8 4 1 0
G2 892.96 902.14 929.81 956.92
AIC 1002.96 990.14 993.81 1008.912
corr(B1:ETAU,Bi:ETAU) 1.0 0.99 0.95 0.90
p-value for change in deviance test 1.0 0.605 0.034 0.0002
13.8 . The above table gives summary statistics for 4 models considered
as nal submodels after performing variable selection. (Several of the predic-
tors were factors, and a factor was considered to have a bad Wald p-value >
0.05 if all of the dummy variables corresponding to the factor had p-values
> 0.05. Similarly the factor was considered to have a borderline p-value with
0.01 p-value 0.05 if none of the dummy variables corresponding to the
factor had a p-value < 0.01 but at least one dummy variable had a p-value
between 0.01 and 0.05.) The response was binary and logistic regression was
used. The response plot for the full model B1 was good. Model B2 was the
minimum AIC model found. There were 1000 cases: for the response, 300
were 0s and 700 were 1s.
a) For the change in deviance test, if the p-value 0.07, there is little
evidence that Ho should be rejected. If 0.01 < p-value < 0.07, then there is
moderate evidence that Ho should be rejected. If p-value 0.01, then there
is strong evidence that Ho should be rejected. For which models, if any, is
there strong evidence that Ho: reduced model is good should be rejected.
446 13 GLMs and GAMs

b) For which plot is corr(B1:ETAU,Bi:ETAU) (using notation from

Arc: T u instead of T x) relevant?

c) Which model should be used as the nal submodel? Explain briey why
each of the other 3 submodels should not be used.

Arc Problems

The following four problems use data sets from Cook and Weisberg (1999a)
Weisberg.

13.9. Activate the banknote.lsp dataset with the menu commands

File > Load > Data > banknote.lsp. Scroll up the screen to read the
data description. Twice you will t logistic regression models and include the
coecients in Word. Print out this output when you are done and include
the output with your homework.

From Graph&Fit select Fit binomial response. Select Top as the predictor,
Status as the response, and ones as the number of trials.

a) Include the output in Word.

b) Predict (x) if x = 10.7.

c) Find a 95% CI for .

d) Perform the 4 step Wald test for Ho : = 0.

e) From Graph&Fit select Fit binomial response. Select Top and Diagonal
as predictors, Status as the response, and ones as the number of trials. Include
the output in Word.

f) Predict (x) if x1 = Top = 10.7 and x2 = Diagonal = 140.5.

g) Find a 95% CI for 1 .

h) Find a 95% CI for 2 .

i) Perform the 4 step Wald test for Ho : 1 = 0.

j) Perform the 4 step Wald test for Ho : 2 = 0.

13.10 . Activate banknote.lsp in Arc. with the menu commands

File > Load > Data > banknote.lsp. Scroll up the screen to read the data
description. From Graph&Fit select Fit binomial response. Select Top and
Diagonal as predictors, Status as the response, and ones as the number of
trials.
a) Include the output in Word.
13.10 Problems 447

b) From Graph&Fit select Fit linear LS. Select Diagonal and Top for pre-
dictors, and Status for the response. From Graph&Fit select Plot of and select
L2:Fit-Values for H, B1:EtaU for V, and Status for Mark by. Include the plot
T T
in Word. Is the plot linear? How are OLS + OLS x and logistic + logistic x
related (approximately)?

13.11 . Activate possums.lsp in Arc with the menu commands

File > Load > Data > possums.lsp. Scroll up the screen to read the data
description.
a) From Graph&Fit select Fit Poisson response. Select y as the response
and select acacia, bark, habitat, shrubs, stags, and stumps as the predictors.
Include the output in Word. This is your full model.

b) Response plot: From Graph&Fit select Plot of. Select P1:EtaU for the
H box and y for the V box. From the OLS popup menu select Poisson and
move the slider bar to 1. Move the lowess slider bar until the lowess curve
tracks the exponential curve well. Include the response plot in Word.
c) From Graph&Fit select Fit Poisson response. Select y as the response
and select bark, habitat, stags, and stumps as the predictors. Include the
output in Word.

d) Response plot: From Graph&Fit select Plot of. Select P2:EtaU for the
H box and y for the V box. From the OLS popup menu select Poisson and
move the slider bar to 1. Move the lowess slider bar until the lowess curve
tracks the exponential curve well. Include the response plot in Word.

e) Deviance test. From the P2 menu, select Examine submodels and click
on OK. Include the output in Word and perform the 4 step deviance test.

f) Perform the 4 step change of deviance test.

g) EE plot. From Graph&Fit select Plot of. Select P2:EtaU for the H box
and P1:EtaU for the V box. Move the OLS slider bar to 1. Click on the
Options popup menu and type y=x. Include the plot in Word. Is the plot
linear?

13.12 . In this problem you will nd a good submodel for the possums
data.

Activate possums.lsp in Arc with the menu commands

File > Load > Data > possums.lsp. Scroll up the screen to read the data
description.

From Graph&Fit select Fit Poisson response. Select y as the response and
select Acacia, bark, habitat, shrubs, stags, and stumps as the predictors.

In Problem 13.11, you showed that this was a good full model.
448 13 GLMs and GAMs

a) Using what you have learned in class nd a good submodel and include
the relevant output in Word.

(Hints: Use forward selection and backward elimination and nd the model
Imin with the smallest AIC. Let (I) = AIC(I) AIC(Imin ). Then nd the
model II with the fewest number of predictors such that (II ) 2. Then
submodel II is the initial submodel to examine. Fit model II and look at the
Wald test pvalues. Try to eliminate predictors with large pvalues but make
sure that the deviance does not increase too much. Also examine submodels
I with fewer predictors than II with (I) 7. You may have several models,
say P2, P3, P4, and P5 to look at. Make a scatterplot matrix of the Pi:ETAU
from these models and from the full model P1. Make the EE and response
plots for each model. The correlation in the EE plot should be at least 0.9 and
preferably greater than 0.95. As a very rough guide for Poisson regression,
the number of predictors in the full model should be less than n/5 and the
number of predictors in the nal submodel should be less than n/10.)

b) Make a response plot for your nal submodel, say P2. From Graph&Fit
select Plot of. Select P2:EtaU for the H box and y for the V box. From
the OLS popup menu select Poisson and move the slider bar to 1. Move the
lowess slider bar until the lowess curve tracks the exponential curve well.
Include the response plot in Word.

c) Suppose that P1 contains your full model and P2 contains your nal
submodel. Make an EE plot for your nal submodel: from Graph&Fit select
Plot of. Select P1:EtaU for the V box and P2:EtaU, for the H box. After
the plot appears, click on the options popup menu. A window will appear.
Type y = x and click on OK. This action adds the identity line to the plot.
Also move the OLS slider bar to 1. Include the plot in Word.

d) Using a), b), c), and any additional output that you desire (e.g.,
AIC(full), AIC(Imin ), AIC(II ), and AIC(nal submodel), explain why your
nal submodel is good.

Warning: The following 5 problems use data from the books

webpage. Save the data les on a ash drive. Get in Arc and use the
menu commands File > Load and a window with a Look in box will appear.
Click on the black triangle and then on Removable Disk (G:). Then click twice
on the data set name.

13.13 . (Response Plot): Activate cbrain.lsp in Arc with the menu com-
mands File > Load > Removable Disk (G:) > cbrain.lsp. Scroll up the
screen to read the data description. From Graph&Fit select Fit binomial
response. Select brnweight, cephalic, breadth, cause, size, and headht as pre-
dictors, sex as the response, and ones as the number of trials. Perform the
logistic regression and from Graph&Fit select Plot of. Place sex on V and
B1:EtaU on H. From the OLS popup menu, select Logistic and move the
13.10 Problems 449

slider bar to 1. From the lowess popup menu select SliceSmooth and move
the slider bar until the t is good. Include your plot in Word. Are the slice
means (observed proportions) tracking the logistic curve (tted proportions)
very well? Use lowess if SliceSmooth does not work.

13.14 . Suppose that you are given a data set, told the response, and
asked to build a logistic regression model with no further help. In this prob-
lem, we use the cbrain data to illustrate the process.

a) Activate cbrain.lsp in Arc with the menu commands

File > Load > Removable Disk (G:) > cbrain.lsp. Scroll up the screen
to read the data description. From Graph&Fit select Scatterplot-matrix of.
Place sex in the Mark by box. Then select age, breadth, cause, cephalic, circum,
headht, height, length, size, and sex. Include the scatterplot matrix in Word.

b) Use the menu commands cbrain>Make factors and select cause. This
makes cause into a factor with 2 degrees of freedom. Use the menu commands
cbrain>Transform and select age and the log transformation.
Why was the log transformation chosen?

c) From Graph&Fit select Plot of and select size in H. Also place sex in
the Mark by box. A plot will come up. From the GaussKerDen menu (the
triangle to the left) select Fit by marks, move the sliderbar to 0.9, and include
the plot in Word.

d) Use the menu commands cbrain>Transform and select size and the
log transformation. From Graph&Fit select Fit binomial response. Select age,
log(age), breadth, {F}cause, cephalic, circum, headht, height, length, size, and
log(size) as predictors, sex as the response, and ones as the number of trials.
This is the full model B1. Perform the logistic regression and include the
relevant output for testing in Word.

e) From Graph&Fit select Plot of. Place sex on V and B1:EtaU on H.

From the OLS popup menu, select Logistic and move the slider bar to 1.
From the lowess popup menu select SliceSmooth and move the slider bar
until the t is good. Include your plot in Word. Are the slice means (observed
proportions) tracking the logistic curve (tted proportions) fairly well? (Use
lowess if SliceSmooth does not work.)

f) From B1 select Examine submodels and select Add to base model (For-
ward Selection). Include the output with the header Base terms: ... and
from Add: length 259 to Add: {F}cause 258 in Word.

g) From B1 select Examine submodels and select Delete from full model
(Backward Elimination). Include the output with df corresponding to the
minimum AIC model in Word. What predictors does this model use?
450 13 GLMs and GAMs

h) As a nal submodel B2, use the model from f): from Graph&Fit select
Fit binomial response. Select age, log(age), circum, height, length, size, and
log(size) as predictors, sex as the response, and ones as the number of trials.
Perform the logistic regression and include the relevant output for testing in
Word.

i) Put the EE plot H B2:EtaU versus V B1:EtaU in Word. Is the plot

linear?

j) From Graph&Fit select Plot of. Place sex on V and B2:EtaU on H.

From the OLS popup menu, select Logistic and move the slider bar to 1.
From the lowess popup menu select SliceSmooth and move the slider bar
until the t is good. Include your plot in Word. Are the slice means (observed
proportions) tracking the logistic curve (tted proportions) fairly well? (Use
lowess if SliceSmooth does not work.)

k) Perform the 4 step change in deviance test using the full model in d)
and the reduced submodel in h).

Now act as if the nal submodel is the full model.

l) From B2 select Examine submodels, click OK, and include the output in
Word. Then use the output to perform a 4 step deviance test on the submodel.

m) From Graph&Fit select Inverse regression. Select age, log(age), circum,

height, length, size, and log(size) as predictors, and sex as the response. From
Graph&Fit select Plot of. Place I3:SIR.p1 on the H axis and B2:EtaU on the
V axis. Include the plot in Word. Is the plot linear?

13.15 . In this problem you will nd a good submodel for the ICU data
obtained from STATLIB or the texts website. This data set will violate some
of the rules of thumb: the model II does not have enough predictors to make
a good EE plot. See Example 13.13.

a) Activate ICU.lsp in Arc with the menu commands

File > Load >. Then use the upper box to navigate to where ICU.lsp is
stored, for example Removable Disk (G:). Scroll up the screen to read the
data description.

b) Use the menu commands ICU>Make factors and select loc and race.

c) From Graph&Fit select Fit binomial response. Select STA as the re-
sponse and ones as the number of trials. The full model will use every predic-
tor except ID, LOC, and RACE (the latter 2 are replaced by their factors):
select AGE, Bic, CAN, CPR, CRE, CRN, FRA, HRA, INF, {F}LOC , PCO,
PH, PO2 , PRE , {F}RACE , SER, SEX, SYS, and TYP as predictors.
13.10 Problems 451

Perform the logistic regression and include the relevant output for testing in
Word.

d) Make the response plot for the full model: from Graph&Fit select Plot
of. Place STA on V and B1:EtaU on H. From the OLS popup menu, select
Logistic and move the slider bar to 1. From the lowess popup menu select
SliceSmooth and move the slider bar until the t is good. Use lowess if Slice-
Smooth does not work. Include your plot in Word. Is the full model good?

e) Using what you have learned in class, nd a good submodel and include
the relevant output in Word.
[Hints: Use forward selection and backward elimination and nd the model
with the minimum AIC. Let (I) = AIC(I) AIC(Imin ). Then nd the
model II with the fewest number of predictors such that (II ) 2. Then
submodel II is the initial submodel to examine. Fit model II and look at the
Wald test pvalues. Try to eliminate predictors with large pvalues but make
sure that the deviance does not increase too much. Also examine submodels
I with fewer predictors than II with (I) 7. WARNING: do not delete
part of a factor. Either keep all 2 factor dummy variables or delete all I-1=2
factor dummy variables. You may have several models, say B2, B3, B4, and
B5 to look at. Make the EE and response plots for each model. WARNING:
if a useful factor is in the full model but not the reduced model, then the EE
plot may have I = 3 lines if the factor should be in the model. See part h)
below.]

f) Make a response plot for your nal submodel.

g) Suppose that B1 contains your full model and B5 contains your nal
submodel. Make an EE plot for your nal submodel: from Graph&Fit select
Plot of. Select B1:EtaU for the V box and B5:EtaU, for the H box. After
the plot appears, click on the options popup menu. A window will appear.
Type y = x and click on OK. This action adds the identity line to the plot.
Include the plot in Word.
If the EE plot is good, then the plotted points will cluster about the
identity line. For model II , some points are far away from the identity line.
At least one variable needs to be added to model II to get a good submodel
and EE plot, violating the rule of thumb that submodels with more predictors
than II should not be examined. Variable selection may be suggesting poor
submodels because of clusters of cases that are given exact probabilities of 0
or 1. Try adding {F}RACE to the predictors in II .

h) Using e), f), g), and any additional output that you desire [e.g.
AIC(full), AIC(Imin ), AIC(II ), and AIC(nal submodel)], explain why your
nal submodel is good.
13.16. In this problem you will examine the museum skull data.
452 13 GLMs and GAMs

Activate museum.lsp in Arc with the menu commands

File > Load > Removable Disk (G:) > museum.lsp. Scroll up the screen
to read the data description.

a) From Graph&Fit select Fit binomial response. Select ape as the response
and ones as the number of trials. Select x5 as the predictor. Perform the
logistic regression and include the relevant output for testing in Word.

b) Make the response plot and place it in Word (the response variable is
ape not y). Is the LR model good?

Now you will examine logistic regression when there is perfect classication
of the sample response variables. Assume that the model used in c)g) is in
menu B2.

c) From Graph&Fit select Fit binomial response. Select ape as the response
and ones as the number of trials. Select x3 as the predictor. Perform the
logistic regression and include the relevant output for testing in Word.

d) Make the response plot and place it in Word (the response variable is
ape not y). Is the LR model good?

e) Perform the Wald test for Ho : = 0.

f) From B2 select Examine submodels and include the output in Word.

Then use the output to perform a 4 step deviance test on the submodel used
in part c).

g) The tests in e) and f) are both testing Ho : = 0 but give dierent

results. Why are the results dierent and which test is correct?

13.17. In this problem you will nd a good submodel for the credit data
from Fahrmeir and Tutz (2001).

Activate credit.lsp in Arc with the menu commands

File > Load > Removable Disk (G:) > credit.lsp. Scroll up the screen to
read the data description. This is a big data set and computations may take
several minutes.

Use the menu commands credit>Make factors and select x1 , x3 , x4 , x6 ,

x7 , x8 , x9 , x10 , x11 , x12 , x14 , x15 , x16 , and x17 . Then click on OK.

a) From Graph&Fit select Fit binomial response. Select y as the response

and ones as the number of trials. Select {F}x1 , x2 , {F}x3 , {F}x4 , x5 , {F}x6 ,
{F}x7 , {F}x8 , {F}x9 , {F}x10 , {F}x11 , {F}x12 , x13 , {F}x14 , {F}x15 , {F}x16 ,
{F}x17 , x18 , x19 , and x20 as predictors. Perform the logistic regression and
include the relevant output for testing in Word. You should get 1000 cases,
df = 945, and a deviance of 892.957.
13.10 Problems 453

b) Make the response plot for the full model: from Graph&Fit select Plot
of. Place y on V and B1:EtaU on H. From the OLS popup menu, select
Logistic and move the slider bar to 1. From the lowess popup menu select
SliceSmooth and move the slider bar until the t is good. Include your plot
in Word. Is the full model good? Use lowess if SliceSmooth does not work.

c) Using what you have learned in class, nd a good submodel and include
the relevant output in Word. (See hints below Problem 13.15e.)

d) Make a response plot for your nal submodel.

e) Suppose that B1 contains your full model and B5 contains your nal
submodel. Make an EE plot for your nal submodel: from Graph&Fit select
Plot of. Select B1:EtaU for the V box and B5:EtaU, for the H box. Place y
in the Mark by box. After the plot appears, click on the options popup menu.
A window will appear. Type y = x and click on OK. This action adds the
identity line to the plot. Also move the OLS slider bar to 1. Include the plot
in Word.

f) Using c), d), e), and any additional output that you desire (e.g.,
AIC(full), AIC(min), and AIC(nal submodel), explain why your nal sub-
model is good.

13.18 . a) This problem uses a data set from Myers et al. (2002). Activate
popcorn.lsp in Arc with the menu commands File > Load > Removable
Disk (G:) > popcorn.lsp. Scroll up the screen to read the data description.
From Graph&Fit select Fit Poisson response. Use oil, temp, and time as
the predictors and y as the response. From Graph&Fit select Plot of. Select
P1:EtaU for the H box and y for the V box. From the OLS popup menu
select Poisson and move the slider bar to 1. Move the lowess slider bar until
the lowess curve tracks the exponential curve. Include the response plot in
Word.

b) From the P1 menu select Examine submodels, click on OK and include

the output in Word.

c) Test whether 1 = 2 = 3 = 0.
d) From the popcorn menu, select Transform and select y. Put 1/2 in the p
box and click on OK. From the popcorn menu, select Add a variate and type
yt = sqrt(y)*log(y) in the resulting window. Repeat three times adding the
variates oilt = sqrt(y)*oil, tempt = sqrt(y)*temp, and timet = sqrt(y)*time.
From Graph&Fit select Fit linear LS and choose y 1/2 , oilt, tempt, and timet
as the predictors, yt as the response and click on the Fit intercept box to
remove the check. Then click on OK. From Graph&Fit select Plot of. Select
L2:Fit-Values for the H box and yt for the V box. A plot should appear.
454 13 GLMs and GAMs

Click on the Options menu and type y = x to add the identity line. Include
the weighted t response plot in Word.

e) From Graph&Fit select Plot of. Select L2:Fit-Values for the H box and
L2:Residuals for the V box. Include the weighted residual response plot in
Word.

f) For the plot in e), highlight the case in the upper right corner of the plot
by using the mouse to move the arrow just above and to the left the case.
Then hold the rightmost mouse button down and move the mouse to the
right and down. From the Case deletions menu select Delete selection from
data set, then from Graph&Fit select Fit Poisson response. Use oil, temp, and
time as the predictors, and y as the response. From Graph&Fit select Plot
of. Select P3:EtaU for the H box and y for the V box. From the OLS popup
menu select Poisson and move the slider bar to 1. Move the lowess slider bar
until the lowess curve tracks the exponential curve. Include the response plot
in Word.

g) From the P3 menu select Examine submodels, click on OK and include

the output in Word.

h) Test whether 1 = 2 = 3 = 0.

i) From Graph&Fit select Fit linear LS. Make sure that y 1/2 , oilt, tempt,
and timet are the predictors, yt is the response, and that the Fit intercept
box does not have a check. Then click on OK. From Graph&Fit select Plot
of. Select L4:Fit-Values for the H box and yt for the V box. A plot should
appear. Click on the Options menu and type y = x to add the identity line.
Include the weighted t response plot in Word.

j) From Graph&Fit select Plot of. Select L4:Fit-Values for the H box and
L4:Residuals for the V box. Include the weighted residual plot in Word.

k) Is the deleted point inuential? Explain briey.

l) From Graph&Fit select Plot of. Select P3:EtaU for the H box and
P3:Dev-Residuals for the V box. Include the deviance residual plot in Word.

m) Is the weighted residual plot from part j) a better lack of t plot than
the deviance residual plot from part l)? Explain briey.

R problems
Use the command source(G:/lregpack.txt) to download the func-
tions and the command source(G:/lregdata.txt) to download the data.
See Preface or Section 14.1. Typing the name of the lregpack function,
e.g. lrdata, will display the code for the function. Use the args command,
e.g. args(lrdata), to display the needed arguments for the function. For some
13.10 Problems 455

of the following problems, the R commands can be copied and pasted from
(http://lagrange.math.siu.edu/Olive/lreghw.txt) into R.

13.19. Obtain the function lrdata from lregpack.txt. Enter the com-
mands
out <- lrdata()
x <- out$x
y <- out$y
Obtain the function lressp from lregpack.txt. Enter the commands
lressp(x,y) and include the resulting plot in Word.

13.20. Obtain the function prdata from lregpack.txt. Enter the com-
mands
out <- prdata()
x <- out$x
y <- out$y
a) Obtain the function pressp from lregpack.txt. Enter the commands
pressp(x,y) and include the resulting plot in Word.

b) Obtain the function prplot from lregpack.txt. Enter the commands

prplot(x,y) and include the resulting plot in Word.

13.21. The and Rousseeuw and Leroy (1987, p. 26) Belgian telephone
data has response Y = number of international phone calls (in tens of mil-
lions) made per year in Belgium. The predictor variable x = year (19501973).
From 1964 to 1969 total number of minutes of calls was recorded instead, and
years 1963 and 1970 were also partially eected. Hence there are 6 large out-
liers and 2 additional cases that have been corrupted.
a) The simple linear regression model is Y = + x + e = SP + e. The
R commands from the URL above Problem 13.19 make a response plot of
ESP = Y = + x versus Y for this model. Include the plot in Word.
b) The additive model is Y = + S(x) + e = AP + e where S is some
unknown function of x. The R commands make a response plot of EAP =
+ S(x) versus Y for this model. Include the plot in Word.
c) The simple linear regression model is a special case of the additive model
with S(x) = x. The additive model is a special case of the additive error
regression model Y = m(x) + e where m(x) = + S(x). The response plots
for these three models are used in the same way as the response plot for the
multiple linear regression model: if the model is good, then the plotted points
should cluster about the identity line with no other pattern. Which response
plot is better for showing that something is wrong with the model? Explain
briey.
456 13 GLMs and GAMs

13.22. In a generalized additive model (GAM), Y x|AP where AP =

k
+ i=1 Si (xi ). In a generalized linear model (GLM), Y x|SP where SP =
+ T x. Note that a GLM is a special case of a GAM where Si (xi ) = i xi .
A GAM is useful for showing that the predictors x1 , ..., xk in a GLM have
the correct form, or if predictor transformations or additional terms such as
x2i are needed. If the plot of Si (xi ) is linear, do not change xi in the GLM,
but if the plot is nonlinear, use the shape of Si to suggest functions of xi to
add to the GLM, such as log(xi ), x2i , and x3i . Ret the GAM to check the
linearity of the terms in the updated GLM. Wood (2006, pp. 8286) describes
heart attack data where the response Y is the number of heart attacks for
mi patients suspected of suering a heart attack. The enzyme ck (creatine
kinase) was measured for the patients. A binomial logistic regression (GLM)
was t with predictors x1 = ck, x2 = [ck]2 , and x3 = [ck]3 . Call this the
Wood model I2 . The predictor ck is skewed suggesting log(ck) should be
added to the model. Then output suggested that ck is not needed in the
model. Let the binomial logistic regression model that uses x = log(ck) as
the only predictor be model I1 . a) The R code for this problem from the URL
above Problem 13.19 makes 4 plots. Plot a) shows S for the binomial GAM
using ck as a predictor is nonlinear. Plot b) shows that S for the binomial
GAM using log(ck) as a predictor is linear. Plot c) shows the EE plot for the
binomial GAM using ck as the predictor and model I1 . Plot d) shows the
response plot of ESP versus Zi = Yi /mi , the proportion of patients suering
a heart attack for each value of xi = ck. The logistic curve = E(Zi |xi ) is
added as a visual aid. Include these plots in Word.
Do the plotted proportions fall about the logistic curve closely?
b) The command for b) gives AIC(outw) for model I2 and AIC(out) for
model I1 . Include the two AIC values below the plots in a).
A model I1 with j fewer predictors than model I2 is better than model
I2 if AIC(I1 ) AIC(I2 ) + 2j. Is model I1 better than model I2 ?

The following problem uses SAS and Arc.

13.23 . SASall subsets: On the webpage (http://lagrange.math.

siu.edu\/Olive/students.htm) there is a le cbrain.txt that will be used for
this problem. The le cbrain.txt contains the cbrain data (that has appeared
in several previous Arc problems) without the header that describes the data.
i) Using a web browser like Firefox, go to the webpage and click on
cbrain.txt. After the le opens, copy and paste the data into Word (or
Notepad). (The commands Edit>Select All and Edit>copy worked. Then
open Word (or Notepad) and enter the command Paste (or Edit>paste)
to make the data set appear.)

ii) SAS needs an end of le marker to determine when the data ends.
SAS uses a period as the end of le marker. Add a period on the line after
the last line of data in Word and save the le as cbrain.dat.txt on your ash
13.10 Problems 457

drive (in the save as type box menu, select plain text and in the File name
box, type cbrain.dat). If these commands fail, get the le cbrain.dat.txt from
the URL and save it on your ash drive. Assume that the ash drive is
Removable Disk (J:)). Warning: make sure that the le has been saved
as cbrain.dat.txt.

iii) Get SAS code for the problem.

iv) Get into SAS, and cut and paste the program into the SAS editor win-
dow. To execute the program, use the top menu commands Run>Submit.
An output window will appear if successful. Warning: if you do not have
the le cbrain.dat.txt on the J drive, then you need to change the in-
le command, e.g. change inle j:cbrain.dat.txt; to inle g:cbrain.dat.txt;
if you are using G drive.

a) To copy and paste relevant output into Word, click on the output win-
dow and use the top menu commands Edit>Select All and then the menu
commands Edit>Copy.
Interesting models have C(p) 2k where k = number in model.
The only SAS output for this problem that should be included
in Word are two header lines (Number in model, R-square, C(p), Variables
in Model) and the rst line with Number in Model = 6 and C(p) = 7.0947.
You may want to copy all of the SAS output into Word, and then cut and
paste the relevant two lines of output into another window of Word.
b) Activate cbrain.lsp in Arc with the menu commands
File > Load >. Then use the upper box to navigate to where cbrain.lsp is
stored, for example Removable Disk (G:). From Graph&Fit select Fit binomial
response. Select age = X2, breadth = X6, cephalic = X10, circum = X9, headht
= X4, height = X3, length = X5, and size = X7 as predictors, sex as the
response, and ones as the number of trials. This is the full logistic regression
model. Include the relevant output in Word. (A better full model was used
in Problem 13.14 .)
c) Response plot. From Graph&Fit select Plot of. Place sex on V and
B1:EtaU on H. From the OLS popup menu, select Logistic and move the
slider bar to 1. From the lowess popup menu select SliceSmooth and move
the slider bar until the t is good. Include your plot in Word. Are the slice
means (observed proportions) tracking the logistic curve (tted proportions)
fairly well? Use lowess if SliceSmooth does not work.
d) From Graph&Fit select Fit binomial response. Select breadth = X6,
cephalic = X10, circum = X9, headht = X4, height = X3, and size = X7 as
predictors, sex as the response, and ones as the number of trials. This is the
best submodel. Include the relevant output in Word.
e) Put the EE plot H B2:EtaU versus V B1:EtaU in Word. Is the plot
linear?
458 13 GLMs and GAMs

f) From Graph&Fit select Plot of. Place sex on V and B2:EtaU on H.

From the OLS popup menu, select Logistic and move the slider bar to 1.
From the lowess popup menu select SliceSmooth and move the slider bar
until the t is good. Include your plot in Word. Are the slice means (observed
proportions) tracking the logistic curve (tted proportions) fairly well? Use
lowess if SliceSmooth does not work.
Chapter 14
Stu for Students

14.1 R and Arc

This chapter gives some information about R and Arc, and some hints for
selected homework problems. As of August 2016, the authors personal com-
puter has Version 3.3.1 (June 21, 2016) of R, and Version 1.06 (July 2004)
of Arc.

Downloading the books data.lsp les into Arc

Many homework problems use data les for Arc contained in the books
website (http://lagrange.math.siu.edu/Olive/lregbk.htm). As an ex-
ample, open the cbrain.lsp le with Notepad. Then use the menu commands
File> Save As. A window appears. On the top Save in box change what
is in the box to Removable Disk (G:) in order to save the le on ash drive
G. Then in Arc activate the cbrain.lsp le with the menu commands File >
Load > Removable Disk (G:) > cbrain.lsp.
Alternatively, open cbrain.lsp le with Notepad. Then use the menu com-
mands File>Save As. A window appears. On the top Save in box change
what is in the box to My Documents. Then go to Arc and use the menu
commands File>Load. A window appears. Change Arc to My Docu-
ments and open cbrain.lsp.

Many of the homework problems use R functions contained in the books

website (http://lagrange.math.siu.edu/Olive/lregbk.htm) under the
le name lregpack.txt. The following two R commands can be copied and
pasted into R from near the top of the le (http://lagrange.math.siu.
edu/Olive/lreghw.txt).
Downloading the books R functions lregpack.txt and data les lreg-
data.txt into R: the commands
source("http://lagrange.math.siu.edu/Olive/lregpack.txt")
source("http://lagrange.math.siu.edu/Olive/lregdata.txt")

Springer International Publishing AG 2017 459

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1 14
460 14 Stu for Students

can be used to download the R functions and data sets into R. Type ls().
Nearly 70 R functions from lregpack.txt should appear. In R, enter the com-
mand q(). A window asking Save workspace image? will appear. Click on
No to remove the functions from the computer (clicking on Yes saves the func-
tions in R, but the functions and data are easily obtained with the source
commands).
This section gives tips on using R, but is no replacement for books such
as Becker et al. (1988), Chambers (2008), Crawley (2005, 2013), Fox and
Weisberg (2010), or Venables and Ripley (2010). Also see MathSoft (1999a,b)
and use the website (www.google.com) to search for useful websites. For
example, enter the search words R documentation.

The command q() gets you out of R.

Least squares regression can be done with the function lst or lm.
The commands help(fn) and args(fn) give information about the function
fn, e.g. if fn = lst.
Type the following commands.
x <- matrix(rnorm(300),nrow=100,ncol=3)
y <- x%*%1:3 + rnorm(100)
out<- lsfit(x,y)
out$coef
ls.print(out)
The rst line makes a 100 by 3 matrix x with N(0,1) entries. The second
line makes y[i] = 0 + 1 x[i, 1] + 2 x[i, 2] + 3 x[i, 2] + e where e is N(0,1). The
term 1:3 creates the vector (1, 2, 3)T and the matrix multiplication operator is
% %. The function lsfit will automatically add the constant to the model.
Typing out will give you a lot of irrelevant information, but out$coef and
out$resid give the OLS coecients and residuals, respectively.
To make a residual plot, type the following commands.
fit <- y - out$resid
plot(fit,out$resid)
title("residual plot")
The rst term in the plot command is always the horizontal axis while the
second is on the vertical axis.

To put a graph in Word, hold down the Ctrl and c buttons simulta-
neously. Then select Paste from the Word menu, or hit Ctrl and v at the
same time.
To enter data, open a data set in Notepad or Word. You need to know
the number of rows and the number of columns. Assume that each case is
entered in a row. For example, assuming that the le cyp.lsp has been saved
on your ash drive from the webpage for this book, open cyp.lsp in Word. It
has 76 rows and 8 columns. In R, write the following command.
14.1 R and Arc 461

cyp <- matrix(scan(),nrow=76,ncol=8,byrow=T)

Then copy the data lines from Word and paste them in R. If a cursor does
not appear, hit enter. The command dim(cyp) will show if you have entered
the data correctly.
Enter the following commands
cypy <- cyp[,2]
cypx<- cyp[,-c(1,2)]
lsfit(cypx,cypy)$coef
to produce the output below.
Intercept X1 X2 X3
205.40825985 0.94653718 0.17514405 0.23415181
X4 X5 X6
0.75927197 -0.05318671 -0.30944144
To check that the data is entered correctly, t LS in Arc with the response
variable height and the predictors sternal height, nger to ground, head length,
nasal length, bigonal breadth, and cephalic index (entered in that order). You
should get the same coecients given by R.

Making functions in R is easy.

For example, type the following commands.

mysquare <- function(x){
# this function squares x
r <- x^2
r }
The second line in the function shows how to put comments into functions.

Modifying your function is easy.

Use the x command.

x(mysquare)
This will open an editor such as Notepad and allow you to make changes. (In
Splus, the command Edit(mysquare) may also be used to modify the function
mysquare.)

To save data or a function in R, when you exit, click on Yes when the
Save worksheet image? window appears. When you reenter R, type ls().
This will show you what is saved. You should rarely need to save anything
for this book. To remove unwanted items from the worksheet, e.g. x, type
rm(x),
pairs(x) makes a scatterplot matrix of the columns of x,
hist(y) makes a histogram of y,
boxplot(y) makes a boxplot of y,
462 14 Stu for Students

stem(y) makes a stem and leaf plot of y,

scan(), source(), and sink() are useful on a Unix workstation.
To type a simple list, use y < c(1,2,3.5).
The commands mean(y), median(y), var(y) are self explanatory.

The following commands are useful for a scatterplot created by the com-
mand plot(x,y).
lines(x,y), lines(lowess(x,y,f=.2))
identify(x,y)
abline(out$coef ), abline(0,1)

The usual arithmetic operators are 2 + 4, 3 7, 8 4, 8/4, and

2^{10}.
The ith element of vector y is y[i] while the ij element of matrix x is x[i, j].
The second row of x is x[2, ] while the 4th column of x is x[, 4]. The transpose
of x is t(x).

The command apply(x,1,fn) will compute the row means if fn = mean.

The command apply(x,2,fn) will compute the column variances if fn = var.
The commands cbind and rbind combine column vectors or row vectors with
an existing matrix or vector of the appropriate dimension.

Transferring Data to and from Arc and R.

For example, suppose that the Belgium telephone data (Rousseeuw and Leroy
1987, p. 26) has the predictor year stored in x and the response number of
calls stored in y in R. Combine the data into a matrix z and then use the
write.table command to display the data set as shown below. The
sep=
separates the columns by two spaces.

> z <- cbind(x,y)

> write.table(data.frame(z),sep= )
row.names z.1 y
1 50 0.44
2 51 0.47
3 52 0.47
4 53 0.59
5 54 0.66
6 55 0.73
7 56 0.81
8 57 0.88
9 58 1.06
10 59 1.2
11 60 1.35
14.1 R and Arc 463

12 61 1.49
13 62 1.61
14 63 2.12
15 64 11.9
16 65 12.4
17 66 14.2
18 67 15.9
19 68 18.2
20 69 21.2
21 70 4.3
22 71 2.4
23 72 2.7073
24 73 2.9

To enter a data set into Arc, use the following template new.lsp.

dataset=new
begin description
Artificial data.
Contributed by David J. Olive.
end description
begin variables
col 0 = x1
col 1 = x2
col 2 = x3
col 3 = y
end variables
begin data

Next open new.lsp in Notepad. (Or use the vi editor in Unix. Sophisticated
editors like Word will often work, but they sometimes add things like page
breaks that do not allow the statistics software to use the le.) Then copy
the data lines from R and paste them below new.lsp. Then modify the le
new.lsp and save it on a ash drive as the le belg.lsp. (Or save it in mdata
where mdata is a data folder added within the Arc data folder.) The header
of the new le belg.lsp is shown below.

dataset=belgium
begin description
Belgium telephone data from
Rousseeuw and Leroy (1987, p. 26)
end description
begin variables
col 0 = case
col 1 = x = year
col 2 = y = number of calls in tens of millions
464 14 Stu for Students

end variables
begin data
1 50 0.44
. . .
. . .
. . .
24 73 2.9
The le above also shows the rst and last lines of data. The header le needs
a data set name, description, variable list, and a begin data command. Often
the description can be copied and pasted from the source of the data, e.g.
from the STATLIB website. Note that the rst variable starts with Col 0.

To transfer a data set from Arc to R, select the item Display data
from the datasets menu. Select the variables you want to save, and then
push the button for Save in R/Splus format. You will be prompted to give
a le name. If you select bodfat, then two les bodfat.txt and bodfat.Rd will
be created. The le bodfat.txt can be read into either R using the read.table
command. The le bodfat.Rd saves the documentation about the data set in
a standard format for R.

Getting information about a library in R

In R, a library is an addon package of R code. The command library()
lists all available libraries, and information about a specic library, such
as leaps for variable selection, can be found, e.g., with the command li-
brary(help=leaps).

Downloading a library into R

Many researchers have contributed a library of R code that can be down-
loaded for use. To see what is available, go to the website
(http://cran.us.r-project.org/) and click on the Packages icon. Suppose
you are interested the Weisberg (2002) dimension reduction library dr.
Following Crawley (2013, p. 8), you may need to Run as administrator
before you can install packages (right click on the R icon to nd this). Then
use the following command to install the dr package.
install.packages("dr")
Open R and type the following command.
library(dr)
Next type help(dr) to make sure that the library is available for use.

Warning: R is free but not fool proof. If you have an old version of R
and want to download a library, you may need to update your version of
R. The libraries for robust statistics may be useful for outlier detection, but
the methods have not been shown to be consistent or high breakdown. All
software has some bugs. For example, Version 1.1.1 (August 15, 2000) of R
had a random generator for the Poisson distribution that produced variates
14.2 Hints for Selected Problems 465

with too small of a mean for 10. Hence simulated 95% condence
intervals might contain 0% of the time. This bug seems to have been xed
in Versions 2.4.1 and later. Also, some functions in lregpack may no longer
work in new versions of R.

14.2 Hints for Selected Problems

1.1 T x = xT

2.1 Fo = 0.904, pvalue > 0.1, fail to reject Ho, so the reduced model is
good.

2.2 a) 25.970
b) Fo = 0.600, pvalue > 0.5, fail to reject Ho, so the reduced model is
good.

2.3 a) [1.229, 3.345]

b) [1.0825, 3.4919]

2.4 c) Fo = 265.96, pvalue = 0.0, reject Ho, there is an MLR relationship

between the response variable height and the predictors sternal height and
nger to ground.
2.6 No, the relationship should be linear.

2.7 No, since 0 is in the CI. X could be a very useful predictor for Y , e.g.
if Y = X 2 .

2.11 a) 7 + Xi
 
b) = (Yi 7)Xi / Xi2
  2
2.14 a) 3 =
 X3i (Yi 10 2X2i )/ X3i . The second partial derivative
2
= X3i > 0.

2.21 d) The rst assumption to check would be the constant variance

assumption.

3.1 The model uses constant, nger to ground, and sternal height. (You
can tell what the variable are by looking at which variables are deleted.)

3.2 Use L3. L1 and L2 have more predictors and higher Cp than L3 while
L4 does not satisfy the Cp 2k screen.

3.3 a) L2.
b) Examine L3 since L1 has too many predictors while L4 does not satisfy
the Cp 2k screen.
466 14 Stu for Students

3.4 Use a constant, A, B, and C since this is the only model that satises
the Cp 2k screen.
b) Use the model with a constant and B since it has the smallest Cp and
the smallest k such that the Cp 2k screen is satised.

3.7 a) The plot looks roughly like the SW corner of a square.

b) No, the plot is nonlinear.
c) Want to spread small values of y, so make smaller. Hence use y (0) =
log(y).

3.8 Several of the marginal relationships are nonlinear, including E(M |H).

3.9 This problem has the student reproduce Example 3.3. Hence log(Y )
is the appropriate response transformation.

3.10 Plots b), c), and e) suggest that log(ht) is needed while plots d),
f), and g) suggest that log(ht) is not needed. Plots c) and d) show that the
residuals from both models are quite small compared to the tted values.
Plot d) suggests that the two models produce approximately the same tted
values. Hence if the goal is prediction, the expensive log(ht) measurement
does not seem to be needed.

3.11 h) The submodel is ok, but the response and residual plots found in
f) for the submodel do not look as good as those for the full model found in
d). Since the submodel residuals do not look good, more terms are probably
needed in the model.

3.12 b) Forward selection gives constant, (size)1/3 , age, sex, breadth, and
cause.
c) Backward elimination gives constant, age, cause, cephalic, headht,
length, and sex.
d) Forward selection is better because it has fewer terms and a smaller Cp .
e) The variables are highly correlated. Hence backward elimination quickly
eliminates the single best predictor (size)1/3 and cannot get a good model
that only has a few terms.
f) Although the model in b) could be used, a better model uses constant,
age, sex, and (size)1/3 .
j) The FF and RR plots are good and so are the response and residual
plots if you ignore the good leverage points corresponding to the 5 babies.

8.3. See Example 8.6.

9.3. See Example 9.2.

10.1 a) X2 N (100, 6).

14.2 Hints for Selected Problems 467

b)      
X1 49 3 1
N2 , .
X3 17 1 4
c) X1 X4 and X3 X4 .
d)
Cov(X1 , X3 ) 1
(X1 , X2 ) =  = = 0.2887.
VAR(X1 )VAR(X3 ) 3 4
10.2 a) Y |X N (49, 16) since Y X. (Or use E(Y |X) = Y +
1
12 22 (X x ) = 49 + 0(1/25)(X 100) = 49 and VAR(Y |X) =
1
11 12 22 21 = 16 0(1/25)0 = 16.)
1
b) E(Y |X) = Y +12 22 (X x ) = 49+10(1/25)(X 100) = 9+0.4X.

1
c) VAR(Y |X) = 11 12 22 21 = 16 10(1/25)10 = 16 4 = 12.

10.4 The proof is identical to that given in Example 10.2. (In addition, it
is fairly simple to show that M1 = M2 M . That is, M depends on but
not on c or g.)

10.6 a) Sort each column, then nd the median of each column. Then
MED(W ) = (1430, 180, 120)T .
b) The sample mean of (X1 , X2 , X3 )T is found by nding the sample mean
of each column. Hence x = (1232.8571, 168.00, 112.00)T .

10.11 B = E[E(X|B T X)X T B)] = E(M B B T XX T B) = M B B T

B. Hence M B = B(B T B)1 .

10.13 a) The 4 plots should look nearly identical with the ve cases 6165
appearing as outliers.

11.1. See the proof of Theorem 11.17.

13.2 a) ESP = 1.11108, exp(ESP ) = 3.0376 and = exp(ESP )/(1 +

exp(ESP )) = 3.0376/(1 + 3.0376) = 0.7523.

13.3 G2 (O|F ) = 62.7188 13.5325 = 49.1863, df = 3, pvalue = 0.00,

reject Ho, there is an LR relationship between ape and the predictors lower
jaw, upper jaw, and face length.

13.4 G2 (R|F ) = 17.1855 13.5325 = 3.653, df = 1, 0.05 < pvalue < 0.1,
fail to reject Ho, the reduced model is good.

13.5a ESP = 0.2812465 and = exp(ESP ) = 1.3248.

13.6 G2 (O|F ) = 187.490138.685 = 48.805, df = 2, pvalue = 0.00, reject

Ho, there is a PR relationship between possums and the predictors habitat
and stags.
468 14 Stu for Students

13.8 a) B4
b) EE plot
c) B3 is best. B3 has 12 fewer predictors than B2 but the AIC increased
by less than 3. B1 has too many predictors with large Wald pvalues, B2 =
II still has too many predictors (want 300/10 = 30 predictors), while B4
has too small of a pvalue for the change in deviance test.

13.12 a) A good submodel uses a constant, Bark, Habitat, and Stags as

predictors.
d) The response and EE plots are good as are the Wald pvalues. Also
AIC(full) = 141.506 while AIC(sub) = 139.644.

13.14 b) Use the log rule: (max age)/(min age) = 1400 > 10.
e) The slice means track the logistic curve very well if 8 slices are used.
i) The EE plot is linear.
j) The slice means track the logistic curve very well if 8 slices are used.

13.15 c) Should have 200 cases, df = 178, and deviance = 112.168.

d) The response plot with 12 slices suggests that the full model is good.
h) The submodel II that uses a constant, AGE, CAN, SYS, TYP, and
FLOC and the submodel I2 that is the same as I1 but also uses FRACE
seem to be competitors. If the factor FRACE is not used, then the response
plot has 3 groups. The Wald pvalues suggest that FRACE is not needed,
but the EE plot suggests that FRACE is needed. Want a good EE plot, so
need to add FRACE to the II model, resulting in model I2 .

13.16 b) The response plot (e.g., with 4 slices) is bad, so the LR model is
bad.
d) Now the response plot (e.g., with 12 slices) is good in that slice smooth
and the logistic curve are close where there is data (also the LR model is
good at classifying 0s and 1s).
f) For this problem, G2 (O|F ) = 62.7188 0.00419862 = 62.7146, df = 1,
pvalue = 0.00, so reject Ho and conclude that there is an LR relationship
between ape and the predictor x3 .
g) The MLE does not exist since there is perfect classication (and the
logistic curve can get close to but never equal a discontinuous step function).
Hence Wald pvalues tend to have little meaning; however, the change in
deviance test tends to correctly suggest that there is an LR relationship
when there is perfect classication.
14.2 Hints for Selected Problems 469

13.18 k) The deleted point is certainly inuential. Without this case,

there does not seem to be a PR relationship between the predictors and the
response.
m) The weighted residual plot suggests that something is wrong with the
model since the plotted points scatter about a line with positive slope rather
than a line with 0 slope. The deviance residual plot does not suggest that
anything is wrong with the model.

13.19 The response plot should look ok, but the function uses a default
number of slices rather than allowing the user to select the number of slices
using a slider bar (a useful feature of Arc).

13.20 a) Since this is simulated PR data, the response plot should look
ok, but the function uses a default lowess smoothing parameter rather than
allowing the user to select the smoothing parameter using a slider bar (a
useful feature of Arc).
b) The data should follow the identity line in the weighted t response
plots. In about 1 in 20 plots there will be a very large count that looks like
an outlier. The weighted residual plot based on the MLE usually looks better
than the plot based on the minimum chi-square estimator (the MLE plot
tends to have less of a left opening megaphone shape).

13.22 b) Model I1 is better since it has fewer predictors and lower AIC
than model I2 .
13.23 a)
Number in Model Rsquare C(p) Variables in model
6 0.2316 7.0947 X3 X4 X6 X7 X9 X10
c) The slice means follow the logistic curve fairly well with 8 slices.
e) The EE plot is linear.
f) The slice means follow the logistic curve fairly well with 8 slices.
470 14 Stu for Students

14.3 Tables

Tabled values are F(k,d, 0.95) where P (F < F (k, d, 0.95)) = 0.95.
00 stands for . Entries were produced with the qf(0.95,k,d) command in
R. The numerator degrees of freedom are k while the denominator degrees of
freedom are d.
k 1 2 3 4 5 6 7 8 9 00
d
1 161 200 216 225 230 234 237 239 241 254
2 18.5 19.0 19.2 19.3 19.3 19.3 19.4 19.4 19.4 19.5
3 10.1 9.55 9.28 9.12 9.01 8.94 8.89 8.85 8.81 8.53
4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 6.00 5.63
5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 4.77 4.37
6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 4.10 3.67
7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 3.68 3.23
8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 3.39 2.93
9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 3.18 2.71
10 4.96 4.10 3.71 3.48 3.33 3.22 3.14 3.07 3.02 2.54
11 4.84 3.98 3.59 3.36 3.20 3.09 3.01 2.95 2.90 2.41
12 4.75 3.89 3.49 3.26 3.11 3.00 2.91 2.85 2.80 2.30
13 4.67 3.81 3.41 3.18 3.03 2.92 2.83 2.77 2.71 2.21
14 4.60 3.74 3.34 3.11 2.96 2.85 2.76 2.70 2.65 2.13
15 4.54 3.68 3.29 3.06 2.90 2.79 2.71 2.64 2.59 2.07
16 4.49 3.63 3.24 3.01 2.85 2.74 2.66 2.59 2.54 2.01
17 4.45 3.59 3.20 2.96 2.81 2.70 2.61 2.55 2.49 1.96
18 4.41 3.55 3.16 2.93 2.77 2.66 2.58 2.51 2.46 1.92
19 4.38 3.52 3.13 2.90 2.74 2.63 2.54 2.48 2.42 1.88
20 4.35 3.49 3.10 2.87 2.71 2.60 2.51 2.45 2.39 1.84
25 4.24 3.39 2.99 2.76 2.60 2.49 2.40 2.34 2.28 1.71
30 4.17 3.32 2.92 2.69 2.53 2.42 2.33 2.27 2.21 1.62
00 3.84 3.00 2.61 2.37 2.21 2.10 2.01 1.94 1.88 1.00
14.3 Tables 471

Tabled values are t,d where P (t < t,d ) = where t has a t distribution
with d degrees of freedom. If d > 29, use the N (0, 1) cutos d = Z = .
alpha pvalue
d 0.005 0.01 0.025 0.05 0.5 0.95 0.975 0.99 0.995 left tail
1 -63.66 -31.82 -12.71 -6.314 0 6.314 12.71 31.82 63.66
2 -9.925 -6.965 -4.303 -2.920 0 2.920 4.303 6.965 9.925
3 -5.841 -4.541 -3.182 -2.353 0 2.353 3.182 4.541 5.841
4 -4.604 -3.747 -2.776 -2.132 0 2.132 2.776 3.747 4.604
5 -4.032 -3.365 -2.571 -2.015 0 2.015 2.571 3.365 4.032
6 -3.707 -3.143 -2.447 -1.943 0 1.943 2.447 3.143 3.707
7 -3.499 -2.998 -2.365 -1.895 0 1.895 2.365 2.998 3.499
8 -3.355 -2.896 -2.306 -1.860 0 1.860 2.306 2.896 3.355
9 -3.250 -2.821 -2.262 -1.833 0 1.833 2.262 2.821 3.250
10 -3.169 -2.764 -2.228 -1.812 0 1.812 2.228 2.764 3.169
11 -3.106 -2.718 -2.201 -1.796 0 1.796 2.201 2.718 3.106
12 -3.055 -2.681 -2.179 -1.782 0 1.782 2.179 2.681 3.055
13 -3.012 -2.650 -2.160 -1.771 0 1.771 2.160 2.650 3.012
14 -2.977 -2.624 -2.145 -1.761 0 1.761 2.145 2.624 2.977
15 -2.947 -2.602 -2.131 -1.753 0 1.753 2.131 2.602 2.947
16 -2.921 -2.583 -2.120 -1.746 0 1.746 2.120 2.583 2.921
17 -2.898 -2.567 -2.110 -1.740 0 1.740 2.110 2.567 2.898
18 -2.878 -2.552 -2.101 -1.734 0 1.734 2.101 2.552 2.878
19 -2.861 -2.539 -2.093 -1.729 0 1.729 2.093 2.539 2.861
20 -2.845 -2.528 -2.086 -1.725 0 1.725 2.086 2.528 2.845
21 -2.831 -2.518 -2.080 -1.721 0 1.721 2.080 2.518 2.831
22 -2.819 -2.508 -2.074 -1.717 0 1.717 2.074 2.508 2.819
23 -2.807 -2.500 -2.069 -1.714 0 1.714 2.069 2.500 2.807
24 -2.797 -2.492 -2.064 -1.711 0 1.711 2.064 2.492 2.797
25 -2.787 -2.485 -2.060 -1.708 0 1.708 2.060 2.485 2.787
26 -2.779 -2.479 -2.056 -1.706 0 1.706 2.056 2.479 2.779
27 -2.771 -2.473 -2.052 -1.703 0 1.703 2.052 2.473 2.771
28 -2.763 -2.467 -2.048 -1.701 0 1.701 2.048 2.467 2.763
29 -2.756 -2.462 -2.045 -1.699 0 1.699 2.045 2.462 2.756
Z -2.576 -2.326 -1.960 -1.645 0 1.645 1.960 2.326 2.576
CI 90% 95% 99%
0.995 0.99 0.975 0.95 0.5 0.05 0.025 0.01 0.005 right tail
0.01 0.02 0.05 0.10 1 0.10 0.05 0.02 0.01 two tail
References

Abraham, B., & Ledolter, J. (2006). Introduction to regression modeling.

Belmont, CA: Thomson Brooks/Cole.
Agresti, A. (2002). Categorical data analysis (2nd ed.). Hoboken, NJ: Wiley.
Agresti, A. (2013). Categorical data analysis (3rd ed.). Hoboken, NJ: Wiley.
Agresti, A. (2015). Foundations of linear and generalized linear models. Hobo-
ken, NJ: Wiley.
Albert, A., & Andersen, J. A. (1984). On the existence of maximum likelihood
estimators in logistic models. Biometrika, 71, 110.
Aldrin, M., Blviken, E., & Schweder, T. (1993). Projection pursuit regression
for moderate non-linearities. Computational Statistics & Data Analysis, 16,
379403.
Allison, P. D. (1999). Multiple regression: A primer. Thousand Oaks, CA:
Pine Forge Press.
Andersen, P. K., & Skovgaard, L. T. (2010). Regression with linear predictors.
New York, NY: Springer.
Anderson, T. W. (1971). The statistical analysis of time series. New York,
NY: Wiley.
Anderson, T. W. (1984). An introduction to multivariate statistical analysis
(2nd ed.). New York, NY: Wiley.
Anderson-Sprecher, R. (1994). Model comparisons and R2 . The American
Statistician, 48, 113117.
Anscombe, F. J. (1961). Examination of residuals. In J. Neyman (Ed.), Pro-
ceedings of the Fourth Berkeley Symposium on Mathematical Statistics and
Probability (pp. 131). Berkeley, CA: University of California Press.
Anscombe, F. J., & Tukey, J. W. (1963). The examination and analysis of
residuals. Technometrics, 5, 141160.
Ashworth, H. (1842). Statistical illustrations of the past and present state of
Lancashire. Journal of the Royal Statistical Society, A, 5, 245256.
Atkinson, A. C. (1985). Plots, transformations, and regression. Oxford:
Clarendon Press.

Springer International Publishing AG 2017 473

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1
474 References

Atkinson, A., & Riani, R. (2000). Robust diagnostic regression analysis.

New York, NY: Springer.
Bartlett, D. P. (1900). General principles of the method of least squares with
applications (2nd ed.). Boston, MA: Massachusetts Institute of Technology.
Beaton, A. E., Martin, M. O., Mullis, I. V. S., Gonzales, E. J., Smith, T. A.,
& Kelly, D. L. (1996). Science Achievement in the Middle School Years:
IEAs Third International Mathematics and Science Study. Chestnut Hill,
MA: TIMSS International Study Center.
Becker, R. A., Chambers, J. M., & Wilks, A. R. (1988). The new S language
a programming environment for data analysis and graphics. Pacic Grove,
CA: Wadsworth and Brooks/Cole.
Belsley, D. A. (1984). Demeaning conditioning diagnostics through centering.
The American Statistician, 38, 7377.
Belsley, D. A., Kuh, E., & Welsch, R. E. (1980). Regression diagnostics: Iden-
tifying inuential data and sources of collinearity. New York, NY: Wiley.
Bennett, C. A., & Franklin, N. L. (1954). Statistical analysis in chemistry
and the chemical industry. New York, NY: Wiley.
Berk, K. N. (1978). Comparing subset regression procedures. Technometrics,
20, 16.
Berk, R. A. (2003). Regression analysis: A constructive critique. Thousand
Oaks, CA: Sage Publications.
Berndt, E. R., & Savin, N. E. (1977). Conict among criteria for testing
hypotheses in the multivariate linear regression model. Econometrika, 45,
12631277.
Bertsimas, D., King, A., & Mazmunder, R. (2016). Best subset selection via
a modern optimization lens. The Annals of Statistics, 44, 813852.
Bickel, P. J., & Doksum, K. A. (1981). An analysis of transformations revis-
ited. Journal of the American Statistical Association, 76, 296311.
Bowerman, B. L., & OConnell, R. T. (2000). Linear statistical models: An
applied approach (2nd ed.). Belmont, CA: Duxbury.
Box, G. E. P. (1979). Robustness in the strategy of scientic model building.
In R. Launer & G. Wilkinson (Eds.), Robustness in statistics (pp. 201235).
New York, NY: Academic Press.
Box, G. E. P. (1984). The importance of practice in the development of
statistics. Technometrics, 26, 18.
Box, G. E. P., & Cox, D. R. (1964). An analysis of transformations. Journal
of the Royal Statistical Society, B, 26, 211246.
Box, G. E. P., & Cox, D. R. (1982). An analysis of transformations revisited,
rebutted. Journal of the American Statistical Association, 77, 209210.
Box, G. E. P, Hunter, J. S., & Hunter, W. G. (2005). Statistics for experi-
menters (2nd ed.). New York, NY: Wiley.
Box, J. F. (1980). R.A. Fisher and the design of experiments 19221926. The
American Statistician, 34, 17.
Brillinger, D. R. (1977). The identication of a particular nonlinear time
series. Biometrika, 64, 509515.
References 475

Brillinger, D. R. (1983). A generalized linear model with Gaussian regres-

sor variables. In P. J. Bickel, K. A. Doksum, & J. L. Hodges (Eds.),
A Festschrift for Erich L. Lehmann (pp. 97114). Pacic Grove, CA:
Wadsworth.
Brooks, D. G., Carroll, S. S., & Verdini, W. A. (1988). Characterizing the
domain of a regression model. The American Statistician, 42, 187190.
Brown, M. B., & Forsythe, A. B. (1974a). The ANOVA and multiple com-
parisons for data with heterogeneous variances. Biometrics, 30, 719724.
Brown, M. B., & Forsythe, A. B. (1974b). The small sample behavior of
some statistics which test the equality of several means. Technometrics,
16, 129132.
Brownlee, K. A. (1965). Statistical theory and methodology in science and
engineering. New York, NY: Wiley.
Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel
inference: A practical information-theoretic approach (2nd ed.). New York,
NY: Springer.
Burnham, K. P., & Anderson, D. R. (2004). Multimodel inference understand-
ing AIC and BIC in model selection. Sociological Methods & Research, 33,
261304.
Buxton, L. H. D. (1920). The anthropology of Cyprus. The Journal of the
Royal Anthropological Institute of Great Britain and Ireland, 50, 183235.
Cambanis, S., Huang, S., & Simons, G. (1981). On the theory of elliptically
contoured distributions. Journal of Multivariate Analysis, 11, 368385.
Cameron, A. C., & Trivedi, P. K. (1998). Regression analysis of count data
(1st ed.). Cambridge, UK: Cambridge University Press.
Cameron, A. C., & Trivedi, P. K. (2013). Regression analysis of count data
(2nd ed.). Cambridge, UK: Cambridge University Press.
Chambers, J. M. (2008). Software for data analysis: Programming with R.
New York, NY: Springer.
Chambers, J. M., Cleveland, W. S., Kleiner, B., & Tukey, P. (1983). Graphical
methods for data analysis. Boston, MA: Duxbury Press.
Chambers, J. M., & Hastie, T. J. (Eds.). (1993). Statistical models in S. New
York, NY: Chapman & Hall.
Chang, J. (2006). Resistant Dimension Reduction. Ph.D. Thesis, South-
ern Illinois University, online at http://lagrange.math.siu.edu/Olive/
sjingth.pdf
Chang, J., & Olive, D. J. (2007). Resistant dimension reduction. Pre-print,
see http://lagrange.math.siu.edu/Olive/preprints.htm
Chang, J., & Olive, D. J. (2010). OLS for 1D regression models. Communi-
cations in statistics: Theory and methods, 39, 18691882.
Chatterjee, S., & Hadi, A. S. (1988). Sensitivity analysis in linear regression.
New York, NY: Wiley.
Chatterjee, S., & Hadi, A. S. (2012). Regression analysis by example (5th
ed.). Hoboken, NJ: Wiley.
476 References

Chen, A., Bengtsson, T., & Ho, T. K. (2009). A regression paradox for lin-
ear models: Sucient conditions and relation to Simpsons paradox. The
American Statistician, 63, 218225.
Chen, C. H., & Li, K. C. (1998). Can SIR be as popular as multiple linear
regression? Statistica Sinica, 8, 289316.
Chihara, L., & Hesterberg, T. (2011). Mathematical statistics with resampling
and R. Hoboken, NJ: Wiley.
Chmielewski, M. A. (1981). Elliptically symmetric distributions: A review
and bibliography. International Statistical Review, 49, 6774.
Christensen, R. (2013). Plane answers to complex questions: The theory of
linear models (4th ed.). New York, NY: Springer.
Christmann, A., & Rousseeuw, P. J. (2001). Measuring overlap in binary
regression. Computational Statistics & Data Analysis, 37, 6575.
Claeskins, G., & Hjort, N. L. (2003). The focused information criterion (with
discussion). Journal of the American Statistical Association, 98, 900916.
Claeskens, G., & Hjort, N. L. (2008). Model selection and model averaging.
New York, NY: Cambridge University Press.
Cleveland, W. (1979). Robust locally weighted regression and smoothing scat-
terplots. Journal of the American Statistical Association, 74, 829836.
Cobb, G. W. (1998). Introduction to design and analysis of experiments.
Emeryville, CA: Key College Publishing.
Cody, R. P., & Smith, J. K. (2006). Applied statistics and the SAS pro-
gramming language (5th ed.). Upper Saddle River, NJ: Pearson Prentice
Hall.
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple
regression/correlation analysis for the behavioral sciences (3rd ed.). Mah-
wah, NJ: Lea, Inc.
Collett, D. (1999). Modelling binary data (1st ed.). Boca Raton, FL: Chap-
man & Hall/CRC.
Collett, D. (2003). Modelling binary data (2nd ed.). Boca Raton, FL: Chap-
man & Hall/CRC.
Cook, R. D. (1977). Deletion of inuential observations in linear regression.
Technometrics,19, 1518.
Cook, R. D. (1993). Exploring partial residual plots. Technometrics, 35, 351
362.
Cook, R. D. (1998). Regression graphics: Ideas for studying regression through
graphics. New York, NY: Wiley.
Cook, R. D., Helland, I. S., & Su, Z. (2013). Envelopes and partial least
squares regression. Journal of the Royal Statistical Society, B, 75, 851877.
Cook, R. D., & Nachtsheim, C. J. (1994). Reweighting to achieve elliptically
contoured covariates in regression. Journal of the American Statistical As-
sociation, 89, 592599.
Cook, R. D., & Olive, D. J. (2001). A note on visualizing response transfor-
mations in regression. Technometrics, 43, 443449.
References 477

Cook, R. D., & Setodji, C. M. (2003). A model-free test for reduced rank
in multivariate regression. Journal of the American Statistical Association,
98, 340351.
Cook, R. D., & Su, Z. (2013). Scaled envelopes: Scale-invariant and ecient
estimation in multivariate linear regression. Biometrika, 100, 929954.
Cook, R. D., & Weisberg, S. (1982). Residuals and inuence in regression.
London: Chapman & Hall.
Cook, R. D., & Weisberg, S. (1994). Transforming a response variable for
linearity. Biometrika, 81, 731737.
Cook, R. D., & Weisberg, S. (1997). Graphics for assessing the adequacy
of regression models. Journal of the American Statistical Association, 92,
490499.
Cook, R. D., & Weisberg, S. (1999a). Applied regression including computing
and graphics. New York, NY: Wiley.
Cook, R. D., & Weisberg, S. (1999b). Graphs in statistical analysis: Is the
medium the message? The American Statistician, 53, 2937.
Cook, R. D., & Zhang, X. (2015). Foundations of envelope models and meth-
ods. Journal of the American Statistical Association, 110, 599611.
Copas, J. B. (1983). Regression prediction and shrinkage (with discussion).
Journal of the Royal Statistical Society, B, 45, 311354.
Cramer, H. (1946). Mathematical Methods of Statistics. Princeton, NJ:
Princeton University Press.
Crawley, M. J. (2005). Statistics: An introduction using R. Hoboken, NJ:
Wiley.
Crawley, M. J. (2013). The R book (2nd ed.). Hoboken, NJ: Wiley.
Croux, C., Dehon, C., Rousseeuw, P. J., & Van Aelst, S. (2001). Robust
estimation of the conditional median function at elliptical models. Statistics
& Probability Letters, 51, 361368.
Daniel, C., & Wood, F. S. (1980). Fitting equations to data (2nd ed.). New
York, NY: Wiley.
Darlington, R. B. (1969). Deriving least-squares weights without calculus.
The American Statistician, 23, 4142.
Datta, B. N. (1995). Numerical linear algebra and applications. Pacic Grove,
CA: Brooks/Cole Publishing Company.
David, H. A. (1995). First (?) occurrences of common terms in mathematical
statistics. The American Statistician, 49, 121133.
David, H. A. (20062007). First (?) occurrences of common terms in statistics
and probability. Publications and Preprint Series, Iowa State University,
www.stat.iastate.edu/preprint/hadavid.html
Dean, A. M., & Voss, D. (2000). Design and analysis of experiments. New
York, NY: Springer.
Dongarra, J. J., Moler, C. B., Bunch, J. R., & Stewart, G. W. (1979). Lin-
packs users guide. Philadelphia, PA: SIAM.
Draper, N. R. (2002). Applied regression analysis bibliography update 2000
2001. Communications in Statistics: Theory and Methods, 31, 20512075.
478 References

Draper, N. R., & Smith, H. (1966). Applied regression analysis (1st ed.).
New York, NY: Wiley.
Draper, N. R., & Smith, H. (1981). Applied regression analysis (2nd ed.).
New York, NY: Wiley.
Draper, N. R., & Smith, H. (1998). Applied regression analysis (3rd ed.). New
York, NY: Wiley.
Driscoll, M. F., & Krasnicka, B. (1995). An accessible proof of Craigs theorem
in the general case. The American Statistician, 49, 5962.
Dunn, O. J., & Clark, V. A. (1974). Applied statistics: Analysis of variance
and regression. New York, NY: Wiley.
Eaton, M. L. (1986). A characterization of spherical distributions. Journal of
Multivariate Analysis, 20, 272276.
Efron, B. (1982). The jackknife, the bootstrap and other resampling plans.
Philadelphia, PA: SIAM.
Efron, B. (2014), Estimation and accuracy after model selection (with dis-
cussion). Journal of the American Statistical Association, 109, 9911007.
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004). Least angle
regression (with discussion). The Annals of Statistics, 32, 407451.
Ernst, M. D. (2009). Teaching inference for randomized experiments. Journal
of Statistical Education, 17 (online).
Ezekiel, M. (1930). Methods of correlation analysis. New York, NY: Wiley.
Ezekiel, M., & Fox, K. A. (1959). Methods of correlation and regression anal-
ysis. New York, NY: Wiley.
Fabian, V. (1991). On the problem of interactions in the analysis of variance
(with discussion). Journal of the American Statistical Association, 86, 362
375.
Fahrmeir, L., & Tutz, G. (2001). Multivariate statistical modelling based on
generalized linear models (2nd ed.). New York, NY: Springer.
Ferrari, D., & Yang, Y. (2015). Condence sets for model selection by
F testing. Statistica Sinica, 25, 16371658.
Fox, J. (1991). Regression diagnostics. Newbury Park, CA: Sage Publications.
Fox, J. (2015). Applied regression analysis and generalized linear models (3rd
ed.). Thousand Oaks, CA: Sage Publications.
Fox, J., & Weisberg, S. (2010). An R companion to applied regression. Thou-
sand Oaks, CA: Sage Publications.
Freedman, D. A. (1983). A note on screening regression equations. The Amer-
ican Statistician, 37, 152155.
Freedman, D. A. (2005). Statistical models theory and practice. New York,
NY: Cambridge University Press.
Frey, J. (2013). Data-driven nonparametric prediction intervals. Journal of
Statistical Planning and Inference, 143, 10391048.
Fujikoshi, Y., Sakurai, T., & Yanagihara, H. (2014). Consistency of high-
dimensional AIC-type and Cp -type criteria in multivariate linear regres-
sion. Journal of Multivariate Analysis, 123, 184200.
References 479

Furnival, G., & Wilson, R. (1974). Regression by leaps and bounds. Techno-
metrics, 16, 499511.
Gail, M. H. (1996). Statistics in action. Journal of the American Statistical
Association, 91, 113.
Gelman, A. (2005). Analysis of varianceWhy it is more important than
ever (with discussion). The Annals of Statistics, 33, 153.
Ghosh, S. (1987), Note on a common error in regression diagnostics using
residual plots. The American Statistician, 41, 338.
Gilmour, S. G. (1996). The interpretation of Mallowss Cp -statistic. The
Statistician, 45, 4956.
Gladstone, R. J. (1905). A study of the relations of the brain to the size of
the head. Biometrika, 4, 105123.
Golub, G. H., & Van Loan, C. F. (1989). Matrix computations (2nd ed.).
Baltimore, MD: John Hopkins University Press.
Graybill, F. A. (1976). Theory and application of the linear model. North
Scituate, MA: Duxbury Press.
Gunst, R. F., & Mason, R. L. (1980). Regression analysis and its application:
A data oriented approach. New York, NY: Marcel Dekker.
Guttman, I. (1982). Linear models: An introduction. New York, NY: Wiley.
Haenggi, J. C. (2009). Plots for the Design and Analysis of Experiments. Mas-
ters Research Paper, Southern Illinois University, at http://lagrange.
math.siu.edu/Olive/sjenna.pdf
Haggstrom, G. W. (1983). Logistic regression and discriminant analysis by
ordinary least squares. Journal of Business & Economic Statistics, 1, 229
238.
Hahn, G. J. (1982). Design of experiments: An annotated bibliography. In S.
Kotz & N. L. Johnson (Eds.), Encyclopedia of statistical sciences (Vol. 2,
pp. 359366). New York, NY: Wiley.
Hamilton, L. C. (1992). Regression with graphics: A second course in applied
statistics. Belmont, CA: Wadsworth.
Harrell, F. E. (2015). Regression modeling strategies with applications to lin-
ear models, logistic and ordinal regression, and survival analysis (2nd ed.).
New York, NY: Springer.
Harrison, D., & Rubinfeld, D. L. (1978). Hedonic prices and the demand
for clean air. Journal of Environmental Economics and Management, 5,
81102.
Harter, H. L. (1974a). The method of least squares and some alternatives.
Part I. International Statistical Review, 42, 147174.
Harter, H. L. (1974b). The method of least squares and some alternatives.
Part II. International Statistical Review, 42, 235165.
Harter, H. L. (1975a). The method of least squares and some alternatives.
Part III. International Statistical Review, 43, 144.
Harter, H. L. (1975b). The method of least squares and some alternatives.
Part IV. International Statistical Review, 43, 125190, 273278.
480 References

Harter, H. L. (1975c). The method of least squares and some alternatives.

Part V. International Statistical Review, 43, 269272.
Harter, H. L. (1976). The method of least squares and some alternatives. Part
VI. International Statistical Review, 44, 113159.
Hastie, T. (1987). A closer look at the deviance. The American Statistician,
41, 1620.
Hastie, T. J., & Tibshirani, R. J. (1990). Generalized additive models. London,
UK: Chapman & Hall.
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical
learning: Data mining, inference and prediction (2nd ed.). New York, NY:
Springer.
Hastie, T., Tibshirani, R., & Wainwright, M. (2015). Statistical learning with
sparsity: The lasso and generalizations. Boca Raton, FL: CRC Press Taylor
& Francis.
Hebbler, B. (1847). Statistics of Prussia. Journal of the Royal Statistical So-
ciety, A, 10, 154186.
Henderson, H. V., & Searle, S. R. (1979). Vec and Vech operators for matrices,
with some uses in Jacobians and multivariate statistics. The Canadian
Journal of Statistics, 7, 6581.
Hilbe, J. M. (2011). Negative binomial regression (2nd ed.). Cambridge, UK:
Cambridge University Press.
Hillis, S. L., & Davis, C. S. (1994). A simple justication of the iterative
tting procedure for generalized linear models. The American Statistician,
48, 288289.
Hinkley, D. V., & Runger, G. (1984). The analysis of transformed data (with
discussion). Journal of the American Statistical Association, 79, 302320.
Hjort, N. L., & Claeskins, G. (2003). Frequentist model average estimators.
Journal of the American Statistical Association, 98, 879899.
Hoaglin, D. C., Mosteller, F., & Tukey, J. W. (Eds.). (1991). Fundamentals
of exploratory analysis of variance. New York, NY: Wiley.
Hoaglin, D. C., & Welsh, R. (1978). The hat matrix in regression and
ANOVA. The American Statistician, 32, 1722.
Hocking, R. R. (2013). Methods and applications of linear models: Regression
and the analysis of variance (3rd ed.). New York, NY: Wiley.
Hoeding, W. (1952). The large sample power of tests based on permutations
of observations. The Annals of Mathematical Statistics, 23, 169192.
Hogg, R. V., Tanis, E. A., & Zimmerman, D. (2014). Probability and statistical
inference (9th ed.). Boston, MA: Pearson.
Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.).
New York, NY: Wiley.
Houseman, E. A., Ryan, L. M., & Coull, B. A. (2004). Cholesky residuals for
assessing normal errors in a linear model with correlated errors. Journal of
the American Statistical Association, 99, 383394.
Hunter, J. S. (1989). Lets all beware the latin square. Quality Engineering,
1, 453465.
References 481

Hunter, W. G. (1977). Some ideas about teaching design of experiments,

with 25 -examples of experiments conducted by students. The American
Statistician, 31, 1217.
Hurvich, C. M., & Tsai, C. L. (1990). The impact of model selection on
inference in linear regression. The American Statistician, 44, 214217.
Hyndman, R. J. (1996). Computing and graphing highest density regions.
The American Statistician, 50, 120126.
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction
to statistical learning with applications in R. New York, NY: Springer.
Joglekar, G., Schuenemeyer, J. H., & LaRiccia, V. (1989). Lack-of-t testing
when replicates are not available. The American Statistician, 43, 135143.
Johnson, M. E. (1987). Multivariate statistical simulation. New York, NY:
Wiley.
Johnson, M. P., & Raven, P. H. (1973). Species number and endemism: The
Galapagos Archipelago revisited. Science, 179, 893895.
Johnson, N. L., & Kotz, S. (1970). Distributions in statistics: Continuous
univariate distributions2. New York, NY: Wiley.
Johnson, R. A., & Wichern, D. W. (1988). Applied multivariate statistical
analysis (2nd ed.). Englewood Clis, NJ: Prentice Hall.
Johnson, R. A., & Wichern, D. W. (2007). Applied multivariate statistical
analysis (6th ed.). Englewood Clis, NJ: Prentice Hall.
Johnson, R. W. (1996). Fitting percentage of body fat to simple body mea-
surements. Journal of Statistics Education, 4, online at www.amstat.org/
publications/jse/
Johnson, W. W. (1892). The theory of errors and method of least squares
New York, NY: Wiley.
Jones, H. L. (1946). Linear regression functions with neglected variables.
Journal of the American Statistical Association, 41, 356369.
Kachigan, S. K. (1982). Multivariate statistical analysis: A conceptual intro-
duction (1st ed.). New York, NY: Radius Press.
Kachigan, S. K. (1991). Multivariate statistical analysis: A conceptual intro-
duction (2nd ed.). New York, NY: Radius Press.
Kakizawa, Y. (2009). Third-order power comparisons for a class of tests for
multivariate linear hypothesis under general distributions. Journal of Mul-
tivariate Analysis, 100, 473496.
Kariya, T., & Kurata, H. (2004). Generalized least squares. New York, NY:
Wiley.
Kay, R., & Little, S. (1987). Transformations of the explanatory variables in
the logistic regression model for binary data. Biometrika, 74, 495501.
Kelker, D. (1970). Distribution theory of spherical distributions and a loca-
tion scale parameter generalization. Sankhya, A, 32, 419430.
Kenard, R. W. (1971). A note on the Cp statistics. Technometrics, 13,
899900.
Khattree, R., & Naik, D. N. (1999). Applied multivariate statistics with SAS
software (2nd ed.). Cary, NC: SAS Institute.
482 References

Kirk, R. E. (1982). Experimental design: Procedures for the behavioral

sciences (2nd ed.). Belmont, CA: Brooks/Cole Publishing Company.
Kirk, R. E. (2012). Experimental design: Procedures for the behavioral
sciences (4th ed.). Thousand Oaks, CA: Sage Publications.
Kleinbaum, D. G., Kupper, L. L., Nizam, A., & Rosenberg, E. S. (2014). Ap-
plied regression analysis and other multivariable methods (5th ed.). Boston,
MA: Cengage Learning.
Kshirsagar, A. M. (1972). Multivariate analysis New York, NY: Marcel
Dekker.
Kuehl, R. O. (1994). Statistical principles of research design and analysis.
Belmont, CA: Duxbury.
Kutner, M. H., Nachtsheim, C. J., Neter, J., & Li, W. (2005). Applied linear
statistical models (5th ed.). Boston, MA: McGraw-Hill/Irwin.
Kvalseth, T. O. (1985). Cautionary note about R2 . The American Statisti-
cian, 39, 279285.
Ledolter, J., & Swersey, A. J. (2007). Testing 1-2-3 experimental design with
applications in marketing and service operations. Stanford, CA: Stanford
University Press.
Leeb, H., & Potscher, B. M. (2006). Can one estimate the conditional dis-
tribution of post-model-selection estimators? The Annals of Statistics, 34,
25542591.
Leger, C., & Altman, N. (1993). Assessing inuence in variable selection
problems. Journal of the American Statistical Association, 88, 547556.
Lehmann, E. L. (2011). Fisher, Neyman, and the creation of classical statis-
tics. New York, NY: Springer.
Lei, J., Robins, J., & Wasserman, L. (2013). Distribution free prediction sets.
Journal of the American Statistical Association, 108, 278287.
Lei, J., & Wasserman, L. (2014). Distribution free prediction bands. Journal
of the Royal Statistical Society, B, 76, 7196.
Leland, O. M. (1921). Practical least squares. New York, NY: McGraw Hill.
Lesno, M., & Lancelot, R. (2010). aod: Analysis of overdispersed data.
R package version 1.2, http://cran.r-project.org/package=aod
Li, K. C., & Duan, N. (1989). Regression analysis under link violation. The
Annals of Statistics, 17, 10091052.
Lindsey, J. K. (2004). Introduction to applied statistics: A modelling approach
(2nd ed.). Oxford, UK: Oxford University Press.
Linhart, H., & Zucchini, W. (1986). Model selection. New York, NY: Wiley.
Long, J. S., & Ervin, L. H. (2000). Using heteroscedasticity consistent stan-
dard errors in the linear regression model. The American Statistician, 54,
217224.
Mallows, C. (1973). Some comments on Cp . Technometrics, 15, 661676.
Marden, J. I. (2012). Mathematical statistics: Old school, unpublished text
online at www.stat.istics.net
References 483

Marden, J. I., & Muyot, E. T. (1995). Rank tests for main and interaction
eects in analysis of variance. Journal of the American Statistical Associ-
ation, 90, 13881398.
Mardia, K. V., Kent, J. T., & Bibby, J. M. (1979). Multivariate analysis.
London, UK: Academic Press.
Maronna, R. A., & Zamar, R. H. (2002). Robust estimates of location and
dispersion for high-dimensional datasets. Technometrics, 44, 307317.
MathSoft (1999a). S-Plus 2000 users guide. Seattle, WA: Data Analysis
Products Division, MathSoft.
MathSoft (1999b). S-Plus 2000 guide to statistics (Vol. 2). Seattle, WA: Data
Analysis Products Division, MathSoft.
Maxwell, S. E., & Delaney, H. D. (2003). Designing experiments and analyzing
data (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.
McCullagh, P., & Nelder, J. A. (1989). Generalized linear models (2nd ed.).
London, UK: Chapman & Hall.
McDonald, G. C., & Schwing, R. C. (1973). Instabilities of regression esti-
mates relating air pollution to mortality. Technometrics, 15, 463482.
McKenzie, J. D., & Goldman, R. (1999). The student edition of MINITAB.
Reading, MA: Addison Wesley Longman.
Mendenhall, W., & Sincich, T. L. (2011). A second course in statistics:
Regression analysis (7th ed.). Boston, MA: Pearson.
Merriman, M. (1907). A text book on the method of least squares (8th ed.).
New York, NY: Wiley.
Mickey, R. M., Dunn, O. J., & Clark, V. A. (2004). Applied statistics: Analysis
of variance and regression (3rd ed.). Hoboken, NJ: Wiley.
Miller, D. M. (1984). Reducing transformation bias in curve tting. The
American Statistician, 38, 124126.
Montgomery, D. C. (1984). Design and analysis of experiments (2nd ed.).
New York, NY: Wiley.
Montgomery, D. C. (2012). Design and analysis of experiments (8th ed.).
New York, NY: Wiley.
Montgomery, D. C., Peck, E. A., & Vining, G. (2001). Introduction to linear
regression analysis (3rd ed.). Hoboken, NJ: Wiley.
Montgomery, D. C., Peck, E. A., & Vining, G. (2012). Introduction to linear
regression analysis (5th ed.). Hoboken, NJ: Wiley.
Moore, D. S. (2000). The basic practice of statistics (2nd ed.). New York,
NY: W.H. Freeman.
Moore, D. S. (2007). The basic practice of statistics (4th ed.). New York, NY:
W.H. Freeman.
Mosteller, F., & Tukey, J. W. (1977). Data analysis and regression. Reading,
MA: Addison-Wesley.
Myers, R. H., & Milton, J. S. (1990). A rst course in the theory of linear
statistical models. Belmont, CA: Duxbury.
484 References

Myers, R. H., Montgomery, D. C., & Vining, G. G. (2002). Generalized linear

models with applications in engineering and the sciences. New York, NY:
Wiley.
Nelder, J. A., & Wedderburn, R. W. M. (1972). Generalized linear models.
Journal of the Royal Statistical Society, A, 135, 370380.
Nishi, R. (1984). Asymptotic properties of criteria for selection of variables
in multiple regression. The Annals of Statistics, 12, 758765.
Norman, G. R., & Streiner, D. L. (1986). PDQ statistics. Philadelphia, PA:
B.C. Decker.
Oehlert, G. W. (2000). A rst course in design and analysis of experiments.
New York, NY: W.H. Freeman.
Olive, D. J. (2002). Applications of robust distances for regression. Techno-
metrics, 44, 6471.
Olive, D. J. (2004a). A resistant estimator of multivariate location and dis-
persion. Computational Statistics & Data Analysis, 46, 99102.
Olive, D. J. (2004b). Visualizing 1D regression. In M. Hubert, G. Pison, A.
Struyf, & S. Van Aelst (Eds.), Theory and applications of recent robust
methods (pp. 221233). Basel, Switzerland: Birkhauser.
Olive, D. J. (2005). Two simple resistant regression estimators. Computa-
tional Statistics & Data Analysis, 49, 809819.
Olive, D. J. (2007). Prediction intervals for regression. Computational Statis-
tics & Data Analysis, 51, 31153122.
Olive, D. J. (2008), Applied robust statistics, online course notes, see http://
lagrange.math.siu.edu/Olive/ol-bookp.htm
Olive, D. J. (2010). Multiple linear and 1D regression models, online course
notes, see http://lagrange.math.siu.edu/Olive/regbk.htm
Olive, D. J. (2013a), Asymptotically optimal regression prediction inter-
vals and prediction regions for multivariate data. International Journal
of Statistics and Probability, 2, 90100.
Olive, D. J. (2013b). Plots for generalized additive models. Communications
in Statistics: Theory and Methods, 42, 26102628.
Olive, D. J. (2014). Statistical theory and inference. New York, NY: Springer.
Olive, D. J. (2016a). Bootstrapping hypothesis tests and condence regions,
preprint, see http://lagrange.math.siu.edu/Olive/ppvselboot.pdf
Olive, D. J. (2016b). Applications of hyperellipsoidal prediction regions. Sta-
tistical Papers, to appear.
Olive, D. J. (2016c). Robust multivariate analysis. New York, NY: Springer,
to appear.
Olive, D. J. (2017). Prediction and statistical learning, online course notes,
see http://lagrange.math.siu.edu/Olive/slearnbk.htm
Olive, D. J., & Hawkins, D. M. (2005). Variable selection for 1D regression
models. Technometrics, 47, 4350.
Olive, D. J., & Hawkins, D.M. (2010). Robust multivariate location and dis-
persion, preprint at http://lagrange.math.siu.edu/Olive/pphbmld.
pdf
References 485

Olive, D. J., & Hawkins, D. M. (2011). Practical high breakdown regression,

preprint at http://lagrange.math.siu.edu/Olive/pphbreg.pdf
Olive, D. J., Pelawa Watagoda, L. C. R., & Rupasinghe Arachchige Don, H.
S. (2015). Visualizing and testing the multivariate linear regression model.
International Journal of Statistics and Probability, 4, 126137.
Pardoe, I. (2012). Applied regression modeling: A business approach, 2nd ed.
Hoboken, NJ: Wiley.
Pelawa Watagoda, L. C. R. (2013). Plots and Testing for Multivariate Lin-
ear Regression. Masters Research Paper, Southern Illinois University, at
http://lagrange.math.siu.edu/Olive/slasanthi.pdf
Pelawa Watagoda, L. C. R. (2017). Inference After Variable Selection. Ph.D.
Thesis, Southern Illinois University, online at http://lagrange.math.
siu.edu/Olive/slasanthiphd.pdf
Pelawa Watagoda, L. C. R., & Olive, D. J. (2017). Inference after variable
selection, preprint at http://lagrange.math.siu.edu/Olive/ppvsinf.
pdf
Pena, E. A., & Slate, E. H. (2006). Global validation of linear model assump-
tions. Journal of the American Statistical Association, 101, 341354.
R Core Team (2016). R: A language and environment for statistical com-
puting. Vienna, Austria: R Foundation for Statistical Computing, www.
R-project.org
Rao, C. R. (1965). Linear statistical inference and its applications (1st ed.).
New York, NY: Wiley.
Rao, C. R. (1973). Linear statistical inference and its applications (2nd ed.).
New York, NY: Wiley.
Ravishanker, N., & Dey, D. K. (2002). A rst course in linear model theory.
Boca Raton, FL: Chapman & Hall/CRC.
Reid, J. G., & Driscoll, M. F. (1988). An accessible proof of Craigs theorem
in the noncentral case. The American Statistician, 42, 139142.
Rencher, A., & Pun, F. (1980). Ination of R2 in best subset regression.
Technometrics, 22, 4953.
Rencher, A. C., & Schaalje, G. B. (2008). Linear models in statistics (2nd
ed.). Hoboken, NJ: Wiley.
Rice, J. (2006). Mathematical statistics and data analysis (3rd ed.). Belmont,
CA: Duxbury.
Robinson, J. (1973). The large sample power of permutation tests for ran-
domization models. The Annals of Statistics, 1, 291296.
Robinson, T. J., Brenneman, W. A., & Myers, W. R. (2009). An intuitive
graphical approach to understanding the split-plot experiment. Journal of
Statistical Education, 17 (online).
Rohatgi, V. K. (1976). An introduction to probability theory and mathematical
statistics. New York, NY: Wiley.
Rounceeld, M. (1995). The statistics of poverty and inequality. Journal of
Statistics and Education, 3, online www.amstat.org/publications/jse/
486 References

Rousseeuw, P. J., & Leroy, A. M. (1987). Robust regression and outlier de-
tection. New York, NY: Wiley.
Rousseeuw, P. J., & Van Driessen, K. (1999). A fast algorithm for the mini-
mum covariance determinant estimator. Technometrics, 41, 212223.
Ryan, T. (2009), Modern regression methods (2nd ed.). Hoboken, NJ: Wiley.
Sadooghi-Alvandi, S. M. (1990). Simultaneous prediction intervals for re-
gression models with intercept. Communications in Statistics: Theory and
Methods, 19, 14331441.
Sall, J. (1990). Leverage plots for general linear hypotheses. The American
Statistician, 44, 308315.
Santer, T. J., & Duy, D. E. (1986). A note on A. Alberts and J. A. An-
dersons conditions for the existence of maximum likelihood estimates in
logistic regression models. Biometrika, 73, 755758.
SAS Institute (1985). SAS users guide: Statistics. Version 5. Cary, NC: SAS
Institute.
Schaahausen, H. (1878). Die Anthropologische Sammlung Des Anatom
ischen Der Universitat Bonn. Archiv fur Anthropologie, 10, 165. Appendix.
Schee, H. (1959). The analysis of variance. New York, NY: Wiley.
Schoemoyer, R. L. (1992). Asymptotically valid prediction intervals for linear
models. Technometrics, 34, 399408.
Searle, S. R. (1971). Linear models. New York, NY: Wiley.
Searle, S. R. (1982). Matrix algebra useful for statistics. New York, NY: Wiley.
Searle, S. R. (1988). Parallel lines in residual plots. The American Statisti-
cian, 42, 211.
Seber, G. A. F., & Lee, A. J. (2003). Linear regression analysis (2nd ed.).
New York, NY: Wiley.
Selvin, H. C., & Stuart, A. (1966). Data-dredging procedures in survey anal-
ysis. The American Statistician, 20 (3), 2023.
Sen, P. K., & Singer, J. M. (1993). Large sample methods in statistics: An
introduction with applications. New York, NY: Chapman & Hall.
Severini, T. A. (1998). Some properties of inferences in misspecied linear
models. Statistics & Probability Letters, 40, 149153.
Sheather, S. J. (2009). A modern approach to regression with R. New York,
NY: Springer.
Shi, L., & Chen, G. (2009). Inuence measures for general linear models with
correlated errors. The American Statistician, 63, 4042.
Simono, J. S. (2003). Analyzing categorical data. New York, NY: Springer.
Snedecor, G. W., & Cochran, W. G. (1967). Statistical methods (6th ed.).
Ames, IA: Iowa State College Press.
Steinberg, D. M., & Hunter, W. G. (1984). Experimental design: Review and
comment. Technometrics, 26, 7197.
Stigler, S. M. (1986). The history of statistics: The measurement of uncer-
tainty before 1900. Cambridge, MA: Harvard University Press.
Su, Z., & Cook, R. D. (2012). Inner envelopes: Ecient estimation in multi-
variate linear regression. Biometrika, 99, 687702.
References 487

Su, Z., & Yang, S.-S. (2006). A note on lack-of-t tests for linear models
without replication. Journal of the American Statistical Association, 101,
205210.
Tremearne, A. J. N. (1911). Notes on some Nigerian tribal marks. Journal
of the Royal Anthropological Institute of Great Britain and Ireland, 41,
162178.
Tukey, J. W. (1957). Comparative anatomy of transformations. Annals of
Mathematical Statistics, 28, 602632.
Tukey, J. W. (1977). Exploratory data analysis. Reading, MA: Addison-
Wesley Publishing Company.
Velilla, S. (1993). A note on the multivariate Box-Cox transformation to
normality. Statistics & Probability Letters, 17, 259263.
Velleman, P. F., & Welsch, R. E. (1981). Ecient computing of regression
diagnostics. The American Statistician, 35, 234242.
Venables, W. N., & Ripley, B. D. (2010). Modern applied statistics with S
(4th ed.). New York, NY: Springer.
Vittingho, E., Glidden, D. V., Shiboski, S. C., & McCulloch, C. E. (2012).
Regression methods in biostatistics: Linear, logistic, survival, and repeated
measures models (2nd ed.). New York, NY: Springer.
Wackerly, D. D., Mendenhall, W., & Scheaer, R. L. (2008). Mathematical
statistics with applications (7th ed.). Belmont, CA: Thomson Brooks/Cole.
Walls, R. C., & Weeks, D. L. (1969). A note on the variance of a predicted
response in regression. The American Statistician, 23, 2426.
Wang, L., Liu, X., Liang, H., & Carroll, R. J. (2011). Estimation and vari-
able selection for generalized additive partial linear models. The Annals of
Statistics, 39, 18271851.
Weisberg, S. (2002). Dimension reduction regression in R. Journal of Statis-
tical Software, 7, webpage www.jstatsoft.org
Weisberg, S. (2014). Applied linear regression (4th ed.). Hoboken, NJ: Wiley.
Welch, B. L. (1947). The generalization of Students problem when several
dierent population variances are involved. Biometrika, 34, 2835.
Welch, B. L. (1951). On the comparison of several mean values: An alternative
approach. Biometrika, 38, 330336.
Welch, W. J. (1990). Construction of permutation tests. Journal of the Amer-
ican Statistical Association, 85, 693698.
Weld, L. D. (1916). Theory of errors and least squares. New York, NY:
Macmillan.
Wilcox, R. R. (2012). Introduction to robust estimation and hypothesis testing
(3rd ed.). New York, NY: Academic Press, Elsevier.
Winkelmann, R. (2000). Econometric analysis of count data (3rd ed.).
New York, NY: Springer.
Winkelmann, R. (2008). Econometric analysis of count data (5th ed.).
New York, NY: Springer.
Wood, S. N. (2006). Generalized additive models: An introduction with R.
Boca Rotan, FL: Chapman & Hall/CRC.
488 References

Wright, T. W. (1884). A treatise on the adjustment of observations, with

applications to geodetic work and other measures of precision. New York,
NY: Van Nostrand.
Yeo, I. K., & Johnson, R. (2000). A new family of power transformations to
improve normality or symmetry. Biometrika, 87, 954959.
Zhang, J., Olive, D. J., & Ye, P. (2012). Robust covariance matrix estimation
with canonical correlation analysis. International Journal of Statistics and
Probability, 1, 119136.
Zuur, A. F., Ieno, E. N., Walker, N. J., Saveliev, A. A., & Smith, G. M.
(2009). Mixed eects models and extensions in ecology with R. New York,
NY: Springer.
Index

1D regression, 2, 144, 338 Bickel, 142

1D regression model, 389, 441 binary regression, 394
binary regression model, 390
Abraham, vi, 402 binomial regression, 394
accelerated failure time models, 442 binomial regression model, 390
added variable plot, 50 bivariate normal, 302
additive error regression, 3 block, 227
additive error regression model, 390 BLUE, 328
additive error single index model, 393 Bowerman, vi
additive predictor, 2 Box, 2, 4, 95, 141, 142, 177, 188, 198200,
Agresti, vii, 14, 390, 391, 407, 408, 440 208, 209, 224, 230, 241, 243, 244, 250,
AIC, 144 256, 275, 279282, 294, 297
Albert, 441 BoxCox transformation, 95
Aldrin, 15 Brillinger, 15
Allison, vi Brooks, 65
Altman, 144 Brown, 200, 201
Andersen, 440, 441 Brownlee, v
Anderson, 143, 144, 340, 361, 421, 441 bulging rule, 88
Anderson-Sprecher, 65 Burnham, 143, 144, 421, 441
ANOVA, 175 Buxton, 41, 81, 117, 132, 135, 137, 312
ANOVA model, 3
Anscombe, 65 Cambanis, 309
ARC, 210 Cameron, 438, 440, 441
Arc, 68 carriers, 4
Ashworth, 117, 147, 159 case, 4, 18
Atkinson, 145, 396 central limit theorem, 28
ceres plots, 141, 393
Bartlett, v Chambers, 65, 129, 285, 296
Beaton, 15 Chang, 15, 33, 142, 338, 394
Becker, 68, 460 Chatterjee, v, 143, 145
Belsley, 144, 145 Chen, 21, 142, 172, 339
Bennett, v Chihara, vi, 28
Berk, vi, 383 Chmielewski, 309
Berndt, 361, 374 Cholesky decomposition, 166
Bertsimas, 146 Cholesky residuals, 172
best linear unbiased estimator, 328 Christensen, 338
betabinomial regression, 390 Christmann, 402

Springer International Publishing AG 2017 489

D.J. Olive, Linear Regression, DOI 10.1007/978-3-319-55252-1
490 Index

CI, 38, 63 dot plot, 179

Claeskins, 143 Draper, v, 65, 133, 169, 173
Clark, 236 Driscoll, 319
Cleveland, 441 Duan, 15, 65, 142, 339
Cobb, 177, 188, 198, 203, 217, 227, 237, Duy, 441
238, 290 Dunn, 236
Cochran, 184, 231, 233, 242, 287, 295 Durbin Watson test, 29
Cody, 68
coecient of multiple determination, 31 Eaton, 304, 309
Cohen, vi EDA, 4
Collett, 14, 402, 438, 440, 441 EE plot, 105, 421
column space, 314, 336 eect, 251
component plus residual plot, 141 Efron, 122, 124, 143, 145
conditional distribution, 302 elliptically contoured, 303, 307, 309
condence region, 120 elliptically symmetric, 303
constant variance MLR model, 18 envelope estimators, 382
Cook, v, vii, viii, 1, 34, 65, 68, 69, 78, Ernst, 198
86, 88, 90, 116, 117, 125, 130132, error sum of squares, 30, 59
141145, 150, 160, 172, 200, 210, 304, Ervin, 172
305, 311, 348, 354, 358, 367, 370, 382, estimable, 335
386, 405, 419, 433, 438, 440, 441, 444, estimated additive predictor, 2, 389
446 estimated sucient predictor, 2, 389
Cooks distance, 130 estimated sucient summary plot, 3
Copas, 383 experimental design, 175
covariance matrix, 130, 163, 301 experimental design model, 3
covariates, 4, 17 explanatory variables, 4, 17
Cox, 95, 141, 142, 199 exploratory data analysis, 248
Cox proportional hazards regression model, exponential family, 391
442 Ezekial, v
Craigs Theorem, 319, 336
Cramer, 31 Fabian, 221
Crawley, 68, 460, 464 factor, 97
critical mix, 249, 291 Fahrmeir, 417, 452
cross validation, 65 feasible generalized least squares, 165
Croux, 306 Ferrari, 143
cube root rule, 88 FF plot, 47, 105, 349
Fisher, vi
Daniel, 10, 108 tted values, 19
Darlington, 66 Forsythe, 200, 201
Datta, 65 Fox, v, 131, 142, 145, 200, 460
David, vi, 294 fractional factorial design, 258
Davis, 440 Franklin, v
DD plot, 135, 308 Freedman, 65, 143, 167, 338
Dean, 198 Frey, 121
degrees of freedom, 32 Fujikoshi, 382
Delaney, 198 full model, 99, 138, 420
dependent variable, 4, 17 full rank, 315
Dey, 338 Furnival, 11, 108, 143
df, 32
diagnostics, 4, 85, 129 Gail, 221
discriminant function, 396 Gamma regression model, 390
DOE, 175 Gauss Markov Theorem, 328
Doksum, 142 Gaussian MLR model, 19
Dongarra, 65 Gelman, 294
Index 491

generalized additive model, 2, 389, 429 James, 2, 37, 65, 144

Generalized Cochrans Theorem, 323 Joglekar, 68
generalized inverse, 315, 336 Johnson, v, 69, 141, 167, 300, 301, 303,
generalized least squares, 165 309, 338, 347, 350, 354
generalized linear model, 2, 389, 391, 392 joint distribution, 301
Ghosh, 65 Jones, 10, 105, 143, 144
Gilmour, 144
Gladstone, 26, 33, 48, 54, 60, 83, 113, 126, Kachigan, vi, 65
312, 416, 435 Kakizawa, vii, 360, 361
GLM, 392, 420 Kariya, 172
Goldman, 162, 202, 211 Kay, 419, 435
Golub, 65 Kelker, 305
Graybill, 318, 338 Kenard, 144
Gunst, 144 Khattree, 360, 361, 383
Guttman, 59, 328, 338 Kirk, 198, 200, 242
Kleinbaum, vi
Hadi, v, 143, 145 Kotz, 338
Haenggi, 202 Krasnicka, 319
Kshirsagar, 360, 374
Haggstrom, 396, 441
Kuehl, 180, 192, 198, 208, 242, 279, 295
Hahn, 294
Kurata, 172
Hamilton, v
Kutner, v, 76, 190, 217, 222
Harrell, vi
Kvalseth, 65
Harrison, 158
Harter, v, 65
Leger, 144
Hastie, 123, 146, 285, 296, 441
ladder of powers, 87
hat matrix, 19, 59, 62, 130
ladder rule, 88, 138
Hawkins, vii, 12, 15, 65, 104, 117, 143, 146,
Lancelot, 442
309, 348, 394, 432, 441
lasso, 2, 4, 65, 144
Hebbler, 52, 158, 366
least squares, 10, 19
Helmreich, 199, 241
least squares estimators, 346
Henderson, 357
Ledolter, vi, 198, 205, 230, 250, 265, 275,
Hesterberg, vi, 28
277, 278, 281, 402
Hilbe, 429, 440, 441
Lee, 32, 55, 103, 122, 123, 166, 324, 327,
Hillis, 440
334, 338, 360
Hinkley, 142
Leeb, 143
Hjort, 143
Lehmann, vi
Hoaglin, 65, 145, 199, 241
Lei, 64, 383
Hocking, 338
Leland, v
Hoeding, 198, 241
Lemeshow, 396, 399, 425, 440
Hogg, vi
Leroy, 131, 455
Hosmer, 396, 399, 425, 440
Lesno, 442
Hossin, 155 leverage, 130
Houseman, 172 Li, 15, 65, 142, 339
Hunter, 241, 294 Lindsey, vi
Hurvich, 143 linear mixed models, 172
Hyndman, 383 linear regression, 3
linear regression model, v, 1
identity line, 6, 21, 105, 348 linearly dependent, 314
iid, 3, 18 linearly independent, 314
independent variables, 4, 17 Linhart, 143
inuence, 130, 131 Little, 419, 435
interaction, 97 location family, 177
interaction plot, 216 location model, 56
492 Index

log rule, 87, 138, 192, 419 Nachtsheim, 141

logistic regression, 394 Naik, 360, 361, 383
Long, 172 Nelder, 440
LR, 394 Neyman, vi
lregpack, viii Nishi, 123
LS CLT, 39, 332 noncentral 2 distribution, 320
nonparametric bootstrap, 122
normal equations, 56
Mahalanobis distance, 130, 135, 299, 304,
normal MLR model, 19
307, 308
Norman, 64
main eects, 97
null space, 315
Mallows, 10, 105, 108, 143
Numrich, 118, 157
Marden, 222, 320
Mardia, 309
OConnell, vi
Maronna, 372
OD plot, 438
Masking, 134
Oehlert, 198
masking, 136
Olive, vii, 2, 5, 12, 15, 33, 40, 6466, 104,
Mason, 144
117, 119, 121, 124, 128, 134, 135,
Mathsoft, 460
141143, 145, 146, 168, 198, 200, 207,
Maxwell, 198
221, 241, 309, 338, 348, 350, 354, 381,
McCullagh, 440
382, 393, 394, 400, 432, 441, 442
McDonald, 111, 156
OLS, 10, 19
McKenzie, 162, 202, 211
outlier, 22, 180, 249
Mendenhall, vi Outliers, 134
Merriman, v overdispersion, 399
Mickey, vi
Miller, 383
Potscher, 143
Milton, 338 parametric model, 2
minimum chisquare estimator, 406 Pardoe, vi
Minitab, 73, 211, 223 Partial F Test Theorem, 333, 337
MLR, 5, 18, 61 partial least squares, 2, 65, 382
MLS CLT, 358 partial residual plot, 141
model, 85 partial residual plots, 393
model checking plot, 65, 145 PCR, 2
model sum of squares, 59 Pena, 68
modied power transformation, 93 Pearson, vi
moment generating function, 320 Pelawa Watagoda, 2, 65, 146, 374
Montgomery, v, 198, 201, 204, 206, 222, PLS, 2
262, 294, 296, 418 Poisson regression, 403, 440
Moore, 71, 183, 202 Poisson regression (PR) model, 390
Mosteller, v, 92, 94 pooled variance estimator, 182
MSE, 32 population correlation, 302
multicollinearity, 50, 144 population mean, 163, 301
multiple linear regression, 3, 5, 18, 99, 344 positive denite, 317
multiple linear regression model, 344 positive semidenite, 317
multivariate linear model, 344 power transformation, 92, 191
multivariate linear regression model, 343 predicted values, 19
multivariate location and dispersion, 299 prediction intervals, 128
multivariate location and dispersion model, prediction region, 120
344 prediction region method, 121
multivariate normal, 299, 300, 304, 308 predictor variables, 2, 17, 61, 343
Muyot, 222 principal component regression, 2
MVN, 299, 301 principal components regression, 65
Myers, 338, 407, 408, 440, 453 projection matrix, 315
Index 493

Projection Matrix Theorem, 316 sample mean, 29

Pruzek, 199, 241 sample size, 4
Pun, 383 Santer, 441
pval, 32, 33, 50, 62, 170, 182, 229, 284 SAS, 68, 75, 209, 222, 456
pvalue, 32, 334 SAS Institute, 161, 185, 210, 297, 386
Savin, 361, 374
quadratic form, 317 scatterplot, 20, 87
qualitative variable, 17 scatterplot matrix, 87, 90, 97
quantitative variable, 17 Schaahausen, 159, 312, 401, 417
Schaalje, 338
R, 68 Schee, 338
R Core Team, viii, 382, 442 Schoemoyer, 66
random vector, 163 Schwing, 111, 156
randomization test, 198, 241 Searle, 65, 318, 323, 338, 357, 384
range rule, 88 Seber, 32, 55, 103, 122, 123, 166, 324, 327,
rank, 315 334, 338, 360
Rank Nullity Theorem, 315 Selvin, 143
rank tests, 222 semiparametric model, 2
Rao, 300, 338 Sen, 39
Ravishanker, 338 Setodji, 382
regression function, 38 Severini, 15
regression graphics, 5 Sheather, v, 169, 172
regression sum of squares, 30 Shi, 172
regression through the origin, 59 Simono, 390, 405, 429, 440
Reid, 319 simple linear regression, 57
Rencher, 338, 383 Singer, 39
residual bootstrap, 122 single index model, 13, 15
residual plot, 5, 21, 348 singular value decomposition, 167
residuals, 4, 19 Sinich, vi
response plot, v, 3, 8, 21, 105, 168, 348, Skovgaard, 440
389, 441 Slate, 68
response transformation, 94 SLR, 57
response transformation model, 390 smallest extreme value distribution, 395
response transformations, 92, 142 Smith, v, 68, 133, 169, 173
response variable, 2, 4, 17, 61 Snedecor, 184, 231, 233, 242, 287, 295
response variables, 343 span, 314, 336
Riani, 396 Spector, 154
Rice, vi Spectral Decomposition Theorem, 317
ridge regression, 2, 65, 144 spectral theorem, 167
Ripley, 442, 460 spherical, 304
Robinson, 241, 294 square root matrix, 167, 317, 336
Rohatgi, 303 STATLIB, 425, 450
Rounceeld, 42, 159 Steinberg, 294
Rousseeuw, 131, 135, 308, 402, 455 Stigler, v
row space, 314 Streiner, 64
RR plot, 34, 105, 349 Stuart, 143
Rubinfeld, 158 Su, vii, 68, 348, 354, 358, 382
rule of thumb, 25 submodel, 99, 138, 420
run, 245 subspace, 314
Runger, 142 sucient predictor, 2, 99, 389
Ryan, v survival regression models, 441
Swamping, 134
Sadooghi-Alvandi, 66 Swersey, 198, 205, 230, 250, 265, 275, 277,
Sall, 66 278, 281
494 Index

Tibshirani, 441 Wedderburn, 440

total sum of squares, 30 Weeks, 143
transformation, 4 Weibull proportional hazards regression
transformation plot, 93, 94, 191 model, 442
Tremearne, 22, 117, 133 weighted least squares, 165
Trivedi, 438, 440, 441 Weisberg, v, viii, 1, 34, 65, 68, 69, 78,
Tsai, 143 86, 88, 90, 116, 117, 125, 130132,
Tukey, v, 65, 88, 9294, 200 142145, 150, 160, 210, 348, 367, 386,
Tutz, 417, 452 405, 419, 438, 440, 441, 444, 446, 460,
464
uncorrected total sum of squares, 59 Welch, 200, 201, 242
unimodal MLR model, 19, 61, 93, 129 Weld, v
unit rule, 87 Welsch, 145
Welsh, 65, 145
Van Driessen, 135, 308 Wichern, 167, 300, 301, 309, 347, 350, 354
Van Loan, 65 Wilcox, 35, 198
variable selection, 9, 99, 420 Wilson, 11, 108, 143
variance ination factor, 144 Winkelmann, 405, 438, 440, 441
vector space, 313 Wood, 10, 108, 435, 441, 456
Velilla, 141 Wright, v
Velleman, 145
Venables, 442, 460
Vittingho, vi Yang, 68, 143
Voss, 198 Yeo, 141

Wackerly, vi Zamar, 372

Walls, 143 Zhang, 146, 172, 309, 440
Wang, 441 Zucchini, 143
Wasserman, 64, 383 Zuur, 429, 440, 441