Applied Statistics and

Probability for
Engineers

Sixth Edition
Douglas C. Montgomery George C.

Chapter 11 Runger

Simple Linear Regression and

Correlation
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11
CHAPTER OUTLINE
Simple Linear
Regression and
Correlation
11-1 Empirical Models 11-6 Prediction of New Observations
11-2 Simple Linear Regression 11-7 Adequacy of the Regression
11-3 Properties of the Least Squares Model
11-7.1 Residual analysis
Estimators 11-7.2 Coefficient of determination
11-4 Hypothesis Test in Simple (R2)
Linear 11-8 Correlation
Regression 11-9 Regression on Transformed
11-4.1 Use of t-tests Variables
11-4.2 Analysis of variance 11-10 Logistic Regression
approach to test
significance of regression
11-5 Confidence Intervals
11-5.1 Confidence intervals on the
slope and
intercept
11-5.2 Confidence interval on the
mean
response
2
Chapter 11 Title and Outline
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Learning Objectives for
Chapter 11
After careful study of this chapter, you should be
able to do
the following:
1. Use simple linear regression for building empirical models
to engineering and scientific data.
2. Understand how the method of least squares is used to
estimate the parameters in a linear regression model.
3. Analyze residuals to determine if the regression model is an
adequate fit to the data or to see if any underlying
assumptions are violated.
4. Test the statistical hypotheses and construct confidence
intervals on the regression model parameters.
5. Use the regression model to make a prediction of a future
observation and construct an appropriate prediction interval
on the future observation.
6. Apply the correlation model.
7. Use simple transformations to achieve a linear regression
model.
Chapter 11 Learning Objectives 3
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-1: Empirical Models
• Many problems in engineering and
science involve exploring the
relationships between two or more
variables.
• Regression analysis is a statistical
technique that is very useful for these
types of problems.
• For example, in a chemical process,
suppose that the yield of the product is
related to the process-operating
temperature.
• Regression analysis can be used to build
a11-1
Sec model to predict yield at a given
Empirical Models 4
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-2: Simple Linear Regression

• The simple linear regression considers

a single regressor or predictor x and a
dependent or response variable Y.
• The expected value of Y at each level
of x is a random variable:
E(Y|x) = b0 + b1x

• We assume that each observation, Y,

can be described by the model
Y = b 0 + b 1x + 

Sec 11-2 Simple Linear Regression 5

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-2: Simple Linear Regression
Least Squares Estimates
The least-squares estimates of the intercept and slope in
the simple linear regression model are
ˆ  y
 ˆ x
 
(11-1)
 n  n 
  yi    xi 
n   
   i 1 
 yi xi  i 1
n
ˆ 
 i 1
 2
 n 
  xi 
n  
(11-2)  xi2   i 1 
i 1 n

y  (1/n)  in1 yi x  (1/n)  in1 xi

where
Sec 11-2 Simple Linear Regression and . 6
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-2: Simple Linear Regression
The fitted or estimated regression line is therefore

ˆ 
yˆ   ˆ x
 
(11-3)

Note that each pair of observations satisfies the

relationship
yi  ˆ   ˆ  xi  ei , i  1, 2,  , n

ei  yi  yˆ i

where is called the residual. The residual

describes the error in the fit of the model to the ith
observation yi.
Sec 11-2 Simple Linear Regression 7
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-2: Simple Linear Regression
Notation
2
 n 
  xi 
n n  
S xx   xi  x 2   xi2   i 1 
i 1 i 1 n

 n  n 
  xi    yi 
n n   
S xy   yi xi  x 2   xi yi   i 1   i 1 
i 1 i 1 n

Sec 11-2 Simple Linear Regression 8

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
EXAMPLE 11-1 Oxygen Purity
We will fit a simple linear regression model to the oxygen purity
data in
Table 11-1. The following
20
quantities
20
may be computed:
n  20  xi  23.92  yi  1,843.21
i 1 i 1
x  1.1960 y  92.1605
20 20
 yi2  170,044.5321  xi2  29.2892
i 1 i 1
20
 xi yi  2,214.6566
i 1
2
 20 
  xi 
20   ( 23.92) 2
S xx   xi 
2  i 1   29.2892 
i 1 20 20
 0.68088
an  20   20 
  xi    yi 
d 20   
S xy   xi yi   i 1   i 1 
i 1 20
( 23.92) (1,843.21)
 2,214.6566   10.17744
20
Sec 11-2 Simple Linear Regression 9
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
EXAMPLE 11-1 Oxygen Purity - continued

Therefore, the least squares estimates of the slope and

intercept are
S xy 10.17744
ˆ 1    14.94748
S xx 0.68088

and
ˆ 0  y  ˆ  x  92.1605  (14.94748)1.196  74.28331

The fitted simple linear regression model (with the

coefficients reportedˆ to three decimal places) is
y  74.283  14.947 x

Sec 11-2 Simple Linear Regression 10

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-2: Simple Linear Regression
Estimating 2
The error sum of squares is

n n
SS E   ei2    yi  yˆ i 2
i 1 i 1

It can be shown that the expected

value of the error sum of squares is
E(SSE) = (n – 2)2.

Sec 11-2 Simple Linear Regression 11

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-2: Simple Linear Regression
Estimating 2
An unbiased estimator of 2 is
SS E
ˆ2 
 (11-4)
n2

where SSE can be easily

computed using
SS E  SST  ̂1S xy (11-5)

Sec 11-2 Simple Linear Regression 12

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-3: Properties of the Least Squares
Estimators

• Slope Properties

2
ˆ )
E ( ˆ ) 
1 1 V (1
S xx

• Intercept Properties

 1 x 2 
E (ˆ 0 )  0 and V (ˆ 0 )   2   
 n S xx 

Sec 11-3 Properties of the Least Squares Estimators 13

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-4: Hypothesis Tests in Simple Linear Regression

11-4.1 Use of t-
Tests
Suppose we wish to test

H0: b1 = b1,0
H1: b1  b1,0

An appropriate test statistic

would be ˆ   1, 0
T0 
(11-6)
ˆ 2 /S xx


Sec 11-4 Hypothesis Tests in Simple Linear Regression 14

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-4: Hypothesis Tests in Simple Linear Regression

11-4.1 Use of t-
Tests
The test statistic could also be
written as: ˆ  ˆ
1 1, 0
T0 
ˆ )
se( 1

We would reject the null

hypothesis if

|t0| > ta/2,n - 2

Sec 11-4 Hypothesis Tests in Simple Linear Regression 15

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-4: Hypothesis Tests in Simple Linear Regression

11-4.1 Use of t-
Tests
Suppose we wish to test

H0: b0 = b0,0
H1: b0  b0,0
An appropriate test statistic
would be ˆ 
0 0, 0 ˆ 
0 0, 0
T0   (11-7)
1 2  ˆ )
se (
2 x 0

ˆ   
n
 S xx 


Sec 11-4 Hypothesis Tests in Simple Linear Regression 16

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-4: Hypothesis Tests in Simple Linear Regression

11-4.1 Use of t-
Tests
We would reject the null
hypothesis if

|t0| > ta/2,n - 2

Sec 11-4 Hypothesis Tests in Simple Linear Regression 17

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-4: Hypothesis Tests in Simple Linear Regression

11-4.1 Use of t-
Tests
An important special case of the
hypotheses of Equation 11-18 is

H0: b1 = 0
H1: b1  0
These hypotheses relate to the
significance of regression.
Failure to reject H0 is equivalent to
concluding that there is no linear
relationship between x and Y.
Sec 11-4 Hypothesis Tests in Simple Linear Regression 18
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-4: Hypothesis Tests in Simple Linear Regression
EXAMPLE 11-2 Oxygen Purity Tests of Coefficients We will test for
significance of regression using the model for the oxygen purity data from
Example 11-1. The hypotheses are
H0: b1 = 0
H1: b1  0
ˆ 1 a
and we will use = .0.01.
 14 947 nFrom
 20, Example ˆ 2 Table
11-1, and
S xx  0.68088 1.18 11-2 we have

ˆ 1
so the t-statistic in Equation ˆ  becomes
11-6 14.947
t0     11 .35
 2
ˆ /S xx se(ˆ 1 ) 1.18/0.68088

Practical Interpretation: Since the reference value of t is t0.005,18 = 2.88, the

P ~ 1.23  10 9
value of the test statistic is very far into the critical region, implying that H0:
b1 = 0 should be rejected. There is strong evidence to support this claim. The
P-value for this test is . This was obtained manually with a
calculator.

Table 11-2 presents the Minitab output for this problem. Notice that the t-
statistic value for the slope is computed as 11.35 and that the reported P-
value is P = 0.000. Minitab also reports the t-statistic for testing the
hypothesis H0: b0Tests
Sec 11-4 Hypothesis = 0.inThis statistic
Simple is computed from Equation 11-7, with b19
Linear Regression
0,0 =
0, as t = 46.62. Clearly,
Copyright © 2014 John
then, the Wiley & Sons, Inc.
hypothesis All rights
that reserved. is zero is
the intercept
11-4: Hypothesis Tests in Simple Linear Regression

11-4.2 Analysis of Variance Approach to

Test Significance of Regression
The analysis of variance identity is
n n n

 iy  y 2
 
 iˆ
y  y 2
 
 i i
y  ˆ
y 2

(11-8)
i 1 i 1 i 1

Symbolically,

SST = SSR + SSE

(11-9)

Sec 11-4 Hypothesis Tests in Simple Linear Regression 20

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-4: Hypothesis Tests in Simple Linear Regression

11-4.2 Analysis of Variance Approach to

Test Significance of Regression
If the null hypothesis, H0: 1 = 0 is true,
the statistic
SS R /1 MS R
F0  
SS E / n  2  MS E
(11-10)
follows the F1,n-2 distribution and we
would reject if f0 > f,1,n-2.

Sec 11-4 Hypothesis Tests in Simple Linear Regression 21

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-4: Hypothesis Tests in Simple Linear Regression

11-4.2 Analysis of Variance Approach to

Test Significance of Regression
The quantities, MSR and MSE are called
mean squares.
Analysis of variance table:
Source of Sum of Degrees of Mean F0
Variation Squares Freedom Square
Regression SS R  ̂1S xy 1 MSR MSR /MSE
SS E  SST  ̂1S xy
Error n-2 MSE
Total SS T n-1

Note that MSEˆ=

2

Sec 11-4 Hypothesis Tests in Simple Linear Regression 22

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-4: Hypothesis Tests in Simple Linear Regression

EXAMPLE 11-3 Oxygen Purity ANOVA We will use the analysis of

variance approach to test for significance of regression using
the oxygen purity data model from 173.38, ˆ 1  14
SST  Example .947 Recall that
11-1.
, Sxy = 10.17744, and n = 20. The regression sum
of squares isSS R  ˆ 1S xy  (14.947) 10.17744  152.13

and the error sum of squares is

SSE = SST - SSR = 173.38 - 152.13 = 21.25

The analysis of variance for testing H0:9b1 = 0 is summarized in

P ~ 1.23  10
the Minitab output in Table 11-2. The test statistic is f0 = MSR/MSE
= 152.13/1.18 = 128.86, for which we find that the P-value is
, so we conclude that b1 is not zero.
There are frequently minor differences in terminology among
computer packages. For example, sometimes the regression
sum of squares is called the “model” sum of squares, and the
error sum of squares is called the “residual” sum of squares.
Sec 11-4 Hypothesis Tests in Simple Linear Regression 23
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-5: Confidence Intervals
11-5.1 Confidence Intervals on the Slope and
Intercept
Definition
Under the assumption that the observation are normally and
independently distributed, a 100(1 - a)% confidence interval on
the slope b1 in simple linear regression is

ˆ t ˆ2
 ˆ t ˆ2

1 /2, n  2  1  1 /2, n  2
(11-11) S xx S xx

Similarly, a 100(1 - a)% confidence interval on the intercept b0 is

ˆ 2 1 x2 
0  t/2, n  2 
ˆ   
 n S xx 


2 1 x2 
 0  ˆ 0  t/2, n  2 
ˆ   
 n S xx 
(11-12)
Sec 11-5 Confidence Intervals 24
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-5: Confidence Intervals
EXAMPLE 11-4 Oxygen Purity Confidence Interval on the Slope
We will find a 95% confidence interval on the slope of the
regression line using the ˆ 1  14.947
data in, SExample ˆ 2  1.18 that
xx  0.6808811-1. 
Recall
, and (see Table 11-2). Then, from
Equation 11-11 we find
ˆ ˆ2 ˆ ˆ2
  t0.025,18  1  1  t0.025,18
S xx S xx

Or 1.18 1.18
14.947  2.101  1  14.947  2.101
0.68088 0.68088

This simplifies to

12.181  b1  17.713

Practical Interpretation: This CI does not include zero, so there

is strong evidence (at a = 0.05) that the slope is not zero. The CI
is reasonably narrow (2.766) because the error variance is
fairly
Sec 11-5 small.
Confidence Intervals 25
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-5: Confidence Intervals
11-5.2 Confidence Interval on the Mean
Response
 ˆ 
ˆ Y | x0   ˆ x
0 1 0

Definition
A 100(1 - a)% confidence interval about the mean response at the
x  x0 of Y | x0
value
, say , is given by

ˆ Y | x0  t/2, n  2  2
1  x0  x 2 
ˆ   

 n S xx 


 Y | x0  
ˆ Y | x0  t/2, n  2  2
1  x0  x 2 
ˆ   

 n S xx 

(11-

ˆ Y | x0 ˆ 
 ˆ x
13) 0 1 0

where is computed from the fitted regression

model.
Sec 11-5 Confidence Intervals 26
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-5: Confidence Intervals
Example 11-5 Oxygen Purity Confidence Interval on the Mean
Response
We will construct a 95% confidence interval about the mean
response for the data in Example  ˆ Y | x0 The
11-1.  14.model
fitted
74.283 947 x0 is
, and the
Y |x0 95% confidence interval on is
found from Equation 11-13 as
1 ( x0  1.1960) 2 
ˆ Y | x0  2.101 1.18   
 20 0.68088 

Suppose that we
ˆ Y |are interested in predicting mean oxygen
x1.00  74.283  14.947(1.00)  89.23
purity when
x0 = 100%. Then
1 (1.00  1.1960) 2 
89.23  2.101 1.18   
and the 95% confidence interval is  20 0 . 68088 

or
89.23  0.75
Therefore, the 95% CI on mY|1.00 is
88.48  mY|1.00  89.98
Sec 11-5 Confidence Intervals 27
This is a reasonable narrow CI.
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-6: Prediction of New
Observations
Prediction Interval
A 100(1 - a) % prediction interval on a future observation Y0 at
the value x0 is given by

2
 1 x0  x 2 
yˆ 0  t/2, n  2 
ˆ 1   
 n S xx 

(11-14) 
2 1 x0  x 2 
 Y0  yˆ 0  t/2, n  2 
ˆ 1   
 n S xx 

ŷ0 yˆ 0  ˆ 0  ˆ 1x0
The value is computed from the regression model
.

Sec 11-6 Prediction of New Observations 28

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-6: Prediction of New
Observations
EXAMPLE 11-6 Oxygen Purity Prediction Interval

To illustrate the construction of a prediction interval, suppose we

use the data in Example 11-1 and find a 95% prediction interval
on the next observation of oxygen purity x0 = 1.00%. Using
yˆ 0  8911-14
Equation .23 and recalling from Example 11-5 that ,
we find that the prediction interval is

89.23  2.101 1.18 1 
1

1.00  1.1960 2 


 20 0 . 68088 


 Y0  89.23  2.101. 1.18 1 
1

1.00  1.1960 2 


 20 0 .68088 


which simplifies to
86.83  y0  91.63

This isPrediction
Sec 11-6 a reasonably narrow prediction interval.
of New Observations 29
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-7: Adequacy of the Regression Model

• Fitting a regression model requires

several assumptions.
1. Errors are uncorrelated random
variables with mean zero;
2. Errors have constant variance; and,
3. Errors be normally distributed.
• The analyst should always consider
the validity of these assumptions to
be doubtful and conduct analyses to
examine the adequacy of the model

Sec 11-7 Adequacy of the Regression Model 30

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-7: Adequacy of the Regression Model

11-7.1 Residual Analysis

• The residuals from a regression model are ei =
yi - ŷi , where yi is an actual observation and ŷi is
the corresponding fitted value from the
regression model.

• Analysis of the residuals is frequently helpful

in checking the assumption that the errors are
approximately normally distributed with
constant variance, and in determining whether
additional terms in the model would be useful.

Sec 11-7 Adequacy of the Regression Model 31

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-7: Adequacy of the Regression Model
EXAMPLE 11-7 Oxygen Purity Residuals
The regression model for the oxygen purity data in Example
11-1
yˆ  is
74.283  14.947 x

Table 11-4 presents the observed and predicted values of y

at each value of x from this data set, along with the
corresponding residual. These values were computed using
Minitab and show the number of decimal places typical of
computer output.

A normal probability plot of the residuals is shown in Fig. 11-

10. Since the residuals fall approximately alongŷia straight
line in the figure, we conclude that there is no severe
departure from normality.

The residuals
Sec 11-7 are
Adequacy of the also plotted
Regression Model against the predicted value 32
in Fig. 11-11 and against the hydrocarbon levels x in Fig. 11-
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-7: Adequacy of the Regression Model

Example 11-7

Sec 11-7 Adequacy of the Regression Model 33

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-7: Adequacy of the Regression Model

Example 11-7

Figure 11-10 Normal

probability plot of
residuals, Example 11-
7.

Sec 11-7 Adequacy of the Regression Model 34

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-7: Adequacy of the Regression Model

Example 11-7

Figure 11-11 Plot of

residuals versus
predicted oxygen
purity, ŷ, Example 11-7.

Sec 11-7 Adequacy of the Regression Model 35

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-7: Adequacy of the Regression Model

11-7.2 Coefficient of Determination

(R
•
2
)
The
quantity 2SS R SS E
R  1
SST SST

is called the coefficient of determination

and is often used to judge the adequacy of
a regression model.
• 0  R2  1;
• We often refer (loosely) to R2 as the
amount of variability in the data explained
or accounted for by the regression model.

Sec 11-7 Adequacy of the Regression Model 36

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
11-7: Adequacy of the Regression Model

11-7.2 Coefficient of Determination

(R2)
• For the oxygen purity regression
model,
R2 = SSR/SST
= 152.13/173.38
= 0.877
• Thus, the model accounts for 87.7%
of the variability in the data.

Sec 11-7 Adequacy of the Regression Model 37

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-8: Correlation
We assume that the joint distribution of Xi and Yi is the bivariate
2 2
normal distribution presentedYin Chapter 5, and mY and are the X
mean and variance of Y, mX and are the mean and variance X, and
r is the correlation coefficient between Y and X. Recall that the
 XY as
correlation coefficient is defined

 X Y
(11-
15)

where sXY is the covariance 

between Yy and X . x  2 
1 1   
fY | x  y   exp    0 1  
The conditional distribution 2Y | x of Y for2 a
 given
Y | xvalue

 of X = x is
 

(11-
16) Y
0  Y   X 
X
where
Y
1  
X
(11-17)
Sec 11-8 Correlation 38
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-8: Correlation
It is possible to draw inferences about the correlation coefficient r
in this model. The estimator of r is the sample correlation
coefficient n
 Yi X i  X  S XY
R i 1 
 n n 
1/ 2
S SS 1/2

 X i  X   Yi  Y  
2 2 XX T
(11-19)
 i 1 i 1 

Note that 1/2

ˆ  SST 
1    R
 S XX 

(11-20)
We may also write:

S XX ˆ S
 SS R
2 ˆ2 1 XY
R  1  
SYY SST SST
Sec 11-8 Correlation 39
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-8: Correlation
It is often useful to test the hypotheses

H0: r = 0
H1: r  0

The appropriate test statistic for these

hypotheses is
R n2
T0  (11-21)
1  R2

Reject H0 if |t0| > t/2,n-2.

Sec 11-8 Correlation 40

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-8: Correlation
The test procedure for the
hypothesis

H0: r = 0
H1: r  0
where 0  0 is somewhat more
complicated. In this case, the
appropriate test statistic
Z0 = (arctanh is r )(n - 3)1/2
R - arctanh 0
(11-22)
Reject H0 if |z0| > z/2.

Sec 11-8 Correlation 41

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-8: Correlation

The approximate 100(1- )% confidence

interval is
 z/2   z/2 
tanh  arctanh r      tanh  arctanh r   (11-23)
 n3 n3
 

Sec 11-8 Correlation 42

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-8: Correlation
EXAMPLE 11-8 Wire Bond Pull Strength In Chapter 1 (Section 1-3)
an application of regression analysis is described in which an
engineer at a semiconductor assembly plant is investigating the
relationship between pull strength of a wire bond and two
factors: wire length and die height. In this example, we will
consider only one of the factors, the wire length. A random
sample of 25 units is selected and tested, and the wire bond pull
strength and wire length arc observed for each unit. The data are
shown in Table 1-2. We assume that pull strength and wire length
are jointly normally distributed.

Figure 11-13 shows a scatter diagram of wire bond strength

versus wire length. We have used the Minitab option of displaying
box plots of each individual variable on the scatter diagram.
There is evidence of a linear relationship between the two
variables.

The Minitab output for fitting a simple linear regression model to

the data is shown below.
Sec 11-8 Correlation 43
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-8: Correlation

Figure 11-13 Scatter plot of wire bond strength versus wire length,
Example 11-8.
Sec 11-8 Correlation 44
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-8: Correlation
Minitab Output for Example 11-8
Regression Analysis: Strength versus Length

The regression equation is

Strength = 5.11 + 2.90 Length

Predictor Coef SE Coef T P

Constant 5.115 1.146 4.46 0.000
Length 2.9027 0.1170 24.80 0.000

S = 3.093 R-Sq = 96.4% R-Sq(adj) = 96.2%

PRESS = 272.144 R-Sq(pred) = 95.54%

Analysis of Variance

Source DF SS MS F P
Regression 1 5885.9 5885.9 615.08
0.000
Residual Error 23 220.1 9.6
Total
Sec 11-8 Correlation 24 6105.9 45
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-8: Correlation
Example 11-8 (continued)

Now Sxx = 698.56 and Sxy = 2027.7132, and the sample

correlation coefficient is

S xy 2027.7132
r   0.9818
S xx SST  1/2
698.5606105.9 1/2

Note that r2 = (0.9818)2 = 0.9640 (which is reported in the

Minitab output), or that approximately 96.40% of the variability
in pull strength is explained by the linear relationship to wire
length.

Sec 11-8 Correlation 46

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-8: Correlation
Example 11-8 (continued)

Now suppose that we wish to test the hypotheses

H0 : r = 0
H1 : r  0

with a = 0.05. We can compute the t-statistic of Equation 11-21 as

r n2 0.9818 23
t0    24.8
1  r2 1  0.9640

This statistic is also reported in the Minitab output as a test of

H0: b1 = 0. Because t0.025,23 = 2.069, we reject H0 and conclude that
the correlation coefficient r  0.

Sec 11-8 Correlation 47

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-8: Correlation
Example 11-8 (continued)

Finally, we may construct an approximate 95% confidence

interval on r from Equation 11-23. Since arctanh r = arctanh
0.9818 = 2.3452, Equation 11-23 becomes
 1.96   1.96 
tanh  2.3452      tanh  2.3452  
 22   22 

which reduces to

0.9585  r  0.9921

Sec 11-8 Correlation 48

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-9: Transformation and Logistic
Regression
We occasionally find that the straight-line
regression model
Y = b0 + b1x +  inappropriate because the true
regression function is nonlinear. Sometimes
nonlinearity is visually determined from the
scatter diagram, and sometimes, because of prior
experience or underlying theory, we know in
advance that the model is nonlinear.

Occasionally, a scatter diagram will exhibit an

apparent nonlinear relationship between Y and x.
In some of these situations, a nonlinear function
can be expressed as a straight line by using a
suitable transformation. Such nonlinear models
are
Sec called intrinsically
11-9 Transformation linear.
and Logistic Regression 49
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-9: Transformation and Logistic
Regression
EXAMPLE 11-9 Windmill Power Observation Wind Velocity DC Output, yi
Number, i (mph), xi
A research engineer is investigating 1 5.00 1.582
the use of a windmill to generate 2 6.00 1.822
electricity and has collected data on 3 3.40 1.057
4 2.70 0.500
the DC output from this windmill and 5 10.00 2.236
the corresponding wind velocity. The 6 9.70 2.386
7 9.55 2.294
data are plotted in Figure 11-14 and 8 3.05 0.558
listed in Table 11-5 9 8.15 2.166
10 6.20 1.866
11 2.90 0.653
12 6.35 1.930
13 4.60 1.562
14 5.80 1.737
15 7.40 2.088
16 3.60 1.137
17 7.85 2.179
18 8.80 2.112
19 7.00 1.800
20 5.45 1.501
Table 11-5 Observed Values and 21 9.10 2.303
22 10.20 2.310
Regressor Variable for Example 23 4.10 1.194
11-9. 24 3.95 1.144
25 2.45 0.123

Sec 11-9 Transformation and Logistic Regression 50

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-9: Transformation and Logistic
Regression
Example 11-9 (Continued)

Figure 11-14 Plot of DC output y versus Figure 11-15 Plot of residuals ei

wind velocity x for the windmill data. versusyˆi fitted values for the
windmill data.

Sec 11-9 Transformation and Logistic Regression 51

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-9: Transformation and Logistic
Regression
Example 11-9 (Continued)

Figure 11-16 Plot of DC output versus x’

= 1/x for the windmill data.

Sec 11-9 Transformation and Logistic Regression 52

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 11-9: Transformation and Logistic
Regression
Example 11-9 (Continued)

Figure 11-17 Plot of residuals versus fitted Figure 11-18 Normal probability plot of the
values
yˆi residuals for the transformed model for the
for the transformed model for the windmill windmill data.
data. ˆ
A plot of the residuals from the transformed model yi versus is shown in Figure 11-17.
This plot does not reveal any serious problem with inequality of variance. The normal
probability plot, shown in Figure 11-18, gives a mild indication that the errors come
from a distribution with heavier tails than the normal (notice the slight upward and
downward curve at the extremes). This normal probability plot has the z-score value
plotted on the horizontal axis. Since there is no strong signal of model inadequacy,
we
Sec conclude that theand
11-9 Transformation transformed model is satisfactory.
Logistic Regression 53
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Important Terms & Concepts of
Chapter 11
Analysis of variance test Odds ratio
in regression Prediction interval on a
Confidence interval on future observation
mean response Regression analysis
Correlation coefficient Residual plots
Empirical model Residuals
Confidence intervals on Scatter diagram
model parameters Simple linear regression
Intrinsically linear model model standard error
Least squares estimation Statistical test on model
of regression model parameters
parameters Transformations
Logistics regression
Model adequacy checking
Chapter 11 Summary 54
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.