Simple Linear Regression

Engineering Mathematics III

Lecturer :

SMA 2217

June 2020

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Simple Linear Regression

The case of simple linear regression considers a single

regressor variable or predictor variable x and a dependent or
response variable Y. Suppose that the true relationship
between Y and x is a straight line and that the observation Y
at each level of x is a random variable.

We assume that each observation, Y, can be described by the model

Y = β0 + β1 x + e
where e is a random error with mean zero

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Simple linear regression

to estimate the parameters β 0 and β 1 we use least square

estimates.

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Least Squares Estimates

Lecturer :
National University of Science and Technology 2020 Figure:
Simple Linear Regression

...continued

The fitted or estimated regression line is therefore

ŷ = βˆ0 + βˆ1 x

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

.....
Example
We will fit a simple linear regression model to the oxygen purity data
in Table below

Lecturer : Figure:
National University of Science and Technology 2020
Simple Linear Regression

Solution

Lecturer :
National University of Science and Technology 2020
Figure:
Simple Linear Regression

solution...continued

Therefore, the least squares estimates of the slope and intercept are

Figure:

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

.....

The fitted simple linear regression model (with the coefficients

reported to three decimal places) is ŷ = 74.283 + 14.947x
Practical Interpretation: Using the regression model, we would
predict oxygen purity of ŷ = 89.23 % when the hydrocarbon level is
x = 1.00 %. The 89.23 % purity may be interpreted as an estimate
of the true population mean purity when x = 1.00 %, or as an
estimate of a new observation when x = 1.00 %. These estimates
are, of course, subject to error; that is, it is unlikely that a future
observation on purity would be exactly 89.23 % when the
hydrocarbon level is 1.00 %. In subsequent sections, we will see how
to use confidence intervals and prediction intervals to describe the
error in estimation from a regression model.

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Properties of the Least Squares Estimators

In simple linear regression, the estimated standard error of the slope

and the estimated standard error of the intercept are

Figure:

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Hypothesis Tests in Simple Linear Regression

To test hypotheses about the slope and intercept of the regression

model, we must make the additional assumption that the error
component in the model, e, is normally distributed.

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Use of t-tests

Suppose that we wish to test the hypothesis that the slope equals a
constant, say, β 1,0 . The appropriate hypotheses are
H0 : β 1 = β 1,0
H1 : β 1 6= β 1,0
A very important special case of the hypotheses of equations above
are
H0 : β 1 = 0 and H1 : β 1 6= 0
These hypotheses relate to the significance of regression.

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Test Statistics

Test Statistic for the Slope is

Figure:

follows the t distribution with n − 2 degrees of freedom under H0 :

β 1 = β 1,0 . We would reject H0 : β 1 = β 1,0 if |t0 | > tα/2,n−2

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Test Statistics

Lecturer :
National University of Science and Technology 2020
Figure:
Simple Linear Regression

Example

We will test for significance of regression using the model for the
oxygen purity data from the previous Example. The hypotheses are
H0 : β 1 = 0
H1 : β 1 6= 0
we will use α = 0.01. From previous Example we have β 1 = 14.947,
n = 20, Sxx = 068088, σ̂ = 1.18

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Example

so the t-statistic

Figure:

Practical Interpretation: Because the reference value of t is

t0 .005, 18 = 2.88, the value of the test statistic is very far into the
critical region, implying that H0 : β 1 = 0 should be rejected. There
is strong evidence to support this claim.

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Analysis of variance approach to test significance of

regression

The analysis of variance identity is as follows

Figure:

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Analysis of variance approach to test significance of

regression

We usually call SSE = ∑ni=1 (yi − yˆi )2 the error sum of squares and
SSE = ∑ni=1 (yˆi − ȳ )2 the regression sum of squares.
Symbolically, the ANOVA identity can be written as
SST = SSR + SSE

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Test for Significance of Regression

The test statistic

Figure:

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Test for Significance of Regression

Figure:

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Example

We will use the analysis of variance approach to test for significance

of regression using the oxygen purity data model
Recall that SST = 173.38, βˆ1 = 14.947, Sxy = 10.17744 and
n = 20.

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Solution
The regression sum of squares is

Figure:
Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Confidence Intervals on Parameters

Figure:

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Prediction Interval

Figure:

Lecturer :
National University of Science and Technology 2020
Simple Linear Regression

Task

Find a 95 % confidence interval on the slope of the regression line

using the data from first Example above
Using the same data, construct the prediction interval and find a 95 %
prediction interval on the next observation of oxygen purity at
x0 = 1.00%.

Lecturer :
National University of Science and Technology 2020