chapter fourteen

Simple and Multiple

Linear Regression
Learning Objectives
• Identify the uses of regression
• Recognize the components of a regression line
• Explain what the best fitting regression line/least squares line
means
• Explain what residuals/errors mean in prediction
• Identify the difference between a simple and a multiple
regression
• Use the slope and Y-intercept formula to calculate a simple
regression line
• Use the regression line model to make a prediction for a
specific case
• Use SPSS printouts to create regression lines
Regression in Real Life

• SAT and GPA in college

• Car insurance rates based on your age
and car driven
Simple Linear Regression

• Linear Regression
– Predictor variable is used to predict a case’s
score on another variable, and the prediction
equation takes the form of a straight line.
– Regression equation helps one arrive at
better decisions overall.
– Simple linear regression (only one predictor
variable) should be used only with a
statistically significant Pearson r.
Joke Time

• What was Sigmund Freud’s favorite

statistical procedure?

– Regression
Using a Regression Line for Prediction
• Relationship between
Fahrenheit and Celsius
– Perfect correlation, r = 1.00.
– All points in the scatterplot fall on
a straight line.
– Six data points are known.
– What about all the other possible
Fahrenheit values? Temperature Measured in
• If an object’s temperature is Fahrenheit and Celsius
measured and found to be 86°
Fahrenheit, what would it be in
Celsius?
Using a Regression Line for Prediction
(continued)
• Regression Line
– Best-fitting straight line for
predicting Y from X

– Regression Line for

Temperature
• Regression line inserted in this
scatterplot makes it easy to
predict an object’s temperature Regression Line for Predicting
in Celsius if we know its Celsius from Fahrenheit
temperature in Fahrenheit • Draw a vertical line from 86° on
• An object that is 86°F would be the X-axis up to the diagonal line
about 30°C
• Draw a horizontal line over to Y-
axis
• Predicted value of Y is 30°
Using a Regression Line for Prediction
(continued 2)
• Marital Satisfaction
Study
– Dr. Paik’s data from Ch. 13
– Strong relationship
between the two variables
in this scatterplot (r = .76)
– Not clear where the best Relationship Between Gender
place is to draw a Role Flexibility and Marital
Satisfaction
regression line for
predicting Y from X
Using a Regression Line for Prediction
(continued 3)
• Marital Satisfaction Study
– Which of these three lines
best “fits” the data in this
scatterplot?
– By what criterion should one
decide?
– Statisticians use the “least
squares” criterion: the best-
fitting line is the one that Three Potential Regression
minimizes the discrepancies Lines for Predicting Marital
between actual Y scores and Satisfaction
predicted Y scores
What is the Best Fit- The Least Squares
Line?
• It is the line that makes
the squared deviations
(residuals) around the
line the smallest
possible
• The minimum value of
the squared deviations
(the least squares line)
How to Judge Whether a Prediction Is
Good (continued)
• Best prediction is the one that yields the smallest
errors between predicted outcomes and actual
outcomes.

• Least Squares Criterion

– Prediction errors are squared and the best-fitting regression
line is the one that has the smallest sum of squared errors

• Standard Error of the Estimate

– Standard deviation of the residual scores, a measure of error
in regression
Residual Plot with SPSS
Outliers and the best fit
The Linear Regression Equation
• Three factors need to be known to apply regression line
formula
1) X value for which one wants to predict a Y value
2) slope, b
3) Y-intercept, a

′
𝑌 =𝑏𝑋 +𝑎
The Linear Regression Equation
(continued)
• Understanding Slope
– Slope
• Tilt of the line; rise over run; how much up or down
change in Y is predicted for each 1-unit change in
X:
– If the slope is positive, the line is moving up and to the
right.
– If the slope is negative, the line is moving down and to
the right.
– If the slope is zero, the line is horizontal.
 Example 1: Marital Satisfaction Study
Step 5

𝑁 ∑ 𝑋𝑌 − ∑ 𝑋 − ∑ 𝑌
𝑟=
√¿ ¿¿
The Linear Regression Equation
(continued 1)
• Formula for Calculating Slope

𝑏=𝑟
( ) 𝑆𝑌
𝑆𝑋
Example 1: The Linear Regression
Equation (continued)

()
• Dr. Paik’s Marital Satisfaction

𝑆𝑌
Study
– r = .76, SY = 0.86, SX = 11.49

𝑏=𝑟
– Value of the slope, 0.06
• For every 1-point

𝑆𝑋
increase in a husband’s
level of gender role
flexibility, there is a
predicted increase of
0.06 points in the wife’s
level of marital
satisfaction
The Linear Regression Equation
(continued 2)
• Understanding the Y-Intercept
– Y-intercept
• Indicates where the regression line would pass
through the Y-axis
• If Y-intercept is positive, line passes through the Y-
axis above zero.
– If Y-intercept is negative, line passes through the Y-axis
below zero.
– The bigger the absolute value of the Y-intercept, the
further away from zero the regression line passes
through the Y-axis.
The Linear Regression Equation
(continued 3)
• Formula for the Y-Intercept

𝑎=𝑀 𝑦 −𝑏 𝑀 𝑋
Example 1: The Linear Regression
Equation (continued 1)
• Dr. Paik’s Marital Satisfaction Study
– b = 0.0568, MY = 2.00, MX = 25.00
– Regression formula, Equation 14.1

𝑎=𝑀 𝑦 −𝑏 𝑀 𝑋 ′
𝑌 =𝑏𝑋 + 𝑎
Example 1: The Linear Regression
Equation (continued 2)
• Predicting Y
– Dr. Paik needs to select an X

′
value for which to predict Y score

𝑌 =𝑏𝑋+𝑎
• Selects gender role flexibility score
of 30 and substitutes that for X
• A man with a gender role flexibility
score of 30 will have a partner who
rates her level of marital
satisfaction as 2.28
• Marital satisfaction rated on a 4-
point scale like GPA
• She would rate her marriage at the
C level
Example 1: The Linear Regression
Equation (continued 3)
• Dr. Paik’s Marital Satisfaction Study
– Dr. Paik will find Y′ for lowest X value (8) and Y′ for largest (38)
– So if a husband’s gender role flexibility is 8, what would their wife’s
marital satisfaction be?
– If a husband’s gender role flexibility is 38, what would their wife’s
marital satisfaction be?

′ ′
𝑌 =𝑏𝑋+𝑎 𝑌 =𝑏𝑋+𝑎
Example 1: Regression Line
• Regression Line for Marital
Satisfaction
– Finding Y′ for the points at
two ends of the range of X
scores allows a researcher to
draw a regression line.
– Any two points can be
connected with a straight line.
– Regression line should only
be used to predict Y′ for the Regression Line for Predicting
range of X scores used to Marital Satisfaction
derive the regression
equation.
 Example 2: Calculating Simple
Regression
• A therapist wants to know if there is a relationship between
perceived stress and the magnitude of the stress symptoms
displayed? They as six clients to report their stress level using
a stress questionnaire and the therapist makes note of their
symptom levels.
Stress Symptom (Y)
(X)
X Y X2 Y2 XY
30 99 900 9801 2970
27 94 729 8836 2538
9 35 81 1225 315
20 70 400 4900 1400
3 30 9 900 90
15 45 225 2025 675
104 373
 Example 2: Stress Study Step 5

𝑁 ∑ 𝑋𝑌 − ∑ 𝑋 − ∑ 𝑌
𝑟=
√¿ ¿¿
Example 2: The Linear Regression
Equation (continued)
• The Therapists’ Stress Study
– r = .9757, SY = 29.9961 , SX =
𝑏=𝑟
( )
𝑆𝑌
𝑆𝑋
10.4051
Example 2: The Linear Regression
Equation (continued 1)
• The Therapists’ Stress Study
– b = 2.8128, MY = 62.1667, MX = 17.3333
– Regression formula, Equation 14.1

𝑎=𝑀 𝑌 − 𝑏 𝑀 𝑋
Example 2: Regression Prediction

• Use the equation from the regression line

to predict symptoms for a new person. For
example, if the stress level is 32, what
would the symptom level be?
Example 2: The Linear Regression
Equation (continued 2)
• Drawing the Regression Line
– Helps to highlight the relationship between the two variables
• The therapists’ stress study
– What if a new client came in and had a stress score of 45, what would
their symptom level be?
– What if a new client came in and had a stress score of 38?
 Example 3: Simple Regression Line

• A professor wants to know if there is a negative relationship

between how much a student studies and the the number
of errors the student makes on a statistics quiz? She asks
four students to report how long they studied and counts
the number of errors made on their statistics quizzes.
Study # of Errors (Y)
Time (X)
X Y X2 Y2 XY
90 25 8100 625 2250
100 28 10000 784 2800
130 20 16900 400 2600
150 20 22500 400 3000
470 93
 Example 3: Professor’s Test Error Study
Step 5

𝑁 ∑ 𝑋𝑌 − ∑ 𝑋 − ∑ 𝑌
𝑟=
√¿ ¿¿
Example 3: The Linear Regression
Equation (continued)
• The Professor’s Test Error
𝑆
𝑏=𝑟
( )
𝑌
𝑆𝑋
Study
– r = -.8509, SY = 3.9476 , SX =
27.5379
Example 3: The Linear Regression
Equation (continued 3)
• The Professor’s Test Error Study
– b = -.1220, MY = 23.25, MX = 117.5
– Regression formula, Equation 14.1

𝑎=𝑀 𝑌 − 𝑏 𝑀 𝑋
Example 3: Regression Prediction

• Use the equation from the regression line

to predict errors for a new person. For
example, if the person studied for 96
minutes, how many errors would we
predict on the exam?
Example 3: The Linear Regression
Equation (continued 4)
• The professor’s test error study
– What if a new student studied for 90 min, how many errors
would they make?
– What if a new student studied for 38 min, how many errors
would they make?
Multiple Regression
• Example: Student GPA
– In which scenario could one more accurately predict a
student’s GPA?
• Knowing how many hours the student spends on schoolwork each
week, OR
• Knowing how many hours the student spends on schoolwork each
week plus high school GPA, IQ, and how much alcohol the student
consumes each week
• Most people believe the additional information in
Scenario B is relevant to predicting academic
performance and they are correct. In Scenario B, the
prediction should be more accurate because more factors
are taken into account.
Multiple Regression (continued)
• The difference between Scenario A and Scenario B is the
difference between simple regression and multiple
regression.
• Simple Linear Regression
– Prediction in which Y′ is predicted from a single independent
variable
– Scenario A
• Multiple Linear Regression
– Prediction in which multiple independent variables are combined
to predict a dependent variable
– Adds together the unique predictive power of each variable
– Scenario B, better at predicting
Multiple Regression (continued 1)

• R2 in Multiple Regression
– r2, the percentage of variability in the dependent
variable that is accounted for by the independent
variable(s), is called R2 in multiple regression
– Better prediction means a higher percentage of
variability is accounted for with multiple
regression than with simple regression
– More powerful technique than simple regression
R2 the proportion of variance that two measures have in common-
overlap determines relationship-explained variance
Multiple Regression (continued 2)
• Example: Admitted Class Evaluation Service (ACES)
– Predicting first-year GPA
• Equation is developed from a first-year class to predict first-year
GPA
• College Board examines four variables; the Pearson r correlation
coefficients for each of these variables predicting GPA by itself are:
– SAT reading test, r = .42
– SAT writing test, r = .42
– SAT math test, r = .39
– High school class rank, r = .52
• When the four College Board variables are combined to predict
GPA, correlation climbs to R = .57
• Percentage of variance explained changes from 27.04% to 32.49%
 Multiple Regression (continued 3)

• Example: Admitted Class Evaluation Service

(continued)
– Equation has “weights” for each of the predictor
variables
• Weights are like the slope in the linear regression equation
– A constant that is like the Y-intercept
– Here is how estimated GPA, GPA′, would be calculated:

′
𝑃 𝐴 =( 𝑆𝐴𝑇 ReadingScore × 𝑊𝑒𝑖𝑔h𝑡 ReadingScore ) + ( 𝑆𝐴𝑇 WritingScore × 𝑊𝑒𝑖𝑔h𝑡WritingSco
Multiple Regression (continued 4)

• Example: Admitted Class Evaluation

Service (continued)
– Here are the four weights and the constant:
• Reading weight = 0.0012
• Writing weight = 0.0013
• Math weight = 0.0006
• HS rank weight = 0.0029
• Constant = 0.7821
 Multiple Regression (continued 5)
• Example: Admitted Class Evaluation Service (continued)
– If an applicant were good at reading (SAT score = 600), not so
good at writing (SAT score = 450), very good at math (SAT score
= 760), and had a very good class rank (90), then her predicted
GPA would be
– A person with those SAT scores and class rank would be
predicted to end up with a GPA of 2.80 at the end of her first
year

′
𝑃𝐴 =(𝑆𝐴𝑇 ReadingScore ×𝑊𝑒𝑖𝑔h𝑡ReadingScore )+(𝑆𝐴𝑇WritingScore×𝑊𝑒𝑖𝑔h𝑡WritingSco