2. Multiple Regression Analysis (MRA)
• Method for studying the relationship
between a dependent variable and two or
more independent variables.
• Purposes:
– Prediction
– Explanation
– Theory building
3. Design Requirements
• One dependent variable (criterion)
• Two or more independent variables
(predictor variables).
• Sample size: >= 50 (at least 10 times as
many cases as independent variables)
4. Assumptions
• Independence: the scores of any particular subject are
independent of the scores of all other subjects
• Normality: in the population, the scores on the dependent
variable are normally distributed for each of the possible
combinations of the level of the X variables; each of the
variables is normally distributed
• Homoscedasticity: in the population, the variances of the
dependent variable for each of the possible combinations of the
levels of the X variables are equal.
• Linearity: In the population, the relation between the dependent
variable and the independent variable is linear when all the other
independent variables are held constant.
5. Simple vs. Multiple Regression
• One dependent variable Y
predicted from one
independent variable X
• One regression coefficient
• r2
: proportion of variation in
dependent variable Y
predictable from X
• One dependent variable Y
predicted from a set of
independent variables (X1,
X2 ….Xk)
• One regression coefficient for
each independent variable
• R2
: proportion of variation in
dependent variable Y
predictable by set of
independent variables (X’s)
7. Example: The Model
• Y’ = a + b1X1 + b2X2 + …bkXk
• The b’s are called partial regression
coefficients
• Our example-Predicting AA:
– Y’= 36.83 + (3.52)XASC + (-.44)XGSC
• Predicted AA for person with GSC of 4 and
ASC of 6
– Y’= 36.83 + (3.52)(6) + (-.44)(4) = 56.23
8. Multiple Correlation Coefficient (R)
and Coefficient of Multiple
Determination (R2)
• R = the magnitude of the relationship
between the dependent variable and the best
linear combination of the predictor variables
• R2
= the proportion of variation in Y
accounted for by the set of independent
variables (X’s).
9. Predictable variation by
the combination of
independent variables
Explaining Variation: How much?
Total Variation in Y
Unpredictable
Variation
10. Proportion of Predictable and
Unpredictable Variation
X1
Y
(1-R2
) = Unpredictable
(unexplained) variation
in Y
X2
Where:
Y= AA
X1 = ASC
X2 =GSC
R2 = Predictable
(explained)
variation in Y
11. Various Significance Tests
• Testing R2
– Test R2
through an F test
– Test of competing models (difference between
R2
) through an F test of difference of R2
s
• Testing b
– Test of each partial regression coefficient (b) by
t-tests
– Comparison of partial regression coefficients
with each other - t-test of difference between
standardized partial regression coefficients ()
12. Example: Testing R2
• What proportion of variation in AA can be
predicted from GSC and ASC?
– Compute R2
: R2
= .16 (R = .41) : 16% of the
variance in AA can be accounted for by the
composite of GSC and ASC
• Is R2
statistically significant from 0?
– F test: Fobserved = 9.52, Fcrit (05/2,100) = 3.09
– Reject H0: in the population there is a
significant relationship between AA and the
linear composite of GSC and ASC
13. Example: Comparing Models -
Testing R2
• Comparing models
– Model 1: Y’= 35.37 + (3.38)XASC
– Model 2: Y’= 36.83 + (3.52)XASC + (-.44)XGSC
• Compute R2
for each model
– Model 1: R2
= r2
= .160
– Model 2: R2
= .161
• Test difference between R2
s
– Fobs = .119, Fcrit(.05/1,100) = 3.94
– Conclude that GSC does not add significantly
to ASC in predicting AA
14. Testing Significance of b’s
• H0: = 0
• tobserved = b -
standard error of b
• with N-k-1 df
15. Example: t-test of b
• tobserved = -.44 - 0/14.24
• tobserved = -.03
• tcritical(.05,2,100) = 1.97
• Decision: Cannot reject the null hypothesis.
• Conclusion: The population for GSC is not
significantly different from 0
16. Comparing Partial Regression
Coefficients
• Which is the stronger predictor? Comparing
bGSC and bASC
• Convert to standardized partial regression
coefficients (beta weights, ’s)
GSC = -.038
ASC = .417
– On same scale so can compare: ASC is stronger
predictor than GSC
• Beta weights (’s ) can also be tested for
significance with t tests.
17. Different Ways of Building Regression
Models
• Simultaneous: all independent variables
entered together
• Stepwise: independent variables entered
according to some order
– By size or correlation with dependent variable
– In order of significance
• Hierarchical: independent variables entered
in stages
18. Practice:
• Grades reflect academic achievement, but also
student’s efforts, improvement, participation, etc.
Thus hypothesize that best predictor of grades
might be academic achievement and general self
concept. Once AA and GSC have been used to
predict grades, academic self-concept (ASC) is not
expected to improve the prediction of grades (I.e.
not expected to account for any additional
variation in grades)
Editor's Notes
#6:Shavelson, Text example p 530 (Table 18.1) (Example 18.1 p 538)
A study of the relation between academic achievement (AA), grades, and general and academic self concept.
General Self concept- one’s perception of him/her self.
Multifaceted: perception of behavior is the base moving to inferences about self in sub-areas (e.g. physical (appearance, ability), academic (math, history) , social , and then to inferences about self in non-academic and academic areas and then to inferences about self in general.
Academic self concept: one’s perception of self in academic areas
The theory leads us to predict that AA and ASC will be more closely related than AA and GSC, But relationship of grades and these other variables is not so clear
Grades certainly reflect academic achievment, but also student’s efforts, improvement and participation. So hypothesize that best predictor of grades might be AA and GSC. Once AA and GSC have been used to predict grades, academic self-concept not expected to improve the prediction of grades (I.e. ASC not expected to account for any additional variation in grades.
MRA allows us to statistically examine these predictions from self-concept theory.
