Applied Econometrics 4th edition

Applied Econometrics 4th edition

MULTIPLE REGRESSION

1. The Multiple Regression Model

2. The OLS Method of Estimation
3. The R2 and the Adjusted R2
4. Hypothesis Testing
5. How to Estimate a Simple Regression in EViews
 Applied Econometrics 4th edition

Learning Objectives
• Derive mathematically the regression coefficients of a
multiple regression model.
• Understand the difference between the R2 and the adjusted
R2 for a multiple regression model.
• Appreciate the importance of the various selection criteria
for the best regression model.
• Conduct hypothesis testing and test linear restrictions,
omitted and redundant variables as well as the overall
significance of the explanatory variables.
 Applied Econometrics 4th edition

Learning Objectives (2)

• Obtain the output of a multiple regression

estimation using econometric software.
• Interpret and discuss the results of a multiple
regression estimation output.
 Applied Econometrics 4th edition

• Definition of the multiple linear regression model

"Explains variable in terms of variables "

Intercept Slope parameters

Dependent variable,
explained variable, Error term,
Independent variables, disturbance,
response variable,… explanatory variables, unobservables,…
regressors,…
 Applied Econometrics 4th edition

• Motivation for multiple regression

– Incorporate more explanatory factors into the model

– Explicitly hold fixed other factors that otherwise would be in
– Allow for more flexible functional forms

• Example: Wage equation

Now measures effect of education explicitly holding experience fixed

All other factors…

Hourly wage Years of education Labor market experience

 Applied Econometrics 4th edition

• Example: Average test scores and per student spending

Other factors

Average standardized Per student spending Average family income

test score of school at this school of students at this school

– Per student spending is likely to be correlated with average family income at

a given high school because of school financing
– Omitting average family income in regression would lead to biased estimate
of the effect of spending on average test scores
– In a simple regression model, effect of per student spending would partly
include the effect of family income on test scores
 Applied Econometrics 4th edition

• Example: Family income and family consumption

Other factors

Family consumption Family income Family income squared

– Model has two explanatory variables: inome and income squared

– Consumption is explained as a quadratic function of income
– One has to be very careful when interpreting the coefficients:
By how much does consumption Depends on how
increase if income is increased much income is
by one unit? already there
 Applied Econometrics 4th edition

• Example: CEO salary, sales and CEO tenure

Log of CEO salary Log sales Quadratic function of CEO tenure with firm

– Model assumes a constant elasticity relationship between CEO salary and

the sales of his or her firm

– Model assumes a quadratic relationship between CEO salary and his or her
tenure with the firm

• Meaning of "linear" regression

– The model has to be linear in the parameters (not in the variables)

 Applied Econometrics 4th edition

• OLS Estimation of the multiple regression model

• Random sample

• Regression residuals

• Minimize sum of squared residuals

Minimization will be carried out by computer

 Applied Econometrics 4th edition

• Interpretation of the multiple regression model

By how much does the dependent variable change if the j-th

independent variable is increased by one unit, holding all
other independent variables and the error term constant

– The multiple linear regression model manages to hold the values of other
explanatory variables fixed even if, in reality, they are correlated with the
explanatory variable under consideration
– "Ceteris paribus"-interpretation
– It has still to be assumed that unobserved factors do not change if the
explanatory variables are changed
 Applied Econometrics 4th edition

Multiple Regression Derivation of the OLS

• The three variables case:
 Applied Econometrics 4th edition

• The k-variables case:

– Requires matrix algebra and it is quite complicated
– Luckily, R, Eviews, Microfit and Stata give results
very quickly and efficiently (always correct
calculations)
 Applied Econometrics 4th edition

Assumptions of the Multiple

Regression Model
Applied Econometrics 4th edition
Applied Econometrics 4th edition
Applied Econometrics 4th edition
Applied Econometrics 4th edition
 Applied Econometrics 4th edition

• Example: Determinants of college GPA

Grade point average at college High school grade point average Achievement test score

• Interpretation
– Holding ACT fixed, another point on high school grade point average is associated with
another .453 points college grade point average
– Or: If we compare two students with the same ACT, but the hsGPA of student A is one
point higher, we predict student A to have a colGPA that is .453 higher than that of
student B
– Holding high school grade point average fixed, another 10 points on ACT are associated
with less than one point on college GPA
 Applied Econometrics 4th edition

2 2
R and adjusted R
• R2 measures the goodness of fit as in Simple
Regression
• However, it cannot be used for comparing two
different equations containing different numbers of
explanatory variables.
• When adding more explanatory variables R2, will
always be increased.
• Therefore, we need a different measure (the adjusted
R2)
 Applied Econometrics 4th edition

Criteria for Model Selection

R2: Measures the proportion of the variation in the regress and explained by
the regressors- ESS/TSS = 1 − RSS/TSS

Adjusted R2: Denoted as , it takes degrees of freedom into account:

 Applied Econometrics 4th edition

Criteria for Model Selection

Akaike’s Information Criterion (AIC): Adds harsher penalty for adding

more variables to the model, defined as:

The model with the lowest AIC is usually chosen.

Schwarz’s Information Criterion (SIC): Alternative to the AIC criterion,
expressed as:

The penalty factor here is harsher than that of AIC.

 Applied Econometrics 4th edition

Criteria for Model Selection

• Finite Prediction Error (FPE)

• Schwarz Bayesian Criterion (SBC)

• Hannan and Quin Criterion (HQC)

Applied Econometrics 4th edition

Multiple Regression
 Applied Econometrics 4th edition

Hypothesis Testing
• Testing Individual Coefficients (t-tests)
• Testing for Linear Restrictions (Wald Test)
– Cobb Douglas Production Function
• Testing for the Overall Significance (F-test)
• Testing for Omitted Variables (Wald Test)
• Testing for Redundant Variables (Wald Test)
– Explain all the tests on board…
 Applied Econometrics 4th edition

Testing Multiple Hypotheses: The F-test

• We used the t-test to test single hypotheses, i.e. hypotheses involving only
one coefficient. But what if we want to test more than one coefficient
simultaneously?

• We do this using the F-test. The F-test involves estimating 2 regressions.

• The unrestricted regression is the one in which the coefficients are freely
determined by the data, as we have done before.

• The restricted regression is the one in which the coefficients are restricted,
i.e. the restrictions are imposed on some βs.
 Applied Econometrics 4th edition
Summary
 F-testApplied
for joint significance of several
Econometrics 4th edition
coefficients example
• Example:
 Applied Econometrics 4th edition

Hypothesis Testing
• Testing for the Overall Significance (F-test)
Applied Econometrics 4th edition
 Applied Econometrics 4th edition

• Adding or deleting explanatory variables

• Consider the following model:

• The F-type test, as we explained above, is based on the difference of the RSS of
the restricted and unrestricted regressions. The LR-statistic is computed as:
• LR = −2(lR − lU)
• where lR and lU are the maximized values of the log-likelihood function of the
unrestricted and restricted equations, respectively. The LR-statistic follows a χ2
distribution with degrees of freedom equal to the number of restrictions (that is the
number of omitted or added variables).
Applied Econometrics 4th edition