Chapter 5:

Further Issues with Multiple Linear Regression

5.1. Changing the scale of variables

5.2. More on functional form

5.3. More on goodness-of-fit and selection of

regressors

5.4. Predictions and residual analysis

5.5. Heteroskedasticity
5.1. Changing the scale of variables
(Changing the unit of measurement)
Change in the measurement unit of the dependent variable 𝑌 Intercept 𝜷𝟎 Slope coefficient 𝛽1
Y is divided by the constant 𝐶: 𝑌/𝐶 𝜷𝟎 /𝐂 𝜷𝟏 /𝐂
Y is multiplied by the constant 𝐶: 𝑌∗𝐶 𝜷𝟎 ∗ 𝐂 𝜷𝟎 ∗ 𝐂

𝑤𝑎𝑔𝑒
ෟ = 608 + 69.6edu wage is measured in 1.000 VND
𝑤𝑎𝑔𝑒
ෟ = 0.608 + 0.0696𝑒𝑑𝑢 wage is measured in 1.000.000 VND: Wage/1000

Change in the measurement unit of the independent variable 𝐗 Intercept 𝜷𝟎 Slope coefficient 𝛽1
X is divided by the constant 𝑪: 𝐗/𝐶 𝜷𝟎 𝛽1 ∗ 𝐶
X is multiplied by the constant 𝑪: 𝐗∗𝐶 𝜷𝟎 𝛽0 /𝐶

𝑤𝑎𝑔𝑒
ෟ = 608 + 70edu education is measured in years.
𝑤𝑎𝑔𝑒
ෟ = 608 + 𝟓. 𝟖𝑒𝑑𝑢 education is measured in months. Edu*12
Changing the unit of measurement
(Cont)

❑ Changing the scale of Y causes a change in the magnitude

of the intercept and slope coefficients and standard
errors, but no change in the statistical significance and
interpretation.
❑ Changing the scale of X causes a change in the magnitude
of the slope coefficient and standard errors, but no change
in the statistical significance and interpretation.
❑ If Y is in log form, changing the scale of Y affects the
intercept but does not affect the slope coefficient
❑ If X is in log form, changing the scale of X affects the
intercept but does not affect the slope coefficient
Standardized or beta coefficient
(Cont)

❑ There are times when the terms "standardized coefficient" or

"beta coefficient" are used, and they have a special meaning.
❑ Replace Y and each X variable with a standardized version:
𝒙𝒊𝒋 −𝒙ഥ𝒋 𝒚𝒊 −ഥ
𝒚
❑ 𝒁𝒙𝒋 = ෞ𝒋
; 𝒁𝒚 = ෝ𝒚
𝝈𝒙 𝝈

❑ Interpretation: If 𝒙 increases by one standard deviation, 𝑦ො changes by

beta standard deviations .
❑ The result allows us to compare the strength of the effect of
each explanatory variable on the dependent variable.
❑ Easy to do with stata: reg y x1 x2 x3, beta
Beta coefficient
(Cont)
It is not easy to compare the effect of each Xj on Y
when looking at the coefficients (unstandardized).

It is easier to compare the effect when looking at the standardized coefficients (beta).
Income has the largest standardized effect, while household size has the smallest standardized effect.

One standard deviation increase in income raises consumption by 0.716 standard deviation.
Beta coefficient
The value of beta coefficients (standardized coefficient) in Table 4 indicates that
forest land plays a larger role in household income than other land types.
5.2. More on functional form

Logarithmic Functional Forms

Benefits:

Simple interpretation of percentage and elasticity

Log variables' slope coefficients are not affected by rescalings.
Taking log frequently prevents or reduces issues with outliers.
Often, taking log aids in ensuring normality and homoscedasticity.

Some notes:
The log of variables with units like years should be avoided.
Additionally, variables expressed in percentage points shouldn't be taken log
Do not take log if the variables contain zero or negative values
It is challenging to predict y by exponentiating the predicted value of log(y).
5.2. More on functional form
 Quadratic Function: y = b0 + b1x + b2x2 + u
 This functional form captures decreasing or increasing marginal effects
 We must not interpret b1 alone as measuring the change in Y with respect to X; we
need to take into account b2 and the value of X as well, since:
∆𝑦ො ≈ 𝛽෠1 + 2𝛽෠2 𝑥 ∆𝑥, 𝑠𝑜 ∆𝑦/∆𝑥
ො ≈ 𝛽෠1 + 2𝛽෠2 𝑥
If X increases by one unit, Y changes by ( 𝛽෠1 + 2𝛽෠2 𝑥) units
Wage = b0+b1𝑒𝑥𝑝𝑒𝑟 + b2𝑒𝑥𝑝𝑒𝑟 2 + u
෣ = 5367.546 + 225.8147𝑒𝑥𝑝𝑒𝑟 − 4.910445𝑒𝑥𝑝𝑒𝑟 2
𝑊𝑎𝑔𝑒

Experience has a diminishing effect on wage

5.2. More on functional form
∆𝑦ො ≈ 𝛽መ1 + 2𝛽መ2 𝑥 ∆𝑥

 ෟ = 5367.546 + 225.8147𝑒𝑥𝑝𝑒𝑟 − 4.910445𝑒𝑥𝑝𝑒𝑟 2

𝑤𝑎𝑔𝑒
 ∆𝑤𝑎𝑔𝑒
ෟ = [225.8147 − 2 4.910445 𝑒𝑥𝑝𝑒𝑟]∆𝑒𝑥𝑝𝑒𝑟
 ∆𝑤𝑎𝑔𝑒
ෟ = (225.8147 − 9.82089𝑒𝑥𝑝𝑒𝑟)∆𝑒𝑥𝑝𝑒𝑟
A calculus approximation: An increase in experience from 10 to 11 years increases wages by about:
225.8147 − 9.82089 ∗ 10 = 127.6058

Stata command: margins, at(exper=10) dydx(exper)

Note: exact calculation: ∆𝑤𝑎𝑔𝑒

ෟ = 𝛽መ1 𝑒𝑥𝑝𝑒𝑟11 − 𝛽መ2 𝑒𝑥𝑝𝑒𝑟11 2 − 𝛽መ1 𝑒𝑥𝑝𝑒𝑟10 − 𝛽መ2 𝑒𝑥𝑝𝑒𝑟10 2 = 122.69535

Stata command: di (225.8147*11-4.910445*11^2)-(225.8147*10-4.910445*10^2)=122.69535

∗ 𝛽1
the turning point is 𝑋 = | |= 225.8147 / 2 ∗ 4.910445 = 22.993303 ≈23 (years)
2𝛽2

Note: If 𝑥 = 0, then 𝛽መ1 can be interpreted as the approximate slope in going from 0 to 1. After that 2𝛽መ2 𝑥
must be accounted for.
5.2. More on functional form
With 𝛽መ1 >0 and 𝛽መ2 <0 , the slope is initially positive, but it diminishes when X increases. X* is
the maximum of the function, and after X*, the slope is negative.

Y= b0 + b1x -b2x2 + u

𝛽෠1 >0 and 𝛽෠2 <0: diminishing effect of X on Y: Inverted U-shape or a parabolic shape
5.2. More on functional form
With 𝛽መ1 <0 and 𝛽መ2 >0 , the slope is initially negative, but it increases (less negative) when X
increases. X* is the minimum of the function, and after X*, the slope is positive.

Y= b0 -b1x +b2x2 + u

𝛽෠1 <0 and 𝛽෠2 >0: increasing effect of X on Y: U-shape

 ෟ = 5367.546 + 225.8147𝑒𝑥𝑝𝑒𝑟 − 4.910445𝑒𝑥𝑝𝑒𝑟 2
𝑤𝑎𝑔𝑒
Stata command
use "D:\Bai giảng kinh te luong_ISVNU_2022\Data for teaching_2023\Chapter5.dta", clear
reg wage c.exper##c.exper
margins, at(exper=(0(1)46))
Reg wage exper exper2
marginsplot
margins, at(exper=22.993303) dydx(exper)
margins, at(exper=15) dydx(exper)

Slope at exper=23 is zero

Slope at exper=15 is 78.5

Quadratic function with the log (y)
෣ = 8.518096 + 0.0319873𝑒𝑥𝑝𝑒𝑟 − 0.0007281𝑒𝑥𝑝𝑒𝑟 2
𝑙𝑛𝑤𝑎𝑔𝑒
A calculus approximation: ≈ 100*[ 𝛽መ1 + 2𝛽መ2 𝑥 ]∆𝑥
%∆𝑤𝑎𝑔𝑒
ෟ ≈ 100 ∗ [0.0319873 − 2 ∗ 0.0007281 𝑒𝑥𝑝𝑒𝑟 ] ∆𝑒𝑥𝑝𝑒𝑟
An increase in experience from 10 to 11 year increases wages by about:
100*[(0.0319873 − 2 ∗ 0.0007281 ∗ 10) = 1.74253%
reg lnwage c.exper##c.exper
margins, at(exper=10) dydx(exper)

Exact calculation: di (0.0319873*11-0.0007281*11^2)-(0.0319873*10-0.0007281*10^2)= .0166972 ≈ 1.67%

H𝐨𝐰 to test a quadratic relationship?
෣ = 8.518096 + 0.0319873𝑒𝑥𝑝𝑒𝑟 − 0.0007281𝑒𝑥𝑝𝑒𝑟 2
𝑙𝑛𝑤𝑎𝑔𝑒
reg lnwage exper exper2
utest exepr exper2

Ho is rejected in favour of H1
There is an inverse U shape: diminishing effect
5.3. More on goodness-of-fit and selection of regressors
 Goodness of Fit: The adjusted R2: 𝑅ത 2
❑ Note that the R2 has a limitation that it will always increase when adding
more explanatory variables to the model.

❑ However, 𝑅ത 2 takes into account the number of variables in a model, and

it may decrease This corrects degrees of freedom of
numerator and denominator

(𝑅𝑆𝑆/(𝑛 − 𝑘 − 1))
𝑅ത 2 = 1 − = 𝑎𝑑𝑗𝑢𝑠𝑡𝑒𝑑 𝑅2
𝑇𝑆𝑆Τ(𝑛 − 1)
❑ 𝑅ത 2 increases if, and only if, the t-statistic of a newly added variable is
greater than one in absolute value.

❑ It is reasonable to compare the fit of 2 models (with the same y) by

comparing 𝑅 ത2
❑ ത 2 to compare models with different dependent
We cannot use 𝑅
variables:
E.g., income vs log(income)
Selection of regresssors
(Cont)
❑ One should not include too many other
independent variables, especially those that are
less relevant.
❑ Adding more (relevant) regressors reduces the
error variance but can lead to multicollinearity
issues.
❑ Relevant variables that are uncorrelated with
the explanatory variables of interest should be
added because they can reduce the error
variance without increasing multicollinearity.
 Selection of regresssors
Four criteria for selecting regressors

❑ Theory: Is the included variable ambiguous or

theoretically sound?
❑ t-test: Is the included variable’s coefficient
statistically significant with the expected sign?
❑ Adjusted R squared (𝑅 ത 2 ): Does 𝑅ത 2 improve when
the variable is included in the model?
❑ Bias: Do other independent variables’
coefficients substantially change when including
the variable in the model?
Selection of regresssors
Four criteria for selecting regressors
5.4 Predictions and residual analysis
❑ Predicting y in a log model

 Exponentiating the predicted values of log(y) will underestimate the expected value
of y
෣
𝑦ො = exp(𝑙𝑜𝑔𝑦)
 We need to scale this up by an estimate of the expected value of exp(u):

𝜎2
 𝑇ℎ𝑒 𝑒𝑥𝑝𝑒𝑐𝑡𝑒𝑑 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 exp 𝑢 = exp 𝒊𝒇𝒖 is normally distributed
2

෣ exp ෝ2
𝜎
The predicted value of Y: 𝑦ො = exp 𝑙𝑜𝑔𝑦 .
2
❑ 𝜎ො 2 : the estimated error variance which is provided by the regression results.
ෝ2
𝜎 1 𝑛
❑ Because 𝜎ො 2 > 0, exp 2
>1 ෝ𝟐 =
𝝈 ෍ 𝑢ො 𝑖2
𝑛 − 𝑘 − 1 𝑖=1
 ෝ2
𝜎
The predicted value of Y: 𝑦 ෣ exp
ො = exp 𝑙𝑜𝑔𝑦
Example: 2

reg lnwage c.edu##c.edu reg lnwage edu edu2

predict lnwage_hat
gen wage_hat = exp(lnwage_hat)*exp(.42958^2/2)
or we can type: gen wage_hat = exp(lnwage_hat)*exp((`e(rmse)'^2)/2)

ෝ=
𝝈 𝜎ො 2 : 𝐑𝐨𝐨𝐭 𝐦𝐞𝐚𝐧 𝐬𝐪𝐮𝐚𝐫𝐞𝐝 𝐞𝐫𝐫𝐨𝐫
Standard error of the regression

1 𝑛
ෝ𝟐 =
𝝈 ෍ 𝑢ො 𝑖2
𝑛 − 𝑘 − 1 𝑖=1
Estimated variance of the error term
Predicting y in a log model

(Cont)

 If u is not normal, E(exp(u)) must be estimated using

an auxiliary regression.
 reg lnwage c.edu##c.edu
 predict lnwage_hat
Create the exponentiation of the predicted ln(wage)
 gen eyhat=exp(lnwage_hat)
Regress wage on eyhat with no intercept
 reg wage eyhat, noconstant
The coefficient on“eyhat: 1.107073” is the estimate of E(exp(u)) that can be
used to scale up the exponentiation of the predicted ln(y) to derive the
predicted wage.
 gen wage_hat = exp(lnwage_hat)*1.107073
Comparing level and log models
reg wage edu vs reg lnwage edu
 Comparing log and level models
reg lnwage edu vs reg wage edu
 ෣ “eyhat” from the log model:
To obtain exp(𝑙𝑜𝑔𝑦)
reg lnwage edu
predict lnwage_hat
gen eyhat=exp(lnwage_hat)
 To regress “wage” on “eyhat” without intercept and then
predict the fitted value of wage “wage_hat”:
reg wage eyhat, noconstant
predict wage_hat
 To calculate the sample correlation between “wage” and
“wage_hat”
corr wage wage_hat
 To square the correlation coefficient: 0.3622^2= 0.13118884
 To compare R squared (0.1190) from the level regression with
the squared correlation coefficient (0.13118884)
 0.1190<0.1312: the log model is better
5.5. Heteroskedasticity:
Var(𝑈|𝑥1, 𝑥2 … 𝑥𝑘 ) is unconstant
❑ Recall from chapter 2 that the assumption of
homoskedasticity implies that, conditional on the
explanatory variables, the variance of the
unobserved error is constant: Var(𝑈|𝑋)=𝝈𝟐
❑ If this does not hold, that is, if the variance of u
varies for varying values of the x’s, then the errors
are heteroskedastic.
Example: in the wage equation, ability is an
unobservable factor, and the variance in ability
differs by educational level.
𝑯𝒆𝒕𝒆𝒓𝒐𝒔𝒌𝒆𝒅𝒂𝒔𝒕𝒊𝒄𝒊𝒕𝒚: The variation of the ability
around the mean is unconstant, increasing with
education level
 What happens if heteroskedasticity occurs?

❑ Heteroskedasticity does not affect the OLS's unbiasedness

and consistency.
❑ The interpretation of R-squared is not changed under
heteroskedasticity.
Consequences of heteroskedascity:
1. The variance formulas for OLS estimators become invalid
under heteroskedasticity.
2. Heteroskedascity makes the F-tests and t-tests invalid
3. Under heteroskedasticity, OLS is no longer the BLUE (best
linear unbiased estimator), and there may be more efficient
linear estimators.
 Heteroskedasticity
(Cont)
 We won’t discuss about the theory behind heteroskedasticity (see Wooldridge,
Chapter 8 for a thorough discussion).
 Fortunately, formulas that are robust to heteroscedasticity of unknown form have
been developed for OLS standard errors and related statistics.

2 2
σ𝑛
𝑖=1 𝑟Ƹ 𝑖𝑗 𝑢
ෝ𝑖
White/Eicker standard errors: ෢ 𝛽መ𝑗 )=
𝑉𝑎𝑟(
𝑅𝑆𝑆𝑗2

Ƹ 2 denotes 𝑡ℎ𝑒 𝑖 𝑠𝑞𝑢𝑎𝑟𝑒𝑑 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 from a regression of 𝑥𝑗 on all other

𝑟𝑖𝑗
explanatory variables while 𝑅𝑆𝑆𝑗2 shows the sum of the squared residuals
from this regression.

 All formulas are only valid in large samples.

 These formulas allow the standard t-test to be asymptotically valid.
 The standard F-statistic does not work in the presence of the heroscedasticity,
however most softwares provide heteroscedasticity robust versions.
 Heteroskedasticity
(Cont)

 There’s a bunch of statistical tests to dedect

heteroskedasticity
 The tests use the information about u that is
contained in the residuals from OLS regression
 Therefore, we always have to run the OLS regression
first
A pair of hypotheses is formulated
Ho: Homoskedasticity or constant variance
H1: Heteroskedasticity or unconstant variance
A p-value<0.05 indicates the presence of heteroskedasticity
Test for heteroscedasticity
reg wage edu exper female married
hettest
estat imtest, white

Breusch-Pagan test for heteroscedasticity White test for heteroscedasticity

Ho is rejected in both tests, which show the presence of heteroskedasticity

 How to do with heteroskedasticity?
(Cont)

❑ Log-transformation.

❑ Using the “robust” option provided by

Stata.
❑ Using other linear regression estimator
(FGLS, [1, chapter 8]): FGLS: Feasible General Least Squares Estimator).
Log-transformation and heteroskedasticity
Level model: wage Log-model: log(wage)
reg wage edu exper female married reg lwage edu exper female married

Ho is rejected Ho is not rejected

Heteroskedastcity in Level and log-models
stata command: rvfplot
Wage Log(wage)

Unconstant variance Constant variance

 Robust standard errors
Heteroscedasticity robust standard errors may be greater or smaller than
their nonrobust counterparts. The differences are often small in practice.

Without “robust” With “robust”

Much differences in the F-test indicate that significant heteroscedasticity is present.

It is recommended to always compute robust standard errors to be on the safe side.
 Exercise 1
 Use "Chapter5.dta."
❑ Please estimate the quadratic relationship between education
and wages.
❑ To test the hypothesis about this relationship using “utest”
❑ Please indicate the year of schooling at which the predicted wage
is at its minimum.
❑ Please interpret the effect of education on wages if education
increases from 10 to 11.
❑ Estimate the slope coefficient of the education variable at year
12
❑ Estimate the slope coefficient of the education variable at year 5
 Exercise 2
Use "Chapter5.dta."
 Please regress wage on gender married edu exper mismatch
❑ Compare the relative importance of each variable in the model
 Please regress lnwage on gender married edu exper mismatch
❑ Compare the goodness of fit between the log and level model.
❑ In the level model, what happens if we change the measurement
units of wage from thousand VND to million VND?
❑ In the log model, what happens if we change the measurement
units of edu from years to months (e.g.,1year=10 months)