Classical linear regression model assumptions

and diagnostics

1
 Violation of the Assumptions of the CLRM

• Recall that we assumed of the CLRM disturbance terms:

1. E(ut) = 0
2. Var(ut) = 2 < 
3. Cov (ui,uj) = 0
4. The X matrix is non-stochastic or fixed in repeated samples
5. ut  N(0,2)
 Investigating Violations of the
Assumptions of the CLRM
• We will now study these assumptions further, and in particular look at:
- How we test for violations
- Causes
- Consequences
in general we could encounter any combination of 3 problems:
- the coefficient estimates are wrong
- the associated standard errors are wrong
- the distribution that we assumed for the
test statistics will be inappropriate
- Solutions
- the assumptions are no longer violated
- we work around the problem so that we
use alternative techniques which are still valid
 Statistical Distributions for Diagnostic Tests

• Often, an F- and a 2- version of the test are available.

• The F-test version involves estimating a restricted and an unrestricted

version of a test regression and comparing the RSS.

• The 2- version is sometimes called an “LM” test, and only has one degree
of freedom parameter: the number of restrictions being tested, m.

• Asymptotically, the 2 tests are equivalent since the 2 is a special case of

the F-distribution: 2
 m 
 F m, T  k  as T  k  
m
• For small samples, the F-version is preferable.
 Assumption 1: E(ut) = 0
• Assumption that the mean of the disturbances is zero.

• For all diagnostic tests, we cannot observe the disturbances and so perform the tests of
the residuals.

• The mean of the residuals will always be zero provided that there is a constant term in
the regression.

• If the regression did not include an intercept, and the average value of the errors was
nonzero, several undesirable consequences could arise. First, R2, defined as ESS/TSS
can be negative, implying that the sample average, ¯y, ‘explains’ more of the variation
in y than the explanatory variables. Second, and more fundamentally, a regression with
no intercept parameter could lead to potentially severe biases in the slope coefficient
estimates
 Assumption 2: Var(ut) = 2 < 

• We have so far assumed that the variance of the errors is constant, 2 - this
is known as homoscedasticity. If the errors do not have a constant
variance, we say that they are heteroscedastic e.g. say we estimate a
regression and calculate the residuals,ut .
t û +

x2t

-
 Detection of Heteroscedasticity: The GQ Test

• Graphical methods

• Formal tests: There are many of them: we will discuss Goldfeld-Quandt test and
White’s test

The Goldfeld-Quandt (GQ) test is carried out as follows.

1. Split the total sample of length T into two sub-samples of length T1 and T2. The
regression model is estimated on each sub-sample and the two residual
variances are calculated.

2. The null hypothesis is that the variances of the disturbances are equal,
H 0: 2
1   22
 The GQ Test (Cont’d)

4. The test statistic, denoted GQ, is simply the ratio of the two residual
variances where the larger of the two variances must be placed in the
numerator.
s12
GQ  2
s2

5. The test statistic is distributed as an F(T1-k, T2-k) under the null of

homoscedasticity.

A problem with the test is that the choice of where to split the sample is
that usually arbitrary and may crucially affect the outcome of the test.
 Detection of Heteroscedasticity using White’s Test

• White’s general test for heteroscedasticity is one of the best approaches

because it makes few assumptions about the form of the
heteroscedasticity.

• The test is carried out as follows:

1. Assume that the regression we carried out is as follows

yt = 1 + 2x2t + 3x3t + ut
And we wantut to test Var(u ) = 2. We estimate the model, obtaining the
t
residuals,

2. Then u 1auxiliary
ˆt2 the
run   2 x2t regression
  3 x3t   4 x22t   5 x32t   6 x2t x3t  vt
 Performing White’s Test for Heteroscedasticity

3. Obtain R2 from the auxiliary regression and multiply it by the

number of observations, T. It can be shown that
T R2  2 (m)
where m is the number of regressors in the auxiliary regression
excluding the constant term.

4. If the 2 test statistic from step 3 is greater than the corresponding

value from the statistical table then reject the null hypothesis that the
disturbances are homoscedastic.
 White’s test in Eviews

• To test for heteroscedasticity using White’s test, click on the View

button in the regression window and select Residual
Tests/Heteroscedasticity Tests
• Select the White specification. You can also select whether to include
the cross-product terms or not (i.e. each variable multiplied by each
other variable) or include only the squares of the variables in the
auxiliary regression. Check/uncheck the ‘Include White cross terms’
given the relatively small/large number of variables in the regression
and then click OK.
 Consequences of Using OLS in the Presence of
Heteroscedasticity

• OLS estimation still gives unbiased coefficient estimates, but they are
no longer BLUE.

• This implies that if we still use OLS in the presence of

heteroscedasticity, our standard errors could be inappropriate and
hence any inferences we make could be misleading.

• Whether the standard errors calculated using the usual formulae are
too big or too small will depend upon the form of the
heteroscedasticity.
 How Do we Deal with Heteroscedasticity?

• If the form (i.e. the cause) of the heteroscedasticity is known, then we can
use an estimation method which takes this into account (called generalised
least squares, GLS).

• A simple illustration of GLS is as follows: Suppose that the error variance

is related to another variable zt by
varut    2 zt2
• To remove the heteroscedasticity, divide the regression equation by zt
yt 1 x x
 1   2 2t   3 3t  vt
u zt zt zt zt
vt  t
where zt is an error term.
 ut  varut   2 zt2
• Now var vt   var   2
 2
  2
for known zt.
 zt  z t z t
 Other Approaches to Dealing
with Heteroscedasticity
• So the disturbances from the new regression equation will be
homoscedastic.

• Other solutions include:

1. Transforming the variables into logs or reducing by some other measure
of “size”.

2. Use White’s heteroscedasticity consistent standard error estimates.

The effect of using White’s correction is that in general the standard errors
for the slope coefficients are increased relative to the usual OLS standard
errors.
This makes us more “conservative” in hypothesis testing, so that we would
need more evidence against the null hypothesis before we would reject it.
 Assumption 3 Cov (ui,uj) = 0

• Assumption 3 that is made of the CLRM’ disturbance terms is that the

covariance between the error terms over time (or cross-sectionally, for
that type of data) is zero. In other words, it is assumed that the errors
are uncorrelated with one another.
• If the errors are not uncorrelated with one another, it would be stated
that they are ‘autocorrelated’ or that they are ‘serially correlated’.
 Background –
The Concept of a Lagged Value

t yt yt-1 yt
1989M09 0.8 - -
1989M10 1.3 0.8 1.3-0.8=0.5
1989M11 -0.9 1.3 -0.9-1.3=-2.2
1989M12 0.2 -0.9 0.2--0.9=1.1
1990M01 -1.7 0.2 -1.7-0.2=-1.9
1990M02 2.3 -1.7 2.3--1.7=4.0
1990M03 0.1 2.3 0.1-2.3=-2.2
1990M04 0.0 0.1 0.0-0.1=-0.1
. . . .
. . . .
. . . .
 Autocorrelation

• We assumed of the CLRM’s errors that Cov (ui , uj) = 0 for ij, i.e.
This is essentially the same as saying there is no pattern in the errors.

• Obviously we never have the actual u’s, so we use their sample

counterpart, the residuals (the ut’s).

• If there are patterns in the residuals from a model, we say that they are
autocorrelated.

• Some stereotypical patterns we may find in the residuals are given on

the next 3 slides.
 Positive Autocorrelation

+
û t ût
+

- +
uˆ t 1 Time

Positive Autocorrelation is indicated by a cyclical residual plot over time.

 Negative Autocorrelation

+ ût
û t
+

- +
uˆ t 1 T
ime

- -

Negative autocorrelation is indicated by an alternating pattern where the residuals

cross the time axis more frequently than if they were distributed randomly
 No pattern in residuals –
No autocorrelation
û t
+
ût +

- +
uˆt 1 Time

-
-

No pattern in residuals at all: this is what we would like to see

 Detecting Autocorrelation:
The Durbin-Watson Test

The Durbin-Watson (DW) is a test for first order autocorrelation - i.e.

it assumes that the relationship is between an error and the previous
one
ut = ut-1 + vt (1)
where vt  N(0, v2).

• The DW test statistic actually tests

H0 : =0 and H1 : 0

• The test statistic is calculated by T

  ut  ut 1 2
DW  t  2 T
 ut 2
t 2
 The Durbin-Watson Test:
Critical Values
• We can also write
DW  2(1   ) (2)
where  is the estimated correlation coefficient. Since  is a
correlation, it implies that  1  pˆ  1.
• Rearranging for DW from (2) would give 0DW4.

• If  = 0, DW = 2. So roughly speaking, do not reject the null

hypothesis if DW is near 2  i.e. there is little evidence of
autocorrelation

• Unfortunately, DW has 2 critical values, an upper critical value (du)

and a lower critical value (dL), and there is also an intermediate region
where we can neither reject nor not reject H0.
 The Durbin-Watson Test: Interpreting the Results

Conditions which Must be Fulfilled for DW to be a Valid Test

1. Constant term in regression
2. Regressors are non-stochastic
3. No lags of dependent variable
 Another Test for Autocorrelation:
The Breusch-Godfrey Test
• It is a more general test for rth order autocorrelation:
ut  1ut 1  2 ut  2  3ut  3 ... r ut  r  vt , vt N(0, v2 )
• The null and alternative hypotheses are:
H0 : 1 = 0 and 2 = 0 and ... and r = 0
H1 : 1  0 or 2  0 or ... or r  0
• The test is carried out as follows:
u
1. Estimate the linear regression using OLS and obtain the residuals, t
u u , u ,..., ut  r
2. Regress t on all of the regressors from stage 1 (the x’s) plus t 1 t  2
Obtain R2 from this regression.
3. It can be shown that (T-r)R2  2(r)
• If the test statistic exceeds the critical value from the statistical tables, reject
the null hypothesis of no autocorrelation.
 Consequences of Ignoring Autocorrelation
if it is Present

• The coefficient estimates derived using OLS are still unbiased, but
they are inefficient, i.e. they are not BLUE, even in large sample sizes.

• Thus, if the standard error estimates are inappropriate, there exists the
possibility that we could make the wrong inferences.

• R2 is likely to be inflated relative to its “correct” value for positively

correlated residuals.
 “Remedies” for Autocorrelation

• If the form of the autocorrelation is known, we could use a GLS

procedure – i.e. an approach that allows for autocorrelated residuals
e.g., Cochrane-Orcutt.

• But such procedures that “correct” for autocorrelation require

assumptions about the form of the autocorrelation.

• If these assumptions are invalid, the cure would be more dangerous

than the disease! - see Hendry and Mizon (1978).

• However, it is unlikely to be the case that the form of the

autocorrelation is known, and a more “modern” view is that residual
autocorrelation presents an opportunity to modify the regression.
 Autocorrelation in Eviews

• The White variance--covariance matrix of the coefficients (that is,

calculation of the standard errors using the White correction for
heteroscedasticity) is appropriate when the residuals of the estimated
equation are heteroscedastic but serially uncorrelated.
• Newey and West (1987) develop a variance--covariance estimator that is
consistent in the presence of both heteroscedasticity and autocorrelation. So
an alternative approach to dealing with residual autocorrelation would be to
use appropriately modified standard error estimates.
• The Newey--West procedure requires the specification of a truncation lag
length to determine the number of lagged residuals used to evaluate the
autocorrelation. In EViews, the Newey--West procedure for estimating the
standard errors is employed by invoking it from the same place as the White
heteroscedasticity correction.
 Dynamic Models

• All of the models we have considered so far have been static, e.g.
yt = 1 + 2x2t + ... + kxkt + ut

• But we can easily extend this analysis to the case where the current
value of yt depends on previous values of y or one of the x’s, e.g.
yt = 1 + 2x2t + ... + kxkt + 1yt-1 + 2x2t-1 + … + kxkt-1+ ut

• We could extend the model even further by adding extra lags, e.g.
x2t-2 , yt-3 .
 Why Might we Want/Need To Include Lags
in a Regression?

• Inertia of the dependent variable

• Over-reactions

• However, other problems with the regression could cause the null
hypothesis of no autocorrelation to be rejected:
– Omission of relevant variables, which are themselves autocorrelated.
– If we have committed a “misspecification” error by using an
inappropriate functional form.
– Autocorrelation resulting from unparameterised seasonality.
 Models in First Difference Form

• Another way to sometimes deal with the problem of autocorrelation is to

switch to a model in first differences.

• Denote the first difference of yt, i.e. yt - yt-1 as yt; similarly for the x-
variables, x2t = x2t - x2t-1 etc.

• The model would now be

yt = 1 + 2 x2t + ... + kxkt + ut

• Sometimes the change in y is purported to depend on previous values of

y or xt as well as changes in x:
yt = 1 + 2 x2t + 3x2t-1 +4yt-1 + ut
 The Long Run Static Equilibrium Solution

• One interesting property of a dynamic model is its long run or static

equilibrium solution.

• “Equilibrium” implies that the variables have reached some steady state
and are no longer changing, i.e. if y and x are in equilibrium, we can say
yt = yt+1 = ... =y and xt = xt+1 = ... =x
Consequently, yt = yt - yt-1 = y - y = 0 etc.

• So the way to obtain a long run static solution is:

1. Remove all time subscripts from variables
2. Set error terms equal to their expected values, E(ut)=0
3. Remove first difference terms altogether
4. Gather terms in x together and gather terms in y together.

• These steps can be undertaken in any order

 The Long Run Static Equilibrium Solution:
An Example

If our model is
yt = 1 + 2 x2t + 3x2t-1 +4yt-1 + ut

then the static solution would be given by

0 = 1 + 3x2t-1 +4yt-1

4yt-1 = - 1 - 3x2t-1
 1 3
y  x2
4 4
 Problems with Adding Lagged Regressors
to “Cure” Autocorrelation

• Inclusion of lagged values of the dependent variable violates the

assumption that the RHS variables are non-stochastic. In small
samples, inclusion of lags of the dependent variable can lead to biased
coefficient estimates, although they are still consistent.

• What does an equation with a large number of lags actually mean?

• Note that if there is still autocorrelation in the residuals of a model

including lags, then the OLS estimators will not even be consistent.
 Autocorrelation and dynamic models in Eviews

• In EViews, the lagged values of variables can be used as regressors or for

other purposes by using the notation x(−1) for a one-period lag, x(−5) for a
five-period lag, and so on, where x is the variable name.
• Eviews will automatically adjust the sample period used for estimation to
take into account the observations that are lost in constructing the lags. For
example, if the regression contains five lags of the dependent variable, five
observations will be lost and estimation will commence with observation
six.
• In EViews, the DW statistic is calculated automatically, and was given in
the general estimation output screens that result from estimating any
regression model.
• The Breusch--Godfrey test can be conducted by selecting View; Residual
Tests; Serial Correlation LM Test . . . In the new window, type again the
number of lagged residuals you want to include in the test and click on OK.
 Multicollinearity

• This problem occurs when the explanatory variables are very highly
correlated with each other.

• Perfect multicollinearity
Cannot estimate all the coefficients
- e.g. suppose x3 = 2x2
and the model is yt = 1 + 2x2t + 3x3t + 4x4t + ut

• Problems if Near Multicollinearity is Present but Ignored

- R2 will be high but the individual coefficients will have high standard errors.
- The regression becomes very sensitive to small changes in the specification
(large changes in coefficient values/significances of other variables)
- Thus confidence intervals for the parameters will be very wide, and significance
tests might therefore give inappropriate conclusions.
 Measuring Multicollinearity

• The easiest way to measure the extent of multicollinearity is simply to

look at the matrix of correlations between the individual variables. e.g.

Corr x2 x3 x4
x2 - 0.2 0.8
x3 0.2 - 0.3
x4 0.8 0.3 -
• But another problem: if 3 or more variables are linear
- e.g. x2t + x3t = x4t

• Note that high correlation between y and one of the x’s is not
muticollinearity.
 Solutions to the Problem of Multicollinearity

• “Traditional” approaches, such as ridge regression or principal

components. But these usually bring more problems than they solve.

• Some econometricians argue that if the model is otherwise OK, just

ignore it. It is worth stating that the presence of near multicollinearity
does not affect the BLUE properties of the OLS estimator -- i.e. it will
still be consistent, unbiased and efficient.

• The easiest ways to “cure” the problems are

- drop one of the collinear variables
- transform the highly correlated variables into a ratio
- go out and collect more data e.g.a longer run of data or switch to a
higher frequency
 Multicollinearity in Eviews

• A correlation matrix for the independent variables can be

constructed in Eviews by clicking Quick/Group
Statistics/Correlations and then entering the list of
regressors (not including the regressand, i.e the y variable)
in the dialog box that appears.
 Adopting the Wrong Functional Form

• We have previously assumed that the appropriate functional form is linear. This
may not always be true.
• We can formally test this using Ramsey’s RESET test, which is a general test for
mis-specification of functional form.

• Essentially the method works by adding higher order terms of the fitted values
(e.g. yt2 , yt3 etc.) into an auxiliary regression:
Regress u on powers of the fitted values:
t
ut  0  1 yt2  2 yt3 ...  p 1 ytp  vt
Obtain R2 from this regression. The test statistic is given by TR2 and is
distributed as a  2 ( p  1) .

• So if the value of the test statistic is greater than a 2 then reject the null
 ( p  1)
hypothesis that the functional form was correct.
 But what do we do if this is the case?

• The RESET test gives us no guide as to what a better specification

might be.

• One possible cause of rejection of the test is if the true model is

yt  1   2 x2t   3 x22t   4 x23t  ut
In this case the remedy is obvious.

• Another possibility is to transform the data into logarithms. This will

linearise many previously multiplicative models into additive ones:

yt  Axt e ut  ln yt     ln xt  ut
 RETEST in Eviews

• Using EViews, the Ramsey RESET test is found in the

View menu of the regression window under Stability
tests/Ramsey RESET test.
• EViews will prompt you for the ‘number of fitted terms’,
equivalent to the number of powers of the fitted value to
be used in the regression; leave the default of 1 to consider
only the square of the fitted values.
 Testing the Normality Assumption

• Why did we need to assume normality for hypothesis testing?

Testing for Departures from Normality

• The Bera Jarque normality test

• A normal distribution is not skewed and is defined to have a

coefficient of kurtosis of 3.

• The kurtosis of the normal distribution is 3 so its excess kurtosis (b2-3)

is zero.

• Skewness and kurtosis are the (standardised) third and fourth moments
of a distribution.
 Normal versus Skewed Distributions

f(x ) f(x )

x x

A normal distribution A skewed distribution

 Leptokurtic versus Normal Distribution

0.5

0.4

0.3

0.2

0.1

0.0
-5.4 -3.6 -1.8 -0.0 1.8 3.6 5.4
 Testing for Normality

• Bera and Jarque formalise this by testing the residuals for normality by
testing whether the coefficient of skewness and the coefficient of excess
kurtosis are jointly zero.

• It can be proved that the coefficients

3 of skewness and 4kurtosis can be
expressed respectively as: E[u ] E[u ]
b1  b 
   
2
2 3/ 2 2 2
 and 

• The Bera Jarque test statistic is given by b 2 b  32 

W T 1  2  ~  2 
2

6 24 

u
• We estimate b1 and b2 using the residuals from the OLS regression, .
 Testing for non-normality in Eviews

• The Bera--Jarque normality tests results can be viewed by

selecting View/Residual Tests/Histogram – Normality
Test. The statistic has a χ2 distribution with 2 degrees of
freedom under the null hypothesis of normally distributed
errors.
• If the residuals are normally distributed, the histogram
should be bell-shaped and the Bera--Jarque statistic would
not be significant.
 What do we do if we find evidence of Non-Normality?

• It is not obvious what we should do!

• Could use a method which does not assume normality, but difficult and
what are its properties?

• Often the case that one or two very extreme residuals causes us to reject
the normality assumption.

• An alternative is to use dummy variables.

e.g. say we estimate a monthly model of asset returns from 1980-1990,
and we plot the residuals, and find a particularly large outlier for October
1987:
 What do we do if we find evidence
of Non-Normality? (cont’d)

û
t
+

Oct T
ime
1
987

• Create a new variable:

D87M10t = 1 during October 1987 and zero otherwise.
This effectively knocks out that observation. But we need a theoretical reason for
adding dummy variables.
• Outliers can be identified (if it is present) by plotting the
actual values, the fitted values and the residuals of the
regression. This can be achieved in Eviews by selecting
View/Actual, Fitted, Residual/Actual, Fitted, Residual
Graph.
• But it is probably easier to just examine a table of values
for the residuals, which can be achieved by selecting
View/Actual, Fitted, Residual/Actual, Fitted, Residual
Table.
 Omission of an Important Variable or
Inclusion of an Irrelevant Variable

Omission of an Important Variable

• Consequence: The estimated coefficients on all the other variables will be
biased and inconsistent unless the excluded variable is uncorrelated with
all the included variables.

• Even if this condition is satisfied, the estimate of the coefficient on the

constant term will be biased.

• The standard errors will also be biased.

Inclusion of an Irrelevant Variable

• Coefficient estimates will still be consistent and unbiased, but the

estimators will be inefficient.