Autocorrelation

DR. INDRA, S.Si, M.Si

OUTLINE

1. What is Autocorrelation
2. What Causes Autocorrelation
3. Types of Autocorrelation:
First and Higher Orders Autocorrelation
4. Consequences of Autocorrelation
5. Detecting Autocorrelation
6. Resolving Autocorrelation

2
Review of the Classical Assumption

1. Linear Regression Model

y =  1 +  2x + e
2. Error Term has a mean of zero:
E(e) = 0 → E(y) = 1 + 2x
3. Data on X are not random and thus are uncorrelated with the
error term: Cov(X,e) = E(Xe) = 0
4. Error term has constant variance: This is the assumption
Var(e) = E(e2) = 2 of no serial correlation
5. Error term is not correlated
with itself (no serial correlation): This assumption states that the disturb
Cov(ei,ej) = E(eiej) = 0 ij ances ut and us are independently distri
buted, which is called serial independ
ence.
 What is Autocorrelation

Autocorrelation is a systematic pattern in the

errors that can be either attracting (positive)
or repelling (negative) autocorrelation.

For efficiency (accurate estimation/prediction)

all systematic information needs to be incor-
porated into the regression model.
 What is Autocorrelation

Yt = 1 + 2X2t + 3X3t + Ut

No autocorrelation: Cov (Ui, Uj)

or E(Ui, Uj) = 0

Autocorrelation: Cov (Ui, Uj)  0

or E(Ui, Uj)  0
Note: i  j

In general

E(Ut, Ut-s)  0
 What is Autocorrelation

• If this assumption is no longer valid, then the

disturbances are not pairwise independent, but
pairwise autocorrelated (or Serially Correlated).
• This means that an error occurring at period t may be
carried over to the next period t+1.
• Autocorrelation is most likely to occur in time series
data.
• In cross-sectional we can change the arrangement of
the data without altering the results.
 What Causes Autocorrelation

• One factor that can cause autocorrelation is omitted

variables.
• Suppose Yt is related to X2t and X3t, but we wrongfully
do not include X3t in our model.
• The effect of X3t will be captured by the disturbances
ut .
• If X3t like many economic series exhibit a trend over
time, then X3t depends on X3t-1, X3t -2 and so on.
• Similarly then ut depends on ut-1, ut-2 and so on.
 What Causes Autocorrelation

• Another possible reason is misspecification.

• Suppose Yt is related to X2t with a quadratic
relationship:
Yt=β1+β2X22t+ut
but we wrongfully assume and estimate a
straight line:
Yt=β1+β2X2t+ut
• Then the error term obtained from the straight
line will depend on X22t.
 What Causes Autocorrelation

• A third reason is systematic errors in measurement.

• Suppose a company updates its inventory at a given
period in time.
• If a systematic error occurred then the cumulative
inventory stock will exhibit accumulated measurement
errors.
• These errors will show up as an autocorrelated
procedure
 Order of Autocorrelation
Yt = 1 + 2X2t + 3X3t + Ut

1st Order: Ut =  Ut−1 + t

2nd Order: Ut = 1 Ut−1 + 2 Ut−2 + t
3rd Order: Ut = 1 Ut−1 + 2 Ut−2 + 3 Ut−3 + t
Where -1 <  < +1

We will assume First Order Autocorrelation:

AR(1) : Ut =  Ut−1 + t
 First-Order Autocorrelation

• The simplest and most commonly observed is the first

-order autocorrelation.
• Consider the multiple regression model:
Yt=β1+β2X2t+β3X3t+β4X4t+…+βkXkt+ut
in which the current observation of the error term ut is
a function of the previous (lagged) observation of the
error term:
ut=ρut-1+et
 First-Order Autocorrelation

• The coefficient ρ is called the first-order autoc

orrelation coefficient and takes values from
-1 to +1.
• It is obvious that the size of ρ will determine
the strength of serial correlation.
• We can have three different cases.
 First-Order Autocorrelation

(a) If ρ is zero, then we have no autocorrelation.

(b) If ρ approaches unity, the value of the previous observa
tion of the error becomes more important in determining
the value of the current error and therefore high degree o
f autocorrelation exists. In this case we have positive au
tocorrelation.
(c) If ρ approaches -1, we have high degree of negative au
tocorrelation.
First-Order Autocorrelation
First-Order Autocorrelation
Higher-Order Autocorrelation

Second-order when:
ut=ρ1ut-1+ ρ2ut-2+et
Third-order when
ut=ρ1ut-1+ ρ2ut-2+ρ3ut-3 +et
p-th order when:
ut=ρ1ut-1+ ρ2ut-2+ρ3ut-3 +…+ ρput-p +et
Consequences of Autocorrelation

1. The OLS estimators are still unbiased and consistent.

This is because both unbiasedness and consistency do not
depend on no serially correlation assumption which is in
this case violated.
2. The OLS estimators will be inefficient and therefore no
longer BLUE.
3. The estimated variances of the regression coefficients will
be biased and inconsistent, and therefore hypothesis testing
is no longer valid. In most of the cases, the R2 will be
overestimated and the t-statistics will tend to be higher.
Consequences of Autocorrelation
^ ^
Yt = 1 + 2Xt + et
Autocorrelated disturbances: E(et, et-s)  0
Formula for ordinary least squares variance
(no autocorrelation in disturbances): Var( ˆ ) =  2

 t
2 2
x

Formula for ordinary least squares variance

(autocorrelated disturbances): ˆ   12 
Var(  2 ) = 1+ 2 xi x j  
k

 xt   xt
2  2 


Therefore when errors are autocorrelated ordinary

least squares estimators are inefficient (i.e. not “best”)
 Detecting Autocorrelation

There are two ways in general:

• The first is the informal way which is done through
graphs and therefore we call it the graphical method.
• The second is through formal tests for autocorrelation,
like the following ones:
1. The Durbin Watson Test
2. The Breusch-Godfrey Test
3. The Durbin’s h Test (for the presence of lagged de
pendent variables)
4. The Engle’s ARCH Test
 Detecting Autocorrelation

• We have the following series (quarterly data from

1985q1 to 1994q2):
• lcons = the consumer’s expenditure on food
• ldisp = disposable income
• lprice = the relative price index of food

Typing in Eviews the following command

ls lcons c ldisp lprice
we get the regression results.
 Detecting Autocorrelation

• Then we can store the residuals of this regression in a vector by typing

the command:
genr res01=resid
• And a plot of the residuals can be obtained by:
plot res01
• While a scatter of the residuals against their lagged terms can be obtained
by:
scat res01(-1) res01
Detecting Autocorrelation
Residual vs Time (t)
.12

.08

.04

.00

-.04

-.08
85 86 87 88 89 90 91 92 93

RES01
Detecting Autocorrelation
Residual (t) vs Residual (t-1)

.12

.08

.04
RES01

.00

-.04

-.08
-.08 -.04 .00 .04 .08 .12

RES01(-1)
 The Durbin Watson Test

The following assumptions should be satisfied:

1. The regression model includes a constant
2. Autocorrelation is assumed to be of first-order only
3. The equation does not include a lagged dependent vari
able as an explanatory variable
 The Durbin Watson Test

• Step 1: Estimate the model by OLS and obtain

the residuals
• Step 2: Calculate the DW statistic
• Step 3: Construct the table with the calculated D
W statistic and the dU, dL, 4-dU and 4-dL critical
values.
• Step 4: Conclude
 The Durbin Watson Test
Value of d relative to dL and dU Decision

d < dL Reject null of no positive

autocorrelation
dL  d  dU No decision

dU < d < 4 - dU Do not reject null of no

positive or negative
autocorrelation

4 – dL < d < 4 - dU No decision

d > 4 - dL Reject null of no negative

autocorrelation
 The Durbin Watson Test

Zone of in Zone of in
decision decision
(+) autoc (-) autoc
No autocorrelation

0 dL dU 2 4-dU 4-dL 4
• Autocorrelation usually exists if:
• The value of d is much smaller than 2 → (+) autocorrelation
• The value of d is much greater than 2 → (-) autocorrelation
• The model is free from autocorrelation if the value of d close to 2.
 The Durbin Watson Test

Drawbacks of the DW test

1. It may give inconclusive results
2. It is not applicable when a lagged dependent variable
is used
3. It can’t take into account higher order of
autocorrelation → only first order of autocorrelation
 The Breusch-Godfrey Test

• It is a Lagrange Multiplier Test that resolves the draw

backs of the DW test
• Consider the model:
Yt=β1+β2X2t+β3X3t+β4X4t+…+βkXkt+ut
where:
ut=ρ1ut-1+ ρ2ut-2+ρ3ut-3 +…+ ρput-p +et
 The Breusch-Godfrey Test

• Combining those two we get:

Yt=β1+β2X2t+β3X3t+β4X4t+…+βkXkt+
+ρ1ut-1+ ρ2ut-2+ρ3ut-3 +…+ ρput-p +et
• The null and the alternative hypotheses are:
H0: ρ1= ρ2=…= ρp=0 no autocorrelation
Ha: at least one of the ρ’s is not zero, thus, autocorrelation
 The Breusch-Godfrey Test

Step 1: Estimate the model and obtain the residuals

Step 2: Run the full LM model with the number of lags
used being determined by the assumed order of autocorr
elation.
Step 3: Compute the LM statistic = (n-ρ)R2 from the L
M model and compare it with the chi-square critical val
ue.
Step 4: Conclude
 The Durbin’s h Test

• When there are lagged dependent variables (i.e. Yt-1) then th

e DW test is not applicable.
• Durbin developed an alternative test statistic, named the h-st
atistic, which is calculated by:

 DW  n
h = 1 − 
 2  1 − n 2ˆ

• Where sigma of gamma hat square is the variance of the estimated

coefficient of the lagged dependent variable.
• This statistic is distributed following the normal distribution
 The Durbin’s h Test
Dependent Variable: LOG(CONS)
Included observations: 37 after adjustments

Variable Coefficient Std. Error t-Statistic Prob.

C 0.834242 0.626564 1.331456 0.1922
LOG(INC) 0.227634 0.188911 1.204981 0.2368
LOG(CPI) -0.259918 0.110072 -2.361344 0.0243
LOG(CONS(-1)) 0.854041 0.089494 9.542982 0.0000

R-squared 0.940878 Mean dependent var 4.582683

Adjusted R-squared 0.935503 S.D. dependent var 0.110256
S.E. of regression 0.028001 Akaike info criterion -4.211360
Sum squared resid 0.025874 Schwarz criterion -4.037207
Log likelihood 81.91016 F-statistic 175.0558
Durbin-Watson stat 1.658128 Prob(F-statistic) 0.000000

 DW  n
h = 1 − 
 2  1 − n 2ˆ
 1.658  37
= 1 −  = 1.2971
 2  1 − 37 * 0.089 2
 Resolving Autocorrelation

We have two different cases:

(a) When ρ is known

(b) When ρ is unknown
 Resolving Autocorrelation
(when ρ is known)

Consider the model

Yt=β1+β2X2t+β3X3t+β4X4t+…+βkXkt+ut

where
ut=ρ1ut-1+et
 Resolving Autocorrelation
(when ρ is known)

Yt=β1+β2X2t+β3X3t+β4X4t+…+βkXkt+ut
Write the model of t-1:
Yt-1=β1+β2X2t-1+β3X3t-1+β4X4t-1+…+βkXkt-1+ut-1
Multiply both sides by ρ to get
ρYt-1= ρβ1+ ρβ2X2t-1+ ρβ3X3t-1+ ρβ4X4t-1
+…+ ρ βkXkt-1+ ρut-1
 Resolving Autocorrelation
(when ρ is known)

Subtract those two equations:

Yt-ρYt-1= (1-ρ)β1+ β2(X2t-ρX2t-1)+ β3(X3t-ρX3t-1)+
+…+ βk(Xkt-ρXkt-1)+(ut-ρut-1)
or
Y*t= β*1+ β*2X*2t+ β*3X*3t+…+ β*kX*kt+et
Where now the problem of autocorrelation is resolved
because et is no longer autocorrelated.
 Resolving Autocorrelation
(when ρ is known)

• Note that because from the transformation we

lose one observation, in order to avoid that loss
we generate Y1 and Xi1 as follows:
Y*1=Y1 sqrt(1- ρ2)
X*i1=Xi1 sqrt(1-ρ2)
• This transformation is known as the
quasi-differencing or generalised differencing.
 Resolving Autocorrelation
(when ρ is unknown))

The Cochrane-Orcutt iterative procedure.

Step 1: Estimate the regression and obtain residuals
Step 2: Estimate ρ from regressing the residuals to its la
gged terms.
Step 3: Transform the original variables as starred varia
bles using the obtained from step 2.
Step 4: Run the regression again with the transformed v
ariables and obtain residuals.
Step 5 and on: Continue repeating steps 2 to 4 for sever
al rounds until (stopping rule) the estimates of from tw
o successive iterations differ by no more than some pres
elected small value, such as 0.001.