Chapter THREE

REGRESSION ANALYIS WITH

TIME SERIES DATA
 3.1. Nature of time series data
• Three types of data may be available for empirical analysis:
time series, cross-section, and pooled (i.e., combination of
time series and cross-section) data.
• A time series is a set of observations on the values that a
variable takes at different times.
• Time series data are data collected on the same
observational unit at multiple time periods. Example
 Aggregate consumption and GDP for a country (for
example, 20 years of quarterly observations = 80
observations)
 Birr/$, pound/$ and Euro/$ exchange rates (daily data for 1
year = 365 observations)
 Why use time series data?
 To develop forecasting models
 What will the rate of inflation be next year?
 To estimate dynamic causal effects
 If the Fed increases the Federal Funds rate now, what will
be the effect on the rates of inflation and unemployment in 3
months? in 12 months?
 Or, because that is your only option …
 Rates of inflation and unemployment can be observed only
over time!
 Most empirical work based on time series data assumes that
the underlying time series is stationary. , a time series is
stationary if its mean and variance do not vary
systematically over time
 1.2. Stationary and non- Stationary
Stochastic Processes

• A random or stochastic process is a collection of random

variables ordered in time

2.1.1.. Stationary Stochastic Processes

 A type of stochastic process that has received a great deal of

attention and scrutiny by time series analysts is the so-called
stationary stochasticprocess.
 2.1.1.. Stationary Stochastic Processes-----
 a stochastic process is said to be stationary if its mean and
variance are constant over time and the value of the
covariance between the two time periods depends only on
the distance or gap or lag between the two time periods
and not the actual time at which the covariance is
computed. that is, they are time invariant

• such a stochastic process is known as a weakly stationary,

or covariance stationary, or second-order stationary, or
wide sense, stochastic process.
 2.1.2.Non-stationary Stochastic Processes

 If a time series is not stationary in the sense just defined, it

is called a non stationary time series .

 In other words, a non stationary time series will have a time

varying mean or a time-varying variance or both.

 Although our interest is in stationary time series, one often

encounters non-stationary time series, the classic example
being the random walk model (RWM).
 Non-stationary Stochastic Processes--

• It is often said that asset prices, such as stock prices or

exchange rates, follow a random walk; that is, they are
non stationary.
• We distinguish two types of random walks:
(1) random walk without a drift (i.e., no constant or
intercept term) and
(2) random walk with a drift (i.e., a constant term is
present).
 1).
Random Walk without Drift
• Suppose ut is a white noise error term with mean 0 and variance σ2. as ut ~
IIDN(0, σ2); that is, ut is independently and identically distributed as a
normal distribution with zero mean and constant variance.
• Then the series Yt is said to be a random walk if
Yt = Yt−1 + ut -----------2.1
• In the random walk model, as (2.1) shows, the value of Y at time t is equal
to its Previous value at time (t − 1) plus a random shock(ut)
 Now from (2.1) we can write. ,Y1 = Y0 + u1, Y2 = Y1 + u2 ,Y0 + u1 +
u2, Y3 = Y2 + u3 = Y0 + u1 + u2 + u3
 In general, if the process started at some time 0 with a value of Y0, we
have, Yt = Y0 + ut --------------(2.2)
• Therefore,
 E(Yt) = E(Y0 + ut ) = Y0 (why?) -----(2.3)
 Similarly , it can be shown that
 As the preceding expression shows, the mean of Y is equal
to its initial, or starting, value, which is constant,

 but as t increases, its variance increases indefinitely,

thus violating a condition of stationary.

 Hence, RWM without a drift is a non-stationary process:

Although its mean is constant over time, its variance
increases over time.
• An interesting feature of RWM is the persistence of random
shocks (i.e.,random errors), which is clear from (2.2):
Yt = Y0 + ut
• As a result, the impact of a particular shock does not die
away.
• For example, if u2 = 2 rather than u2 = 0, then all Yt ’s from
Y2 onward will be 2 units higher and the effect of this shock
never dies out.
• That is why random walk is said to have an infinite
memory.
• Interestingly, if you write (2.1) as
(Yt − Yt−1) = Yt = ut ------(2.5)
• where  is the first difference operator .
• It is easy to show that, while Yt is non stationary, its first
difference is stationary.
• In other words, the first differences of a random walk time
series are stationary.
 2)
Random Walk with Drift.

• Let us modify (2.1) as follows:

Yt = δ + Yt−1 + ut ----------(2.6)
• where δ is known as the drift parameter. The name drift
comes from the fact that if we write the preceding equation
as
Yt − Yt−1 =  Yt = δ + ut -------------(2.7
• it shows that Yt drifts upward or downward, depending on
δ being positive or negative.
• E(Yt) = Y0 + t · δ ------(2.8))proof)
• var (Yt) = tσ2 ------(2.9)(proof)
• As you can see, for RWM with drift the mean as well as the
variance increases over time, again violating the
conditions of (weak) stationary.

• In short, RWM, with or without drift, is a non-

stationary stochastic process.
 TREND STATIONARY (TS) AND DIFFERENCE
STATIONARY (DS)
STOCHASTIC PROCESSES

• If a time series has deterministic trend it

can be made stationary by regressing it
on time or trend variable. The residuals
from his regression will tend represent a
time series that is trend free

• However If a time series has stochastic

trend it can be made stationary by
differencing it on one or more times
• If the trend in a time series is a deterministic
function of time, such --as t and t2, we call it a
deterministic trend (predictable).

• If it is not predictable, we have a stochastic trend.

Consider the following model:

• (iid) independently and identically distributed

• Pure Random Walk: this is non-
stationary as we get

 If we difference, we get
• Note that the differenced series is stationary (DS) because
both are time invariant.
 Hence, a random walk without a drift is difference-stationary
(DS).
 Random Walk with a drift: and this is again
non-stationary.
• If we difference, we get
• Note that differenced series is again stationary (DS)
because
• Both are again time-invariant. Hence, a random
walk with a drift is also difference-stationary (DS).

• Also note that in this case, Yt is trending upward or

downward depending on the sign of the drift (α) but
this will be called a stochastic trend (rather than
deterministic).
 A Test of Stationarity: unit root
test (Dickey-Fuller Test:
• There are several tests of stationarity, we will focus on a test
which became popular over the past years: This is the unit
root tests (Dickey-Fuller tests).
• Let start with autoregressive process
Yt = ρYt−1 + ut , 1 ≤ ρ ≤ 1----------- (21.4.1)
where ut is a white noise error term.
• We know that if ρ = 1, that is, in the case of the unit root,
(21.4.1) becomes a random walk model without drift,
which we know is a non stationary stochastic process.
• Therefore, why not simply regress Yt on its (one period)
lagged value Yt−1 and find out if the estimated ρ is
statistically equal to 1? Answer, If it is, then Yt is non
stationary.
 A Test of Stationarity: unit root
test (Dickey-Fuller Test
• This is the general idea behind the unit root test
of stationarity.
• So what we can do????

We will transform time series variables using

lags, first differences, logarithms, & growth rates
 A Test of Stationarity: unit root
test (Dickey-Fuller Test
I. Lag of Yt is Yt-1, its jth lag is Yt-j.
II. The first difference of series is change between periods t-1
and t. i,e, Yt= Yt- Yt-1
III. First difference of logarithms of Yt is , ln(Yt)=ln(Yt)-
ln(Yt-t)
IV. The percentage change of time series Yt b/n periods t and
t-1 is approximately 100ln(Yt), where approximation is
most accurate when the percentage change is small
 A Test of Stationarity: unit root test
(Dickey-Fuller Test
 Equation 21.4.1. can be manipulated as
Yt − Yt−1 = ρYt−1 − Yt−1 + ut---- (21.9.1)
= (ρ − 1)Yt−1 + ut

 which can be alternatively written as:

Yt = δYt−1 + ut -------------(21.9.2)

where δ = (ρ − 1) and , as usual, is the first-difference

operator.
 A Test of Stationarity: unit root
test (Dickey-Fuller Test
 In practice, therefore, instead of estimating (21.4.1), we
estimate (21.9.2) and test the (null) hypothesis that δ = 0.

 If δ = 0, then ρ = 1, that is we have a unit root, meaning

the time series under consideration is non stationary.

• Now let us turn to the estimation of (21.9.2). take the first

differences of Yt and regress them on Yt−1 and see if the
estimated slope coefficient in this regression (= ˆ δ) is
zero or not.
• Yt ˆ = δ ˆ Yt−1 + ut
A Test of Stationarity: unit root test (Dickey-Fuller Test

• In practice, we can estimate this model

(in Stata, reg d.Y l.Y, noconstant) and
obtain δ and test the hypothesis:
– H0 : δ = 0 (i.e. there is a unit root, and
the series is non-stationary) against
– HA : δ < 0 (i.e. there is no unit root and
the series is stationary).
 If δ ˆ is zero, we conclude that Yt is
non stationary.

 But if it is negative, we conclude

 A Test of Stationary: unit root
test (Dickey-Fuller Test
• The problem is that we can not rely on the
usual t-test on the significance of !!!

• The alternative is to use the Dickey-Fuller test

statistic with its own critical values for each of
the following three specifications.
 Integrated time series
• In time series literature you will often come
across the term integrated time series .if
such a time series become stationary after
differencing it once, it is said to be
integrated of order one denoted as I(1)
• IF it has t be differenced twice (difference
of difference)to make it stationary it said to
reintegrated of order two, denoted as I(2)
• It has to be differenced n time to make it
stationary it say to be integrated of order
n, denoted as I(n)
• Stationary time series is integrated
of order zero, denoted as I(0).
Therefore the term stationary time serirs
and time series integrated of order zero
mean the same
• By the same token if the time series is
integrated it is non stationary
• To sum-up, anon stationary time series is
known variously as an integrated time
series or a series with stochastic trend.
• Thank you