Forecasting

Francis X. Diebold
University of Pennsylvania

August 11, 2015

1 / 323
Copyright
c 2013 onward, by Francis X. Diebold.

These materials are freely available for your use, but be warned:
they are highly preliminary, significantly incomplete, and rapidly
evolving. All are licensed under the Creative Commons
Attribution-NonCommercial-NoDerivatives 4.0 International
License. (Briefly: I retain copyright, but you can use, copy and
distribute non-commercially, so long as you give me attribution and
do not modify. To view a copy of the license, visit
http://creativecommons.org/licenses/by-nc-nd/4.0/.) In return I
ask that you please cite the books whenever appropriate, as:
”Diebold, F.X. (year here), Book Title Here, Department of
Economics, University of Pennsylvania,
http://www.ssc.upenn.edu/ fdiebold/Textbooks.html.”

The painting is Enigma, by Glen Josselsohn, from Wikimedia

Commons.

2 / 323
Elements of Forecasting in Business, Finance, Economics
and Government

1. Forecasting in Action
1.1 Operations planning and control
1.2 Marketing
1.3 Economics
1.4 Financial speculation
1.5 Financial risk management
1.6 Capacity planning
1.7 Business and government budgeting
1.8 Demography
1.9 Crisis management

3 / 323
Forecasting Methods: An Overview

Review of probability, statistics and regression

Six Considerations Basic to Successful Forecasting

1. Forecasts and decisions
2. The object to be forecast
3. Forecast types
4. The forecast horizon
5. The information set
6. Methods and complexity
6.1 The parsimony principle
6.2 The shrinkage principle

4 / 323
Statistical Graphics for Forecasting

I Why graphical analysis is important

I Simple graphical techniques
I Elements of graphical style
I Application: graphing four components of real GNP

5 / 323
Modeling and Forecasting Trend

I Modeling trend
I Estimating trend models
I Forecasting trend
I Selecting forecasting models using the Akaike and Schwarz
criteria
I Application: forecasting retail sales

6 / 323
Modeling and Forecasting Seasonality

I The nature and sources of seasonality

I Modeling seasonality
I Forecasting seasonal series
I Application: forecasting housing starts

7 / 323
Characterizing Cycles

I Covariance stationary time series

I White noise
I The lag operator
I Wold’s theorem, the general linear process, and rational
distributed lags
I Estimation and inference for the mean, autocorrelation and
partial autocorrelation functions
I Application: characterizing Canadian employment dynamics

8 / 323
Modeling Cycles: MA, AR and ARMA Models

I Moving-average (MA) models

I Autoregressive (AR) models
I Autoregressive moving average (ARMA) models
I Application: specifying and estimating models for forecasting
employment

9 / 323
Forecasting Cycles

I Optimal forecasts
I Forecasting moving average processes
I Forecasting infinite-ordered moving averages
I Making the forecasts operational
I The chain rule of forecasting
I Application: forecasting employment

10 / 323
Putting it all Together: A Forecasting Model with Trend,
Seasonal and Cyclical Components

I Assembling what we’ve learned

I Application: forecasting liquor sales
I Recursive estimation procedures for diagnosing and selecting
forecasting models

11 / 323
Forecasting with Regression Models

I Conditional forecasting models and scenario analysis

I Accounting for parameter uncertainty in confidence intervals
for conditional forecasts
I Unconditional forecasting models
I Distributed lags, polynomial distributed lags, and rational
distributed lags
I Regressions with lagged dependent variables, regressions with
ARMA disturbances, and transfer function models
I Vector autoregressions
I Predictive causality
I Impulse-response functions and variance decomposition
I Application: housing starts and completions

12 / 323
Evaluating and Combining Forecasts

I Evaluating a single forecast

I Evaluating two or more forecasts: comparing forecast accuracy
I Forecast encompassing and forecast combination
I Application: OverSea shipping volume on the Atlantic East
trade lane

13 / 323
Unit Roots, Stochastic Trends, ARIMA Forecasting
Models, and Smoothing

I Stochastic trends and forecasting

I Unit roots: estimation and testing
I Application: modeling and forecasting the yen/dollar
exchange rate
I Smoothing
I Exchange rates, continued

14 / 323
Volatility Measurement, Modeling and Forecasting

I The basic ARCH process

I The GARCH process
I Extensions of ARCH and GARCH models
I Estimating, forecasting and diagnosing GARCH models
I Application: stock market volatility

15 / 323
Useful Books, Journals and Software

Books

Statistics review, etc.:

I Wonnacott, T.H. and Wonnacott, R.J. (1990), Introductory
Statistics, Fifth Edition. New York: John Wiley and Sons.
I Pindyck, R.S. and Rubinfeld, D.L. (1997),Econometric Models
and Economic Forecasts, Fourth Edition. New York:
McGraw-Hill.
I Maddala, G.S. (2001), Introduction to Econometrics, Third
Edition. New York: Macmillan.
I Kennedy, P. (1998), A Guide to Econometrics, Fourth Edition.
Cambridge, Mass.: MIT Press.

16 / 323
Useful Books, Journals and Software cont.

Time series analysis:

I Chatfield, C. (1996), The Analysis of Time Series: An
Introduction, Fifth Edition. London: Chapman and Hall.
I Granger, C.W.J. and Newbold, P. (1986), Forecasting
Economic Time Series, Second Edition. Orlando, Florida:
Academic Press.
I Harvey, A.C. (1993), Time Series Models, Second Edition.
Cambridge, Mass.: MIT Press.
I Hamilton, J.D. (1994), Time Series Analysis, Princeton:
Princeton University Press.

17 / 323
Useful Books, Journals and Software cont.

Special insights:
I Armstrong, J.S. (Ed.) (1999), The Principles of Forecasting.
Norwell, Mass.: Kluwer Academic Forecasting.
I Makridakis, S. and Wheelwright S.C. (1997), Forecasting:
Methods and Applications, Third Edition. New York: John
Wiley.
I Bails, D.G. and Peppers, L.C. (1997), Business Fluctuations.
Englewood Cliffs: Prentice Hall.
I Taylor, S. (1996), Modeling Financial Time Series, Second
Edition. New York: Wiley.

18 / 323
Useful Books, Journals and Software cont.

Journals
I Journal of Forecasting
I Journal of Business Forecasting Methods and Systems
I Journal of Business and Economic Statistics
I Review of Economics and Statistics
I Journal of Applied Econometrics

19 / 323
Useful Books, Journals and Software cont.

Software
I General:
I Eviews
I S+
I Minitab
I SAS
I R
I Python
I Many more...
I Cross-section:
I Stata
I Open-ended:
I Matlab

20 / 323
Useful Books, Journals and Software cont.

Online Information
I Resources for Economists:

21 / 323
A Brief Review of Probability, Statistics, and Regression
for Forecasting
Topics
I Discrete Random Variable
I Discrete Probability Distribution
I Continuous Random Variable
I Probability Density Function
I Moment
I Mean, or Expected Value
I Location, or Central Tendency
I Variance
I Dispersion, or Scale
I Standard Deviation
I Skewness
I Asymmetry
I Kurtosis
I Leptokurtosis
22 / 323
A Brief Review of Probability, Statistics, and Regression
for Forecasting
Topics cont.
I Skewness
I Asymmetry
I Kurtosis
I Leptokurtosis
I Normal, or Gaussian, Distribution
I Marginal Distribution
I Joint Distribution
I Covariance
I Correlation
I Conditional Distribution
I Conditional Moment
I Conditional Mean
I Conditional Variance
23 / 323
A Brief Review of Probability, Statistics, and Regression
for Forecasting cont.
Topics cont.
I Population Distribution
I Sample
I Estimator
I Statistic, or Sample Statistic
I Sample Mean
I Sample Variance
I Sample Standard Deviation
I Sample Skewness
I Sample Kurtosis
I χ2 Distribution
I t Distribution
I F Distribution
I Jarque-Bera Test
24 / 323
Regression as Curve Fitting

Least-squares estimation:
T
X
min [ yt − β0 − β1 xt ] 2
β
t = 1

Fitted values:

ŷt = β̂0 + β̂1 xt

Residuals:

et = yt − ŷt

25 / 323
Regression as a probabilistic model
Simple regression:

y t = β 0 + β 1 x t + εt

iid
εt ∼ (0, σ 2 )

Multiple regression:

yt = β0 + β1 xt + β2 zt + εt

iid
εt ∼ (0, σ 2 )

26 / 323
Regression as a probabilistic model cont.

Mean dependent var 10.23

T
1X
ȳ = yt
T
t=1

S.D. dependent var 1.49

s
PT
− ȳ)2
t=1 (yt
SD =
T−1
Sum squared resid 43.70
T
X
SSR = e2t
t=1

27 / 323
Regression as a probabilistic model cont.

F−statistic 30.89

(SSRres − SSR) / (k − 1)
F =
SSR / (T − k)
S.E. of regression 0.99
PT 2
2 t=1 et
s =
T−k

s
√
PT 2
t=1 et
SER = s2 =
T−k

28 / 323
Regression as a probabilistic model cont.

R−squared 0.58
PT 2
2 t=1 et
R = 1 − PT
t=1 (yt − ȳt )2
or
1 PT 2
2 T t=1 et
R = 1 − 1 PT
T t=1 (yt − ȳt )2
Adjusted R−squared 0.56
1 PT 2
2 T−k t=1 et
R̄ = 1 − 1 PT
T−1 t=1 (yt − ȳt )2

29 / 323
Regression as a probabilistic model cont.

Akaike info criterion 0.03

PT 2
2k
t=1 et
AIC = e( T )
T
Schwarz criterion 0.15
PT 2
k
t=1 et
SIC = T( T )
T

30 / 323
Regression as a probabilistic model cont.

Durbin – Watson stat 1.97

y t = β 0 + β 1 x t + εt

iid
vt ∼ N(0, σ 2 )

εt = φεt−1 + vt

PT 2
t=2 (et − et−1 )
DW = PT 2
t=1 et

31 / 323
Regression of y on x and z

32 / 323
Scatterplot of y versus x

33 / 323
Scatterplot of y versus x – Regression Line Superimposed

34 / 323
Scatterplot of y versus z – Regression Line Superimposed

35 / 323
Residual Plot – Regression of y on x and z

36 / 323
Six Considerations Basic to Successful Forecasting

1. The Decision Environment and Loss Function

L(e) = e2
L(e) = |e|
2. The Forecast Object
I Event outcome, event timing, time series.
3. The Forecast Statement
I Point forecast, interval forecast, density forecast, probability
forecast

37 / 323
Six Considerations Basic to Successful Forecasting cont.

4. The Forecast Horizon

I h-step ahead forecast
I h-step-ahead extrapolation forecast
5. The Information Set

Ωunivariate
T = {yT , yT−1 , ..., y1 }

Ωmultivariate
T = {yT , xT , yT−1 , xT−1 , ..., y1 , x1 }
6. Methods and Complexity, the Parsimony Principle, and the
I Shrinkage Principle
I Signal vs. noise
I Smaller is often better
I Even incorrect restrictions can help

38 / 323
Six Considerations Basic to Successful Forecasting cont.

Decision Making with Symmetric Loss

Demand High Demand Low
Build Inventory 0 $10,000
Reduce Inventory $10,000 0

Decision Making with Asymmetric Loss

Demand High Demand Low
Build Inventory 0 $10,000
Reduce Inventory $20,000 0

39 / 323
Six Considerations Basic to Successful Forecasting cont.

Forecasting with Symmetric Loss

High Actual Sales Low Actual Sales
High Forecasted
Sales 0 $10,000
Low Forecasted
Sales $10,000 0

Forecasting with Asymmetric Loss

High Actual Sales Low Actual Sales
High Forecasted
Sales 0 $10,000
Low Forecasted
Sales $20,000 0

40 / 323
Quadratic Loss

41 / 323
Absolute Loss

42 / 323
Asymmetric Loss

43 / 323
Forecast Statement

44 / 323
Forecast Statement cont.

45 / 323
Extrapolation Forecast

46 / 323
Extrapolation Forecast cont.

47 / 323
Statistical Graphics For Forecasting
1. Why Graphical Analysis is Important
I Graphics helps us summarize and reveal patterns in data
I Graphics helps us identify anomalies in data
I Graphics facilitates and encourages comparison of different
pieces of data
I Graphics enables us to present a huge amount of data in a
small space, and it enables us to make huge data sets coherent
2. Simple Graphical Techniques
I Univariate, multivariate
I Time series vs. distributional shape
I Relational graphics
3. Elements of Graphical Style
I Know your audience, and know your goals.
I Show the data, and appeal to the viewer.
I Revise and edit, again and again.
4. Application: Graphing Four Components of Real GNP
48 / 323
Anscombe’s Quartet

(1) (2) (3) (4)

x1 y1 x2 y2 x3 y3 x4
10.0 8.04 10.0 9.14 10.0 7.46 8.0
8.0 6.95 8.0 8.14 8.0 6.77 8.0
13.0 7.58 13.0 8.74 13.0 12.74 8.0
9.0 8.81 9.0 8.77 9.0 7.11 8.0
11.0 8.33 11.0 9.26 11.0 7.81 8.0
14.0 9.96 14.0 8.10 14.0 8.84 8.0
6.0 7.24 6.0 6.13 6.0 6.08 8.0
4.0 4.26 4.0 3.10 4.0 5.39 19
12.0 10.84 12.0 9.13 12.0 8.15 8.0
7.0 4.82 7.0 7.26 7.0 6.42 8.0
5.0 5.68 5.0 4.74 5.0 5.73 8.0

49 / 323
Anscombe’s Quartet

50 / 323
Anscombe’s Quartet – Bivariate Scatterplot

51 / 323
1-Year Treasury Bond Rates

52 / 323
Change in 1-Year Treasury Bond Rates

53 / 323
Liquor Sales

54 / 323
Histogram and Descriptive Statistics – Change in 1-Year
Treasury Bond Rates

55 / 323
Scatterplot 1-Year vs. 10-year Treasury Bond Rates

56 / 323
Scatterplot Matrix – 1-, 10-, 20-, and 30-Year Treasury
Bond Rates

57 / 323
Time Series Plot – Aspect Ratio 1:1.6

58 / 323
Time Series Plot – Banked to 45 Degrees

59 / 323
Time Series Plot – Aspect Ratio 1:1.6

60 / 323
Graph

61 / 323
Graph

62 / 323
Graph

63 / 323
Graph

64 / 323
Graph

65 / 323
Components of Real GDP (Millions of Current Dollars,
Annual

66 / 323
Modeling and Forecasting Trend

1. Modeling Trend

Tt = β0 + β1 TIMEt

Tt = β0 + β1 TIMEt + β2 TIME2t

Tt = β0 eβ1 TIMEt

ln(Tt ) = ln(β0 ) + β1 TIMEt

67 / 323
Modeling and Forecasting Trend
2. Estimating Trend Models
T
X
(β̂0 , β̂1 ) = arg min [ yt − β0 − β1 TIMEt ] 2
β0 ,β1
t = 1

T
X
yt − β0 − β1 TIMEt − β2 TIME2t

(β̂0 , β̂1 , β̂2 ) = arg min
β0 ,β1 ,β2
t = 1

T h
X i2
(β̂0 , β̂1 ) = arg min yt − β0 eβ1 TIMEt
β0 ,β1
t = 1

T
X
(β̂0 , β̂1 ) = arg min [ ln yt − ln β0 − β1 TIMEt ] 2
β0 ,β1
t = 1

68 / 323
Modeling and Forecasting Trend

3. Forecasting Trend

yt = β0 + β1 TIMEt + εt

yT+h = β0 + β1 TIMET+h + εT+h

yT+h,T = β0 + β1 TIMET+h

ŷT+h,T = β̂0 + β̂1 TIMET+h

69 / 323
Modeling and Forecasting Trend

3. Forecasting Trend cont.

yT+h,T ± 1.96σ

ŷT+h,T ± 1.96σ̂

N(yT+h,T , σ 2 )

N(ŷT+h,T , σ̂ 2 )

70 / 323
Modeling and Forecasting Trend

4. Selecting Forecasting Models

PT 2
t=1 et
MSE =
T
PT 2
t=1 et
R2 = 1 − PT
t=1 (yt − ȳ)2

71 / 323
Modeling and Forecasting Trend

4. Selecting Forecasting Models cont.

PT 2
2 t=1 et
s =
T−k
  PT 2
T t=1 et
s2 =
T−k T

PT 2 2
2 t=1 et / T−k PT 2
R̄ = 1 − PT = 1 − s t=1 (yt − ȳ) / T−

t=1 (yt − ȳ)2 / T − 1

72 / 323
Modeling and Forecasting Trend

4. Selecting Forecasting Models cont.

PT 2
2k
t=1 et
AIC = e( T )
T
PT 2
k
t=1 et
SIC = T( T )
T
I Consistency
I Efficiency

73 / 323
Labor Force Participation Rate

74 / 323
Increasing and Decreasing Labor Trends

75 / 323
Labor Force Participation Rate

76 / 323
Linear Trend – Female Labor Force Participation Rate

77 / 323
Linear Trend – Male Labor Force Participation Rate

78 / 323
Volume on the New York Stock Exchange

79 / 323
Various Shapes of Quadratic Trends

80 / 323
Quadratic Trend – Volume on the New York Stock
Exchange

81 / 323
Log Volume on the New York Stock Exchange

82 / 323
Various Shapes of Exponential Trends

83 / 323
Linear Trend – Log Volume on the New York Stock
Exchange

84 / 323
Exponential Trend – Volume on the New York Stock
Exchange

85 / 323
Degree-of-Freedom Penalties – Various Model Selection
Criteria

86 / 323
Retail Sales

87 / 323
Retail Sales – Linear Trend Regression

88 / 323
Retail Sales – Linear Trend Residual Plot

89 / 323
Retail Sales – Quadratic Trend Regression

90 / 323
Retail Sales – Quadratic Trend Residual Plot

91 / 323
Retail Sales – Log Linear Trend Regression

92 / 323
Retail Sales – Log Linear Trend Residual Plot

93 / 323
Retail Sales – Exponential Trend Regression

94 / 323
Retail Sales – Exponential Trend Residual Plot

95 / 323
Model Selection Criteria

Linear, Quadratic and Exponential Trend Models

Linear Trend Quadratic Trend Exponential Trend

AIC 19.35 15.83 17.15
SIC 19.37 15.86 17.17

96 / 323
Retail Sales – History January, 1990 – December, 1994

97 / 323
Retail Sales – History January, 1990 – December, 1994

98 / 323
Retail Sales – History January, 1990 – December, 1994

99 / 323
Retail Sales – History January, 1990 – December, 1994

100 / 323
Modeling and Forecasting Seasonality
1. The Nature and Sources of Seasonality
2. Modeling Seasonality
D1 = (1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, ...)
D2 = (0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, ...)
D3 = (0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, ...)
D4 = (0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, ...)

s
X
yt = γi Dit + εt
i=1
s
X
yt = β1 TIMEt + γi Dit + εt
i=1

s
X v1
X v2
X
yt = β1 TIMEt + γi Dit + δiHD HDVit + δiTD TDVi
i=1 i=1 i=1

101 / 323
Modeling and Forecasting Seasonality
3. Forecasting Seasonal Series

s
X v1
X v2
X
yt = β1 TIMEt + γi Dit + δiHD HDVit + δiTD TDVi
i=1 i=1 i=1

s
X v1
X
yT+h = β1 TIMET+h + γi Di,T+h + δiHD HDVi,T+h +
i=1 i=1

s
X v1
X
yT+h,T = β1 TIMET+h + γi Di,T+h + δiHD HDVi,T+h +
i=1 i=1

sγ̂i
X v1
X
ŷT+h,T = β̂1 TIMET+h + Di,T+h + δ̂iHD HDVi,T+h +
i=1 i=1

102 / 323
Modeling and Forecasting Seasonality

3. Forecasting Seasonal Series cont.

yT+h,T ± 1.96σ

ŷT+h,T ± 1.96σ̂

N(yT+h,T , σ 2 )

N(ŷT+h,T , σ̂ 2 )

103 / 323
Gasoline Sales

104 / 323
Liquor Sales

105 / 323
Durable Goods Sales

106 / 323
Housing Starts, January, 1946 – November, 1994

107 / 323
Housing Starts, January, 1990 – November, 1994

108 / 323
Housing Starts Regression Results - Seasonal Dummy
Variable Model

109 / 323
Residual Plot

110 / 323
Housing Starts – Estimated Seasonal Factors

111 / 323
Housing Starts

112 / 323
Housing Starts

113 / 323
Characterizing Cycles
1. Covariance Stationary Time Series
I Realization
I Sample Path
I Covariance Stationary

Eyt = µt
Eyt = µ

γ(t, τ ) = cov(yt , yt−τ ) = E(yt − µ)(yt−τ − µ)

γ(t, τ ) = γ(τ )

cov(yt , yt−τ ) γ(τ ) γ(τ )

ρ(τ ) = p p = p p =
var(yt ) var(yt−τ ) γ(0) γ(0) γ(0)
114 / 323
1.Characterizing Cycles Cont.

cov(x, y)
corr(x, y) =
σx σy
γ(τ )
ρ(τ ) = , τ = 0, 1, 2, ....
γ(0)

cov(yt , yt−τ ) γ(τ ) γ(τ )

ρ(τ ) = p p = p p =
var(yt ) var(yt−τ ) γ(0) γ(0) γ(0)

I p(τ ) regression of yt on yt−1 , ..., yt−τ

115 / 323
2.White Noise

yt ∼ WN(0, σ 2 )

iid
yt ∼ (0, σ 2 )

iid
yt ∼ N(0, σ 2 )

116 / 323
2.White Noise Cont.

E(yt ) = 0
var(yt ) = σ 2

E(yt |Ωt−1 ) = 0
var(yt |Ωt−1 ) = E[(yt − E(yt |Ωt−1 ))2 |Ωt−1 ] = σ 2

117 / 323
3.The Lag Operator

L yt = yt−1
L2 yt = L(L(yt )) = L(yt−1 ) = yt−2
B(L) = b0 + b1 L + b2 L2 + ... bm Lm
Lm yt = yt−m
∆yt = (1 − L)yt = yt − yt−1
(1 + .9L + .6L2 )yt = yt + .9yt−1 + .6yt−2
∞
X
B(L) = b0 + b1 L + b2 L2 + ... = bi Li
i=0
∞
X
B(L) εt = b0 εt + b1 εt−1 + b2 εt−2 + ... = bi εt−i
i=0

118 / 323
4.Wold’s Theorem, the General Linear Process, and
Rational Distributed Lags
Wold’s Theorem

Let {yt } be any zero-mean covariance-stationary process. Then:

∞
X
yt = B(L)εt = bi εt−i
i=0

εt ∼ WN(0, σ 2 )
where
b0 = 1
and
∞
X
b2i < ∞
i=0
119 / 323
The General Linear Process

∞
X
yt = B(L)εt = bi εt−i
i=0

iid
εt ∼ WN(0, σ 2 ),

where b0 = 1 and
∞
X
b2i < ∞
i=0

120 / 323
The General Linear Process Cont.

∞
X ∞
X ∞
X
E(yt ) = E( bi εt−i ) = bi Eεt−i = bi • 0 = 0
i=0 i=0 i=0

X∞ ∞
X ∞
X ∞
X
2 2 2 2
var(yt ) = var( bi εt−i ) = bi var(εt−i ) = bi σ = σ b
i=0 i=0 i=0 i=0

E(yt |Ωt−1 ) = E(εt |Ωt−1 ) + b1 E(εt−1 |Ωt−1 ) + b2 E(εt−2 |Ωt−1 ) + ... =

var(yt |Ωt−1 ) = E[(yt − E(yt |Ωt−1 ))2 |Ωt−1 ] = E(ε2t |Ωt−1 ) = E(ε2t )

121 / 323
Rational Distributed Lags

Θ (L)
B(L) =
Φ (L)
q
X
Θ(L) = θi Li
i=0
p
X
Φ(L) = φi Li
i=0

Θ (L)
B(L) ≈
Φ (L)

122 / 323
5.Estimation and Inference for the Mean, Auto Correlation
and Partial Autocorrelation Functions
T
1X
ȳ = yt
T
t=1

E [(yt − µ) (yt−τ − µ)]

ρ(τ ) =
E[(yt − µ)2 ]
1 PT PT
T t = τ +1 [(yt − ȳ) (yt−τ − ȳ)] t = τ +1 [(yt − ȳ) (
ρ̂(τ ) = 1 PT
= PT
T t=1 (yt − ȳ)2 t=1 (yt −

ŷt = ĉ + β̂1 yt−1 + ... + β̂τ yt−τ

p̂(τ ) ≡ β̂τ
 
1
ρ̂(τ ), p̂(τ ) ∼ N 0,
T
123 / 323
5.Estimation and Inference for the Mean, Auto Correlation
and Partial Autocorrelation Functions Cont.

1
ρ̂(τ ) ∼ N(0, )
T

rootTρ̂(τ ) ∼ N(0, 1)

T ˆρ2 (τ ) ∼ χ21

m
X
QBP = T ρ̂2 (τ )
τ =1
m  
X 1
QLB = T (T + 2) ρ̂2 (τ )
T−τ
τ =1

124 / 323
A Rigid Cycle Pattern

125 / 323
Autocorrelation Function, One-Sided Gradual Damping

126 / 323
Autocorrelation Function, Non-Damping

127 / 323
Autocorrelation Function, Gradual Damped Oscillation

128 / 323
Autocorrelation Function, Sharp Cutoff

129 / 323
Realization of White Noise Process

130 / 323
Population Autocorrelation Function of White Noise
Process

131 / 323
Population Partialautocorrelation Function of White Noise
Process

132 / 323
Canadian Employment Index

133 / 323
Canadian Employment Index Correlogram
Sample: 1962:1 1993:4
Included observations: 128
Acorr. P. Acorr. Std. Error Ljung-Box p-v
v
1 0.949 0.949 .088 118.07 0.000
2 0.877 −0.244 .088 219.66 0.000
3 0.795 −0.101 .088 303.72 0.000
4 0.707 −0.070 .088 370.82 0.000
5 0.617 −0.063 .088 422.27 0.000
6 0.526 −0.048 .088 460.00 0.000
7 0.438 −0.033 .088 486.32 0.000
8 0.351 −0.049 .088 503.41 0.000
9 0.258 −0.149 .088 512.70 0.000
10 0.163 −0.070 .088 516.43 0.000
11 0.073 −0.011 .088 517.20 0.000
12 −0.005 0.016 .088 517.21 0.000
134 / 323
Canadian Employment Index, Sample Autocorrelation and
Partial Autocorrelation Functions

135 / 323
Modeling Cycles: MA,AR, and ARMA Models

The MA(1) Process

yt = εt + θεt−1 = (1 + θL)εt

εt ∼ WN(0, σ 2 )

If invertible:

yt = εt + θyt−1 − θ2 yt−2 + θ3 yt−3 − ...

136 / 323
Modeling Cycles: MA,AR, and ARMA Models Cont.

Eyt = E(εt ) + θE(εt−1 ) = 0

var(yt ) = var(εt ) + θ2 var(εt−1 ) = σ 2 + θ2 σ 2 = σ 2 (1 + θ2 )

E(yt |Ωt−1 ) = E((εt + θεt−1 )|Ωt−1 ) = E(εt |Ωt−1 ) + θE(εt−1 |Ωt−1 )

var(yt |Ωt−1 ) = E[(yt − E(yt |Ωt−1 ))2 |Ωt−1 ] = E(ε2t |Ωt−1 ) = E(ε2t )

137 / 323
The MA(q) Process

yt = εt + θ1 εt−1 + ... + θq εt−q = Θ(L)εt

εt ∼ WN(0, σ 2 )

where

Θ(L) = 1 + θ1 L + ... + θq Lq

138 / 323
The AR(1) Process

yt = φyt−1 + εt
εt ∼ WN(0, σ 2 )

If covariance stationary:

yt = εt + φεt−1 + φ2 εt−2 + ...

139 / 323
Moment Structure

E (yt ) = E (εt + φεt−1 + φ2 εt−2 + ...)

= E (εt ) + φE (εt−1 ) + φ2 E (εt−2 ) + ...

= 0

var (yt ) = var (εt + φεt−1 + φ2 εt−2 + ...)

= σ 2 + φ2 σ 2 + φ4 σ 2 + ...
P∞
= σ2 i=0 φ
2i

2
= σ 1−φ2

140 / 323
Moment Structure Cont.

E (yt |yt−1 ) = E ((φyt−1 + εt ) | yt−1 )

= φE (yt−1 |yt−1 ) + E (εt |yt−1 )

= φyt−1 + 0

= φyt−1

var (yt |yt−1 ) = var ((φyt−1 + εt ) | yt−1 )

= φ2 var (yt−1 |yt−1 ) + var (εt |yt−1 )

= 0 + σ2

= σ2

141 / 323
Moment Structure Cont.
Autocovariances and autocorrelations:

yt = φyt−1 + εt
yt yt−τ = φyt−1 yt−τ + εt yt−τ
For
τ ≥1
,

γ(τ ) = φγ(τ − 1).

(Yule-Walker equation) But
2
γ(0) = σ 1−φ2
142 / 323
Moment Structure Cont.

. Thus

2
γ(τ ) = φτ σ 1−φ2 , τ = 0, 1, 2, ....

and

ρ(τ ) = φτ , τ = 0, 1, 2, ....

Partial autocorrelations:

143 / 323
The AR(p) Process

yt = φ1 yt−1 + φ2 yt−2 + ... + φp yt−p + εt

εt ∼ WN(0, σ 2 )

144 / 323
The ARMA(1,1) Process

yt = φyt−1 + εt + θεt−1
εt ∼ WN(0, σ 2 )

MA representation if invertible:

(1 + θ L)
yt = εt
(1 − φ L)

AR representation of covariance stationary:

(1 − φ L)
y t = εt
(1 + θ L)

145 / 323
The ARMA(p,q) Process

yt = φ1 yt−1 + ... + φp yt−p + εt + θ1 εt−1 + ... + θq εt−q

εt ∼ WN(0, σ 2 )

Φ(L)yt = Θ(L)εt

146 / 323
Realization of Two MA(1) Processes

147 / 323
Population Autocorrelation Function MA(1) Process

θ = .4

148 / 323
Population Autocorrelation Function MA(1) Process

θ = .95

149 / 323
Population Partial Autocorrelation Function MA(1)
Process

θ = .4

150 / 323
Population Partial Autocorrelation Function MA(1)
Process

θ = .95

151 / 323
Realization of Two AR(1) Processes

152 / 323
Population Autocorrelation Function AR(1) Process

φ = .4

153 / 323
Population Autocorrelation Function AR(1) Process

φ = .95

154 / 323
Population Partial Autocorrelation Function AR(1) Process

φ = .4

155 / 323
Population Partial Autocorrelation Function AR(1) Process

φ = .95

156 / 323
Population Autocorrelation Function AR(2) Process with
Complex Roots

157 / 323
Employment: MA(4) Model

Sample: 1962:1 1993:4

Included observations: 128

158 / 323
Employment MA(4) Residual Plot

159 / 323
Employment: MA(4) Model
Residual Correlogram Sample: 1962:1 1993:4
Included observations: 128
Q−statistic probabilities adjusted for 4 ARMA term(s)
Acorr. P. Acorr. Std. Error Ljung-Box p-v
1 0.345 0.345 .088 15.614
2 0.660 0.614 .088 73.089
3 0.534 0.426 .088 111.01
4 0.427 −0.042 .088 135.49
5 0.347 -0.398 .088 151.79 0.000
6 0.484 0.145 .088 183.70 0.000
7 0.121 −0.118 .088 185.71 0.000
8 0.348 −0.048 .088 202.46 0.000
9 0.148 −0.019 .088 205.50 0.000
10 0.102 −0.066 .088 206.96 0.000
11 0.081 −0.098 .088 207.89 0.000
12 0.029 −0.113 .088 208.01 0.000
160 / 323
Employment: MA(4) Model
Residual Sample Autocorrelation and Partial Autocorrelation
Functions, With Plus or Minus Two Standard Error Bands

161 / 323
Employment: AR(2) Model

LS // Dependent Variable is CANEMP

Sample: 1962:1 1993:4
Included observations: 128
Convergence achieved after 3 iterations
Variable Coefficient Std. Error t−Statistic Prob.

162 / 323
Employment AR(2) Model Residual Plot

163 / 323
Employment AIC Values of Various ARMA Models

MA Order
0 1 2 3 4
0 2.86 2.32 2.47 2.20
1 1.01 .83 .79 .80 .81
AR Order 2 .762 .77 .78 .80 .80
3 .77 .761 .77 .78 .79
4 .79 .79 .77 .79 .80

164 / 323
Employment SIC Values of Various ARMA Models

MA Order
0 1 2 3 4
0 2.91 2.38 2.56 2.31
1 1.05 .90 .88 .91 .94
AR Order 2 .83 .86 .89 .92 .96
3 .86 .87 .90 .94 .96
4 .90 .92 .93 .97 1.00

165 / 323
Employment: ARMA(3,1) Model

LS // Dependent Variable is CANEMP

Sample: 1962:1 1993:4
Included observations: 128
Convergence achieved after 17 iterations
Variable Coefficient Std. Error t−Statistic Prob.

166 / 323
Employment ARMA(3) Model Residual Plot

167 / 323
Employment: ARMA(3,1) Model Residual Correlogram
Sample: 1962:1 1993:4
Included observations: 128
Q−statistic probabilities adjusted for 4 ARMA term(s)
Acorr. P. Acorr. Std. Error Ljung-Box p-v
1 −0.032 −0.032 .09 0.1376
2 0.041 0.040 .09 0.3643
3 0.014 0.017 .09 0.3904
4 0.048 0.047 .09 0.6970
5 0.006 0.007 .09 0.7013 0.402
6 0.013 0.009 .09 0.7246 0.696
7 −0.017 −0.019 .09 0.7650 0.858
8 0.064 0.060 .09 1.3384 0.855
9 0.092 0.097 .09 2.5182 0.774
10 0.039 0.040 .09 2.7276 0.842
11 −0.016 −0.022 .09 2.7659 0.906
12 −0.137 −0.153 .09 5.4415 0.710
168 / 323
Employment: ARMA(3) Model

Residual Sample Autocorrelation and Partial Autocorrelation

Functions, With Plus or Minus Two Standard Error Bands

169 / 323
Forecasting Cycles

ΩT = {yT , yT−1 , yT−2 , ...},

ΩT = {εT , εT−1 , εT−2 , ...}.

Optimal Point Forecasts for Infinite-Order Moving Averages
∞
X
yt = bi εt−i ,
i=0

where

εt ∼ WN(0, σ 2 )

170 / 323
Forecasting Cycles Cont.

b0 = 1
, and
∞
X
σ2 b2i < ∞
i=0
.

yT+h = εT+h + b1 εT+h−1 + ... + bh εT + bh+1 εT−1 + ...

yT+h,T = bh εT + bh+1 εT−1 + ...

h−1
X
eT+h,T = (yT+h − yT+h,T ) = bi εT+h−i ,
i=0
h−1
X
σh2 = σ 2 b2i .
i=0
171 / 323
Interval and Density Forecasts

yT+h = yT+h,T + eT+h,T .

95% h-step-ahead interval forecast:

yT+h,T ± 1.96σh

h-step-ahead density forecast:

N(yT+h,T , σh2 )

Making the Forecasts Operational

The Chain Rule of Forecasting

172 / 323
Employment History and Forecast MA(4) Model

173 / 323
Employment History and Long-Horizon Forecast MA(4)
Model

174 / 323
Employment History, Forecast and Realization MA(4)
Model

175 / 323
Employment History and Forecast AR(2) Model

176 / 323
Employment History and Long-Horizon Forecast AR(2)
Model

177 / 323
Employment History and Very Long-Horizon Forecast
AR(2) Model

178 / 323
Employment History, Forecast and Realization AR(2)
Model

179 / 323
Putting it all Together
A Forecast Model with Trend, Seasonal and Cyclical Components

The full model:

s
X v1
X v2
X
yt = Tt (θ) + γi Dit + δiHD HDVit + δiTD TDVit + εt
i=1 i=1 i=1

Φ(L)εt = Θ(L)vt
Φ(L) = 1 − φ1 L − ... − φp Lp
Θ(L) = 1 + θ1 L + ... + θq Lq
vt ∼ WN(0, σ 2 ).

180 / 323
Point Forecasting

s
X v1
X v2
X
yT+h = TT+h (θ) + γi Di,T+h + δiHD HDVi,T+h + δiTD T
i=1 i=1 i=1

s
X v1
X v2
X
yT+h,T = TT+h (θ) + γi Di,T+h + δiHD HDVi,T+h + δiTD
i=1 i=1 i=1

sγ̂i
X v1
X v2
X
ŷT+h,T = TT+h (θ̂) + Di,T+h + δ̂iHD HDVi,T+h + δ̂iTD T
i=1 i=1 i=1

181 / 323
Interval Forecasting and Density Forecasting

Interval Forecasting:

ŷT+h,T ± zα/2 σ̂h

e.g.: (95% interval)

ŷT+h,T ± 1.96σ̂h

Density Forecasting:

N(ŷT+h,T , σ̂h2 )

182 / 323
Recursive Estimation

K
X
yt = βk xkt + εt
k=1

εt ∼ iidN(0, σ 2 ),
t = 1, ..., T .
OLS estimation uses the full sample, t = 1, ..., T .

Recursive least squares uses an expanding sample.

Begin with the first K observations and estimate the model.
Then estimate using the first K + 1 observations, and so on.
At the end we have a set of recursive parameter estimates:
β̂k,t , for k = 1, ..., K and t = K , ..., T .

183 / 323
Recursive Residuals

At each t, t = K , ..., T − 1, compute a 1-step forecast,

K
X
ŷt+1,t = β̂kt xk,t+1 .
k=1

The corresponding forecast errors, or recursive residuals, are

êt+1,t = yt+1 − ŷt+1,t .

êt+1,t ∼ N(0, σ 2 rt )

where rt > 1 for all t

184 / 323
Standardized Recursive Residuals and CUSUM

êt+1,t
wt+1,t ≡ √ ,
σ rt

t = K , ..., T − 1.

Under the maintained assumptions,

wt+1,t ∼ iidN(0, 1).

Then
t ∗
X
CUSUMt∗ ≡ wt+1,t , t ∗ = K , ..., T − 1
t=K

is just a sum of iid N(0, 1)’s.

185 / 323
Liquor Sales, 1968.1-1993.12

186 / 323
Log Liquor Sales, 1968.01 - 1993.12

187 / 323
Log Liquor Sales: Quadratic Trend Regression

188 / 323
Liquor Sales Quadratic Trend Regression Residual Plot

189 / 323
Liquor Sales Quadratic Trend Regression Residual
Correlogram
Acorr. P. Acorr. Std. Error Ljung-Box
1 0.117 0.117 .056 4.3158 0.0
2 −0.149 −0.165 .056 11.365 0.0
3 −0.106 −0.069 .056 14.943 0.0
4 −0.014 −0.017 .056 15.007 0.0
5 0.142 0.125 .056 21.449 0.0
6 0.041 −0.004 .056 21.979 0.0
7 0.134 0.175 .056 27.708 0.0
8 −0.029 −0.046 .056 27.975 0.0
9 −0.136 −0.080 .056 33.944 0.0
10 −0.205 −0.206 .056 47.611 0.0
11 0.056 0.080 .056 48.632 0.0
12 0.888 0.879 .056 306.26 0.0
13 0.055 −0.507 .056 307.25 0.0
14 −0.187 −0.159 .056 318.79 0.0
15 −0.159 −0.144 .056 327.17 0.0
190 / 323
Liquor Sales Quadratic Trend Regression Residual Sample
Autocorrelation Functions

191 / 323
Liquor Sales Quadratic Trend Regression Residual Partial
Autocorrelation Functions

192 / 323
Log Liquor Sales: Quadratic Trend Regression With
Seasonal Dummies and AR(3) Disturbances

193 / 323
Liquor Sales Quadratic Trend Regression with Seasonal
Dummies Residual Plot

194 / 323
Liquor Sales Quadratic Trend Regression with Seasonal
Dummies Residual Correlogram
Acorr. P. Acorr.Std.Error Ljung-Box
p-value
1 0.700 0.700 .056 154.34 0.000
2 0.686 0.383 .056 302.86 0.000
3 0.725 0.369 .056 469.36 0.000
4 0.569 −0.141 .056 572.36 0.000
5 0.569 0.017 .056 675.58 0.000
6 0.577 0.093 .056 782.19 0.000
7 0.460 −0.078 .056 850.06 0.000
8 0.480 0.043 .056 924.38 0.000
9 0.466 0.030 .056 994.46 0.000
10 0.327 −0.188 .056 1029.1 0.000
11 0.364 0.019 .056 1072.1 0.000
12 0.355 0.089 .056 1113.3 0.000
13 0.225 −0.119 .056 1129.9 0.000
14 0.291 0.065 .056 1157.8 0.000
15 0.211 −0.119 .056 1172.4 0.000 195 / 323
Liquor Sales Quadratic Trend Regression with Seasonal
Dummies Residual Sample Autocorrelation Functions

196 / 323
Liquor Sales Quadratic Trend Regression with Seasonal
Dummies Residual Sample Partial Autocorrelation
Functions

197 / 323
Liquor Sales Quadratic Trend Regression with Seasonal
Dummies and AR(3) Disturbances Residual Plot

198 / 323
Liquor Sales Quadratic Trend Regression with Seasonal
Dummies and AR(3) Disturbances Residual Correlogram
Acorr. P. Acorr. Std. Error Ljung-Box p-v
1 0.056 0.056 .056 0.9779 0.323
2 0.037 0.034 .056 1.4194 0.492
3 0.024 0.020 .056 1.6032 0.659
4 −0.084 −0.088 .056 3.8256 0.430
5 −0.007 0.001 .056 3.8415 0.572
6 0.065 0.072 .056 5.1985 0.519
7 −0.041 −0.044 .056 5.7288 0.572
8 0.069 0.063 .056 7.2828 0.506
9 0.080 0.074 .056 9.3527 0.405
10 −0.163 −0.169 .056 18.019 0.055
11 −0.009 −0.005 .056 18.045 0.081
12 0.145 0.175 .056 24.938 0.015
13 −0.074 −0.078 .056 26.750 0.013
14 0.149 0.113 .056 34.034 0.002
15 −0.039 −0.060 .056 34.532 0.003
199 / 323
Liquor Sales Quadratic Trend Regression with Seasonal
Dummies and AR(3) Disturbances Residual Sample
Autocorrelation Functions

200 / 323
Liquor Sales Quadratic Trend Regression with Seasonal
Dummies and AR(3) Disturbances Residual Sample Partial
Autocorrelation Functions

201 / 323
Liquor Sales Quadratic Trend Regression with Seasonal
Dummies and AR(3) Disturbances Residual Histogram and
Normality Test

202 / 323
Log Liquor Sales History and 12-Month-Ahead Forecast

203 / 323
Log Liquor Sales History, 12-Month-Ahead Forecast, and
Realization

204 / 323
Log Liquor Sales History and 60-Month-Ahead Forecast

205 / 323
Log Liquor Sales Long History and 60-Month-Ahead
Forecast

206 / 323
Liquor Sales Long History and 60-Month-Ahead Forecast

207 / 323
Recursive Analysis Constant Parameter Model

208 / 323
Recursive Analysis Breaking Parameter Model

209 / 323
Log Liquor Sales: Quadratic Trend Regression with
Seasonal Dummies and AR(3) Residuals and Two
Standard Errors Bands

210 / 323
Log Liquor Sales: Quadratic Trend Regression with
Seasonal Dummies and AR(3) Disturbances Recursive
Parameter Estimates

211 / 323
Log Liquor Sales: Quadratic Trend Regression with
Seasonal Dummies and AR(3) Disturbances CUMSUM
Analysis

212 / 323
Forecasting with Regression Models

Conditional Forecasting Models and Scenario Analysis

y t = β 0 + β 1 x t + εt
εt ∼ N(0, σ 2 )
yT+h,T |x∗T+h = β0 + β1 x∗T+h

Density forecast:

N(yT+h,T |x∗T+h , σ 2 )

• “Scenario analysis,” “contingency analysis”

• No “forecasting the RHS variables problem”

213 / 323
Unconditional Forecasting Models

yT+h,T = β0 + β1 xT+h,T

• “Forecasting the RHS variables problem”

• Could fit a model to x (e.g., an autoregressive model)
• Preferably, regress y on

xt−h , xt−h−1, ...

• No problem in trend and seasonal models

214 / 323
Distributed Lags

Start with unconditional forecast model:

yt = β0 + δxt−1 + εt

Generalize to

Nx
X
yt = β0 + δi xt−i + εt
i=1

• “distributed lag model”

• “lag weights”
• “lag distribution”

215 / 323
Polynomial Distributed Lags

T
" Nx
#2
X X
min yt − β 0 − δi xt−i
β0 , δi t = Nx +1 i=1

subject to

δi = P(i) = a + bi + ci2 , i = 1, ..., Nx

• Lag weights constrained to lie on low-order polynomial

• Additional constraints can be imposed, such as

P(Nx ) = 0

• Smooth lag distribution

• Parsimonious

216 / 323
Rational Distributed Lags

A(L)
yt = x t + εt
B(L)
Equivalently,

B(L)yt = A(L)xt + B(L) εt

• Lags of x and y included

• Important to allow for lags of y, one way or another

217 / 323
Another way:
distributed lag regression with lagged dependent variables

Ny Nx
X X
yt = β 0 + αi yt−i + δj xt−j + εt
i=1 j=1

Another way:
distributed lag regression with ARMA disturbances

Nx
X
yt = β0 + δi xt−i + εt
i=1

Θ(L)
εt = vt
Φ(L)
vt ∼ WN(0, σ 2 )
218 / 323
Another Way: The Transfer function Model and Various
Special Cases
Univariate ARMA
C(L)
yt = εt
D(L)
A(
Distributed Lag with
B(L) yt = A(L) xt + εt
, or
Lagged Dep. Variables

A(L) 1
yt = xt + εt
B(L) B(L)
C(
Distributed Lag with 219 / 323
Vector Autoregressions
e.g., bivariate VAR(1)

y1,t = φ11 y1,t−1 + φ12 y2,t−1 + ε1,t

y2,t = φ21 y1,t−1 + φ22 y2,t−1 + ε2,t
ε1,t ∼ WN(0, σ12 )
• Estimation by OLS

ε2,t ∼ WN(0, σ22 )

cov(ε1,t , ε2,t ) = σ12
• Order selection by information criteria
• Impulse-response functions, variance decompositions, predictive
causality
• Forecasts via Wold’s chain rule

220 / 323
Point and Interval Forecast

221 / 323
U.S. Housing Starts and Completions, 1968.01-1996.06

222 / 323
Starts Correlogram
Sample: 1968:01 1991:12
Included observations: 288
Acorr. P. Acorr. Std. Error Ljung-Box p-v
1 0.937 0.937 0.059 255.24 0.000
2 0.907 0.244 0.059 495.53 0.000
3 0.877 0.054 0.059 720.95 0.000
4 0.838 −0.077 0.059 927.39 0.000
5 0.795 −0.096 0.059 1113.7 0.000
6 0.751 −0.058 0.059 1280.9 0.000
7 0.704 −0.067 0.059 1428.2 0.000
8 0.650 −0.098 0.059 1554.4 0.000
9 0.604 0.004 0.059 1663.8 0.000
10 0.544 −0.129 0.059 1752.6 0.000
11 0.496 0.029 0.059 1826.7 0.000
12 0.446 −0.008 0.059 1886.8 0.000
13 0.405 0.076 0.059 1936.8 0.000
14 0.346 −0.144 0.059 1973.3 0.000
223 / 323
Starts Sample Autocorrelations

224 / 323
Starts Sample Partial Autocorrelations

225 / 323
Completions Correlogram
Completions Correlogram
Sample: 1968:01 1991:12
Included observations: 288
Acorr. P. Acorr. Std. Error Ljung-Box p-v
1 0.939 0.939 0.059 256.61 0.000
2 0.920 0.328 0.059 504.05 0.000
3 0.896 0.066 0.059 739.19 0.000
4 0.874 0.023 0.059 963.73 0.000
5 0.834 −0.165 0.059 1168.9 0.000
6 0.802 −0.067 0.059 1359.2 0.000
7 0.761 −0.100 0.059 1531.2 0.000
8 0.721 −0.070 0.059 1686.1 0.000
9 0.677 −0.055 0.059 1823.2 0.000
10 0.633 −0.047 0.059 1943.7 0.000
11 0.583 −0.080 0.059 2046.3 0.000
12 0.533 −0.073 0.059 2132.2 0.000
226 / 323
Completions Sample Autocorrelations

227 / 323
Completions Partial Autocorrelations

228 / 323
Starts and Completions: Sample Cross Correlations

229 / 323
VAR Order Selection with AIC and SIC

230 / 323
VAR Starts Equation

231 / 323
VAR Start Equation Residual Plot

232 / 323
VAR Starts Equation Residual Correlogram
Sample: 1968:01 1991:12
Included observations: 284
Acorr. P. Acorr. Std. Error Ljung-Box p-v
1 0.001 0.001 0.059 0.0004 0.985
2 0.003 0.003 0.059 0.0029 0.999
3 0.006 0.006 0.059 0.0119 1.000
4 0.023 0.023 0.059 0.1650 0.997
5 −0.013 −0.013 0.059 0.2108 0.999
6 0.022 0.021 0.059 0.3463 0.999
7 0.038 0.038 0.059 0.7646 0.998
8 −0.048 −0.048 0.059 1.4362 0.994
9 0.056 0.056 0.059 2.3528 0.985
10 −0.114 −0.116 0.059 6.1868 0.799
11 −0.038 −0.038 0.059 6.6096 0.830
12 −0.030 −0.028 0.059 6.8763 0.866
13 0.192 0.193 0.059 17.947 0.160
14 0.014 0.021 0.059 18.010 0.206
233 / 323
VAR Starts Equation Residual Sample Autocorrelations

234 / 323
Var Starts Equation Residual Sample Partial
Autocorrelations

235 / 323
Evaluating and Combining Forecasts
Evaluating a single forecast Process:
yt = µ + εt + b1 εt−1 + b2 εt−2 + ...

εt ∼ WN(0, σ 2 ),

h-step-ahead linear least-squares forecast:

yt+h,t = µ + bh εt + bh+1 εt−1 + ...
Corresponding h-step-ahead forecast error:
et+h,t = yt+h − yt+h,t = εt+h + b1 εt+h−1 + ... + bh−1 εt+1
with variance
h−1
X
σh2 2
= σ (1 + b2i )
i=1
236 / 323
Evaluating and Combining Forecasts

So, four key properties of optimal forecasts:

a. Optimal forecasts are unbiased
b. Optimal forecasts have 1-step-ahead errors that are white noise
c. Optimal forecasts have h-step-ahead errors that are at most
MA(h-1)
d. Optimal forecasts have h-step-ahead errors with variances that
are non-decreasing in h and that converge to the unconditional
variance of the process
1. All are easily checked. How?

237 / 323
Assessing optimality with respect to an information set
Unforecastability principle: The errors from good forecasts are not
be forecastable!
Regression:
k−1
X
et+h,t = α + αi xit + ut
i=1

1. Test whether α0 , ..., αk−1 are 0

Important case:
et+h,t = α + α1 yt+h,t + ut

1. Test whether (α0 , α1 ) = (0, 0)

Equivalently,
yt+h,t = β + β1 yt+h,t + ut

1. Test whether (β0 , β1 ) = (0, 1)

238 / 323
Evaluating multiple forecasts: comparing forecast accuracy
Forecast errors, et+h,t = yt+h − yt+h,t Forecast percent errors,
y −y
pt+h,t = t+hyt+ht+h,t
T
1X
ME = et+h,t
T
t=1
T
1 X
EV = (et+h,t − ME)2
T
1

T
1X 2
MSE = et+h,t
T
t=1
T
1X 2
MSPE = pt+h,t
T
t=1
v
u
u1 X T
RMSE = t e2t+h,t
T
t=1 239 / 323
Forecast encompassing

a b
yt+h = βa yt+h,t + βb yt+h,t + εt+h,t

1. If (βa , βb ) = (1, 0), model a forecast-encompasses model b

2. If (βa , βb ) = (0, 1), model b forecast-encompasses model a
3. Otherwise, neither model encompasses the other
Alternative approach:
a b
(yt+h − yt = β(yt+h,t − yt ) + βb (yt+h,t − yt ) + εt+h,t

1. Useful in I(1) situations

240 / 323
Variance-covaiance forecast combination

Composite formed from two unbiased forecasts:

c a b
yt+h = ωyt+h,t + (1 − ω)yt+h,t

ect+h = ωeat+h,t + (1 − ω)ebt+h,t

σc2 = ω 2 σaa
2
+ (1 − ω)2 σbb
2 2
+ 2ω(1 − ω)σab

2 − σ2
σbb
ω∗ = ab
2 + σ 2 − 2sigma2
σbb aa ab
2 − σ̂ 2
σ̂bb
ω̂ ∗ = ab
2 + σ̂ 2 − 2σ̂ 2
σ̂bb aa ab

241 / 323
Regression-based forecast combination

a b
yt+h = β0 + β1 yt+h,t + β2 yt+h,t + εt+h,t

1. Equivalent to variance-covariance combination if weights sum

to unity and intercept is excluded
2. Easy extension to include more than two forecasts
3. Time-varying combining weights
4. Dynamic combining regressions
5. Shrinkage of combining weights toward equality
6. Nonlinear combining regressions

242 / 323
Unit Roots, Stochastic Trends, ARIMA Forecasting
Models, and Smoothing

1. Stochastic Trends and Forecasting

Φ(L)yt = Θ(L)εt

Φ(L) = Φ0 (L)(1 − L)

Φ0 (L)(1 − L)yt = Θ(L)εt

Φ0 (L)∆yt = Θ(L)εt
I(0) vs I(1) processes

243 / 323
Unit Roots, Stochastic Trends, ARIMA Forecasting
Models, and Smoothing Cont.
I Random Walk

yt = yt−1 + εt

εt ∼ WN(0, σ 2 ),

I Random walk with drift

yt = δ + yt−1 + εt

εt ∼ WN(0, σ 2 ),

Stochastic trend vs deterministic trend

244 / 323
Properties of random walks

yt = yt−1 + εt

εt ∼ WN(0, σ 2 ),

With time 0 value y0 :

t
X
yt = y0 + εi
i=1

E(yt ) = y0
var(yt ) = tσ 2
lim var(yt ) = ∞
t→∞

245 / 323
Random walk with drift
Random walk with drift

yt = δ + yt−1 + εt

εt ∼ WN(0, σ 2 ),

Assuming time 0 value y0 :

t
X
yt = tδ + y0 + εi
i=1

E(yt ) = y0 + tδ
var(yt ) = tσ 2
lim var(yt ) = ∞
t→∞
246 / 323
ARIMA(p,1,q) model

Φ(L)(1 − L)yt = c + Θ(L)εt

or

(1 − L)yt = cΦ−1 (L) + Φ−1 (L)Θ(L)εt

where
Φ(L) = 1 − Φ1 L − ... − Φp Lp

Θ(L) = 1 − Θ1 L − ... − Θq Lq
and all the roots of both lag operator polynomials are outside the
unit circle.

247 / 323
ARIMA(p,d,q) model

Φ(L)(1 − L)d yt = c + Θ(L)εt

or

(1 − L)d yt = cΦ−1 (L) + Φ−1 (L)Θ(L)εt

where
Φ(L) = 1 − Φ1 L − ... − Φp Lp

Θ(L) = 1 − Θ1 L − ... − Θq Lq
and all the roots of both lag operator polynomials are outside the
unit circle.

248 / 323
Properties of ARIMA(p,1,q) processes

I Appropriately made stationary by differencing

I Shocks have permanent effects
I Forecasts don’t revert to a mean
I Variance grows without bound as time progresses
I Interval forecasts widen without bound as horizon grows

249 / 323
Random walk example

Point forecast
Recall that for the AR(1) process,

yt = φyt−1 + εt
yt ∼ WN(0, σ 2 )
the optimal forecast is

yT+h,T = φh yT
Thus in the random walk case,

yT+h,T = yT , for all h

250 / 323
Random walk example Cont.
Interval and density forecasts
Recall error associated with optimal AR(1) forecast:
eT+h,T = (yT+h − yT+h,T ) = εT+h + φεT+h−1 + ... + φh−1 εT+1
with variance
h−1
X
σh2 = σ 2
φ2i
i=0
Thus in the random walk case,
h−1
X
eT+h,T = εT+h−i
i=0

σh2 = hσ 2
√
h − step − ahead 95% interval : yT ± 1.96σ h

h − step − ahead density forecast : N(yT , hσ 2 )

251 / 323
Effects of Unit Roots

I Sample autocorrelation function “fails to damp“

I Sample partial autocorrelation function near 1 for τ = 1, and
then damps quickly
I Properties of estimators change
e.g., least-squares autoregression with unit roots
True process:
yt = yt−1 + εt
Estimated model:
yt = φyt−1 + εt
Superconsistency:T (φ̂LS − 1) stabilizes as sample size grows
Bias: E (φ̂LS ) < 1
I Offsetting effects of bias and superconsistency

252 / 323
Unit Root Tests

yt = φyt−1 + εt
iid
εt ∼ N(0, σ 2 )

φ̂ − 1
τ̂ = q
s PT 1 2
t=2 yt−1

“Dickey-Fuller τ̂ distribution“
Trick regression:

yt − yt = (φ − 1)yt−1 + εt

253 / 323
Allowing for nonzero mean under the alternative

Basic model:
(yt − µ) = φ(yt−1 − µ) + εt
which we rewrite as

yt = α + φyt−1 + εt

where
α = µ(1 − φ)

I α vanishes when φ = 1 (null)

I α is nevertheless present under the alternative,
so we include an intercept in the regression
Dickey-Fuller τˆµ distribution

254 / 323
Allowing for deterministic linear trend under the alternative
Basic model:

(yt − a − bTIMEt ) = φ(yt−1 − a − b TIMEt−1 ) + εt

or
yt = α + βTIMEt + φyt−1 + εt
where α = a(1 − φ) + bφ and β = b(1 − φ).
I Under the null hypothesis we have a random walk with drift,
yt = b + yt−1 + εt

I Under the deterministic-trend alternative hypothesis both the

intercept and the trend enter and so are included in the
regression.

255 / 323
Allowing for higher-order autoregressive dynamics

AR(p) process:
p
X
yt + φj yt−j = εt
j=1

Rewrite:
p
X
yt = ρ1 yt−1 + ρj (yt−j+1 − yt−j ) + εt
j=2
Pp Pp
where p ≥ 2, ρ1 = − j=1 φj , and ρi = j=1 φj , i = 2, ..., p.

Unit root: ρ1 = 1 (AR(p − 1) in first differences)

τ̂ distribution holds asymptotically.

256 / 323
Allowing for a nonzero mean in the AR(p) case

p
X
(yt − µ) + φj (yt−j − µ) = εt
j=1
or
p
X
yt = α + ρyt−1 + ρj (yt−j+1 − yt−j ) + εt ,
j=2

where α = µ(1 + pj=1 φj ), and the other parameters are as

P
above.
Pp In the unit root case, the intercept vanishes, because
j=1 j = −1. τ̂µ distribution holds asymptotically.
φ

257 / 323
Allowing for trend under the alternative
p
X
(yt − a − bTIMEt ) + φj (yt−j − a − bTIMEt−j ) = εt
j=1
or
p
X
yt = k1 + k2 TIMEt + ρ1 yt−1 + ρj (yt−j+1 − yt−j ) + εt
j=2

where
p
X p
X
k1 = a(1 + φi ) − b iφi
i=1 i=1
and
p
X
k2 = bTIMEt (1 + φi )
i=1
Pp
In the unit root case, k1 = −b i=1 iφi and k2 = 0.

τ̂τ distribution holds asymptotically.

258 / 323
General ARMA representations: augmented Dickey-Fuller
tests

k−1
X
yt = ρ1 yt−1 + ρj (yt−j+1 − yt−j ) + εt
j=2

k−1
X
yt = α + ρ1 yt−1 + ρj (yt−j+1 − yt−j ) + εt
j=2

k−1
X
yt = k1 + k2 TIMEt + ρ1 yt−1 + ρj (yt−j+1 − yt−j ) + εt
j=2

I k-1 augmentation lags have been included

I τ̂ , τ̂µ , and τ̂τ hold asymptotically under the null

259 / 323
Simple moving average smoothing

1. Original data: {yt }T

t=1

2. Smoothed data: {ȳt }

3. Two-sided moving average is ȳt = (2m + 1)−1 m
P
i=−m yt−i

4. One-sided moving average is ȳt = (m + 1)−1 m

P
i=0 yt−i
Pm
5. One-sided weighted moving average is ȳt = i=0 wi yt−i

I Must choose smoothing parameter, m

260 / 323
Exponential Smoothing

Local level model:

yt = c0t + εt
c0t = c0,t−1 + ηt
ηt ∼ WN(0, ση2 )

I Exponential smoothing can construct the optimal estimate of

c0 - and hence the optimal forecast of any future value of y -
on the basis of current and past y
I What if the model is misspecified?

261 / 323
Exponential smoothing algorithm

I Observed series, {yt }T

t=1

I Smoothed series, {ȳt }T

t=1 (estimate of the local level)

I Forecasts, ŷT +h,T

1. Initialize at t = 1: ȳ1 = y1
2. Update: ȳt = αyt + (1 − α)ȳt−1 , t = 2, ...T
3. Forecast: ŷT + h, T = ȳT

I Smoothing parameter α ∈ [0, 1]

262 / 323
Demonstration that the weights are exponential

Start:
ȳt = αyt + (1 − α)ȳt−1
Substitute backward for ȳ :
t−1
X
ȳt = wj yt−j
j=0

where
wj = α(1 − α)j

I Exponential weighting, as claimed

I Convenient recursive structure

263 / 323
Holt-Winters Smoothing

yt = c0t + c1t TIMEt + εt

c0t = c0,t−1 + ηt

c1t = c1,t−1 + νt

I Local level and slope model

I Holt-Winters smoothing can construct optimal estimates of c0
and c1 - hence the optimal forecast of any future value of y by
extrapolating the trend - on the basis of current and past y

264 / 323
Holt-Winters smoothing algorithm
1. Initialize at t = 2:

ȳ2 = y2
F2 = y2 − y1
2. Update:

ȳt = αyt + (1 − α)(ȳt−1 + Ft−1 ), 0 < α < 1

Ft = β(y¯t − ȳt−1 ) + (1 − β)Ft−1 , 0 < β < 1

t = 3, 4, ..., T.
3. Forecast: ŷT +h,T = ȳT + hFT

I ȳt is the estimated level at time t

I Ft is the estimated slope at time t

265 / 323
Random Walk – Level and Change

266 / 323
Random Walk With Drift – Level and Change

267 / 323
U.S. Per Capita GNP – History and Two Forecasts

268 / 323
U.S. Per Capita GNP – History, Two Forecasts, and
Realization

269 / 323
Random Walk, Levels – Sample Autocorrelation Function
(Top Panel) and Sample Partial Autocorrelation Function
(Bottom Panel)

270 / 323
Random Walk, First Differences – Sample Autocorrelation
Function (Top Panel) and Sample Partial Autocorrelation
Function (Bottom Panel)

271 / 323
Log Yen / Dollar Exchange Rate (Top Panel) and Change
in Log Yen / Dollar Exchange Rate (Bottom Panel)

272 / 323
Log Yen / Dollar Exchange Rate – Sample
Autocorrelations (Top Panel) and Sample Partial
Autocorrelations (Bottom Panel)

273 / 323
Log Yen / Dollar Exchange Rate, First Differences –
Sample Autocorrelations (Top Panel) and Sample Partial
Autocorrelations (Bottom Panel)

274 / 323
Log Yen / Dollar Rate, Levels – AIC and SIC Values of
Various ARMA Models

275 / 323
Log Yen / Dollar Exchange Rate – Best-Fitting
Deterministic-Trend Model

276 / 323
Log Yen / Dollar Exchange Rate – Best-Fitting
Deterministic-Trend Model : Residual Plot

277 / 323
Log Yen / Dollar Rate – History and Forecast : AR(2) in
Levels with Linear Trend

278 / 323
Log Yen / Dollar Rate – History and Long-Horizon
Forecast : AR(2) in Levels with Linear Trend

279 / 323
Log Yen / Dollar Rate – History, Forecast and Realization :
AR(2) in Levels with Linear Trend

280 / 323
Log Yen / Dollar Exchange Rate – Augmented
Dickey-Fuller Unit Root Test

281 / 323
Log Yen / Dollar Rate, Changes – AIC and SIC Values of
Various ARMA Models

282 / 323
Log Yen / Dollar Exchange Rate – Best-Fitting
Stochastic-Trend Model

283 / 323
Log Yen / Dollar Exchange Rate – Best-Fitting
Stochastic-Trend Model : Residual Plot

284 / 323
Log Yen / Dollar Rate – History and Forecast : AR(1) in
Differences with Intercept

285 / 323
Log Yen / Dollar Rate – History and Long-Horizon
Forecast : AR(1) in Differences with Intercept

286 / 323
Log Yen / Dollar Rate – History, Forecast and Realization :
AR(1) in Differences with Intercept

287 / 323
Log Yen / Dollar Exchange Rate – Holt-Winters
Smoothing

288 / 323
Log Yen / Dollar Rate – History and Forecast :
Holt-Winters Smoothing

289 / 323
Log Yen / Dollar Rate – History and Long-Horizon
Forecast : Holt-Winters Smoothing

290 / 323
Log Yen / Dollar Rate – History, Forecast and Realization :
Holt-Winters Smoothing

291 / 323
Volatility Measurement, Modeling and Forecasting
The main idea:

εt | Ωt−1 ∼ (0, σt2 )

Ωt−1 = {εt−1 , εt−2 , ...}

We’ll look at:
I Basic Structure and properties
I Time variation in volatility and prediction-error variance
I ARMA representation in squares
I GARCH(1,1) and exponential smoothing
I Unconditional symmetry and leptokurtosis
I Convergence to normality under temporal aggregation
I Estimation and testing
292 / 323
Basic Structure and Properties

Standard models (e.g., ARMA):

I Unconditional mean: constant
I Unconditional variance: constant
I Conditional mean: varies
I Conditional variance: constant (unfortunately)

I k-step-ahead forecast error variance: depends only on k, not

on Ωt (again unfortunately)

293 / 323
The Basic ARCH Process

yt = B(L)εt
∞
X ∞
X
i
B(L) = bi L b2i < ∞ b0 = 1
i = 0 i = 0

εt | Ωt−1 ∼ N(0, σt2 )

σt2 = ω + γ(L)ε2t

p
X X
ω>0 γ(L) = γi Li γi ≥ 0 for all i γi < 1.
i = 1

294 / 323
The Basic ARCH Process cont.

ARCH(1) process:

rt | Ωt−1 ∼ (0, σt2 )

σt2 = ω + αr2t−1

I Unconditional mean: E (rt ) = 0

ω
I Unconditional variance: E (rt − E (rt ))2 = 1−α
I Conditional mean: E (rt | Ωt−1 ) = 0
I Conditional variance:
E ([rt − E (rt | Ωt−1 )]2 | Ωt−1 ) = ω + αrt−1
2

295 / 323
The GARCH Process

yt = εt

εt | Ωt−1 ∼ N(0, σt2 )

σt2 = ω + α(L)ε2t + β(L)σt2

p
X q
X
i
α(L) = αi L , β(L) = βi Li
i = 1 i = 1
X X
ω > 0, αi ≥ 0, βi ≥ 0, αi + βi < 1.

296 / 323
Time Variation in Volatility and Prediction Error Variance

Prediction error variance depends on Ωt−1

I e.g. 1-step-ahead prediction error variance is now

σt2 = ω + αr2t−1 + βσt−1

2

Conditional variance is a serially correlated RV

I Again, follows immediately from

σt2 = ω + αr2t−1 + βσt−1

2

297 / 323
ARMA Representation in Squares

rt2 has the ARMA(1,1) representation:

r2t = ω + (α + β)r2t−1 − βνt−1 + νt

where νt = rt2 − σt2

Important result:
The above equation is simply

r2t = (ω + (α + β)r2t−1 − βνt−1 ) + νt

= σt2 + νt
Thus rt2 is a noisy indicator of σt2

298 / 323
GARCH(1,1) and Exponential Smoothing
Exponential smoothing recursion:

r̄2t = γr2t + (1 − γ)r̄2t−1

Back substitution yields:

X
r̄2t = wj r2t−j

where
wj = γ(1 − γ)j
GARCH(1,1)
σt2 = ω + αr2t−1 + βσt−1
2

Back substitution yields:

ω X
σt2 = +α β j−1 r2t−j
1−β

299 / 323
Unconditional Symmetry and Leptokurtosis

I Volatility clustering produces unconditional leptokurtosis

I Conditional symmetry translates into unconditional symmetry
Unexpected agreement with the facts!

Convergence to Normality under Temporal Aggregation

I Temporal aggregation of covariance stationary GARCH
processes produces convergence to normality.
Again, unexpected agreement with the facts!

300 / 323
Estimation and Testing
Estimation: easy!
Maximum Likelihood Estimation

L(θ; r1 , ..., rT ) = f(rT | ΩT−1 ; θ) f(rT−1 | ΩT−2 ; θ)...

If the conditional densities are Gaussian,

1 r2
 
1
f(rt | Ωt−1 ; θ) = √ σt2 (θ)−1/2 exp − 2 t .
2π 2 σt (θ)

We can ignore the f (rp , ..., r1 ; θ) term, yielding the likelihood:

T T
T−p 1 X 1 X r2t
− ln(2π) − ln σt2 (θ) − .
2 2
t=p+1
2 σ 2 (θ)
t=p+1 t

Testing: likelihood ratio tests

Graphical diagnostics: Correlogram of squares, correlogram of
squared standardized residuals
301 / 323
Variations on Volatility Models

We will look at:

I Asymmetric response and the leverage effect
I Exogenous variables
I GARCH-M and time-varying risk premia

302 / 323
Asymmetric Response and the Leverage Effect:

TGARCH and EGARCH

Asymmetric response I: TARCH
Standard GARCH:

σt2 = ω + αr2t−1 + βσt−1

2

TARCH:
σt2 = ω + αr2t−1 + γr2t−1 Dt−1 + βσt−1
2

where
positive return (good news): α effect on volatility
negative return (bad news): α + γ effect on volatility
γ 6= 0 : Asymmetric news response
γ > 0 : ”Leverage effect”

303 / 323
Asymmetric Response and the Leverage Effect Cont.

Asymmetric Response II: E-GARCH

 
 rt−1 
2
ln(σt ) = ω + α    + γ rt−1 + β ln(σt−1
2
)
σt−1  σt−1

I Log specification ensures that the conditional variance is

positive.
I Volatility driven by both size and sign of shocks
I Leverage effect when γ < 0

304 / 323
Introducing Exogenous Variables

rt | Ωt−1 ∼ N(0, σt2 )

σt2 = ω + αr2t−1 + βσt−1

2
+ γXt
where:
γ is a parameter vector
X is a set of positive exogenous variables.

305 / 323
Component GARCH

Standard GARCH:

(σt2 − ω̄) = α(r2t−1 − ω̄) + β(σt−1

2
− ω̄),

for constant long-run volatility ω̄.

Component GARCH:

(σt2 − qt ) = α(r2t−1 − qt−1 ) + β(σt−1

2
− qt−1 ),

for time-varying long-run volatility qt , where

qt = ω + ρ(qt−1 − ω) + φ(r2t−1 − σt−1

2
)

306 / 323
Component GARCH Cont.

I Transitory dynamics governed by α + β

I Persistent dynamics governed by ρ
I Equivalent to nonlinearly restricted GARCH(2,2)
I Exogenous variables and asymmetry can be allowed:
(σt2 − qt ) = α(r2t−1 − qt−1 ) + γ(r2t−1 − qt−1 )Dt−1 + β(σt−1
2
− qt−1 ) + θ

307 / 323
Regression with GARCH Disturbances

yt = x0t β + εt

εt | Ωt−1 ∼ N(0, σt2 )

308 / 323
GARCH-M and Time-Varying Risk Premia
Standard GARCH regression model:

yt = x0t β + εt

εt | Ωt−1 ∼ N(0, σt2 )

GARCH-M model is a special case:

yt = x0t β + γσt2 + εt

εt | Ωt−1 ∼ N(0, σt2 )

I Time-varying risk premia in excess returns

309 / 323
Time Series Plot – NYSE Returns

310 / 323
Histogram and Related Diagnostic Statistics – NYSE
Returns

311 / 323
Correlogram – NYSE Returns

312 / 323
Time Series Plot – Squared NYSE Returns

313 / 323
Correlogram – Squared NYSE Returns

314 / 323
AR(5) Model – Squared NYSE Returns

315 / 323
ARCH(5) Model – NYSE Returns

316 / 323
Correlogram – Standardized ARCH(5) Residuals : NYSE
Returns

317 / 323
GARCH(1,1) Model – NYSE Returns

318 / 323
Correlogram – Standardized GARCH(1,1) Residuals :
NYSE Returns

319 / 323
Estimated Conditional Standard Deviation – GARCH(1,1)
Model : NYSE Returns

320 / 323
Estimated Conditional Standard Deviation – Exponential
Smoothing : NYSE Returns

321 / 323
Conditional Standard Deviation – History and Forecast :
GARCH(1,1) Model

322 / 323
Conditional Standard Deviation – Extended History and
Extended Forecast : GARCH(1,1) Model

323 / 323