Time Series Analysis:

An Introduction
 WHAT IS A TIME SERIES?
• Series of observations recorded sequentially over
a period of time (i.e. a collection of observations
recorded along with the time stamp)
• Represented as
(t,Yt ); t = 1,...,n
• Yt may be univariate (single variable) or
multivariate (collection of variables of interest)
• Frequency of observations: yearly, quarterly,
monthly, weekly, daily or tick (i.e. as and when
data changes)
8/2/23 Amit Mitra 2
 ANALYSIS OF TIME SERIES
• Identify the dominant feature (s) of an observed
time series
• Explain the movements of the time series
through an “appropriate” stochastic process and
study the characteristics of the process
• Identify possible cause-effect or lead-lag
relationship between related time series
• Fit (through estimation of parameters) a
statistical model and build a predictive model for
forecasting
• Build early warning prediction system for possible
future adverse events
8/2/23 Amit Mitra 3
 APPLICATION AREAS
• FINANCIAL: Exchange rate forecasting, share price
movements, credit card fraud detection
• ECONOMIC: Modeling of Economic fundamentals data,
e.g. GDP, GNP, inflation, employment rate
• BIO-MEDICAL: Monitoring/modeling ECG, EEG,
Diastolic & systolic pressure
• SPEECH: Digitized speech signal data-synthesis and
analysis
• METEOROLOGICAL: Forecasting and/or relationship
between rainfall, temperature, pressure, air pollution
• Wireless communication, radar signal, nowcasting in
aviation, demographic, ……….. ……..

8/2/23 Amit Mitra 4

 FINANCIAL TIME SERIES
DJIA (Dow Jones): daily Series

FTSE tick series

8/2/23 Amit Mitra 5

 FINANCIAL TIME SERIES
NSE Nifty: Daily Return Series

8/2/23 Amit Mitra 6

 FINANCIAL TIME SERIES
FOREIGN EXCHANGE RATES

INR/EURO Daily

INR/USD Daily
8/2/23 Amit Mitra 7
 FINANCIAL TIME SERIES
U.S. Weekly Interest Rates
Red line is 1-year; Blue line is 3-year
16
14
12
percent
10
8
6
4

01/05/1962 07/18/1969 01/28/1977 08/10/1984 02/21/1992 09/03/1999

Time in weeks

8/2/23 Amit Mitra 8

 ECONOMIC TIME SERIES
US Gross National Product (GNP) series

8/2/23 Amit Mitra 9

 Market Research Time Series

Walmart Daily Sales Data

2400000

2200000

2000000

1800000

1600000

1400000

1200000

1000000
22/01/10 10/08/10 26/02/11 14/09/11 01/04/12 18/10/12

8/2/23 Amit Mitra 10

SUNSPOT SERIES

The Wolf number (also known

as the International sunspot
number, relative sunspot
number, or Zürich number) is a
quantity that measures the
number of sunspots and
groups of sunspots present on
the surface of the sun. The
Sunspot Number is a crucial
tool used to study the solar
dynamo, space weather and
climate change.
11
 BIO-MEDICAL TIME SERIES
Digitized ECG signal data

8/2/23 Amit Mitra 12

 SPEECH PROCESSING
Digitized Speech signal

8/2/23 Amit Mitra 13

 SPEECH PROCESSING

Bat squeak/chirp Digitized Data

8/2/23 Amit Mitra 14

 Communication Signals

8/2/23 Amit Mitra 15

 MULTIVARIATE METEOROLOGICAL
TIME SERIES

8/2/23 Amit Mitra 16

 MULTIVARIATE TIME SERIES

8/2/23 Amit Mitra 17

WHAT TYPE OF DOMINANT FEATURES
CAN BE PRESENT IN A TIME SERIES?

• TIME TREND COMPONENT

• SEASONAL VARIATION COMPONENT
• LONG TERM CYCLICAL COMPONENT
• IRREGULAR COMPONENT

8/2/23 Amit Mitra 18

 Components of a Time Series
• Trend – smooth long term characteristics of a time series
• Seasonal Variation – Patterns of change in a time series within
a year which tends to repeat each year
• Cyclical Variation – the rise and fall of a time series over
periods longer than one year
• Irregular Variation – Random stochastic component

Additive Model OR Multiplicative Model of components

o STATISTICAL HYPOTHESIS TESTING APPROACH FOR TESTING EXISTENCE

OF THE COMPONENTS
o ESTIMATION OF SIGNIFICANT COMPONENTS

8/2/23 Amit Mitra 19

 Dominant Trend Component
• US annual population series

300000
Thousands

250000

200000

150000

100000

50000

0
0 5 10 15 20

8/2/23 Amit Mitra 20

 Dominant Seasonal and Trend
Component
Monthly Airline Passengers in USA

Observe the “structural break” in the time series

8/2/23 Amit Mitra 21

 Dominant Cyclical and Trend
Components

Positions of cycle:
Prosperity/Peak, recession, depression/ trough, recovery

8/2/23 Amit Mitra 22

 Aim of decomposition

Deterministic
components

Stochastic
component:

8/2/23 Amit Mitra 23

 Trend Estimation
Simple Moving Average Method
• Useful in smoothing time series to find its trend
• Equal importance to all values inside the MA
window
• 2 sided or 1-sided
• Padding, if required

Extensions
• Weighted MA
• Exponential Weighted MA

8/2/23 Amit Mitra 24

 Simple Moving Average

Year Values MA(3)

1995 20 NA
1996 24 (20+24+22)/3 = 22
1997 22 (24+22+26)/3 = 24
1998 26 (22+26+25)/3 = 24
1999 25 NA

8/2/23 Amit Mitra 25

 Three-year and Five-Year Moving Averages

8/2/23 Amit Mitra 26

 Weighted Moving Average
• A simple moving average assigns the same weight
to each observation inside the MA window
• Weighted moving average assigns different
weights to the observations inside the MA
window
• Most recent observation receives the highest
weight, the weights decrease for older data
values
• The sum of the weights = 1
8/2/23 Amit Mitra 27
 Weighted Moving Average

8/2/23 Amit Mitra 28

 Trend estimation using
least squares fitting

• A Linear time trend model : long term trend

of many time series may often be
approximated by a straight line

yt = a + b t
• Unknown constants estimated using least
squares

Amit Mitra
29
 Beyond Linear Trend
• Quadratic and higher order trends can also be
fitted using OLS

• Degree of the polynomial trend line may be

arrived at using statistical hypothesis testing
approach

• Nonlinear trend lines may also be appropriate

and can be fitted using least squares
8/2/23 Amit Mitra 30
 Seasonal Component Estimation
• Estimation depends on whether trend is present
or not.
• In case trend is present, we first estimate trend
(slow or fast changing) and then seasonal factors
are extracted from the de-trended times series.
• In the absence of trend, seasonal factors from
monthly or quarterly data are extracted directly
from the observed data.
• Estimation of the seasonal factors would depend
on the assumed additive or multiplicative
component model
8/2/23 Amit Mitra 31
Probabilistic Formulation of Time Series

• Time series is an example of a

stochastic process (collection of
random variables)

• Stochastic process is an indexed

collection of random variables
{Xt :t ÎT }
8/2/23 Amit Mitra 32
 Probabilistic Formulation of Time Series

𝑋 𝜔, 𝑡

belongs to sample space In time index set

• For a fixed t, 𝑋 𝜔, 𝑡 is a random variable

• For a given 𝜔, 𝑋 𝜔, 𝑡 is a sample function or
a realization as a function of t.

33
• We are concerned with processes
indexed by time, either continuous
time or discrete time processes

{Xt :t Î(-¥,¥)}= {Xt :-¥ < t < ¥}

or
{Xt :t Î{0,1,2,...}} = {X0 , X1, X2 ,...}

8/2/23 Amit Mitra 34

 Process Specification
• To describe a stochastic process
completely, we must specify all the
finite dimensional distributions, i.e.
the joint distribution of the random
variables for any finite set of time
points
{Xt , Xt , Xt ,.... Xt }
1 2 3 n
• e.g. A Gaussian process

8/2/23 Amit Mitra 35

 Process Specification

• A simpler approach is to only

specify the moments of the
underlying random process
• The mean and variance functions
are given by
µt = E ( X t ) and s t2 = V ( X t )

8/2/23 Amit Mitra 36

 Auto Covariance Function
• Because the random variables
comprising the system are usually not
independent (they are observed
sequentially over time), we must also
specify their covariance to extract the
time dependence structure
• ACF (
g t1 ,t2 = Cov X t1 , X t2 )
8/2/23 Amit Mitra 37
 Forms of Stationarity
• Strict stationarity: joint distribution
function of any collection of random
variables does not change with
location shift

• Weak stationarity or covariance

stationarity: mean and autocovariance
functions do not depend on time
points of reference
8/2/23 Amit Mitra 38
 Stochastic nature of Time Series:
Stationary Vs. Non-Stationary
STATIONARITY
• No change in means, no systematic change in
variability
• Process in a state of statistical equilibrium

• Analysis and modeling requires the time series to be

stationary (through transformation, if required)

• Co-integration analysis uses non-stationarity of a

particular nature for meaningful analysis

8/2/23 Amit Mitra 39

Examples of stationary time series
0.3

0.25

0.2

0.15

0.1

0.05

-0.05

-0.1

-0.15

-0.2

-0.25
0 10 20 30 40 50 60 70

White Noise
0.6
0.4
0.2
0
-0.2
-0.4
-0.6

8/2/23 Amit Mitra 40

 Some non-stationary time series
2.5

.7
2.25

2 .65

1.75 .6

1.5 Exchange Rate

Share Price
.55

1.25
.5
1
.45
.75
.4
.5

.25 .35

0 50 100 150 200 250 300 350 400 450 0 50 100 150 200 250

9.3

9.2
GNP
9.1

8.9

8.8

8.7

1960 1965 1970 1975

8/2/23 Amit Mitra 41

 APPROACHES OF TIME SERIES
ANALYSIS

• TIME DOMAIN ANALYSIS

• FREQUENCY DOMAIN ANALYSIS

8/2/23 Amit Mitra 42

 TIME DOMAIN ANALYSIS
• Study the covariance/correlation structure
between observations separated by a fixed unit
of time
• Use the above information to build models of the
form
Yt = f1 Yt -1 + f2 Yt -2 + ..... + f p Yt - p
+ e t + q1 e t -1 + q2 e t -2 + ...... + qq e t -q
• Auto correlation function (ACF) plays a central
role in time domain analysis

8/2/23 Amit Mitra 43

 TIME DOMAIN ANALYSIS
• Use the Auto Correlation structure for building
forecasting models based on the historical
data that can be used for forecasting future
path of the observed time series.
• Parameter estimation, model selection, model
validation and residual diagnostics are
important concepts under this approach.

8/2/23 Amit Mitra 44

 FREQUENCY DOMAIN ANALYSIS
• Extract hidden periodicities in the time series
• Spectral Density Function plays a central role
• Study the structure of spectral density
function of standard probability models (WN,
AR, MA, ARMA); estimation of spectral density
function and periodogram analysis are
important concepts.

8/2/23 Amit Mitra 45

 FREQUENCY DOMAIN ANALYSIS
8 4

6 3.5

4 3

2 2.5

0 2

-2 1.5

-4 1

-6 0.5

-8
0 20 40 60 80 100 120 140 160 180 200 0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

Time series Periodogram

• 2-component sinusoidal model: frequencies at
0.25 & 0.30
( ))
2
( )
y ( t ) = å a k0 Cos 2pwk0t + b k0 Sin 2pwk0t + e ( t ) ; t = 1,..., n (
k =1

8/2/23 Amit Mitra 46

 FREQUENCY DOMAIN ANALYSIS
15 4.5

4
10
3.5

5 3

2.5
0
2

-5 1.5

1
-10

0.5

-15
0 50 100 150 200 250 300 350 400 450 500 0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

Time series Periodogram

• 4-component sinusoidal model: frequencies at
0.14, 0.15, 0.21 & 0.27

8/2/23 Amit Mitra 47

 FREQUENCY DOMAIN ANALYSIS
• Time domain analysis and frequency domain
analysis are not mutually exclusive

• Any covariance stationary process has both

time domain as well as frequency domain
representation
• Inter relationship between ACF and spectral
density function

8/2/23 Amit Mitra 48

 STATISTICAL MODELS OF TIME SERIES
LINEAR TIME SERIES MODELS

• AUTO REGRESSIVE (AR)

• MOVING AVERAGE (MA)
• AUTO REGRESSIVE MOVING AVERAGE (ARMA)
• AUTO REGRESSIVE INTEGRATED MOVING AVERAGE (ARIMA)
• SEASONAL ARMA (S-ARMA)
• SEASONAL ARIMA (S-ARIMA)
• ARMA EXOGENOUS (ARMAX)

MULTIVARIATE VERSION OF THE ABOVE MODELS

• VAR, VMA………

8/2/23 Amit Mitra 49

 NONLINEAR MODELS

• THRESHOLD AR (TAR)
• SELF EXCITED TAR (SETAR)
• AUTOREGRESSIVE CONDITIONAL
HETEROSKADASTIC (ARCH)
• GENERALIZED ARCH (GARCH)

• SUPERIMPOSED SINUSOIDAL MODEL

• ARTIFICIAL NEURAL NETWORK (ANN) MODELS

8/2/23 Amit Mitra 50
 Box-Jenkins Approach of Time Series
Modelling
• The approach involves the following
steps:
—model identification
—model fitting
—diagnostic checking

8/2/23 Amit Mitra 51

 Forecasting Guidelines
• Simplest (least number of parameters) model
to describe the data
– principle of parsimony

• No pattern in forecast error

– ei = (Actual Yi - Forecast Yi)
– Seen in plots of errors over time
 Principal of Parsimony
• Suppose two or more models provide
“good fit” for data

• Select the Simplest Model-less number of

model parameters
 Pattern of Forecast Error
Trend Not Fully Accounted
for Desired Pattern

Error Error

0 0

Time (Years) Time (Years)

 Residual Analysis
e e

0 0

T T
Random errors periodic effects not accounted for

T
Trend not accounted for
 MODELING OF TIME SERIES
350

300

250

200

150

100

50

0
May-83

May-85

May-87

May-89

May-91

May-93

May-95

May-97

May-99

May-01

May-03

May-05

May-07
• Actual monthly index for industrial production (IIP) and
fitted seasonal-arima model
8/2/23 Amit Mitra 56
 FITTING OF ECG SIGNAL
Fitted Against Observed
500

400

300

200

100

-100

-200
0 100 200 300 400 500 600

• A MULTIPLE SINUSOIDAL MODEL FIT OF ECG DATA

8/2/23 Amit Mitra 57
 MODELING OF SPEECH SIGNAL DATA

MLE BASED MULTIPLE SINUSOIDAL MODEL FIT

8/2/23 Amit Mitra 58

Neuro-Genetic forecasting model of NIFTY
NSE Nifty

1300
Index

1100

900

700

Neural Net Actual

8/2/23 Amit Mitra 59

8/2/23
-40
-30
-20
0

-10
20
30

10
Aug-82

Feb-83

Aug-83

Feb-84

Aug-84

Feb-85

Aug-85

Feb-86

Aug-86

Feb-87

Aug-87

Feb-88

Aug-88

Feb-89
Actual

Aug-89

Feb-90

Amit Mitra
Aug-90

Feb-91

Aug-91
Predicted

Feb-92

Aug-92
IIP (growth rate)

Feb-93

Aug-93

Feb-94

Aug-94

Feb-95

Aug-95

Feb-96
Neuro-Genetic forecasting model of

Aug-96

Feb-97
60
 Some Standard Text Books
• Introduction to Time Series and Forecasting-
P.J.Brockwell & R.A.Davies

• Time Series: Theory & Methods- P.J.Brockwell &

R.A.Davies

• Time Series Analysis- J.D. Hamilton

• Introduction to Statistical Time Series – Wayne A.

Fuller
8/2/23 Amit Mitra 61
8/2/23 Amit Mitra 62