L-3: IMF’s Vulnerability

y Exercise
for Emerging Economies (VEE)

IMF Institute for Capacity Development

Early Warning Exercise Presenter
HQ14.17 Seung Mo Choi

This training material is the property of the IMF’s Institute for Capacity Development
(ICD) and is intended for use in ICD courses. Any reuse requires the permission of ICD.
 Content Outline

I Structure of the VEE

I.

II. Indicators and Their Use

III. Translating Indicators to Vulnerabilities

IV. Other Components: Global Environment,

Spillovers, Bubbles

Appendix: Vulnerability Exercise for Advanced

Economies (VEA)

2
 Early Work on Early Warning Models
• Interest in Early Warning Models increased in the aftermath
off crises
i iin the
h 1990
1990s
– Frankel and Rose (1996 JIE) use probit regressions to
estimate currency crises
– Kaminsky, Lizondo and Reinhart (IMF SP 1998) develop an
early warning system for currency crises based on a
collection
ll off signals
l ffor whether
h h indicators
d cross a
threshold value
– Berg and Patillo (IMF SP 1999) uses monthly data and a
regression framework to predict currency crises
– Berg et al. (IMF OP 186, 1999) reviews the performance of
various models.
3
 What is the VEE?
• Assessment of underlying vulnerabilities and crisis risks in
Emerging Market (EM) economies
– Classifies an economy as having a high (red), medium
(yellow), or low (green) vulnerability in each sector
(external, public, financial and real) and overall.
• Began 2001. (Methods periodically updated, especially in
2007, 2011 and 2014.)
– Semi-annual
Semi annual exercise
– Collaboration between functional and area departments
– Increased
I d iinteraction
t ti with
ith market
k t participants
ti i t
4
 What is the VEE?
• Informs IMF staff’s
staff s surveillance of EM economies.
economies
– Not published.
– Th
The purpose off thi
this llecture
t iis nott tto llearn h
how th
the IMF
exactly does the VEE.
–R
Rather,
th it iis tto understand
d t d th the overallll methodology
th d l ffor
reference when you contribute to developing your
country’s
y EWE.

5
 Performance of the VEE
• Independent Evaluation Office’s
Office s report (2011) on the
IMF’s performance in the run-up to the crisis points to
the success of the VEE in identifying crisis-prone
emerging markets
– Report is fairly critical of the IMF’s failures in bilateral
andd multilateral
ltil t l surveillance
ill i th
in the run-up to
t the
th
crisis, with the VEE being one of the few exceptions

• Unfortunately, exercise did not cover advanced countries

prior to the crisis
p

6
 Predicting Crises is Hard!
• Predictingg Crises (timing)
( g) is veryy hard

• Even when vulnerabilities identified, it may take several

years for
f a crisis
i i to materialize
i li

• Instead of trying to predict timing of crises,

crises it seems
more promising to focus on identifying vulnerabilities

• Vulnerabilities are necessary but not sufficient for crises

– For example, global conditions matter

7
 Predicting Crises and
Identifying
d if i Vulnerabilities
l bili i
• Suppose focus is only on predicting crises
crises:
– Crises are rare (e.g., 5% of sample). If a country has
predicted probability of 10% we may not “flag”
flag it
because there is a 90% chance it will be OK

• If instead we focus on vulnerabilities:

– That country likely has many indicators that point to
weaknesses vis-à-vis other countries
– Focusing on those vulnerabilities helps identify
problems
bl ahead
h d off time,
i andd inform
i f policy
li advice
d i
8
 Process
Vulnerability Indicators
There are 3 main choices
“Crisis Risk Model” involved in the exercise:

Vulnerability Index (VI) - Crisis definition

- Indicators to be
conside ed
considered
Indicator-Based Vulnerability Rating
- How to combine
(High Medium
(High, Medium, Low) information from
- Judgment indicators into a measure
- Area Departments’ “Intelligence” of vulnerability

Final Vulnerability Rating

9
 Content Outline

I
I. Structure of the VEE

II. Indicators and Their Use

III. Translating Indicators to Vulnerabilities

IV. Other Components: Global Environment,

Spillovers, Bubbles

Appendix: Vulnerability Exercise for Advanced

Economies (VEA)

10
 Types of Crises
• Recall
eca our
ou discussions
d scuss o s oon types
ypes o
of ccrises
ses in L-1.
• For Emerging Markets (EMs), our focus is on capital
account crises
• Capital account crises are marked by a “sudden stop” in
capital flows, and often involve
– exchange rate pressure (sometimes leading to a
devaluation)
– debt sustainability concerns (sometimes leading to a
default)
– financial sector fragility

11
 Types of Crises
• Although the VEE’s
VEE s focus is on capital account crisis, the
following three crisis types are also considered. (These are
formally analyzed in VE for Advanced Economies.)
– Fi
Financial
i l Crisis―either
Cii ith a systemic
t i b banking
ki crisis
i i or a
currency crisis (Laeven and Valencia 2012 IMF WP)
– Growth Crisis―significant
Crisis significant slowdown in growth rela ve to
trend (difference in growth between year t and average in
years t-5 to t-1 in the bottom 5 percent of sample as a whole)
– Fi
Fiscall Consolidation
C lid ti Pressures―an
P i
increase iin th
the cyclically
li ll
adjusted primary balance as a ratio to GDP of at least 2.5
percentage points, from a negative balance of at least 2.5
percentage
t points,
i t dduring
i ththe course off th
the year
12
 Process
4 sectors: External,
public, financial,
Vulnerability Indicators
and real sectors.

“Crisis Risk Model” Each sector has

several indicators.
Vulnerability Index (VI)
Advantages of looking
at various indicators
i diff
in differentt sectors:
t
- More information
Indicator-Based Vulnerability Rating
- Reduce overall
(High Medium
(High, Medium, Low) measurement error
- Judgment - Minimize large
- Area Departments’ “Intelligence” jumps in overall
indicator
Final Vulnerability Rating
13
Indicators

14
Indicators

15
Indicators

16
 Content Outline

I
I. Structure of the VEE

II. Indicators and Their Use

III.Translating Indicators to
Vulnerabilities

IV. Other Components: Global Environment,

Spillovers, Bubbles

Appendix: Vulnerability Exercise for Advanced

Economies (VEA)
17
 Process
Vulnerability Indicators
Combines information
for a wide set of
“Crisis Risk Model” vulnerability
indicators.
Vulnerability Index

Calculated
l l d at sectorall
Indicator-Based Vulnerability Rating and economy-wide
level.
(High Medium
(High, Medium, Low)
- Judgment
- Area Departments’ “Intelligence”

Final Vulnerability Rating

18
 Crisis-prone?
Structure of
Apply threshold.
threshold
0=No or 1=Yes V l
Vulnerability
bilit IIndex
d
Indicator 1 0 or 1
Indicator 2 0 or 1
External
Indicator 3 0 or 1 Weighted =
Average
Sector
Indicator 4 0 or 1
Index
… 0 or 1

Indicator 1 0 or 1
Indicator 2 0 or 1
Indicator 3 0 or 1 Weighted = Public Sector
Average Index
Indicator 4 0 or 1
Overall
… 0 or 1 Weighted
Average = Vulnerability
Indicator 1 0 or 1
Indicator 2 0 or 1 Financial
Index
Weighted =
Indicator 3 0 or 1
Average
Sector
Indicator 4 0 or 1 Index
… 0 or 1

Indicator 1 0 or 1
Indicator 2 0 or 1
Weighted =
Real Sector
Indicator 3 0 or 1 Average Index
Indicator 4 0 or 1
19
… 0 or 1
 Threshold Approach: Overview
• Identify a threshold for an indicator above/below which
crises are more prevalent.
• Pick threshold that minimizes the percentage of crises
missed
i d (“t
(“type 1 error”)”) and
d th
the percentage
t off non-crises
i
misclassified (false alarms) (“type 2 error”).

Loss Function: Minimize the sum of two

errors
Missed crises False Alarms
(Type I error) (Type II error)

20
 PDF and CDF
• PDF (probability density function): Describes the relative
likelihood for a random variable X to take on a given
value x.

• CDF (cumulative distribution function): Illustrates a

probability that a random variable X will be found to
have a value less than or equal to x.

21
 Missed Crises (Type I Error)
Crisis subsample
Threshold
CDF PDF Threshold

Missed
Crisis
(Type I
Error)

HIGH VULNERABILITY

Increased risk of crisis, Increased risk of crisis,

e.g., current account deficit e.g., current account deficit

22
 False Alarms (Type II Error)
Non-crisis subsample
p
Threshold
CDF PDF Threshold

False
Alarms
(Type II
Error)

HIGH VULNERABILITY

Increased risk of crisis, Increased risk of crisis,

e.g., current account deficit e.g., current account deficit

23
 Optimal Threshold
CDF
Threshold

Non crises Crises

Threshold
maximizes the
distance
between the two
CDFs

Increased risk of crisis,

e.g., current account deficit

24
 Threshold Approach
• Good:
ood Threshold
es o d rule
u e minimizes
es problems
p ob e s due to o out
outliers
es
– It signals that a 5% CA deficit is a source of concern
without imposing functional forms on the data (e.g.,
imposing a 10% deficit to be twice as bad)

• Bad: Thresholds can be sensitive to relatively small

changes in data
– Since we willing to trade type 1 for type 2 error,
error
different thresholds that are far apart may give similar
((type
yp 1 + type
yp 2)) but with errors concentrated in a
particular type. (See the next slide.)
25
 Sensitivity of Thresholds: Example
Reserve Cover

1
Crises

.8

Non-Crises
.6
.4
.2
0

0 100 200 300

(% of Ext. Debt + Curr. Acct. Deficit)

• Gap between the lines is similar at two different points.

26
 Aggregation

• Combining 0-10 1 Values into Sector Index: Define zi=

(fraction crises missed) + (fraction of non-crises
misclassified).
– Use within-sector weights wi = (1-zi)/zi.
– So lower errors get higher weights.

• Combining Sector Index into the Overall Vulnerability

Index: Sectoral weights determined by judgment.
– Focusing on capital account crisis, put a higher weight
on external
t l sector.
t
27
 What Are Pros of This Approach?
• Allows
o s to
o co
consider
s de large
a ge number
u be ofo variables
a ab es
– More traditional regression approach would limit
number of variables that could be meaningfully
considered
– Inclusion of correlated variables does not affect
univariate
i i t thresholds
th h ld

• Minimizes information lost due to missing data.

data
Traditional regression/probit approach would drop a lot
of data ggiven heterogeneous
g data coverage
g across
countries/time
28
 What Are Pros of This Approach?
• Threshold
es o d rule
u e minimizes
es p
problems
ob e s due to outliers
out e s
– It signals that a 5% CA deficit is a source of concern
without imposing functional forms on the data (e.g.,
imposing a 10% deficit to be twice as bad)

• While individual thresholds are sensitive, the overall

aggregate score (combining different indicators) tends to
be fairly robust
robust. Composite score yields fairly sensible
results.

29
 Pruning the Model
• Some
So e o of thee weaker
ea e indicators
d ca o s have
a e a very
e y poor
poo fit,, and
a d
will not influence the aggregate score much even if kept

• But some “cleaning-up” is appropriate, particularly if

several alternative versions of the same variable are
considered
id d

30
 Process
Vulnerability Indicators

“Crisis Risk Model”

Vulnerability Index

Use some cutoffs to

the overall
Indicator-Based Vulnerability Rating vulnerability index to
(High Medium
(High, Medium, Low) have “high”
high ,
- Judgment “medium”, or “low”
- Area Departments’ “Intelligence” vulnerability rating.

Final Vulnerability Rating

31
 Determining the Ratings
• The cutoffs between the ratings are arbitrary by nature, but
here is the general rule.

• The “high”
high vs.
vs “medium”
medium cutoff is (again!) the minimization of
type-1 and type-2 errors when the overall VI is used as an
indicator.

• The “medium” vs. “low” cutoff is the level in which type-1

errors are roughly a half of “high” vs. “medium” cutoff.

• These cutoffs (for overall VI) are (usually) applied for sectoral
VIs too
VIs, too, to determine sectoral ratings.
ratings

32
 n.a.
L
Country Heat Map
M
H

Countries with high underlying vulnerabilities overall Countries with medium underlying vulnerabilities overall
External Public Financial Real External Public Financial Real
Country Overall Country Overall
sector sector sector sector sector sector sector sector

Country 14
C M M H H M
Country 1 H H H H H
Country 15 M M H L H
Country 2 H H H H H Country 16 M M M H n.a.
Country 3 H H H H H Country 17 M M M M H
Country 4 H H H H M Country 18 M H M M na
n.a.
Country 5 H H H H n.a. Country 19 M M M H L
Country 6 H H H M L Country 20 M M H L n.a.
Country 7 H H H M n.a. Country 21 M H M L L
Country 22 M M H L n.a.
Country 8 M M L M H
Country 23 M L H L n.a.
Country 9 M H L H M Country 24 M H L L n.a.
Country 10 H H M H n.a. Country 25 M M M M n.a.
Country 11 H H M M H Country 26 M M M M L
Country 12 H M H M L Country 27 M L M M M
Country 13 H M L H L Country 28 M M M L L

33
 Sectoral Risk
Fiscal Risks 0.8 External Risks 0.8 Financial Risks
0.8

0.7 0.7
0.7

0.6 0.6 0.6

0.5 0.5 FLAT 0.5

0.4 0.4 0.4

'07F'08S'08F'09S'09F'10S'10Fn '07F '08S '08F '09S '09F '10S'10Fn '07F '08S '08F '09S '09F '10S'10Fn
1/ Final ratings. Computed as the unweighted averages across countries with the following
weights:
i ht HiHighh=1 1, M
Medium
di =0
0.5,
5 LLow = 0
0. Based
B d on Fi
Finall R
Ratings
ti i f
informed db
by th
the new
methodology which includes new indicators for the financial sector.

34
 VEE “Riskiest
Riskiest Countries
Countries”
Fiscal

Countries rated “M” or

“H”

External Financial

35
 Content Outline

I
I. Structure of the VEE

II. Indicators and Their Use

III. Translating Indicators to Vulnerabilities

IV. Other Components: Global

Environment, Spillovers, Bubbles

Appendix: Vulnerability Exercise for Advanced

Economies (VEA)

36
 Process
Vulnerability Indicators

“Crisis Risk Model”

Vulnerability Index

Indicator-Based Vulnerability Rating Global Environment,

Spillovers, Bubbles
(High Medium
(High, Medium, Low)
- Judgment
- Area Departments’ “Intelligence”

Final Vulnerability Rating

37
 Global Environment
• Global
oba liquidity
qu d y fuels
ue s
capital flows.

• This may affect the

likelihood of crisis.

38
 Spillovers
• AE to
o EM sp
spillovers
o es
– Example: As the probability of a U.S. recession rose in
2010, financing costs for EMs rose, too.
– So the VE for Advanced Economies should be
integrated with VEE.

39
 Bursting Bubbles? China Asset Price
I l i
Implosion
China’s property
price
i G 20 EMs: Trade exposure to China
G-20
50 (Average 2008-09)
bubble bursts…
45

40 Exports to China (in total exports)

…depressing
domestic demand 35 Exports of goods to GDP

30

25
Growth slowdown
to 4 percent, 20
from 10 percent 15

10

EM equity and bond 5

sell-off
sell off akin
0
to Asia crisis

40
 …With Large Impact on EM Activity and
Commodity
di Prices
i
Peak Output Loss Impact on Commodities Prices
(percent of GDP))
(p
(percent)
3.5 0

Financial Channel
3.0
-10
Trade Channel
2.5
-20
2.0
-30
30
1.5

-40
1.0

0.5 -50

0.0 -60
Metals prices Oil prices

41
 Conclusion
• VEE is a component of the IMF’s
IMF s EWE. You can try to develop a
similar method (or a better one) for your purpose.
• We learned:
– How to construct thresholds and aggregate indicators
– Characteristics of the VEE
• Advertisement:
– Hands-on exercise in the afternoon workshop
p
– Will cover sectoral issues (fiscal, external and financial) on
Wed and Thu (L-4, L-5 and L-6)
– Will discuss more on EWE in general on Fri (L-8 and L-9)
42
 Content Outline

I
I. Structure of the VEE

II. Indicators and Their Use

III. Translating Indicators to Vulnerabilities

IV. Other Components: Global Environment,

Spillovers, Bubbles

Appendix: Vulnerability Exercise for

Advanced Economies (VEA)

43
 Vulnerability Indicators
Prior to Prior to
2014 2014
Indicator AEs EMs Indicator AEs EMs
External Sector Financial Sector
Reserve Coverage (ST Debt and Projected CA Deficit) √ Inflation √
Current Account/GDP √ √ Capital Adequacy Ratio (Banks) √ √
External Debt/GDP √ Return on Assets (Banks) √ √
External Debt/Exports of Goods and Services √ Nonperforming loans (% of total) √
Real export growth Foreign liability (% of domestic credit) √

Real Effective Exchange Rate Overvaluation √ Loan/deposits ratio √

CGER Current Account Norm Deviation √

Private sector external debt √

Public Sector Corporate/Real Sector

General Government Balance/GDP
/ √ √ Black-Scholes-Merton Default Probabilityy √ √
Primary Gap √ √ Return on Assets (Corporate) √
General Government Gross Debt/GDP √ √ Price/Earnings Ratio (Corporate) √ √
Public Debt Exposed to Currency Risk √ Interest Coverage ratio √
Public Debt Exposed to Rollover Risk √ Real GDP Growth √

44
 Vulnerability Indicators
Prior to
2014
Indicator AEs EMs
Medium Term Variables

House Prices √
Stock Prices √
Private Credit √ √
Construction sector contribution to GDP growth √
Financial sector contribution to GDP growth √
Prior to
2014
AEs EMs
Household Sector
House Price Acceleration √
Stock Price Acceleration √
H
Household
h ld Li
Liabilities/GDP
biliti /GDP √
Interaction (Household Liabilities)*(Medium-Term House Price Increase) √
Interaction (Household Liabilities)*( House Price Acceleration) √

45
 Constructing the Vulnerability Index
• Crises in advanced economies are less well-understood than
EM crises, so we let the data speak more
– Unlike in the VEE,, weights
g are determined solelyy byy
goodness of fit wi = (1-zi)/zi, not allocated across sectors
by judgment
• Adjustment for collinearity
– Group indicators whenever correlation between any two
sets of 0-1
0 1 flags is above 0.4,
0 4 reassign weights so total
weight is equal to highest individual weight within the
g p
group

46
Crisis Risk Models—Performance
Based on data through 2006,
would have identified vulnerabilities in advanced
economies
i att the
th epicenter
i t off th the crisis
i i
30
Financial crisis vulnerability
25

20

15 IIceland,
l d U U.S.
S
and U.K.
10

Other
countries
5

0
2003 2004 2005 2006 2007

47
Crisis Risk Models—Performance
Fall 2009 Predicted Values for Growth Crisis Over 2-year Horizon
25
Growth Crisis Vulnerability

20 Greece, Italy,
Portugal and Spain
Portugal,

15

10
Other
countries

0
2003 2004 2005 2006 2007 2008 2009 2010

48