Unit 13: Problems and tricks with IV estimation

C. Zulehner: Introductory Econometrics 1 / 32

Outline

1 Exogenous and Poor Instruments

2 “Testing” the power of the instruments

3 “Testing” for endogeneity

4 Testing over-identifying restrictions

5 Other tests on assumptions

C. Zulehner: Introductory Econometrics 2 / 32

1. Exogenous and Poor Instruments

Can we say something about the quality of our instruments?

We can test whether the correlation between the instruments and the endogenous
variables is high (i.e. we can check the condition E (Z0 X) = ΣZX 6= 0
What if our assumption that E (Z0 u) = 0 is false?
I The IV estimator will be inconsistent, too
I As in the OLS case we cannot test whether E (u|Z) = 0 since u are
unobservable and û are derived by using the condition we would like to test
I The defense of your identification assumptions (i.e. E (u|X) = 0 or E (u|Z) = 0
) has always to be of an economic nature!
So when should we use OLS and when IV?
We can compare the asymptotic biases of OLS and IV

C. Zulehner: Introductory Econometrics 3 / 32

The Effect of Poor Instruments
Let’s take the plims (and consider the SLR for simplicity)
p
Cov (x, u) Cov (x, u) Var (u)
OLS: plimβ̃1 = β1 + plim = β1 + plim p p p
Var (x) Var (x) Var (u) Var (x)
σu
= β1 + ρ(x, u) ·
σx
√ Cov (z,u)
√ p
Cov (z, u) Var (z) Var (u) Var (u)
IV: plimβ̂1 = β1 + plim = β1 + plim Cov (z,x)
p
Cov (z, x) √ √ Var (x)
Var (z) Var (x)

ρ(z, u) σu
= β1 + ·
ρ(z, x) σx

So we should:
I Prefer IV over OLS if Corr (z, u)/Corr (z, x) < Corr (x, u)
I But in the other case, IV makes thing even worse!
I This is for instance the case when the Corr (z, x) is very small, i.e. when the
instruments are “poor”

C. Zulehner: Introductory Econometrics 4 / 32

The Effect of Poor Instruments: Relative Bias

The variance of the IV estimator is always larger than the variance of the OLS
estimator
By how much depends on the partial R 2 of the first stage regression and thus on
the predictive power of the instrument
IV more biased than OLS when:

Bias(β̂IV ) ρ(z, u)
= >1
Bias(β̂OLS ) ρ(z, x)ρ(x, u)

I pay a lot of attention to the failure of orthogonality of the instrument as even

a small departure may lead to serious inconsistency
I double your attention if instrument is weak, which you can check
I ceteris paribus, rely on IV when you deem endogeneity is really serious (ρ(x, u)
is large)

C. Zulehner: Introductory Econometrics 5 / 32

C. Zulehner: Introductory Econometrics 6 / 32
The Effect of Poor Instruments

The key element to understand the “quality” of our IV procedure is the relative bias
2. Instrument Exogeneity
z }| {
ρ(z, u)
ρ(z, x) ρ(x, u)
| {z } | {z }
3. Instrument Power 1. Endogeneity

where
0
1 endogeneity: plim XNu = ρ(x, u) 6= 0
0
2 instrument exogeneity:plim ZNu = ρ(z, u) = 0
0
3 instrument power: plim XNZ = ρ(z, x) = large

C. Zulehner: Introductory Econometrics 7 / 32

2.“Testing” the power of the instruments

The assumption that the instrument(s) is correlated with the potentially

endogenous variables should always be tested
How do we do this?
1 Regress the endogenous variable on all the excluded instruments (variables
excluded in the second stage) and all the included instruments (the
exogeneous variable that also appear in the second stage)
2 If we have multiple endogenous variables, we repeat this for all endogenous
variables
3 Look at the partial R 2 of the excluded instruments: a "high"partial R 2 ⇒
instruments are not weak
But how high should it be?

C. Zulehner: Introductory Econometrics 8 / 32

Power of the Instrument: An alternative

An alternative strategy is to test the joint significance of the excluded instruments

by computing a statistic
A large F (or partial R 2 ) is evidence of non-weak instruments
In the over-identified case one can show (Bound, Jaeger, Baker) that:

Bias(β̂IV ) 1
∼
Bias(β̂OLS ) F

where F is the statistic of excluded instruments. Hence

1 F = 1: IV is as biased as OLS
2 F = 2: IV bias is half of OLS bias
3 F = 100: IV bias is 1/100=1% of OLS bias
One can use this simple measure to judge about the power of the instruments

C. Zulehner: Introductory Econometrics 9 / 32

Power of the Instrument: More Formally
Stock and Yogo (2005) provide a test (and its critical values) for the null
hypotheses that the instrument is "too weak"to be reliable
The null hypothesis of the test is

Bias(β̂IV )
H0 : > k%
Bias(β̂OLS )

1 Compute the F for the excluded instruments in the 1st stage regression
2 Check whether F > critical value (computed by Stock & Watson). Critical
values depend on the number of instruments and the number of endogenous
variables

Power of the Instrument: Example

Suppose k = 10% and one endogenous variable
With 3 instruments the critical value is 9.08; with 4 instruments it is 10.27
If F > 9.08 you reject H0 that instruments are weak

Rule of thumb: no weak instruments if F > 11 (sometimes one sees F > 10)

C. Zulehner: Introductory Econometrics 10 / 32

C. Zulehner: Introductory Econometrics 11 / 32
Example: Testing the Putnam’s conjecture
Putnam argues that much of the differences in economic development between
north and southern Italy are due to different endowments of social capital
But where is social capital coming from?
He conjectures regional differences in social capital can be traced back to the free
city state experience in 1000-1300 which was absent in the south
But many differences between south and north so conjecture is untested
Guiso, Sapienza, and Zingalesa focus on within northern Italy variation in history of
independence
I Not all major cities in Center North became free cities between XII-IVX century
I Use heterogeneity in history across cities within Center-North
I Correlate social capital today to history as a city state
a
See, Guiso, Sapienza, Zingales, 2009, Long Term Persistence,, NBER Working Papers, 14278

C. Zulehner: Introductory Econometrics 12 / 32

Free cities in medieval Italy

C. Zulehner: Introductory Econometrics 13 / 32

Example: Testing the Putnam’s conjecture – OLS Estimates
Social capital measure: number of voluntary organizations
Main explanatory variable: Free city
Controls
I Geography: altitude, steepness, proximity to the sea, location on Roman road
I City size : population linear, square
I Inequality in endowments: income and land ownership

OLS Estimates Controls No large towns No province capitals

Free City 1.1736*** 0.9293** 1.6961***
Problem with OLS: Free city may be picking up some unobserved variable
I Must be very persistent, e.g. an important geographical control
I Other reasons for endogeneity likely unimportant (e.g. reverse causality? What
about measurement error? )

C. Zulehner: Introductory Econometrics 14 / 32

Example: Testing the Putnam’s conjecture – OLS Estimates
History suggests two potential instruments
1 Whether a city was a Bishop city in year 1,000
I Lack of authority was initially made up with an informal agreement among the
main families to run the city (Patto giurato)
I Presence of Bishop facilitated coordination and made easier to run the pact
and obtain independence; Bishop towns in year 1,000 became such in IV-VII
century
2 Whether it was founded by the Etruscans
I Etruscans were organized in independent city states
I For this they built cities that were easier to defend ⇒ an easy-to-defend city is
more likely to gain independence

C. Zulehner: Introductory Econometrics 15 / 32

How to defend your instruments

Use and check intuition: a picture of Orvieto, the capital of the Etruscans! Do whatever you can to
persuade reader your instruments are good

C. Zulehner: Introductory Econometrics 16 / 32

First-stage regression: Free city

C. Zulehner: Introductory Econometrics 17 / 32

Example: Testing the Putnam’s conjecture – Remarks on First-stage
No evidence that instruments are week:
I The partial R 2 for the excluded instruments is high
I The F test is very high
But do not limit to just checking the size of F
Is effect of instruments on endogenous variables consistent with priors?
I In this case yes
I This lends support to the identification strategy as sign of effect of
instruments is as expected
In general first stage is more than just a way of getting an F-test: it is informative
about whether the chosen instruments are meaningful

C. Zulehner: Introductory Econometrics 18 / 32

Should we always add instruments?

Sometimes we face the issue of whether to add instruments. There is a trade off:
I An extra instrument may help identification because adds an extra source of
variation
I But if extra instrument weak, it may worsen the bias of IV!

C. Zulehner: Introductory Econometrics 19 / 32

3. “Testing” for endogeneity: Durbin-Wu-Hausman
Test

in absence of endogeneity problems, we would prefer OLS to IV because, while

they are both consistent, the former is more efficient (OLS is BLUE!)
While we cannot really test endogeneity, we can test whether OLS is better than IV
I H0 : No endogeneity – OLS and IV are consistent but OLS is more efficient –
they should be similar (i.e. β̂ IV = β̂ OLS )
I H1 : Endogeneity – OLS is NOT consistent while IV is consistent – they should
differ
The test statistic is:

H = (β̂ IV − β̂ OLS )0 [Var (β̂ IV − β̂ OLS )]−1 (β̂ IV − β̂ OLS ) ∼ χ2q

I with degrees of freedom determined by the dimension of β

A high value of H implies rejection of the null of exogeneity of X

C. Zulehner: Introductory Econometrics 20 / 32

A generalization of the Hausman Test

The idea of the Hausman test is therefore to see if the estimates from OLS and IV
are similar. If they are different, we conclude that the variable(s) that we
instrumented for must be endogenous
Where is the problem with this test?
I We still have to assume that the instrument are exogenous, i.e. that
E [u|Z] = 0
I If this is not true (i.e. the instruments are invalid), IV is also not consistent
Hence, the Hausman test cannot differentiate between these two failures: the test
of the assumption about the exogeneity of X is conditional on the instruments
being truly exogenous

C. Zulehner: Introductory Econometrics 21 / 32

Example: Testing the Putnam’s conjecture – Hausman Test
IV regression: ivreg2 totassoc_p (freecity = bishop etruscan)..
Store results: est store freecity
OLS regression: reg totassoc_p freecity ...
Test: Hausman freecity
I b: consistent under H(0) and H(A), obtained from IV regression
I B: inconsistent under H(A), efficient under H(0), obtained from OLS
I Test: H(0): difference (b-B) in coefficients not systematic
I χ2 (q) = (b − B)0 (Vb − VB )−1 (b − B)
I if χ2 (q) ≥ critical value, then reject H(0)
I b=1.0285 and B=1.1736
I χ2 (q) = 0.18 with Prob> χ2 = 0.6748

C. Zulehner: Introductory Econometrics 22 / 32

Testing for endogeneity - A regression example

While it is a good idea to see if IV and OLS have different implications, it is easier
to use a regression “test” for endogeneity
If x2 is endogenous, then η2 (residuals from the reduced form equation) and u1
from the structural model will be correlated
I save the residuals from the first stage
I include the residuals in the structural equation (which of course has x2 in it)
and estimate it with OLS
I if the coefficient on the (predicted) residuals η̂2 is statistically different from
zero, reject the null of exogeneity
I we may want to use a heteroskedasticity-robust t-test
I if there are multiple endogenous variables, jointly test the residuals from each
first stage

C. Zulehner: Introductory Econometrics 23 / 32

4. Testing over-identifying restrictions

Can we test the exogeneity of the instruments, i.e. plim(Z0 u/N) = 0?

In general not because the disturbances are not observed
Furthermore, in the exact identified case (i.e. when we only have one instrument for
a potentially endogenous variable) we cannot use û to replace u the instrumental
variable estimator is constructed precisely imposing Z0 u = 0
If we have multiple instruments, however, it is possible to use these over-identifying
restrictions to see if some of the instruments are correlated with the error

C. Zulehner: Introductory Econometrics 24 / 32

The over identifiation test: Intuition

Suppose you just have one endogenous regressor and two instruments Z1 and Z2
We can estimate the model using only Z1 , compute the residuals ^
u(Z1 ) and then
test whether Z2 0^
u/N = 0
We could do also the opposite to test wether Z1 is uncorrelated with the error term
In practice we test each instrument against the residuals that we obtain in the 2SLS
estimator using X̂ as instrument

C. Zulehner: Introductory Econometrics 25 / 32

The over-identifying test: Sargan J statistic

If each instrument is valid then they should all give a consistent estimate of β
Hence all should converge to the same estimate
Problem: if the null that instruments are valid is rejected how should we interpret
it?
I It could be that instruments are invalid
I But it could also be that the model is incorrectly specified because instruments
belong to the second stage
The test statistic that we use (Sargan J statistic) is

(y − Xβ̂ IV )0 PZ (y − Xβ̂ IV )
J= ∼ χ2q−k
σû2

This is the minimized value of the objective function in the IV divided by the
variance of the residual
High values reject the null of validity

C. Zulehner: Introductory Econometrics 26 / 32

Example: Testing the Putnam’s conjecture – Sargan Test

Second stage output

• Number of obs = 401
• F( 10, 390) = 15.62
• Prob > F = 0.0000
• Total (centered) SS = 1185.026061 Centered R2 = 0.3017
• Total (uncentered) SS = 12440.64984 Uncentered R2 = 0.9335
• Residual SS = 827.5467269 Root MSE = 1.4
• ------------------------------------------------------------------------------
• totassoc_p | Coef. Std. Err. z P>|z| [95% Conf. Interval]
• -------------+----------------------------------------------------------------
• Freecity | 1.028478 .2867427 3.59 0.000 .4664723 1.590483
• ------------------------------------------------------------------------------
• Sargan statistic (overidentification test of all instruments): 2.420
• Chi-sq(1) P-val = 0.11979
• Instrumented: free city
• Instruments: sedevescovile citta_etrusca altitudine escursione costal nearsea
• crossroad population pop2 gini_land gini_income

C. Zulehner: Introductory Econometrics 27 / 32

Testing for over-identification

Estimate the structural model using IV and obtain the residuals

Regress the residuals on all the exogenous variables and obtain the R 2 to form NR 2
Under the null that all instruments are uncorrelated with the error, LM ∼ χ2(q−k)
where (q − k) is the number of additional instruments

C. Zulehner: Introductory Econometrics 28 / 32

5. Testing for heteroskedasticity and serial
correlation

When using 2SLS, we need a slight adjustment to the Breusch-Pagan test

I Get the residuals from the IV estimation
I Regress these residuals squared on all of the exogenous variables in the model
(including the instruments)
I Test for the joint significance
I In general, also with IV estimation is good practice to use robust/clustered
standard errors!
When using 2SLS, we need a slight adjustment to the test for serial correlation.
I Get the residuals from the IV estimation.
I Re-estimate the structural model by 2SLS, including the lagged residuals, and
using the same instruments as originally.

C. Zulehner: Introductory Econometrics 29 / 32

Summary on IV estimation

Instrumental variable estimates are a powerful tool when we are interested in

identifying causal links
But the power of IV hinges on the quality of the instruments
Two features are key:
1 Must be powerful ⇒ can be tested
2 Must be orthogonal to the error term ⇒ cannot be fully tested
Weak instruments can have unpleasant consequences unless one can swear on the
orthogonality of the instruments ⇒ always test the power of the instruments

C. Zulehner: Introductory Econometrics 30 / 32

Finding good Instruments

But how can we find valid instruments if they cannot be tested?

There is no prescription as this is not a technical issue, it is a deep economic
problem
In general, good instruments are variables that satisfy the orthogonality condition
because this appears plausible and self evident
The assumption of orthogonality is like an axiom: when good is self evident. This
happens rarely
Finding instruments is a matter of economic reasoning, ingenuity, and a lot of
thinking
1 Economic theory is often of help but not always theory gives clear exclusion
restrictions
2 A good understanding of the context at end is always very helpful
3 All this implies that you have to find good arguments to convince that your
instruments are indeed good

C. Zulehner: Introductory Econometrics 31 / 32

Some Practical Rules

1 Discuss the plausibility of your instruments

I Why they should be excluded from the regression?
I Why they should be orthogonal to the error term?

2 Test when possible the over-identifying restrictions

I Remember that it rests on the assumption that at least one instrument is valid
I This is less likely to be the case – and thus OIR test to fail – when instruments are devised
from the same logic (highly correlated) ⇒ if one is invalid also the other is
3 Always report the first stage and discuss the effects of the instruments on X? are they consistent
with the priors?
4 Discuss and address possible links between the instruments and the disturbances
5 Be diligent about omitted explanations: IV estimates are inconsistent if a relevant variable is
omitted and is correlated with the instrument
6 When possible rely on alternative instruments: getting similar results from alternative instruments
lends credibility to the results
7 Use and check intuition for instrument validity: one check is to run reduced form regressions and
show that instruments have signs that are consistent with the identification strategy

C. Zulehner: Introductory Econometrics 32 / 32