Imbens/Wooldridge, Lecture Notes 10, Summer 07

Whats New in Econometrics?

NBER, Summer 2007
Lecture 10, Tuesday, July 31st, 4.30-5.30 pm

Difference-in-Differences Estimation
These notes provide an overview of standard difference-in-differences methods that have
been used to study numerous policy questions. We consider some recent advances in Hansen
(2007a,b) on issues of inference, focusing on what can be learned with various group/time
period dimensions and serial independence in group-level shocks. Both the repeated cross
sections and panel data cases are considered. We discuss recent work by Athey and Imbens
(2006) on nonparametric approaches to difference-in-differences, and Abadie, Diamond, and
Hainmueller (2007) on constructing synthetic control groups.

1. Review of the Basic Methodology

Since the work by Ashenfelter and Card (1985), the use of difference-in-differences
methods has become very widespread. The simplest set up is one where outcomes are observed
for two groups for two time periods. One of the groups is exposed to a treatment in the second
period but not in the first period. The second group is not exposed to the treatment during
either period. In the case where the same units within a group are observed in each time period,
the average gain in the second (control) group is substracted from the average gain in the first
(treatment) group. This removes biases in second period comparisons between the treatment
and control group that could be the result from permanent differences between those groups, as
well as biases from comparisons over time in the treatment group that could be the result of
trends. We will treat the panel data case in Section 4.
With repeated cross sections, we can write the model for a generic member of any of
groups as
y 0 1 dB 0 d2 1 d2 dB u

(1.1)

where y is the outcome of interest, d2 is a dummy variable for the second time period. The
dummy variable dB captures possible differences between the treatment and control groups
prior to the policy change. The time period dummy, d2, captures aggregate factors that would
cause changes in y even in the absense of a policy change. The coefficient of interest, 1 ,
multiplies the interaction term, d2 dB, which is the same as a dummy variable equal to one
for those observations in the treatment group in the second period. The
difference-in-differences estimate is
1 y B,2 y B,1 y A,2 y A,1 .

(1.2)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

Inference based on even moderate sample sizes in each of the four groups is straightforward,
and is easily made robust to different group/time period variances in the regression framework.
In some cases a more convincing analysis of a policy change is available by further
refining the definition of treatment and control groups. For example, suppose a state
implements a change in health care policy aimed at the elderly, say people 65 and older, and
the response variable, y, is a health outcome. One possibility is to use data only on people in
the state with the policy change, both before and after the change, with the control group being
people under 65 and the treatment group being people 65 and older. The potential problem
with this DD analysis is that other factors unrelated to the states new policy might affect the
health of the elderly relative to the younger population, for example, changes in health care
emphasis at the federal level. A different DD analysis would be to use another state as the
control group and use the elderly from the non-policy state as the control group. Here, the
problem is that changes in the health of the elderly might be systematically different across
states due to, say, income and wealth differences, rather than the policy change.
A more robust analysis than either of the DD analyses described above can be obtained by
using both a different state and a control group within the treatment state. If we again label the
two time periods as one and two, let B represent the state implementing the policy, and let E
denote the group of elderly, then an expanded verson of (1.1) is
y 0 1 dB 2 dE 3 dB dE 0 d2 1 d2 dB 2 d2 dE 3 d2 dB dE u

(1.3)

The coefficient of interest is now 3 , the coefficient on the triple interaction term, d2 dB dE.
The OLS estimate 3 can be expressed as follows:
3 y B,E,2 y B,E,1 y A,E,2 y A,E,1 y B,N,2 y B,N,1
where the A subscript means the state not implementing the policy and the N subscript
represents the non-elderly. For obvious reasons, the estimator in (1.4) is called the
difference-in-difference-in-differences (DDD) estimate. [The population analog of (1.4) is
easily established from (1.3) by finding the expected values of the six groups appearing in
(1.4).] If we drop either the middle term or the last term, we obtain one of the DD estimates
described in the previous paragraph. The DDD estimate starts with the time change in averages
for the elderly in the treatment state and then nets out the change in means for elderly in the
control state and the change in means for the non-elderly in the treatment state. The hope is
that this controls for two kinds of potentially confounding trends: changes in health status of

(1.4)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

elderly across states (that would have nothing to do with the policy) and changes in health
status of all people living in the policy-change state (possibly due to other state policies that
affect everyones health, or state-specific changes in the economy that affect everyones
health). When implemented as a regression, a standard error for 3 is easily obtained, including
a heteroskedasticity-robust standard error. As in the DD case, it is straightforward to add
additional covariates to (1.3) and inference robust to heteroskedasticity.

2. How Should We View Uncertainty in DD Settings?

The standard approach just described assumes that all uncertainty in inference enters
through sampling error in estimating the means of each group/time period combination. This
approach has a long history in statistics, as it is equivalent to analysis of variance. Recently,
different approaches have been suggest that focus on different kinds of uncertainty perhaps
in addition to sampling error in estimating means. Recent work by Bertrand, Duflo, and
Mullainathan (2004), Donald and Lang (2007), Hansen (2007a,b), and Abadie, Diamond, and
Hainmueller (2007) argues for additional sources of uncertainty. In fact, in most cases the
additional uncertainty is assumed to swamp the sampling error in estimating group/time period
means. We already discussed the DL approach in the cluster sample notes, although we did not
explicitly introduce a time dimension. One way to view the uncertainty introduced in the DL
framework and a perspective explicitly taken by ADH is that our analysis should better
reflect the uncertainty in the quality of the control groups.
Before we turn to a general setting, it is useful to ask whether introducing more than
sampling error into DD analyses is necessary, or desirable. As we discussed in the cluster
sample notes, the DL approach does not allow inference in the basic comparison-of-mean case
for two groups. While the DL estimate is the usual difference in means, the error variance of
the cluster effect cannot be estimated, and the t distribution is degenerate. It is also the case
that the DL approach cannot be applied to the standard DD or DDD cases covered in Section 1.
We either have four different means to estimate or six, and the DL regression in these cases
produces a perfect fit with no residual variance. Should we conclude nothing can be learned in
such settings?
Consider the example from Meyer, Viscusi, and Durbin (1995) on estimating the effects of
benefit generosity on length of time a worker spends on workers compensation. MVD have a
before and after period, where the policy change was to raise the cap on covered earnings. The
treatment group is high earners, and the control group is low earners who should not have

Imbens/Wooldridge, Lecture Notes 10, Summer 07

been affected by the change in the cap. Using the state of Kentucky and a total sample size of
5,626, MVD find the DD estimate of the policy change is about 19.2% (longer time on
workers compensation). The t statistic is about 2.76, and the estimate changes little when
some controls are added. MVD also use a data set for Michigan. Using the same DD approach,
they estimate an almost identical effect: 19.1%. But, with only 1,524 observations, the t
statistic is 1.22. It seems that, in this example, there is plenty of uncertainty in estimation, and
one cannot obtain a tight estimate without a fairly large sample size. It is unclear what we gain
by concluding that, because we are just identifying the parameters, we cannot perform
inference in such cases. In this example, it is hard to argue that the uncertainty associated with
choosing low earners within the same state and time period as the control group somehow
swamps the sampling error in the sample means.

3. General Settings for DD Analysis: Multiple Groups and Time

Periods
The DD and DDD methodologies can be applied to more than two time periods. In the first
case, a full set of time-period dummies is added to (1.1), and a policy dummy replaces d2 dB;
the policy dummy is simply defined to be unity for groups and time periods subject to the
policy. This imposes the restriction that the policy has the same effect in every year, and
assumption that is easily relaxed. In a DDD analysis, a full set of dummies is included for each
of the two kinds of groups and all time periods, as well as all pairwise interactions. Then, a
policy dummy (or sometimes a continuous policy variable) measures the effect of the policy.
See Gruber (1994) for an application to mandated maternity benefits.
With many time periods and groups, a general framework considered by BDM (2004) and
Hansen (2007b) is useful. The equation at the individual level is
y igt t g x gt z igt gt v gt u igt , i 1, . . . , M gt ,

(3.1)

where i indexes individual, g indexes group, and t indexes time. This model has a full set of
time effects, t , a full set of group effects, g , group/time period covariates, x gt (these are the
policy variables), individual-specific covariates, z igt , unobserved group/time effects, v gt , and
individual-specific errors, u igt . We are interested in estimating . Equation (3.1) is an example
of a multilevel model.
One way to write (3.1) that is useful is
y igt gt z igt gt u igt , i 1, . . . , M gt ,

(3.2 )

Imbens/Wooldridge, Lecture Notes 10, Summer 07

which shows a model at the individual level where both the intercepts and slopes are allowed
to differ across all g, t pairs. Then, we think of gt as
gt t g x gt v gt .

(3.3)

Equation (3.3) is very useful, as we can think of it as a regression model at the group/time
period level.
As discussed by BDM, a common way to estimate and perform inference in (3.1) is to
ignore v gt , in which case the observations at the individual level are treated as independent.
When v gt is present, the resulting inference can be very misleading. BDM and Hansen (2007b)
allow serial correlation in v gt : t 1, 2, . . . , T and assume independence across groups, g.
A simple way to proceed is to view (3.3) as ultimately of interest. We observe x gt , t is
handled with year dummies,and g just represents group dummies. The problem, then, is that
we do not observe gt . But we can use the individual-level data to estimate the gt , provided
the group/time period sizes, M gt , are reasonably large. With random sampling within each
g, t, the natural estimate of gt is obtained from OLS on (3.2) for each g, t pair, assuming
that Ez igt u igt 0. (In most DD applications, this assumption almost holds by definition, as
the individual-specific controls are included to improve estimation of gt .) If a particular model
of heteroskedasticity suggests itself, and Eu it |z igt 0 is assumed, then a weighted least
squares procedure can be used. Sometimes one wishes to impose some homogeneity in the
slopes say, gt g or even gt in which case pooling can be used to impose such
restrictions. In any case, we proceed as if the M gt are large enough to ignore the estimation
error in the gt ; instead, the uncertainty comes through v gt in (3.3). Hansen (2007b) considers
adjustments to inference that accounts for sampling error in the gt , but the methods are more
complicated. The minimum distance approach we discussed in the cluster sampling notes,
applied in the current context, effectively drops v gt from (3.3) and views gt t g x gt
as a set of deterministic restrictions to be imposed on gt . Inference using the efficient
minimum distance estimator uses only sampling variation in the gt , which will be independent
across all g, t if they are separately estimated, or which will be correlated if pooled methods
are used.
Because we are ignoring the estimation error in gt , we proceed simply by analyzing the
panel data equation
gt t g x gt v gt , t 1, . . . , T, g 1, . . . , G,

(3.4)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

where we keep the error as v gt because we are treating gt and gt interchangeably. If we
assume that We can apply the BDM findings and Hansen (2007a) results directly to this
equation. Namely, if we estimate (3.4) by OLS which means full year and group effects,
along with x gt then the OLS estimator has satisfying properties as G and T both increase,
provided v gt : t 1, 2, . . . , T is a weakly dependent (mixing) time series for all g. The
simulations in BDM and Hansen (2007a) indicate that cluster-robust inference, where each
cluster is a set of time periods, work reasonably well when v gt follows a stable AR(1) model
and G is moderately large.
Hansen (2007b), noting that the OLS estimator (the fixed effects estimator) applied to (3.4)
is inefficient when v gt is serially uncorrelated (and possibly heteroskedastic), proposes feasible
GLS. As is well known, if T is not large, estimating parameters for the variance matrix
g Varv g , where v g is the T 1 error vector for each g, is difficult when group effects
have been removed. In other words, using the FE residuals, v gt , to estimate g can result in
severe bias for small T. Solon (1984) highlighted this problem for the homoskedastic AR(1)
model. Of course, the bias disappears as T , and regression packages such as Stata, that
have a built-in command to do fixed effects with AR(1) errors, use the usual AR(1) coefficient
, obtained from
v gt on v g,t1 , t 2, . . . , T, g 1, . . . , G.
As discussed in Wooldridge (2003) and Hansen (2007b), one way to account for the bias in
is to still use a fully robust variance matrix estimator. But Hansens simulations show that this
approach is quite inefficient relative to his suggestion, which is to bias-adjust the estimator
and then use the bias-adjusted estimator in feasible GLS. (In fact, Hansen covers the general
ARp model.) Hansen derives many attractive theoretical properties of his the estimator.An
iterative bias-adjusted procedure has the same asymptotic distribution as in the case should
work well: G and T both tending to infinity. Most importantly for the application to DD
problems, the feasible GLS estimator based on the iterative procedure has the same asymptotic
distribution as the GLS etsimator when G and T is fixed. When G and T are both large,
there is no need to iterated to achieve efficiency.
Hansen further shows that, even when G and T are both large, so that the unadjusted AR
coefficients also deliver asymptotic efficiency, the bias-adusted estimates deliver higher-order
improvements in the asymptotic distribution. One limitation of Hansens results is that they
assume x gt : t 1, . . . , T are strictly exogenous. We know that if we just use OLS that is,
6

(3.5)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

the usual fixed effects estimate strict exogeneity is not required for consistency as T .
GLS, in exploiting correlations across different time periods, tends to exacerbate bias that
results from a lack of strict exogeneity. In policy analysis cases, this is a concern if the policies
can switch on and off over time, because one must decide whether the decision to implement
or remove a program is related to past outcomes on the response.
With large G and small T, one can estimate an unstricted variance matrix g and proceed
with GLS this is the approach suggested by Kiefer (1980) and studied more recently by
Hausman and Kuersteiner (2003). It is equivalent to dropping a time period in the
time-demeaned equation and proceeding with full GLS (and this avoids the degeneracy in the
variance matrix of the time-demeaned errors). Hausman and Kuersteiner show that the Kiefer
approach works pretty well when G 50 and T 10, although substantial size distortions
exist for G 50 and T 20.
Especially if the M gt are not especially large, we might worry about ignoring the estimation
error in the gt . One simple way to avoid ignoring the estimation error in gt is to aggregate
equation (3.1) over individuals, giving
y gt t g x gt z gt v gt gt , t 1, . . , T, g 1, . . . , G.
Of course, this equation can be estimated by fixed effects, too, and fully robust inference is
available using Hansen (2007a) because the composite error, r gt v gt gt , is weakly
dependent. Fixed Effects GLS using an unrestricted variance matrix can be used with large G
and small T. The complication with using specific time series model for the error is the
presence of gt . With different M gt , Var gt is almost certainly heteroskedastic (and might be
with the same M gt , of course). So, even if we specify, say, an AR(1) model v gt v g,t1 e gt ,
the variance matrix of r g is more complicated. One possibility is to just assume the composite
error, r gt , follows a simple model, implement Hansens methods, but then use fully robust
inference.
The Donald and Land (2007) approach applies in the current setting by using finite sample
analysis applied to the pooled regression (3.4). However, DL assume that the errors v gt are
uncorrelated across time, and so, even though for small G and T it uses small
degrees-of-freedom in a t distribution, it does not account for uncertainty due to serial
correlation in v gt : t 1, . . . , T.

4. Individual-Level Panel Data

Individual-level panel data is a powerful tool for estimating policy effects. In the simplest
7

(3.6)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

case we have two time periods and a binary program indicator, w it , which is unity if unit i
participates in the program at time t. A simple, effective model is
y it d2 t w it c i u it , t 1, 2,

(4.1)

where d2 t 1 if t 2 and zero otherwise, c i is an observed effect, and u it are the idiosyncratic
errors. The coefficient is the treatment effect. A simple estimation procedure is to first
difference to remove c i :
y i2 y i1 w i2 w i1 u i2 u i1

(4.2)

y i w i u i .

(4.3)

or

If Ew i u i 0, that is, the change in treatment status is uncorrelated with changes in the
idiosyncratic errors, then OLS applied to (4.3) is consistent. The leading case is when w i1 0
for all i, so that no units we exposed to the program in the initial time period. Then the OLS
estimator is
y treat y control ,

(4.4)

which is a difference-in-differences estimate except that we different the means of the same
units over time.This same estimate can be derived without introducing heterogeneity by simply
writing the equation for y it with a full set of group-time effects. Also, (4.4) is not the same
estimate obtained from the regression y i2 on 1, y i1 , w i2 that is, using y i1 as a control in a cross
section regression. The estimates can be similar, but their consistency is based on different
assumptions.
More generally, with many time periods and arbitrary treatment patterns, we can use
y it t w it x it c i u it , t 1, . . . , T,
which accounts for aggregate time effects and allows for controls, x it . Estimation by FE or FD
to remove c i is standard, provided the policy indicator, w it , is strictly exogenous: correlation
beween w it and u ir for any t and r causes inconsistency in both estimators, although the FE
estimator typically has smaller bias when we can assume conteporaneous exogeneity,
Covw it , u it 0. Strict exogeneity can be violated if policy assignment changes in reaction to
past outcomes on y it . In cases where w it 1 whenever w ir 1 for r t, strict exogeneity is
usually a reasonable assumption.
Equation (4.5) allows policy designation to depend on a level effect, c i , but w it might be

(4.5)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

correlated with unit-specific trends in the response, too. This suggests the correlated random
trend model
y it c i g i t t w it x it u it , t 1, . . . , T,

(4.6)

where g i is the trend for unit i. A general analysis allows arbitrary corrrelation between c i , g i
and w it , which requires at least T 3. If we first difference, we get
y it g i t w it x it u it , t 2, . . . , T,

(4.7)

where t t t1 is a new set of time effects. We can estimate (4.7) by differencing again,
or by using FE. The choice depends on the serial correlation properties in u it (assume strict
exogeneity of treatment and covariates). If u it is roughly uncorrelated, FE is preferred. If the
original errors u it are essentially uncorrelated, applying FE to (4.6), in the general sense of
sweeping out the linear trends from the response, treatment, and covariates, is preferred. Fully
robust inference using cluster-robust variance estimators is straightforward. Of course, one
might want to allow the effect of the policy to change over time, which is easy by interacting
time dummies with the policy indicator.
We can derive standard panel data approaches using the counterfactural framework from
the treatment effects literature.For each i, t, let y it 1 and y it 0 denote the counterfactual
outcomes, and assume there are no covariates. One way to state the assumption of
unconfoundedness of treatment is that, for time-constant heterogeneity c i0 and c i1 ,
Ey it0 |w i , c i0 , c i1 Ey it0 |c i0

(4.8)

Ey it1 |w i , c i0 , c i1 Ey it1 |c i1 ,

(4.9)

where w i w i1 , . . . , w iT is the time sequence of all treatments. We saw this kind of strict
exogeneity assumption conditional on latent variables several times before. It allows treatment
to be correlated with time-constant heterogeneity, but does not allow treatment in any time
period to be correlated with idiosyncratic changes in the counterfactuals. Next, assume that the
expected gain from treatment depends at most on time:
Ey it1 |c i1 Ey it0 |c i0 t , t 1, . . . , T.

(4.10)

Writing y it 1 w it y it0 w it y i1 , and using (4.8), (4.9), and (4.10) gives

Ey it |w i , c i0 , c i1 Ey it0 |c i0 w it Ey it1 |c i1 Ey it0 |c i0
Ey it0 |c i0 t w it .
If we now impose an additive structure on Ey it0 |c i0 , namely,

(4.11)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

Ey it0 |c i0 t0 c i0 ,

(4.12)

Ey it |w i , c i0 , c i1 t0 c i0 t w it ,

(4.13)

then we arrive at

an estimating equation that leads to well-known procedures. Because w it : t 1, . . . , T is

strictly exogenous conditional on c i0 , we can use fixed effects or first differencing, with a full
set of time period dummies. A standard analysis would use t , but, of course, we can
easily allow the effects of the policy to change over time.
Of course, we can add covariates x it to the conditioning sets and assume linearity, say
Ey it0 |x it , c i0 t0 x it 0 c i0 . If (4.8) becomes
Ey it0 |w i , x i , c i0 , c i1 Ey it0 |x it , c i0 ,

(4.14)

and similarly for (4.9), then the estimating equation simply adds x it 0 to (4.13). More
interesting models are obtained by allowing the gain from treatment to depend on
heterogeneity. Suppose we assume, in addition to the ignorability assumption in (4.14) (and the
equivalent condition for y it1
Ey it1 y it0 |x it , c i0 , c i1 t a i x it t

(4.15)

where a i is a function of c i0 , c i1 normalized so that Ea i 0 and t Ex it . Equation

(4.15) allows the gain from treatment to depend on time, unobserved heterogeneity, and
observed covariates. Then
Ey it |w i , x i , c i0 , a i t0 t w it x it 0

(4.16)

w it x it t c i0 a i w it .
This is a correlated random coefficient model because the coefficient on w it is t a i , which
has expected value t . Generally, we want to allow w it to be correlated with a i and c i0 . With
small T and large N, we do not try to estimate the a i (nor the c i0 ). But an extension of the
within transformation effectively eliminates a i w it . Suppose we simplify a bit and assume
t and drop all other covariates. Then, a regression that appears to suffer from an
incidental parameters problem turns out to consistently estimate : Regress y it on year
dummies, dummies for each cross-sectional observation, and latter dummies interacted with
w it . In other words, we estimate
it t0 i0 i w it .
While i is usually a poor estimate of i a i , their average is a good estimator of :
10

(4.17)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

N

N 1 i .

(4.18)

i1

A standard error can be calculated using Wooldridge (2002, Section 11.2) or bootstrapping.
We can apply the results from the linear panel data notes to determine when the usual FE
estimator that is, the one that ignores a i w it is consistent for . In addition to the
unconfoundedness assumption, sufficient is
it E i , t 1, . . . , T,
E i |w

(4.19)

it w it w
where w
i . Essentially, the individual-specific treatment effect can be correlated
with the average propensity to recieve treatment, w
i , but not the deviations for any particular
time period.
Assumption (4.19) is not completely general, and we might want a simple way to tell
whether the treatment effect is heterogeneous across individuals. Here, we an exploit
correlation between the i and treatment. Recalling that i a i , a useful assumption (that
need not hold for obtaining a test) is
Ea i |w i1 , . . . w iT Ea i |w
i w
i w i ,

(4.20)

where other covariates have been suppressed. Then we can estimate the equation (with
covariates)
y it t0 w it x it 0 w it x it x t

(4.21)

w it w
i w
c i0 e it
by standard fixed effects. Then, we use a simple t test on , robust to heteroskedasticity and
serial correlation. If we reject, it does not mean the mean usual FE estimator is inconsistent,
but it could be.

5. Semiparametric and Nonparametric Approaches

Return to the setting with two groups and two time periods. Athey and Imbens (2006)
generalize the standard DD model in several ways. Let the two time periods be t 0 and 1 and
label the two groups g 0 and 1. Let Y i 0 be the counterfactual outcome in the absense of
intervention and Y i 1 the counterfactual outcome with intervention. AI assume that
Y i 0 h 0 U i , T i ,

(5.1)

where T i is the time period and

h 0 u, t strictly increasing in u for t 0, 1
11

(5.2)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

The random variable U i represents all unobservable characteristics of individual i. Equation
(5.1) incorporates the idea that the outcome of an individual with U i u will be the same in a
given time period, irrespective of group membership. The strict monotonicity assumption in
(5.2) rules out discrete responses, but Athey and Imbens (2006) provide bounds under weak
monotonicity, and show how, with additional assumptions, point identification be be
recovered.
The distribution of U i is allowed to vary across groups, but not over time within groups, so
that
DU i |T i , G i DU i |G i .

(5.3)

This assumption implies that, within group, the population distribution is stable over time.
The standard DD model can be expressed in this way, with
h 0 u, t u t

(5.4)

U i G i V i , V i G i , T i

(5.5)

and

although, because of the linearity, we can get by with the mean independence assumption
EV i |G i , T i 0. If the treatment effect is constant across individuals, Y i 1 Y i 0, then
we can write
Y i T i G i G i T i V i ,

(5.6)

where Y i 1 G i T i Y i 0 G i T i Y i 1 is the realized outcome. Because EV i |G i , T i 0, the

parameters in (5.6) can be estimated by OLS.
Athey and Imbens call the extension of the usual DD model the changes-in-changes (CIC)
model. They show not only how to recover the average treatment effect, but also that the
distribution of the counterfactual outcome conditional on intervention, that is
DY i 0|G i 1, T i 1,

(5.7)

is identified. The distribution of DY i 1|G i 1, T i 1 is identified by the data because

Y i Y i 1 when G i T i 1. The extra condition AI use is that the support of the distribution
of DU i |G i 1 is contained in the support of DU i |G i 0, written as
U1 U0.
Let F 0gt y the be cumulative distribution function of DY i 0|G i g, T i t for g 1, 2
and t 1, 2, and let F gt y be the cdf for the observed outcome Y i conditional on G i g and
12

(5.8)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

T i t. By definition, F gt y is generally identified from the data, assuming random sampling
for each g, t pair. AI show that, under (5.1), (5.2), (5.3), and (5.8),
0

F 11 y F 10 F 1
00 F 01 y,

(5.9)

1
where F 1
00 is the inverse function of F 00 , which exists under the strict monotonicity

assumption. Notice that all of the cdfs appearing on the right hand size of (5.9) are estimable
from the data; they are simply the cdfs for the observed outcomes conditional on different g, t
pairs. Because F 1
11 y F 11 y, we can estimate the entire distributions of both
counterfactuals conditional on intervention, G i T i 1.
The average treatment effect in the CIC framework as
CIC EY1|G 1, T 1 EY0|G 1, T 1.

(5.10)

EY 11 1 EY 11 0,
where we drop the i subscript, Y gt 1 is a random variable having distribution DY1|G g, t,
and Y gt 0 is a random variable having distribution DY0|G g, t. Under the same
assumptions listed above,
CIC EY 11 EF 1
01 F 00 Y 10

(5.11)

where Y gt is a random variable with distribution DY|G g, t. Given random samples from
each subgroup, a generally consistent estimator of CIC is
N 11

CIC

N 1
11

Y 11,i

N 10

N 1
10

i1

F 101 F 00 Y 10 , i ,
i1

for consistent estimators F 00 and F 01 of the cdfs for the control groups in the initial and later
time periods, respectively. Now, Y 11,i denotes a random draw on the observed outcome for the
g 1, t 1 group and similarly for Y 10,i . Athey and Imbens establish weak conditions under
which CIC is N -asymptotically normal (where, naturally, observations must accumulate
within each of the four groups). In the case where the distributions of Y 10 and Y 00 are the same,
a simple difference in means for the treatment group over time.
The previous approach can be applied either with repeated cross sections or panel data.
Athey and Imbens discuss how the assumptions can be relaxed with panel data, and how
alternative estimation strategies are available. In particular, if U i0 and U i1 represent
unobservables for unit i in the initial and later time periods, respectively, then (5.3) can be
modified to

13

(5.12)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

DU i0 |G i DU i1 |G i ,

(5.13)

which allows for unobservd components structures U it C i V it where V it has the same
distribution in each time period.
As discussed by AI, with panel data there are other estimation approaches. As discussed
earlier, Altonji and Matzkin (2005) use exchangeability assumptions to identify average partial
effects. To illustrate how their approach might applie, suppose the counterfactuals satisfy the
ignorability assumption
EY it g|W i1 , . . . , W iT , U i h tg U i , t 1, . . . , T, g 0, 1.

(5.14)

The treatment effect for unit i in period t is h t1 U i h t0 U i , and the average treatment effect
is
t Eh t1 U i h t0 U i , t 1, . . . , T.

(5.15)

Suppose we make the assumption

i ,
DU i |W i1 , . . . , W iT DU i |W

(5.16)

which means that only the intensity of treatment is correlated with heterogeneity. Under (5.14)
and (5.16), it can be shown that
i .
EY it |W i EEY it |W i , U i |W i EY it |W it , W

(5.17)

The key is that EY it |W i does not depend on W i1 , . . . , W iT in an unrestricted fashion; it is a

i . If W it are continuous, or take on numerous values, we can use local
function only of W it , W
i . In the treatment effect case, estimation is very
smoothing methods to estimate Ey it |W it , W
i can take on only 2T. The average treatment effect can be estimated as
simple because W it , W
n

t N

Yt 1, W i Yt 0, W i .
i1

If we pool across t (as well as i) and use a linear regression, Y it on

i , t 1, . . . , T; i 1, . . . , N, we obtain the usual fixed effects estimate FE
1, d2 t , . . . , dT t , W it , W
as the coefficient on W it . Wooldridge (2005) describes other scenarios and compares this
strategy to other approaches. As we discussed earlier, a condtional MLE logit can estimate
parameters by not generally ATEs, and require conditional independence. Chamberlains
correlated random effects probit models the heterogeneity as
U i |W i Normal 0 1 W i1 . . . T W iT , 2 , which identifies the ATEs without assuming
exchangeability but maintaining a distributional assumption (and functional form for the
14

(5.18)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

response probability).
For the leading case of two time periods, where treatment does not occur in the initial time
period for any unit, but does for some units in the second time period, Abadie (2005) provides
methods for both repeated cross sections and panel data that use unconfoundedness
assumptions on changes over time. Here we describe the panel data approach. Omitting the
i subscript, for any unit from the population there are counterfactual outcomes, which we write
as Y t w, where t 0, 1 are the two time periods and w 0, 1 represent control and treatment.
In this setup, interest lies in two parameters, the average treatment effect in the second time
period,
ATE EY 1 1 Y 1 0,

(5.19)

or the average treatment effect on the treated,

ATT EY 1 1 Y 1 0|W 1.

(5.20)

Remember, in the current setup, no units are treated in the initial time period, so W 1 means
treatment in the second time period.
As in Heckman, Ichimura, Smith, and Todd (1997), Abadie uses unconfoundedness
assumptions on changes over time to identify ATT , and straightforward extensions serve to
identify ATE . Given covariates X (that, if observed in the second time period, should not be
influenced by the treatment), Abadie assumes
EY 1 0 Y 0 0|X, W EY 1 0 Y 0 0|X,

(5.21)

so that, conditional on X, treatment status is not related to the gain over time in the absense of
treatment. In addition, the overlap assumption,
0 PW 1|X 1

(5.22)

is critical. (Actually, for estimating ATT , we only need PW 1|X 1.) Under (5.21) and
(5.22), it can be shown that
W pXY 1 Y 0
1 pX

ATT PW 1 1 E

where Y 1 is the observed outcome in period 1, Y 0 , is the outcome in period 0, and

pX PW 1|X is the propensity score. Dehejia and Wahba (1999) derived (5.23) for the
cross-sectional case; see also Wooldridge (2002, Chapter 18). All quantities in (5.23) are
observed or, in the case of the pX and PW 1, can be estimated. As in Hirano,
Imbens, and Ridder (2003), a flexible logit model can be used for pX; the fraction of units
15

Imbens/Wooldridge, Lecture Notes 10, Summer 07

treated would be used for . Then
N

W i p X i Y i
1 p X i

ATT 1 N 1
i1

(5.23)

is consistent and N -asymptotically normal. HIR discuss variance estimation. Imbens and
Wooldridge (2007) provide a simple adjustment available in the case that p is treated as a
parametric model.
If we also add
EY 1 1 Y 0 1|X, W EY 1 0 Y 0 0|X,

(5.24)

so that treatment is mean independent of the gain in the treated state, then
ATE E

W pXY 1 Y 0
pX1 pX

(5.25)

which dates back to Horvitz and Thompson (1952); see HIR. Now, to estimate the ATE over
the specified population, the full overlap assumption in (5.22) is needed, and
N

ATE N

i1

W i p X i Y i
p X i 1 p X i

Hirano, Imbens, and Ridder (2003) study this estimator in detail where p x is a series logit
estimator. If we treat this estimator parametrically, a simple adjustment makes valid inference
i be the summand in (5.26) less ATE , and let D
i hX i W i hX i
on ATE simple. Let K
be the gradient (a row vector) from the logit estimation. Compute the residuals, R i from the
i on D
i , i 1, . . . , N. Then, a consistent estimator of Avar N ATE ATE is
OLS regression K
just the sample variance of the R i . This is never greater than if we ignore the estimation of px
i themselves.
and just use the sample variance of the K
Under the unconfoundedness assumption, other strategies are available for estimating the
ATE and ATT. One possibility is to run the regression
Y i on 1, W i , p X i , i 1, . . . , N,
which was studied by Rosenbaum and Rubin (1983) in the cross section case. The coefficient
on W i is the estimated ATE, although it requires some functional form restrictions for
consistency. This is much preferred to pooling across t and running the regression Y it on 1, d1 t ,
d1 t W i , p X i . This latter regression requires unconfoundedness in the levels, and as
dominated by the basic DD estimate from Y i on 1, W i : putting in any time-constant function

16

(5.26)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

as a control in a pooled regression is always less general than allowing an unobserved effect
and differencing it away.
Regression adjustment is also possible under the previous assumptions. As derived by
HIST,
EY 1 1 Y 0 1|X, W 1 EY 1 |X, W 1 EY 1 |X, W 0

(5.27)

EY 0 |X, W 1 EY 0 |X, W 0
where, remember, Y t denotes the observed outcome for t 0, 1. Each of the four conditional
expectations on the right hand side is estimable using a random sample on the appropriate
subgroup. Call each of these wt x for w 0, 1 and t 0, 1. Then a consistent estimator of
ATT is
N

N 1
1

W i 11 X i 01 X i

10 X i 00 X i .

i1

Computationally, this requires more effort than the weighted estimator proposed by Abdadie.
Nevertheless, with flexible parametric functional forms that reflect that nature of Y,
implementing (5.28) is not difficult. If Y is binary, then the wt should be obtained from binary
response models; if Y is nonnegative, perhaps a count variable, then wt x expx wt is
attractive, with estimates obtained via Poisson regression (quasi-MLE).

6. Synthetic Control Methods for Comparative Case Studies

In Section 3 we discussed difference-in-differences methods that ignore sampling uncertain
in the group/time period means (more generally, regression coefficients). Abadie, Diamond,
and Haimmueller (2007), building on the work of Abadie and Gardeazabal (2003), argue that
in policy analysis at the aggregate leve, there is no estimation uncertainty: the goal is to
determine the effect of a policy on an entire population say, a state and the aggregate is
measured without error (or very little error). The application in ADH is the effects of
Californias tobacco control program on state-wide smoking rates.
Of course, one source of uncertainty in any study using data with a time series dimension is
the change in outcomes over time, even if the outcomes are aggregates measured without error.
Event study methodology is one such example: often, time series regressions for a single
entity, such as a state, are used to determine the effect of a policy (speed limit change, tobacco
control program, and so on) on an aggregate outcome. But such event studies can suffer
because they do not use a control group to account for aggregate effects that have nothing to

17

(5.28)

Imbens/Wooldridge, Lecture Notes 10, Summer 07

do with the specific state policy.
In the context of case control studies, where a time series is available for a particular unit
the treatment group there are often many potential control groups. For example, in the
tobacco control example, each state in the U.S. is a potential control for California (provided a
state did not undergo a similar policy). ADH study this setup and emphasize the uncertainty
associated with choosing suitable control groups. They point out that, even in the absense of
sampling error, surely someone analyzing a state-level policy must nevertheless seal with
uncertainty.
The approach of ADH is to allow one to select a synthetic control group out of a collection
of possible controls. For example, in the California tobacco control case, ADH identify 38
states that did not implement such programs during the time period in question. Rather than
just use a standard fixed effects analysis which effectively treats each state as being of equal
quality as a control group ADH propose choosing a weighted average of the potential
controls. Of course, choosing a suitable control group or groups is often done informally,
including matching on pre-treatment predictors. ADH formalize the procedure by optimally
choosing weights, and they propose methods of inference.
Consider a simple example, with only two time periods: one before the policy and one
after. Let y it be the outcome for unit i in time t, with i 1 the (eventually) treated unit.
Suppose there are J possible controls, and index these as 2, . . . , J 1. Let x i be observed
covariates for unit i that are not (or would not be) affected by the policy; x i may contain period
t 2 covariates provided they are not affected by the policy. Generally, we can estimate the
effect of the policy as
J1

y 12 w j y j2 ,
j2

where w j are nonnegative weights that add up to one. The question is: how can we choose the
weights that is, the synthetic control to obtain the best estimate of the intervention effect?
ADH propose choosing the weights so as to minimize the distance between, in this simple
J1

case, y 11 , x 1 and j2 w j y j1 , x j , or some linear combinations of elements of y 11 , x 1 and

y j1 , x j . The optimal weights which differ depending on how we define distance produce
the synthetic control whose pre-intervention outcome and predictors of post-intervention
outcome are closest. With more than two time periods, one can use averages of

18

Imbens/Wooldridge, Lecture Notes 10, Summer 07

pre-intervention outcomes, say, or weighted averages that give more weight to more recent
pre-intervention outcomes.
ADH propose permutation methods for inference, which require estimating a placebo
treatment effect for each region (potential control), using the same synthetic control method as
for the region that underwent the intervention. In this way, one can compare the estimated
intervention effect using the synthetic control method is substantially larger than the effect
estimated from a region chosen at random. The inference is exact even in the case the
aggregate outcomes are estimated with error using individual-level data.

References
(To be added.)

19