Holger Sieg

Carnegie Mellon University

Econometrics III: Problem Set # 2

INSTRUCTIONS:
This problem set was originally created by Ron Goettler. The objective of this problem set
is to estimate a machine replacement problem (similar to Rust 1987) using Rusts nested
fixed-point algorithm, and using Hotz-Millers conditional choice probability method. Please
answer the following question and hand in the solution by Monday, Feb 25, 2008.

THE MODEL:
Firms each use one machine to produce output in each period. The machines age, becoming
more likely to breakdown, and in each time period the firms have the option of replacing the
machines. Let at be the age of the machine at time t and let the expected current period
payoffs from using a machine of age at be given by:

1 at + 0t

if it = 0
(at , it , 0t , 1t ) = ,
R + 1t

if it = 1

where it = 1 if the firm decides to replace the machine at t, R is the net cost of a new
machine, and t are time specific shocks to the payoffs from replacing and not replacing.
Assume t are i.i.d. logit errors (i.e., type 1 extreme value). This payoff function may be
interpreted as a normalized profit function, or a cost function.

Assume a simple state evolution equation for age:

That is, if not replaced, the machine ages by one year, up to a maximum of 5 years; after 5
years the machines age is fixed at 5 until replacement. If replaced in the current year, the
machines age next year is 1. Hence, at is one of five possible values: 1, 2, 3, 4, or 5.
Part A: Rusts Nested Fixed Point Algorithm

1. Write down the sequence problem for a firm maximizing the EDV of future payoffs
(assume an infinite horizon).

2. Write down Bellmans equation (the functional equation) for the value function of the
firm. Use the alternative-specific value functions method from lecture, i.e., define
V0 (at ) and V1 (at ), where V0 (at ) is the firms value at time t if it chooses not to replace
the machine and V1 (at ) is the firms value at time t if it replaces the machine. The
functional equation should be a mapping from this pair of equations to itself. (For
the parts that follow, remember from class that there are (at least) two ways to write
the functional equation, and that one of them will be easier to use than the other.)

3. On the computer, write a procedure that solves this dynamic programming problem
given values of the parameters (1 , R). Assume = .9. Your procedure should iterate
the contraction mapping on the two alternative-specific value functions. Each of these
functions is represented by a five element vector since there are five values of a. Iterate
until the V functions do not change very much (say 106 ). Remember that given the
logit error assumption there is an analytic solution to the expectation of the max in
these equations and that Eulers constant is approximately 0.5775. (You could also
iterate on the smooth value function V (at ) that is not alternative-specific, and then
derive V1 (at ), where V0 (at ) from V (at ) and .)

4. Solve the model for the parameters (1 = 1, R = 3). Suppose at = 2. Will the
firm replace its machine in period t? For what value of 0t 1t is the firm indifferent
between replacing its machine or not? What is the probability (to an econometrician
who doesnt observe the s) that this firm will replace its machine? What is the
EDV of future payoffs for a firm at state {at = 4, 0t = 1, 1t = 1.5}? (Remember the
constant term has been normalized out so this EDV could be negative.)

5. Download the data for this problem set from the course web page. The dataset has
two columns: at and it . Consider this to be cross-sectional data (i.e., there is only
one data point per firm). Estimate (1 , R) using maximum likelihood. The best way
to do this is to write a function that returns ln(L), where L is the likelihood function
 and then use a package (e.g. gauss or matlab) to maximize the this function (or
minimize the negative log-likelihood). Also compute (asymptotic) standard errors.
The function itself should follow these steps:

(a) Start with arbitrary (1 , R). That is, these should be the arguments of the
function.

(b) Solve the dynamic programming problem given these parameters. That is, com-
pute the functions V0 (at ) and V1 (at ) using your procedure from part 3.

(c) Using V0 (at ) and V1 (at ), compute the probability of replacement for each possible
at .

(d) Compute the likelihood of each observation using the above probabilities. Form
the log-likelihood function by summing the logs of the likelihoods across obser-
vations (i.e., assume that each data point is an independent observation).

(e) Return ln(L) for a maximization routine or ln(L) for a minimization routine.

6. Describe (you do NOT have to do this on the computer) how you would need to change
your model (either the dynamic programming problem, the estimation procedure, or
both) to accommodate the following perturbations:

(a) Consider an alternative empirical model. Suppose firms differ in their value of
1 . Proportion of firms have 1 = 1A , proportion (1 ) have 1 = 1B . How
would you change both the dynamic programming problem and the likelihood
function?

(b) What if you had the model in (a) but you have panel data (i.e., multiple obser-
vations on each firm)? Write down the likelihood function.

(c) What if instead of firms differing in 1 , machines differ in 1 (i.e., when a firm
replaces an old machine, the new machine may have 1 = 1A (with probability
) or it may have 1 = 1B (with probability 1 )? Now how would you need
to change your program (go back to the original data with one observation per
firm)?
Part B: Hotz-Miller Algorithm

1. Using the data, calculate the machine replacement probabilities at each state at using
the average replacement rate in the sample (cell averages). These estimates are
nonparametric estimates of the replacement probabilities. Also calculate the standard
errors of your estimates (using the Bernoulli formula from first year statistics).

2. Define the normalized conditional value function as 1 (at ) = V1 (at ) V0 (at ). Write
down the Hotz and Miller inversion formula for this model. That is, write down the
formula for 1 (at ) as a function of the conditional choice probabilities. Calculate the
normalized conditional value functions using the choice probabilities you estimated
above.

3. Write down current period utility (i.e., payoff) for each choice in terms of the parame-
ters and the current state. Call this u0 and u1 . Define wkt (at ) = E [kt |at , dkt = 1] for
k {0, 1}. In the special case of the logit, wkt = ln(pkt ), where = 0.5775 is Eulers
constant. You can work this out for yourself mathematically, but it is tedious.

What is the formula for expected utility at time t conditional on reaching state at ,
as a function of current period utilities (u0 , u1 ), the conditional choice probabilities,
P r(iit = 1|at ), and w0t and w1t ? Call the vector of expected utilities (a vector across
states at ) u.

4. Construct the two 5x5 conditional transition matrices F0 and F1 which give the tran-
sition probabilities conditional on the firms {0, 1} replacement choice. Write them
down.

5. Using the estimated conditional choice probabilities and the F0 and F1 matrices above,
calculate the 5x5 unconditional transition matrix of states (which takes into account
the probability of replacement as well as the transition probabilities conditional on
replacement). Call this matrix T and write it down.

6. Write down the formula for the conditional value functions, V0 (at ) and V1 (at ), in terms
of current period utilities (u0 , u1 ), expected utility (u), the unconditional transition
 probabilities T , the discount factor ( = 0.9), and the conditional transition probabil-
ities F0 and F1 . On the computer, write a procedure which takes a parameter vector
(1 , R) as inputs and returns the conditional value functions using this formula.

7. Estimate (1 , R) using maximum likelihood. Again, write a function that returns

(a) Start with arbitrary (1 , R). That is, these should be the arguments of the
function.

(b) Use the procedure that you wrote above to calculate V0 (at ) and V1 (at ).

(c) Using V0 (at ) and V1 (at ) compute the probability of replacement for each possible
at .

(d) Compute the likelihood of each observation using the above probabilities. Form
the log likelihood function by summing the logs of the likelihoods across obser-
vations (i.e., assume that each data point is an independent observation).

(e) Return ln(L) for a maximization routine or ln(L) for a minimization routine.

8. Compare your results here in part B with those of Part A.