Scribd
Upload
116 views

MIT18.650. Statistics For Applications Fall 2016. Problem Set 4

This document contains a statistics problem set with 6 problems. Problem 1 asks to find maximum likelihood estimators and Fisher information for several probability distributions including Bernoulli, Poisson, exponential, Gaussian, shifted exponential, and log-normal. Problem 2 asks to find moment estimators for the parameters of the distributions in Problem 1. Problem 3 considers a censored data problem where individuals report whether their salary exceeds a threshold, assuming salaries are exponentially distributed. It asks questions about the distribution of responses, consistent estimators of the parameter λ, asymptotic normality, and compares the Fisher information when salaries are directly observed versus when only responses are observed.

Uploaded by

yilvas
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
116 views

MIT18.650. Statistics For Applications Fall 2016. Problem Set 4

This document contains a statistics problem set with 6 problems. Problem 1 asks to find maximum likelihood estimators and Fisher information for several probability distributions including Bernoulli, Poisson, exponential, Gaussian, shifted exponential, and log-normal. Problem 2 asks to find moment estimators for the parameters of the distributions in Problem 1. Problem 3 considers a censored data problem where individuals report whether their salary exceeds a threshold, assuming salaries are exponentially distributed. It asks questions about the distribution of responses, consistent estimators of the parameter λ, asymptotic normality, and compares the Fisher information when salaries are directly observed versus when only responses are observed.

Uploaded by

yilvas
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

18.650.

Statistics for Applications


Fall 2016. Problem Set 4
Due Friday, Oct. 7 at 12 noon

NOTE: there was a typo in the definition of the Log-normal pdf

Problem 1 Maximum likelihood and Fisher information


For each of the following distributions, compute the maximum likelihood estimator for
the unknown (one or two dimensional) parameter, based on a sample of n i.i.d. random
variables with that distribution. In each case, is the Fisher information well defined ? If
yes, compute it.
1. Ber(p), 0 < p < 1;
2. Poisson with parameter λ > 0:
k
−λ λ
Pλ [X = k] = e , ∀k ∈ IN;
k!

3. Exponential with parameter λ > 0, with density

fλ (x) = λe−λx , ∀x > 0;

4. Gaussian with parameters µ ∈ IR, σ 2 > 0 (recall that the second parameter is σ 2 ,
not σ);
5. Shifted exponential distribution with parameters a ∈ IR, λ > 0 with density

fa,λ (x) = λe−λ(x−a) 1x≥a , ∀x ∈ IR;

6. Log-normal distribution with parameters µ ∈ IR and σ 2 > 0, with density


1 1 2
fµ,σ2 (x) = √ e− 2σ2 (ln x−µ) , ∀x > 0.
x 2πσ 2

Problem 2 Method of moments


For each distribution of Problem 1, find the moment estimator for the unknown pa­
rameter, based on a sample of n i.i.d. random variables.

Problem 3 Censored data


In a given population, n individuals are sampled randomly, with replacement, and each
sampled individual is asked whether his/her salary is greater than some fixed threshold z.
Assume that the salary of a randomly chosen individual has the exponential distribution
with unknown parameter λ. Asking whether the salary overcomes a given threshold rathen
than directly asking for the salary increases the number people that are willing to answer
and decreases the number of mistakes in the collected answers. Denote by X1 , . . . , Xn the
binary responses (Xi ∈ {0, 1}, i = 1, . . . , n) of the n sampled individuals.
1. What is the distribution of the Xi ’s ?
2. Let X̄n be the proportion of sampled individuals whose response was 1 (correspond­
ing to Yes). Prove that X̄n is asymptotically normal and compute the asymptotic
variance.
3. Find a function f such that f (X ¯ n ) is a consistent estimator of λ.

4. Prove that f (X ¯
n ) is asymptotically normal and compute the asymptotic variance.
5. What equation must z satisfy in order to minimize the asymptotic variance com­
puted in Question 4 ? Write this equation in the form gλ (z) = z, where gλ is a
function that depends on the unknown parameter λ.
6. Let Y1 , . . . , Yn be the salaries of the n sampled people.
a) If one could actually observe Y1 , . . . , Yn , what would be the statistical model
?
b) In that case, what would be the Fisher information (as a function of the
unknown parameter λ ? Denote it by IY (λ).
c) In the model where only the Xi ’s are observed (with fixed threshold z), what
is the Fisher information ? Denote it by IX (λ).
d) Compare IY (λ) and IX (λ): Which one is the largest ? How do you interpret
this fact ?

MIT OpenCourseWare
https://ocw.mit.edu

18.650 / 18.6501 Statistics for Applications


Fall 2016

For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.

You might also like