Scribd
Upload
19 views

hw_2

The document outlines a homework assignment for an M Tech (CS) course covering statistical methods, including topics such as sufficiency, Fisher information, method of moments estimators (MME), and maximum likelihood estimators (MLE). It consists of various problems requiring students to find sufficient statistics, evaluate Fisher information, and derive estimators for different probability distributions. The assignment includes specific cases for uniform, normal, exponential, and logistic distributions, among others.

Uploaded by

Soham Chakraborty
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
19 views

hw_2

The document outlines a homework assignment for an M Tech (CS) course covering statistical methods, including topics such as sufficiency, Fisher information, method of moments estimators (MME), and maximum likelihood estimators (MLE). It consists of various problems requiring students to find sufficient statistics, evaluate Fisher information, and derive estimators for different probability distributions. The assignment includes specific cases for uniform, normal, exponential, and logistic distributions, among others.

Uploaded by

Soham Chakraborty
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Statistical Methods: M Tech (CS) Jan-May 2025

Homework 2
(sufficiency, Fisher information, MME, MLE, exponential family, consistency )

1. Let f (x, y|θ1 , θ2 , θ3 , θ4 ) be the bivariate pdf for the uniform distribution on the rectangle with lower left
corner (θ1 , θ2 ) and upper right corner (θ3 , θ4 ) in R2 . The parameters satisfy θ1 < θ3 and θ2 < θ4 . Let
(X1 , Y1 ), . . . , (Xn , Yn ) be a random sample from this pdf. Find a four dimensional sufficient statistic
for θ = (θ1 , θ2 , θ3 , θ4 ).

2. Show that a one-to-one function of a sufficient statistic is also sufficient.

3. Let X1 , . . . , Xn be i.i.d. observations from pmf or pdf f (x| θ). For each of the cases, either find a
sufficient statistic for θ or show that such statistic does not exist (except for the order statistics or
the full data). Which of the following pdfs and pmfs belong to exponential family, location or scale
family?
2x
(a) f (x| θ) = θ2
, 0 < x < θ, θ>0
θ
(b) f (x| θ) = (1+x)1+θ
, 0 < x < ∞, θ>0
(log θ)θx
(c) f (x| θ) = θ−1 , 0 < x < 1, θ > 1
f (x| θ) = x2 θx (1 − θ)2−x , x = 0, 1, 2, 0 <

(d) θ<1
(e) f (x| θ) = 1θ , x = 1, 2, . . . , θ, θ ∈ {1, 2, . . .}
f (x| θ) = r+x−1
 r
(f) x θ (1 − θ)x , x = 0, 1, 2, . . . , 0 < θ < 1, r known
(g) f (x|θ) = e−(x−θ) I{x≥θ} , where −∞ < θ < ∞

4. Let X1 , . . . , Xn be i.i.d. N (µ, σ 2 ) where θ = (µ, σ 2 ) ∈ R × (0, ∞).

(a) Find the Fisher information of θ contained in sample mean X and sample variance S 2 , i.e., find
IX (θ) and IS 2 (θ).
(b) Show that T = (X, S 2 ) is a sufficient statistic for θ by computing its Fisher Information IT (θ).

5. Suppose X has a density f (x|θ) where θ ∈ Θ ⊆ R. Under appropriate regularity conditions on f (x|θ),
show that the Fisher information IX (θ) can be written as
 2 
∂
IX (θ) = −Eθ ln f (X|θ) .
∂θ2

State sufficient regularity conditions required for this to hold.


6. Let X1 , X2 be i.i.d Geometric(p) random variables where 0 < p < 1. Let X = (X1 , X2 ) and T (X) =
X1 + X2 . Evaluate IX (p) and IT (p) and then conclude whether T is sufficient for p.

7. Let X1 , . . . , Xn be i.i.d. random variables with common PMF/PDF f (x|θ). Find a method of moment
estimator (MME) and a maximu likelihood estimator (MLE) for θ in each of the following cases:
α
(a) f (x|θ) = θαxα−1 e−θx I{x>0} where θ > 0 and α(> 0) known.
(b) f (x|θ) = θ(1 − x)θ−1 I{0<x<1} where θ > 1.

8. Let X1 , . . . , Xn be a i.i.d. sample from log-normal(µ, σ 2 ) distribution with density


√ 1 2
f (x|µ, σ 2 ) = (σx 2π)−1 e− 2σ2 (log x−µ) I{x>0} where µ ∈ R.

Find the method of moments estimator and the maximum likelihood estimator for (µ, σ 2 ).

9. Let X1 , . . . , X60 be random sample from a discrete distribution with pmf



1−θ
 2
 if x=1
Pθ (X = x) = p(x|θ) = 2 1
if x=2

θ
2 if x=3

where 0 < θ < 1. In the above sample, 1 appears 10 times, 2 appears 20 times and 3 appears 30 times.
θ
(a) Find the maximum likelihood estimate of θ and log{ 1−θ }.
(b) Find another estimate of θ using method of moments.

10. Let X1 , . . . , Xn be i.i.d. observations from U nif orm(−θ, θ) distribution where θ > 0. Find the MLE
of θ. Is it a function of sufficient statistic of θ?

11. Let X1 , . . . , Xn be random sample from exponential distribution with location and scale parameters θ
and σ respectively. The pdf of Xi is given by
 
1 x−θ
f (x|θ, σ) = exp − I{x≥θ} ,
σ σ

where −∞ < θ < ∞ and σ > 0.

(a) Find the MLE of (θ, σ) and g(θ, σ) = θ + σ.


(b) Show that the MLE of (θ, σ) is a consistent estimator of (θ, σ).
(c) Find the MME of (θ, σ) and show that it is also consistent.

12. Let X1 , . . . , Xn be i.i.d. random variables from N (µ, σ 2 ), where µ is known to be restricted in [0, ∞)
and σ > 0. Find the MLE of (µ, σ 2 ).
13. Let X1 , . . . , Xn be random sample (i.i.d.) from logistic distribution with pdf

e−(x−θ)
f (x; θ) = , −∞ < θ < ∞.
{1 + e−(x−θ) }2

(a) Show that the likelihood equation has a unique solution and that solution is the MLE of θ.
(b) Argue whether the MLE is consistent (even though the MLE can not be obtained in closed form).

You might also like