0% found this document useful (0 votes)
4 views

revision class 4

note

Uploaded by

hollis.chung93
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
4 views

revision class 4

note

Uploaded by

hollis.chung93
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

Lectures 17 to 21: Section VII

VII Estimation

Method of moments estimation

Maximum likelihood estimation

Estimates and estimators

Sampling distribution of an estimator

Bias, precision and mean square error

Goodness of fit: quantile-quantile plots

Maren and Kuntalee MA10213 Revision Class 4


Example Sheet 6, Problem P3
Suppose X1 , . . . , Xn are i.i.d. Exp(λ) RVs, so each Xi has PDF

fX (x) = λ e −λ x for x > 0.

(a) Derive the maximum likelihood estimate of λ based on the


observed values x1 , . . . , xn .
Define the maximum likelihood estimator of λ in terms of
X1 , . . . , Xn .

(b) Since each Xi ∼ Exp(λ) = Gamma(λ, 1), it follows from


properties of the Gamma distribution that
Xn
S = Xi ∼ Gamma(λ, n).
i=1

Use this fact to show that E(S −1 ) = λ/(n − 1).


Is the maximum likelihood estimator of λ unbiased?
Maren and Kuntalee MA10213 Revision Class 4
Example Sheet 6, Problem P3, continued

(c) The exponential distribution can also be parameterised by


θ = 1/λ, in which case it has PDF

fX (x) = θ−1 e −x/θ , for x > 0.

Derive the maximum likelihood estimator of θ based on


X1 , . . . , Xn .
Is this estimator unbiased?
What is its mean square error?

Maren and Kuntalee MA10213 Revision Class 4


Example Sheet 6, Problem P3
Plot of Pn
1 xi
−n log(θ) − .
θ

−20
−25
lhd(x, n, sumx)

−30
−35
−40

0 5 10 15 20 25 30

Maren and Kuntalee MA10213 Revision Class 4


Example Sheet 6, Problem P4
Suppose X1 , . . . , Xn are i.i.d. Unif(a, b) random variables.

(a) Show that the method of moments estimates of a and b are


the solutions to the equations
n
1 X a+b
xi =
n 2
i=1

and
n
(b − a)2 a+b 2
 
1 X 2
xi = + .
n 12 2
i=1

Maren and Kuntalee MA10213 Revision Class 4


Example Sheet 6, Problem P4, continued
(b) Let Ta (X1 , . . . , Xn ) and Tb (X1 , . . . , Xn ) denote the
corresponding estimators of a and b, respectively.
Deduce that

Tb (X1 , . . . , Xn ) − Ta (X1 , . . . , Xn ) =
 
n n
!2  1/2
1 X 1 X 
12 Xi2 − Xi  .
n n 
i=1 i=1

(c) Can you create an example of a data set where some of the
observed values lie outside the interval (â, b̂), where â and b̂ are
the estimates of a and b?
Why might you regard the estimates â and b̂ as unsatisfactory
in such a case?

Maren and Kuntalee MA10213 Revision Class 4


Example Sheet 7, Problem A5

Quantile-quantile plots for checking goodness of fit

(a) Suppose X is a continuous RV with CDF FX (x) and define

Y = FX (X ).
Show that
Y ∼ Unif(0, 1).

Maren and Kuntalee MA10213 Revision Class 4


Example Sheet 7, Problem A5, continued
(b) Let X(1) , . . . , X(n) denote the random variables taking the
values of X1 , . . . , Xn arranged in increasing order. So,

X(1) = min(X1 , . . . , Xn ), . . . , X(n) = max(X1 , . . . , Xn ).

Define Y(1) , . . . , Y(n) by

Y(i) = FX (X(i) ), i = 1, . . . , n.

It follows from part (a) that, for i = 1, . . . , n, the distribution of


Y(i) is the same as that of U(i) where U1 , . . . , Un are i.i.d.
Unif(0, 1) RVs and U(1) , . . . , U(n) are their ordered values.
Find the CDF of U(i) for a general value of i ∈ {1, . . . , n}.
Hence find the PDF of U(i) and show that
i
E(U(i) ) = .
n+1
Maren and Kuntalee MA10213 Revision Class 4
Example Sheet 7, Problem A5, continued
Recall that in a quantile-quantile plot, we plot x(i) against
FX−1 (i/(n + 1)) for i = 1, . . . , n.
The above argument implies that i/(n + 1) is the expectation of
the random variable FX (X(i) ).
Since FX (X(i) ) is likely to be close to its expectation, i/(n + 1),
then it is likely that X(i) will be close to FX−1 (i/(n + 1)).

(c) Experiment with quantile-quantile plots by generating samples


of data . . .
You may use the R code for “Problem Sheet 6, Question 5” which
is loaded on the Moodle page . . .

Maren and Kuntalee MA10213 Revision Class 4


Exam question: 2009, Question 5
Let X be a Poisson(µ) random variable. Derive the maximum
likelihood estimator of µ, based on the realisations X1 , X2 , . . . , Xn .
What is the sampling distribution of your estimator? [6]

Maren and Kuntalee MA10213 Revision Class 4


Exam question: 2009, Question 8
Suppose that the random variable X has probability density
function  2
2 −x
f (x) = x exp , x > 0,
λ λ
where λ > 0 is a parameter.
(a) Find the maximum likelihood estimator of λ based on the
realisations X1 , . . . , Xn . [5]
(b) Is your estimator unbiased? [5]
(c) Find the precision of your estimator. [5]

Maren and Kuntalee MA10213 Revision Class 4

You might also like