Scribd
Upload
12 views

Problem Sheet 3.7

This document contains a multi-part stochastic modeling problem set. It addresses topics such as: 1. Unbiased estimators and their mean square errors. It asks to show certain estimators for a density's parameter θ are unbiased, and that the mean square error of an unbiased estimator is equal to its variance. 2. Properties of sample means and variances when observations are independent and identically distributed. It asks to derive an unbiased estimator for the variance and find the covariance between sample mean and sample variance. 3. Independence of sample mean and variance when observations are normally distributed. 4. Bounds on probabilities involving probability generating functions. 5. An inequality relating expectation of maximum

Uploaded by

Niladri Dutta
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
12 views

Problem Sheet 3.7

This document contains a multi-part stochastic modeling problem set. It addresses topics such as: 1. Unbiased estimators and their mean square errors. It asks to show certain estimators for a density's parameter θ are unbiased, and that the mean square error of an unbiased estimator is equal to its variance. 2. Properties of sample means and variances when observations are independent and identically distributed. It asks to derive an unbiased estimator for the variance and find the covariance between sample mean and sample variance. 3. Independence of sample mean and variance when observations are normally distributed. 4. Bounds on probabilities involving probability generating functions. 5. An inequality relating expectation of maximum

Uploaded by

Niladri Dutta
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

E1 222 Stochastic Models and Applications

Problem Sheet 3.7

1. Let f be a density function with a parameter θ. (For example, f could


be Gaussian with mean θ). Let X1 , X2 , · · · , Xn be iid with density f .
These are said to be an iid sample from f or said to be iid realizations
of X which has density f . Any function T (X1 , · · · , Xn ) is called a
statistic. Any estimator for θ is such a statistic. We choose a function
based on what we think is the best guess for θ based on the sample.
An estimator T (X1 , · · · , Xn ) is said to be unbiased if E[T (X1 , · · · , Xn )] =
θ. Let us write X for (X1 , · · · , Xn ) and T (X) for any statistic.
Suppose θ is the mean of the density f . Show that T1 (X) = (X2 +
X5 )/2, T2 (X) = X1 , T3 (X) = ( ni=1 Xi )/n are all unbiased estimators
P

for θ.
If T is an estimator for θ, the mean square error of the estimator is
E(T − θ)2 . Show that if T is unbiased then the mean square error is
equal to the variance of the estimator.
Among the the three estimators T1 , T2 , T3 for the mean, listed earlier,
which one has least mean square error?

2. Let X1 , · · · , Xn be iid with mean µ and variance σ 2 . Let X̄ = n1 (X1 +


· · · + Xn ). Show that
n
!
(Xk − X̄)2 = (n − 1)σ 2 .
X
E
k=1

(Hint: Write (Xk − X̄) = (Xk − µ) − (X̄ − µ) and note that (X̄ − µ) =
k (Xk − µ)/n and that E(Xk − µ)(Xj − µ) = 0 for k 6= j).
P

Based on this, suggest an unbiased estimator for the variance.


Let Z = nk=1 (Xk − X̄)2 . Suppose the first and third moments of Xi
P

are zero. Find the covariance between X̄ and Z.

3. Let X1 , X2 , · · · , Xn be iid random variables with mean µ and variance


σ 2 . Let X̄ = ( ni=1 Xi )/n and S 2 = nk=1 (Xk − X̄)2 /(n − 1) be the
P P

sample mean and sample variance respectively. As we have seen, these


are unbiased estimators of mean and variance.

1
Show that cov(X̄, Xi − X̄) = 0, i = 1, 2, · · · , n. (Hint: Note that Xi X̄
can be written as sum of terms like Xi Xj ; note that EXi Xj = µ2 if
i 6= j and is µ2 +σ 2 if i = j; note also that you know mean and variance
of X̄).
Now suppose that the iid random variables Xi have normal distribu-
tion. Show that X̄ and S 2 are independent random variables. (Hint:
Try to use the result that for jointly Gaussian random variables, un-
correlatedness implies independence).

4. Let X be a nonnegative integer valued random variable. Let ΦX (t) =


EtX be its probability generating function and assume that ΦX (t) is
finite for all t. By arguing as in the proof of Chebeshev inequality, show
that for any positive integer, y,
ΦX (t)
a. P [X ≤ y] ≤ ty
, 0 ≤ t ≤ 1;
ΦX (t)
b. P [X ≥ y] ≤ ty
, t ≥ 1.

Now suppose X is a Poisson random variable with parameter λ. Use


the above to show that
" #  λ/2
λ 2
P X≤ ≤ .
2 e

5. Let X, Y be two random variables each having mean zero and variance
one. Let ρ be the correlation coefficient of X, Y . Show that
q
E[max(X 2 , Y 2 )] ≤ 1 + 1 − ρ2

(Hint: You can use the identity max(a, b) = (a + b + |a − b|)/2).

6. Let X1 , · · · , Xn be iid random variables with a density (or mass) func-


tion having a parameter θ. Let X = (X1 , · · · , Xn ). Let T (X) be a
function of X. As mentioned earlier such T is called a statistic. If the
conditional distribution of X given T (X) does not depend on θ then
T (X) is called a sufficient statistic for θ. Show that T (X) = ni=1 Xi
P

is a sufficient statistic when Xi are poisson with mean θ.

7. Suppose you have to play the following game. You are going to be
shown N prizes in sequence. At any time you can either accept the

2
one that is being offered or reject it and choose to see the next prize.
Once you reject a prize you cannot go back to it. At any time you are
seeing a prize, all the information you have is the relative rank of the
prize that you are being offered, with respect to all the ones that have
gone by. That is, when you are seeing the third prize, you know how it
ranks with respect to the first and second ones that you have already
seen and rejected. Consider the following strategy. You fix an integer
k between 1 and N . You reject the first k prizes and then accept the
first one that you see which is better than all the ones rejected till
that point. (If after the first k, in the remaining N − k chances, you
never see a prize that is better than all the ones you had rejected till
then, then you would end up rejecting all the prizes). Assuming that
all possible orderings of the N prizes are equally likely, calculate the
probability that this strategy would get you the best prize. Based on
this, suggest what is a good value of k to choose.

You might also like