STAT6201 Lecture Note 5 Fall 2021

5 Properties of a Random Sample

5.1 Basic Concepts of Random Sample
Definition 1 (Random Sample, Textbook Definition 5.1.1, pp.207) If each RV has a pdf/pmf f (x),
then the joint pdf or pmf of (X1 , . . . , Xn ) is given by
n
Y n
Y
f (x1 , · · · , xn ) = f (x1 )f (x2 ) · · · f (xn ) = f (xi ), or f (x1 , · · · , xn | θ) = f (xi | θ).
i=1 i=1

Therefore, (X1 , . . . , Xn ) can be regarded as a very special multivariate random vector.

Example 5.1.1 Textbook Example 5.1.2, pp.208.

5.2 Summary of A Random Sample

5.2.1 Statistics
Definition 2 (Statistic, Sampling Distribution, Textbook Definition 5.2.1, pp.211)
Remark: A statistic is a function of the sample, NOT a function of parameters.
Examples of commonly statistics:
Pn
sample sum T = i=1P Xi .
sample mean X̄ = n1 iPXi .
1
sample variance S 2 = n−1 i (Xi − X̄) .
2
√
sample standard deviation S = S2.
minimum T = min(X1 , . . . , Xn ).
maximum T = max(X 1 ,P. . . , Xn ).
a two-dimensional statistic T = ( i Xi , i Xi2 ).
P

Example 5.2.1 Let X1 , . . . , Xn be a random sample from Exp(β). Let T1 = max(X1 , . . . , Xn ). Find the
sampling distribution of T1 . What about T2 = min(X1 , . . . , Xn )?

5.2.2 Sample Mean and Sample Variance

Lemma 1 Textbook Lemma 5.2.5, pp.213.
Theorem 1 (Textbook Theorem 5.2.6, pp.213)
Proof required!
Remark 1 Conclusions from the theorem above
(a) The sample mean is an unbiased estimator of the population mean.
(b) The variance of the sample mean decreases when the sample size increases; a large sample size will lead
to a more accurate sample mean (in the sense of repeated sampling!).
(c) The sample variance is an unbiased estimator of the population variance.
Theorem 2 (Textbook Theorem 5.2.7, pp.215) Furthermore, let Y = X1 + · · · + Xn . Then
n
MY (t) = {MX (t)} .
Example 5.2.2
Pn
Xi ∼ Binomial (k, p) i.i.d. −→ i=1 Xi ∼ Binomial (nk, p)
Pn
Xi ∼ Poisson (λ) i.i.d. −→ i=1 Xi ∼ Poisson (nλ)
Pn
Xi ∼ N(µ, σ 2 ) i.i.d. −→ i=1 Xi ∼ N(nµ, nσ 2 )
Pn
Xi ∼ Gamma (α, β) i.i.d. −→ i=1 Xi ∼ Gamma (nα, β)
STAT6201 Lecture Note 5 Page 2 of 2

5.3 Sampling from the Normal Distribution

5.3.1 Properties of the Sample Mean and Variance
Theorem 3 (Textbook Theorem 5.3.1, pp.218)

Proof. Proof sketch (different from the Textbook proof):

(a). First prove (X̄, X1 − X̄, . . . , Xn−1 − X̄) is jointly normal (any linear combination of them is normal
since it can be written as a weighted sum of i.i.d. X1 , . . . , Xn ). Then prove Cov(X̄, Xi − X̄) = 0 for any
i = 1, . . . , n−1, so X̄ is independent of all (X1 − X̄, . . . , Xn−1 − X̄). S 2 is a function of (X1 − X̄, . . . , Xn−1 − X̄)
Pn−1
(note that Xn − X̄ = − j=1 (Xj − X̄)), so it is independent of X̄.

(b). Obvious, so omitted.

(c). First
n
X n
X n
X
(Xi − µ)2 = (Xi − X̄ + X̄ − µ)2 = (Xi − X̄)2 + n(X̄ − µ)2 .
i=1 i=1 i=1

Thus
n  2
. X Xi − µ
W = ∼ χ2n
i=1
σ
n 2
(Xi − X̄)2 (X̄ − µ)2 (n − 1)S 2 .

X X̄ − µ .
= +n = (= V ) + √ (= Z).
i=1
σ2 σ 2 σ2 σ/ n

By (b),
 2
X̄ − µ
Z= √ ∼ χ21 .
σ/ n
Due to the independence of S 2 and X̄, MW (t) = MV (t)MZ (t). Therefore

MW (t) (1 − 2t)−n/2 (n − 1)S 2

MV (t) = = = (1 − 2t)−(n−1)/2 , i.e., V = ∼ χ2n−1 .
MZ (t) (1 − 2t)−1/2 σ2

Lemma 2 (Other Properties, Textbook Lemma 5.3.3, pp.220)

Acknowledgement
The lecture notes of this course are based on the textbook and Prof. Huixia Judy Wang’s lecture slides. The
instructor thanks Prof. Wang for kindly sharing them.

The End.