Parameter, Statistic and Random Samples

• A parameter is a number that describes the population. It is a fixed

number, but in practice we do not know its value.

• A statistic is a function of the sample data, i.e., it is a quantity

whose value can be calculated from the sample data. It is a random
variable with a distribution function. Statistics are used to make
inference about unknown population parameters.

• The random variables X1, X2,…, Xn are said to form a (simple)

random sample of size n if the Xi’s are independent random
variables and each Xi has the sample probability distribution. We say
that the Xi’s are iid.

week1 1
 Example – Sample Mean and Variance

• Suppose X1, X2,…, Xn is a random sample of size n from a population

with mean μ and variance σ2.

• The sample mean is defined as

1 n
X   Xi.
n i 1

• The sample variance is defined as

S 
2 1 n
 X i  X  .
2

n  1 i 1

week1 2
 Goals of Statistics

• Estimate unknown parameters μ and σ2.

• Measure errors of these estimates.

• Test whether sample gives evidence that parameters are (or are
not) equal to a certain value.

week1 3
 Sampling Distribution of a Statistic
• The sampling distribution of a statistic is the distribution of values
taken by the statistic in all possible samples of the same size from
the same population.
• The distribution function of a statistic is NOT the same as the
distribution of the original population that generated the original
sample.
• The form of the theoretical sampling distribution of a statistic will
depend upon the distribution of the observable random variables in
the sample.

week1 4
 Sampling from Normal population

• Often we assume the random sample X1, X2,…Xn is from a normal

population with unknown mean μ and variance σ2.

• Suppose we are interested in estimating μ and testing whether it is

equal to a certain value. For this we need to know the probability
distribution of the estimator of μ.

week1 5
 Claim
• Suppose X1, X2,…Xn are i.i.d normal random variables with
unknown mean μ and variance σ2 then
 2 
X ~ N   , 
 n 

• Proof:

week1 6
 Recall - The Chi Square distribution

• If Z ~ N(0,1) then, X = Z2 has a Chi-Square distribution with

parameter 1, i.e., X ~  21 .

• Can proof this using change of variable theorem for univariate

random variables.
1/ 2
 1 
• The moment generating function of X is m X t    
 1  2t 

• If X 1 ~  2v1  , X 2 ~  2v2  , , X k ~  2vk  , all independent then

k
T   X i ~  2k v
1 i
i 1

• Proof…

week1 7
 Claim
• Suppose X1, X2,…Xn are i.i.d normal random variables with mean μ
and variance σ2. Then, Z i  X i   are independent standard normal

variables, where i = 1, 2, …, n and
 Xi   
n n 2


i 1
Z i   
i 1   
 ~  2
n 

• Proof: …

week1 8
 t distribution
• Suppose Z ~ N(0,1) independent of X ~ χ2(n). Then, T  Z
~ t v  .
X /v
• Proof:

week1 9
 Claim
• Suppose X1, X2,…Xn are i.i.d normal random variables with mean μ
and variance σ2. Then,
X 
~ tn1
S/ n
• Proof:

week1 10
 F distribution
• Suppose X ~ χ2(n) independent of Y ~ χ2(m). Then,

X /n
~ Fn ,m 
Y /m

week1 11
 Properties of the F distribution
• The F-distribution is a right skewed distribution.

 1 
i.e. PFn ,m   a   P

1 1 1
• Fm,n      P Fm,n   
Fn,m  F   a
  n ,m  a 

• Can use Table 7 on page 796 to find percentile of the F- distribution.

• Example…

week1 12
 The Central Limit Theorem
• Let X1, X2,…be a sequence of i.i.d
n
random variables with E(Xi) = μ < ∞
and Var(Xi) = σ2 < ∞. Let S n   X i
i 1

 S n  n 
lim P
Then, n    z   PZ  z   z  for - ∞ < x < ∞
  n 

where Z is a standard normal random variable and Ф(z)is the cdf for the
standard normal distribution.
S n  n
• This is equivalent to saying that Z n  converges in distribution to
Z ~ N(0,1).  n

 Xn   
• Also, n 
lim P   x   x 

 n 
Xn  
i.e. Z n  converges in distribution to Z ~ N(0,1).
 n
week1 13
 Example
• Suppose X1, X2,…are i.i.d random variables and each has the Poisson(3)
distribution. So E(Xi) = V(Xi) = 3.

 
• The CLT says that P X 1    X n  3n  x 3n  x as n  ∞.

week1 14
 Examples
• A very common application of the CLT is the Normal approximation to the
Binomial distribution.
• Suppose X1, X2,…are i.i.d random variables and each has the Bernoulli(p)
distribution. So E(Xi) = p and V(Xi) = p(1- p).

 
• The CLT says that P X 1    X n  np  x np1  p  x as n  ∞.

• Let Yn = X1 + … + Xn then Yn has a Binomial(n, p) distribution.

 y  np   y  np 
So for large n, PYn  y   P Yn  np    
   
 np1  p  np1  p    np1  p  

• Suppose we flip a biased coin 1000 times and the probability of heads on
any one toss is 0.6. Find the probability of getting at least 550 heads.

• Suppose we toss a coin 100 times and observed 60 heads. Is the coin fair?

week1 15