Scribd
Upload
60 views

Variance Estimation

This document discusses estimating the mean and variance of a population from a sample. It states that the sample mean is an unbiased estimator of the population mean, with its expectation equal to the population mean and variance equal to the population variance divided by the sample size. The sample variance is also an unbiased estimator of the population variance. It requires a denominator of N-1 rather than N to properly account for the degrees of freedom used in estimating the variance.

Uploaded by

Malkin Divya
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
60 views

Variance Estimation

This document discusses estimating the mean and variance of a population from a sample. It states that the sample mean is an unbiased estimator of the population mean, with its expectation equal to the population mean and variance equal to the population variance divided by the sample size. The sample variance is also an unbiased estimator of the population variance. It requires a denominator of N-1 rather than N to properly account for the degrees of freedom used in estimating the variance.

Uploaded by

Malkin Divya
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

ACM/ESE 118

Mean and variance estimation

Consider a sample x1 , . . . , xN from a random variable X. The sample may have


been obtained through N independent but statistically identical experiments. From
this sample, we want to estimate the mean and variance 2 of the random variable
X (i.e., we want to estimate population quantities).
The sample mean
N
1 X
x = xi
N
i=1

is an estimate of the mean . The expectation value of the sample mean is the
population mean, E(x) = , and the variance of the sample mean is var(x) =
2 /N . Since the expectation value of the sample mean is the population mean,
the sample mean is said to be an unbiased estimator of the population mean. And
since the variance of the sample mean approaches zero as the sample size increases
(i.e., fluctuations of the sample mean about the population mean decay to zero with
increasing sample size), the sample mean is said to be a consistent estimator of the
population mean.
These properties of the sample mean are a consequence of the fact that if
x1 , . . . , xN are mutually uncorrelated random variables with variances 12 , . . . , N
2 ,

the variance of their sum z = x1 + + xN is

z2 = 12 + + N
2
. (1)

If we view the members of the sample x1 , . . . , xN as realizations of identically


distributed random variables with mean E(xi ) = and variance var(xi ) = 2 ,
it follows by the linearity of the expectation value operation that P
the expectation
value of the sample mean is the population mean: E(x) = N P i E(xi ) = .
1

The variance of the sample mean follows from (1): var(x) = N 2 i i2 = 2 /N .


Moreover, the Central Limit Theorem states that, under fairly general con-
ditions, the distribution of the sample mean x approaches a normal distribution
N (, 2 /N ) with mean and variance 2 /N as the sample size N increases [see,
e.g., Johnson and Wichern (2002, chapter 4.5) or Papoulis (1991, chapter 8)].
The sample variance
N
1 X
s2 = (xi x)2
N 1
i=1

is an unbiased estimator of the population variance 2 , that is, E(s2 ) = 2 . To see

1
this, we calculate
N
1 X
E [(xi )(x )] = E [(xi )(xj )]
N
j=1
1
= E(xi )2
N
2
= ,
N
where we have
 used 2the
 assumption that the xi are mutually uncorrelated. With
2
var(x) = E (x ) = /N , it then follows that
n o
E (xi x)2 = E [(xi ) (x )]2
 

2 2
= 2 + 2
N N
N 1 2
= .
N
Thus,
N
1 X 
E(s2 ) = E (xi x)2 = 2 .

N 1
i=1
The denominator N 1 in the sample variance is necessary to ensure unbi-
asedness of the variance estimator. The denominator N would only be correct if
fluctuations about the population mean and not about the sample mean x would
appear in the expression for the sample variance. With the denominator N 1,
one obtains an indefinite sample variance for a sample of size N = 1, as expected.
With the denominator N , the sample variance would vanish, yielding an obviously
incorrect estimate of the population variance. The denominator N 1 appears
because, after estimation of the sample mean, only N 1 degrees of freedom are
available for the estimation of the variance, since the variables x 1 , . . . , xN and the
sample mean satisfy the constraint
N
X
(xi x) = 0.
i=1

References
Johnson, R. A., and D. W. Wichern, 2002: Applied Multivariate Statistical Anal-
ysis. 5th ed., Prentice-Hall, 767 pp.
Papoulis, A., 1991: Probability, Random Variables, and Stochastic Processes. 3rd
ed., McGraw Hill, 666 pp.

You might also like