Scribd
Upload
94 views2 pages

Law of Large Numbers

The law of large numbers states that if an experiment is repeated independently many times, the average of the results will be close to the expected value. There are two versions: the weak law of large numbers and the strong law. The weak law says that as the number of trials increases, the probability that the sample mean will differ from the expected value by a given amount decreases toward zero. It is proved using Chebyshev's inequality and assuming the variables have finite variance.

Uploaded by

bossishere
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
94 views2 pages

Law of Large Numbers

The law of large numbers states that if an experiment is repeated independently many times, the average of the results will be close to the expected value. There are two versions: the weak law of large numbers and the strong law. The weak law says that as the number of trials increases, the probability that the sample mean will differ from the expected value by a given amount decreases toward zero. It is proved using Chebyshev's inequality and assuming the variables have finite variance.

Uploaded by

bossishere
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Law of Large Numbers https://www.probabilitycourse.com/chapter7/7_1_1_law_of_large_numb...

7.1.1 Law of Large Numbers


The law of large numbers has a very central role in probability and statistics. It states that if you repeat an
experiment independently a large number of times and average the result, what you obtain should be close to
the expected value. There are two main versions of the law of large numbers. They are called the weak and
strong laws of the large numbers. The difference between them is mostly theoretical. In this section, we state
and prove the weak law of large numbers (WLLN). The strong law of large numbers is discussed in Section
7.2. Before discussing the WLLN, let us define the sample mean.

⎯⎯⎯⎯
Definition 7.1. For i.i.d. random variables X1 , X2 , . . . , Xn , the sample mean, denoted by X , is
defined as

⎯⎯⎯⎯ X1 + X2 +. . . +Xn
X = .
n
Another common notation for the sample mean is Mn . If the Xi 's have CDF FX (x), we might show
the sample mean by Mn (X) to indicate the distribution of the Xi 's.

⎯⎯⎯⎯
Note that since the Xi 's are random variables, the sample mean, X = Mn (X), is also a random variable. In
particular, we have

⎯⎯⎯⎯ EX1 + EX2 +. . . +EXn


E[X ] = (by linearity of expectation)
n
nEX
= (since EXi = EX)
n
= EX.
⎯⎯⎯⎯
Also, the variance of X is given by

⎯⎯⎯⎯ Var(X1 + X2 +. . . +Xn )


Var(X ) = (since Var(aX) = a2 Var(X))
n2
Var(X1 ) + Var(X2 )+. . . +Var(Xn )
= (since the Xi 's are independent)
n2
nVar(X)
= (since Var(Xi ) = Var(X))
n2
Var(X)
= .
n
Now let us state and prove the weak law of large numbers (WLLN).

1 of 2 10/12/22, 3:03 pm
Law of Large Numbers https://www.probabilitycourse.com/chapter7/7_1_1_law_of_large_numb...

The weak law of large numbers (WLLN)

Let X1 , X2 , ... , Xn be i.i.d. random variables with a finite expected value EXi = μ < ∞. Then, for
any ϵ > 0,
⎯⎯⎯⎯
lim P(|X − μ| ≥ ϵ) = 0.
n→∞

• Proof
◦ The proof of the weak law of large number is easier if we assume Var(X) = σ 2 is finite. In
this case we can use Chebyshev's inequality to write
⎯⎯⎯⎯
⎯⎯⎯⎯ Var(X )
P(|X − μ| ≥ ϵ) ≤
ϵ2
Var(X)
= ,
nϵ2
which goes to zero as n → ∞.

← previous
next →

The print version of the book is available through Amazon here.

2 of 2 10/12/22, 3:03 pm

You might also like