Chapter 4: Interval estimation

STAT2602A Probability and statistics II

STAT3902 Statistical Models
(2023-2024 1st Semester)
Contents

4.1 Basic concepts

4.2 Confidence intervals for means

4.3 Confidence intervals for variances

4.4 Confidence intervals: Large samples

4.1 Basic concepts

I Motivation: A point estimator for θ does not provide much

information about the accuracy of the estimator. It is
desirable to generate a narrow interval that will cover the
unknown parameter θ with a large probability (confidence).
I Definition 4.1. (Interval estimator)An interval estimator of θ
is a random interval [L(X), U(X)], where
L(X) := L(X1 , · · · , Xn ) and U(X) := U(X1 , · · · , Xn ) are two
statistics such that L(X) ≤ U(X) with probability one.
I Definition 4.2. (Interval estimate) If X = x is observed,
[L(x), U(x)] is the interval estimate of θ.
I Remark: it will sometimes be more natural to use an open
interval (L(X), U(X)), a half-open and half-closed interval
(L(X), U(X)] (or [L(X), U(X))), or an one-sided interval
(∞, U(X)] (or [L(X), ∞)).
4.1 Basic concepts
Example 4.1.For an independent random sample X1 , X2 , X3 , X4
from N(µ, 1), consider an interval estimator of µ by [X − 1, X + 1].
Then, the probability that µ is covered by the interval
[X − 1, X + 1] can be calculated by



P µ ∈ [X − 1, X + 1] = P X − 1 ≤ µ ≤ X + 1
 p 
= P −2 ≤ (X − µ)/ 1/4 ≤ 2
= P (−2 ≤ Z ≤ 2)
≈ 0.9544,

where Z ∼ N(0, 1) and with the fact that X ∼ N(µ, 1/4). Thus,
we have over a 95% change of covering the unknown parameter
with our interval estimator. Note that for any point estimator µ̂ of
µ, we have P(µ̂ = µ) = 0. Sacrificing some precision in the interval
estimator, in moving from a point to an interval, has resulted in
increased confidence that our assertion about µ is correct.
4.1 Basic concepts
I Definition 4.3. (Confidence coefficient) For an interval
estimator [L(X), U(X)] of θ, the confidence coefficient of
[L(X), U(X)], denoted by (1 − α), is

1 − α = P(θ ∈ [L(X), U(X)]),

where P(θ ∈ [L(X), U(X)]) is the coverage probability of

[L(X), U(X)].
I Remark: In some situations, the coverage probability
P(θ ∈ [L(X), U(X)]) may depend on θ, and then the
confidence coefficient is defined as

1 − α = inf P(θ ∈ [L(X), U(X)]).

θ

I Interval estimator, together with a measure of confidence (say,

the confidence coefficient), is sometimes known as confidence
interval. A confidence interval with confidence coefficient
equal to 1 − α, is called a 1 − α confidence interval.
4.1 Basic concepts

I Definition 4.4. (Pivotal Quantity) A random variable

Q(X, θ) = Q(X1 , · · · , Xn , θ) is a pivotal quantity if the
distribution of Q(X, θ) is free of θ. That is, regardless of the
distribution of X, Q(X, θ) has the same distribution for all
values of θ.
I Remark: Logically, when Q(X, θ) is a pivotal quantity, we can
easily construct a 1 − α confidence interval for Q(X, θ) by
 
1 − α = P Leα ≤ Q(X, θ) ≤ Ueα , (1.1)

where Leα and Ueα do not depend on θ. Suppose that the

inequalities Leα ≤ Q(X, θ) ≤ Ueα in (1.1) are equivalent to the
inequalities L(X) ≤ θ ≤ U(X), Then, from (1.1), a 1 − α
confidence interval of θ is [L(X), U(X)].
4.2 Confidence intervals for means - One-sample case
Formula derivation Let X = {X1 , · · · , Xn } be an independent
random sample from the population N(µ, σ 2 ). We first consider
the interval estimator of µ when σ 2 is known. Note that
X ∼ N(µ, σ 2 /n). (2.1)
X −µ
Hence, when σ 2 is known, Z = √ ∼ N(0, 1) is a pivotal
σ/ n
quantity involving µ. Let


1 − α = P −zα/2 ≤ Z ≤ zα/2
 
X −µ
= P −zα/2 ≤ √ ≤ zα/2
σ/ n
 
σ σ
= P X − zα/2 √ ≤ µ ≤ X + zα/2 √ ,
n n
where zα satisfies
P(Z ≥ zα ) = α
for Z ∼ N(0, 1)
4.2 Confidence intervals for means - One-sample case
Formula derivation (con’t) Usually, we call zα the upper
percentile of N(0, 1) at the level α. When σ 2 is known, a 1 − α
confidence interval of µ is
 
σ σ
X − zα/2 √ , X + zα/2 √ . (2.2)
n n

Given the observed value of X = x and the value of zα/2 , we can

calculate the interval estimate of µ by
 
σ σ
x − zα/2 √ , x + zα/2 √ .
n n

As the point estimator, the 1 − α confidence interval is also not

unique. Ideally, we should choose it as narrow as possible in some
sense, but in practice, we usually choose the equal-tail confidence
interval as in (2.2) for convenience, since tables for selecting equal
probabilities in the two tails are readily available.
4.2 Confidence intervals for means - One-sample case
Example 4.2. A publishing company has just published a new
college textbook. Before the company decides the price of the
book, it wants to know the average price of all such textbooks in
the market. The research department at the company took a
sample of 36 such textbooks and collected information on their
prices. This information produced a mean price of $48.40 for this
sample. It is known that the standard deviation of the prices of all
such textbooks is $4.50. Construct a 90% confidence interval for
the mean price of all such college textbooks assuming that the
underlying population is normal.
Solution. From the given information, n = 36, x = 48.40 and
σ = 4.50. Now, 1 − α = 0.9, i.e., α = 0.1, and by (2.2), the 90%
confidence interval for the mean price of all such college textbooks
is given by
σ σ 4.50 4.50
[x − zα/2 √ , x + zα/2 √ ] = [48.40 − z0.05 √ , 48.40 + z0.05 √ ]
n n 36 36
≈ [47.1662, 49.6338].
4.2 Confidence intervals for means - One-sample case
Example 4.3. Suppose the bureau of the census and statistics of a
city wants to estimate the mean family annual income µ for all
families in the city. It is known that the standard deviation σ for
the family annual income is 60 thousand dollars. How large a
sample should the bureau select so that it can assert with
probability 0.99 that the sample mean will differ from µ by no
more than 5 thousand dollars?
Solution. From the construction of a confidence interval, we have
−zα/2 σ zα/2 σ zα/2 σ
   
1−α=P √ ≤X −µ≤ √ = P |X − µ| ≤ √ ,
n n n
where 1 − α = 0.99 and σ = 60 thousand dollars. It suffices to
have zα/2 √σn ≤ 5 or

2
n ≥ 60zα/2 /5 = (60 × 2.576/5)2 ≈ 955.5517.
Thus, the sample size should be at least 956. (Note that we have
to round 955.5517 up to the next higher integer. This is always the
case when determining the sample size.)
4.2 Confidence intervals for means - One-sample case
Case: consider the interval estimator of µ when σ 2 is
unknown.
I Property 4.1. (i) X and S 2 are independent;
Pn
2
nS 2 i=1 Xi − X
(ii) 2 = is χ2n−1 , where χ2k is a chi-square
σ σ2
distribution with k degrees of freedom;
X −µ
(iii) T = √ is tn−1 , where tk is a t distribution with
S/ n − 1
k degrees of freedom.
I Property 4.2. (i) If Z1 , · · · , Zk are k independent N(0, 1)
X k
random variables, then Zi2 is χ2k ;
i=1
(ii) If Z is N(0, 1), U is χ2k , and Z and U are independent,
Z
then T = p is tk .
U/k
The proof of Property 4.1. could be found in the lecture notes.
4.2 Confidence intervals for means - One-sample case

Derivation of the interval estimator of µ when σ 2 is unknown.

From Property 4.1(iii), we know that T is a pivotal quantity of µ.
Let


1 − α = P −tα/2,df =n−1 ≤ T ≤ tα/2,df =n−1
 
X −µ
= P −tα/2,df =n−1 ≤ √ ≤ tα/2,df =n−1
S/ n − 1
 
S S
= P X − tα/2,df =n−1 √ ≤ µ ≤ X + tα/2,df =n−1 √
n−1 n−1
where tα,df =k satisfies

P(T ≥ tα,df =k ) = α

for a random variable T ∼ tk .

4.2 Confidence intervals for means - One-sample case
Derivation con’t
Therefore, when σ 2 is unknown, a 1 − α confidence interval of µ is
 
S S
X − tα/2,df =n−1 √ , X + tα/2,df =n−1 √ . (2.3)
n−1 n−1

Given the observed value of X = x, S = s, and the value of

tα/2,df =n−1 , we can calculate the interval estimate of µ by
 
s s
x − tα/2,df =n−1 √ , x + tα/2,df =n−1 √ .
n−1 n−1

I Remark: Usually there is a row with ∞ degrees of freedom in

a t-distribution table, which actually shows values of zα . In
fact, when n → ∞, the distribution function of tn tends to
that of N(0, 1). That is, in tests or exams, if n is so large that
the value of tα,df =n cannot be found, you may use zα instead.
4.2 Confidence intervals for means - One-sample case

Example 4.4 A paint manufacturer wants to determine the average

drying time of a new brand of interior wall paint. If for 12 test areas
of equal size he obtained a mean drying time of 66.3 minutes and
a standard deviation of 8.4 minutes, construct a 95% confidence
interval for the true population mean assuming normality.

Solution. As n = 12, x = 66.3, s = 8.4, α = 1 − 0.95 = 0.05 and

tα/2,df =n−1 = t0.025,11 ≈ 2.201, the 95% confidence interval for µ
is
 
8.4 8.4
66.3 − 2.201 × √ , 66.3 + 2.201 × √ ,
12 − 1 12 − 1

that is, [61.1722, 71.4278].

4.2 Confidence intervals for means - One-sample case

Example 4.5 Construct a 95% confidence interval for the mean

hourly wage of apprentice geologists employed by the top 5 oil
companies. For a sample of 50 apprentice geologists, x = 14.75
and s = 3.0 (in dollars).

Solution. As n = 50, α = 1 − 0.95 = 0.05, t0.025,df =49 ≈ 2.010, we

have
s 3.0
tα/2,df =n−1 √ ≈ 2.010 × √ = 0.8614.
n−1 50 − 1

Thus, the 95% confidence interval is [14.75 − 0.86, 14.75 + 0.86],

or [13.89, 15.59].
4.2 Confidence intervals for means - Two-sample case
In the next part, we shall consider the problem of constructing
confidence intervals for the difference of the means of two normal
distributions when the variances are unknown.
Formula derivation
Let X = {X1 , X2 , · · · , Xn } and Y = {Y1 , Y2 , · · · , Ym } be random
samples from independent distributions N(µX , σX2 ) and N(µY , σY2 ),
respectively. We are of interest to construct the confidence interval
for µX − µY when σX2 = σY2 = σ 2 .
First, we can show that
(X − Y ) − (µX − µY )
Z= p
σ 2 /n + σ 2 /m
is N(0, 1). Also, by the independence of X and Y, from Property
4.1(ii), we know that
nSX2 mSY2
U= +
σ2 σ2
is χ2n+m−2 .
4.2 Confidence intervals for means - Two-sample case
Formula derivation con’t
Moreover, by Property 4.1(i), Z and U are independent. Hence,

Z
T =p
U/(n + m − 2)
p
[(X − Y ) − (µX − µY )]/ σ 2 /n + σ 2 /m
= q
(nSX2 + mSY2 )/[σ 2 (n + m − 2)]
(X − Y ) − (µX − µY )
=
R
is tn+m−2 , where
s
nSX2 + mSY2
 
1 1
R= + .
n+m−2 n m

That is, T is a pivotal quantity of µX − µY .

4.2 Confidence intervals for means - Two-sample case

Formula derivation con’t

Let


1 − α = P −tα/2,df =n+m−2 ≤ T ≤ tα/2,df =n+m−2
 
(X − Y ) − (µX − µY )
= P −t ≤ ≤t
R


= P (X − Y ) − tR ≤ µX − µY ≤ (X − Y ) + tR ,

So, when σX2 = σY2 = σ 2 is unknown, a 1 − α confidence interval

of µX − µY is
 
(X − Y ) − tα/2,df =n+m−2 R, (X − Y ) + tα/2,df =n+m−2 R . (2.4)
4.2 Confidence intervals for means - Two-sample case

Formula derivation con’t

Given the observed value of X = x, Y = y , SX = sX , SY = sY ,
and the value of tα/2,df =n+m−2 , we can calculate the interval
estimate of µX − µY by
 
(x − y ) − tα/2,df =n+m−2 r , (x − y ) + tα/2,df =n+m−2 r .

where s
nsX2 + msY2
 
1 1
r= + .
n+m−2 n m
4.2 Confidence intervals for means - Two-sample case

Example 4.6 Suppose that scores on a standardized test in

mathematics taken by students from large and small high schools
are N(µX , σ 2 ) and N(µY , σ 2 ), respectively, where σ 2 is unknown. If
a random sample of n = 9 students from large high schools yielded
x̄ = 81.31, sX2 = 60.76 and a random sample of m = 15 students
from small high schools yielded ȳ = 78.61, sY2 = 48.24, the
endpoints for a 95% confidence interval for µX − µY are given by
s  
9 × 60.76 + 15 × 48.24 1 1
81.31 − 78.61 ± 2.074 + ,
22 9 15

since P(T ≤ 2.074) = 0.975. So, the 95% confidence interval is

[−3.95, 9.35].
4.3 Confidence intervals for variances - One-sample case
Formula derivation First, we consider the one-sample case. By
Property 4.1(ii),
nS 2
∼ χ2n−1
σ2
is a pivotal quantity involving σ 2 . Let

nS 2
 
2 2
1 − α = P χ1−α/2,df =n−1 ≤ 2 ≤ χα/2,df =n−1
σ
!
nS 2 nS 2
2
=P ≤σ ≤ 2 ,
χ2α/2,df =n−1 χ1−α/2,df =n−1

where χ2α,df =n satisfies

P(T ≥ χ2α,df =n ) = α

for a random variable T ∼ χ2n .

4.3 Confidence intervals for variances - One-sample case

Formula derivation con’t

Therefore, a 1 − α confidence interval of σ 2 is
" #
nS 2 nS 2
, . (3.1)
χ2α/2,df =n−1 χ21−α/2,df =n−1

Given the observed value of S = s and the values of χ2α/2,df =n−1

and χ21−α/2,df =n−1 , we can calculate the interval estimate of σ 2 by
" #
ns 2 ns 2
, .
χ2α/2,df =n−1 χ21−α/2,df =n−1
4.3 Confidence intervals for variances - One-sample case
Example 4.7. A machine is set up to fill packages of cookies. A
recently taken random sample of the weights of 25 packages from
the production line gave a variance of 2.9 g2 . Construct a 95%
confidence interval for the standard deviation of the weight of a
randomly selected package from the production line.

Solution. As n = 25, s 2 = 2.9, α = 0.05,

ns 2 25(2.9) 25(2.9)
= ≈ ≈ 1.8420,
χ2α/2,df =n−1 χ20.025,df =24 39.36
ns 2 25(2.9) 25(2.9)
= ≈ ≈ 5.8468,
χ21−α/2,df =n−1 χ20.975,df =24 12.40

the 95% confidence interval for the population variance is

(1.8420, 5.8468). Taking positive square roots, we obtain the 95%
confidence interval for the population standard deviation to be
(1.3572, 2.4180).
4.3 Confidence intervals for variances - Two-sample case

I Basic settings: Consider the two-sample case. Let

X = {X1 , X2 , · · · , Xn } and Y = {Y1 , Y2 , · · · , Ym }

be random samples from independent distributions N(µX , σX2 )

and N(µY , σY2 ), respectively. We are of interest to construct
the confidence interval for σX2 /σY2 .
I Property 4.3.Suppose that U ∼ χ2r1 and V ∼ χ2r2 are
independent. Then,

U/r1
Fr1 ,r2 =
V /r2

has an Fr1 ,r2 distribution with r1 and r2 degrees of freedom.

4.3 Confidence intervals for variances - Two-sample case

Formula derivation
By Property 4.1(ii),

nSX2 2 mSY2
∼ χ n−1 and ∼ χ2m−1 .
σX2 σY2

Then, by Property 4.3, it follows that

mSY2 nSX2
 . 
∼ Fm−1,n−1 ,
σY2 (m − 1) σX2 (n − 1)

which is a pivotal quantity involving σX2 /σY2 .

4.3 Confidence intervals for variances - Two-sample case

Formula derivation con’t

Let
 
mSY2
h i
σY2 (m−1)
1 − α = P F1−α/2,df =(m−1,n−1) ≤ h i ≤ Fα/2,df =(m−1,n−1) 
 
nSX2
σX2 (n−1)

n(m − 1)SX2 σX2 n(m − 1)SX2

 
=P F ≤ ≤ F ,
m(n − 1)SY2 σY2 m(n − 1)SY2

where Fα,df =(m,n) satisfies

P(T ≥ Fα,df =(m,n) ) = α

for a random variable T ∼ Fm,n .

4.3 Confidence intervals for variances - Two-sample case

Formula derivation con’t

Therefore, a 1 − α confidence interval of σX2 /σY2 is

n(m − 1)SX2 n(m − 1)SX2

 
F1−α/2,df =(m−1,n−1) , Fα/2,df =(m−1,n−1) .
m(n − 1)SY2 m(n − 1)SY2
(3.2)

Given the observed value of SX = sX , SY = sY , and the values of

Fα/2,df =(m−1,n−1) and F1−α/2,df =(m−1,n−1) , we can calculate the
interval estimate of σX2 /σY2 by

n(m − 1)sX2 n(m − 1)sX2

 
F1−α/2,df =(m−1,n−1) , Fα/2,df =(m−1,n−1) .
m(n − 1)sY2 m(n − 1)sY2
4.4 Confidence intervals: Large samples

I Motivation: When the population is not normal, we can

make use of the CLT to propose confidence intervals, which
has an approximated confidence coefficient 1 − α for large n.
I Theorem 4.1. (Slutsky’s theorem) If Xn →d X and Yn →p C
(a constant), then
(i) Xn + Yn →d X + C ;
(ii) Xn · Yn →d X · C ;
(iii) Xn /Yn →d X /C provided that C 6= 0.
Remark: Note that Theorem 4.1 fails if C is not a constant.
4.4 Confidence intervals: Large samples

Formula derivation
Let X be an independent random sample from a population, which
has the mean µ and the variance σ 2 < ∞. According to Theorem
4.1, CLT and the fact that S →p σ, we have
√ √
n(X − µ) n(X − µ) S
= · →d N(0, 1), (4.1)
S σ σ
for large n. Hence, by (4.1), it follows that for large n,
 √ 
n(X − µ)
1 − α ≈ P −zα/2 ≤ ≤ zα/2
S
 
S S
= P X − zα/2 √ ≤ µ ≤ X + zα/2 √ .
n n
4.4 Confidence intervals: Large samples

Formula derivation con’t

So, an approximated 1 − α confidence interval of µ is
 
S S
X − zα/2 √ , X + zα/2 √ . (4.2)
n n

Given the observed value of x and s, we can calculate the interval

estimate of µ by
 
s s
x − zα/2 √ , x + zα/2 √ .
n n

Note that the confidence interval in (4.2) only requires a large n

but not the normal population assumption. Clearly, the similar idea
can be applied to the two-sample case.
4.4 Confidence intervals: Large samples
End of the Chapter
To end this chapter, we consider the interval estimator for
percentage p, where

p = P (X ∈ (a, b)) .

Define ξ = I(a < X < b). Then, E(ξ) = p. This indicates that p is
the theoretical mean of ξ. Hence, by (4.2), an approximated 1 − α
confidence interval of p is
 
Sξ Sξ
ξ − zα/2 √ , ξ + zα/2 √ , (4.3)
n n

where ξ = n1 ni=1 ξi and Sξ2 = n1 ni=1 (ξi − ξ)2 = ξ(1 − ξ) with

P P
ξi = I(a < Xi < b).
In general, we can treat the interval (a, b) as “success”,
p = P(“success”), and ξ = relative frequence of “success”.
4.4 Confidence intervals: Large samples

Example 4.8. In a certain political campaign, one candidate has a

poll taken at random among the voting population. The results are
n = 112 and y = 59 (for “Yes”). Should the candidate feel very
confident of winning?

Solution. Let p = P(“the condidate wins the campaign”). Then,

ξ = 59/112 ≈ 0.527. According to (4.3), since z0.025 ≈ 1.96, an
approximated 95% confident interval estimate for p is
" r r #
0.527 ∗ (1 − 0.527) 0.527 ∗ (1 − 0.527)
0.527 − z , 0.527 + z
112 112
≈ [0.435, 0.619].

There has certain possibility that p is less than 50%, and the
candidate should take this into account in campaigning.