Design and Analysis of Experiments

04 - Statistical Intervals

Version 2.12.2018b

Felipe Campelo
http://orcslab.cpdee.ufmg.br/

Graduate Program in Electrical Engineering

Belo Horizonte
September 2018
 “I attribute my success to this: I never
gave or took an excuse.”

“I think one’s feelings waste themselves in words;

they ought all to be distilled into actions,
and into actions which bring results.”

Florence Nightingale
1820-1910
English statistician and founder of modern nursing

Image: https://www.pinterest.com.au/pin/572238696374683644/
Statistical Intervals
Introduction

Statistical intervals are important in quantifying the uncertainty

associated to a given estimate;

As an example, consider the situation: a coaxial cable manufacturing

operation produces cables with a target resistance of 50Ω and a
known standard deviation of 2Ω. Assume that the resistance values
can be well modeled by a normal distribution.

Let us now suppose that a sample mean of n = 25 observations of

resistance yields x̄ = 48. Given the sampling variability, it is very likely
that this value is not exactly the true value of µ, but we are so far
unable quantify how much uncertainty there is in this estimate.
Statistical Intervals
Definition

Statistical intervals define regions that are likely to contain the true
value of an estimated quantity.

Intervals are used to quantify the uncertainty associated with a given

estimate, allowing the derivation of statements at quantifiable levels of
confidence.

Three (plus one) of the most common types of interval are:

Tolerance intervals;

Prediction intervals;

Highest density intervals;

Confidence intervals.
Tolerance Intervals
Definition

“A tolerance interval is an enclosure interval for a specified proportion of

the sampled population, not its mean or standard deviation. For a specified
confidence level, you may want to determine lower and upper bounds such
that a given percent of the population is contained within them.”[1] .

95% enclosure of a population

(Normal distribution, mean = 50, sd = 2)
0.20

0.15
P(X)

0.10

0.05

0.00
45 50 55
X
[1] J.G. Ramírez: https://git.io/v5ZFh
Tolerance Intervals
Definition

The common practice in engineering of defining specification limits by

adding ±3σ to a given estimate of the mean arises from this definition -
for a Normal population, ≈ 99.75% of observations fall within µ ± 3σ.

Since σ 2 is usually unknown, we use s2 and compensate for the

additional uncertainty in this estimation.
Tolerance Intervals
Calculation (Normal Population)

For a Normal population, the two-sided tolerance interval for a given

population proportion γ at the confidence level 1 − α is given as:[2]
v  
u
n + z 2
u (n − 1)
u
(α/2)
X̄ ± S t (n−1)
n χ2 (γ)

where z(q) is the q-quantile of the standard normal distribution; and

(n−1)
χ2 (q) is the q-quantile of the χ2 distribution with n − 1 degrees of
freedom.

[2] NIST Engineering Statistics Handbook, https://goo.gl/m6cxC6

Tolerance Intervals
Quantiles

Quantiles of the standard Normal distribution

0.4

0.3
P(X)

0.2

0.1
0.025

0.975
0.01

0.05

0.95

0.99
0.1

0.5

0.9
0.0
−4 −2 0 2 4
X

Quantiles of the Chi−squared distribution

with 14 degrees−of−freedom
0.08

0.06
P(X)

0.04

0.02
0.025

0.975
0.01

0.05

0.95

0.99
0.1

0.5

0.9

0.00
0 10 20 30 40 50
X
Confidence Intervals
Definition

Confidence intervals quantify the degree of uncertainty associated with

the estimation of population parameters such as the mean or the
variance.

Can be defined as “the interval that contains the true value of a given
population parameter with a confidence level of 100(1 − α)”;

Another useful definition is to think about confidence intervals in terms

of confidence in the method: “The method used to derive the interval
has a hit rate of 95%” - i.e., the interval generated has a 95% chance of
‘capturing’ the true population parameter.”
Confidence Intervals
Example: 100 CI.95 for a sample of 25 observations

Confidence Intervals

53

52

51

50
X

49

48

47

0 20 40 60 80 100

Interval

For an interactive demonstration of the factors involved in the definition of a confidence interval, download the files from
https://git.io/vxXGj and run on RStudio.
Confidence Intervals
CI on the Mean of a Normal Variable

The two-sided CI(1−α) for the mean of a normal population with known
variance σ 2 is given by:
σ σ
x̄ + zα/2 √ ≤ µ ≤ x̄ + z1−α/2 √
n n

where (1 − α) is the confidence level and zx is the x-quantile of the

standard normal distribution.

For the more usual case with an unknown variance,

(n−1) s (n−1) s
x̄ + tα/2 √ ≤ µ ≤ x̄ + t1−α/2 √
n n
(n−1)
where tx is the x-quantile of the t distribution with n − 1 degrees of
freedom.
Confidence Intervals
CI on the Variance and Standard Deviation of a Normal Variable

A two-sided confidence interval on the variance of a normal variable

can also be easily calculated:

(n − 1)s2 (n − 1)s2
(n−1)
≤ σ2 ≤ (n−1)
χ2 1−α/2 χ2 α/2

For the standard deviation one simply needs to take the squared root
of the confidence limits.
Statistical Intervals
Wrapping up

Statistical intervals quantify the uncertainty associated with different

aspects of estimation;

Reporting intervals is always better than point estimates, as it provides

the necessary information to quantify the location and uncertainty of
your estimated values;

The correct interpretation is a little tricky (although not very difficult)[3] ,

but it is essential in order to derive the correct conclusions based on
the statistical interval of interest.

Intervals for other distributions (and even distribution-free intervals)

can be obtained using analytical or resampling methods.

[3] See the table at the end of https://git.io/v5ZFh

Bibliography

Required reading
1 J.G. Ramírez, Statistical Intervals: Confidence, Prediction, Enclosure:
https://git.io/v5ZFh
2 D.C. Montgomery and G.C. Runger, Applied Statistics and Probability for Engineers,
Chapter 8. 3rd Ed., Wiley 2005.
3 J. Orloff and J. Bloom, Bootstrap confidence intervals: https://goo.gl/XrT1ao
Recommended reading
1 Simply Statistics (blog) - http://simplystatistics.org
2 R. Dawkins, Climbing Mount Improbable, W.W.Norton&Co.,1997.
About this material
Conditions of use and referencing

This work is licensed under the Creative Commons CC BY-NC-SA 4.0 license
(Attribution Non-Commercial Share Alike International License version 4.0).
http://creativecommons.org/licenses/by-nc-sa/4.0/

Please reference this work as:

Felipe Campelo (2018), Lecture Notes on Design and Analysis of Experiments.

Online: https://github.com/fcampelo/Design-and-Analysis-of-Experiments
Version 2.12. Creative Commons BY-NC-SA 4.0.
@Misc{Campelo2018,
title={Lecture Notes on Design and Analysis of Experiments},
author={Felipe Campelo},
howPublished={\url{https://github.com/fcampelo/Design-and-Analysis-of-Experiments}},
year={2018},
note={Version 2.12. Creative Commons BY-NC-SA 4.0.},
}