Probability, Statistics and

Estimation

Propagation of Uncertainties in Experimental

Measurement

(Short Edition)

Mathieu ROUAUD
Physics and Chemistry Lecturer in Preparatory Classes for
the Engineering Schools. Associate Professor.
MSc Degree in Theoretical Physics.
Phd Student in Particle Physics.
 To share the knowledge and provide access to the greatest number of reader, the
book license is free, the digital book is free and to minimize the cost of the paper
version, it is printed in black and white and on economic paper.

Complete and High Quality

Digital and Paper Books
with all the exercises corrected
on www.lulu.com

To contact the author:

[email protected]
Boudiguen 29310 Querrien
France

This book is licensed under a Creative Commons Attribution-NonCommercial 4.0

International License (CC BY-NC 4.0).

You are free to:

Share — copy and redistribute the Adapt — remix,
material in any medium or format transform, and build upon
the material

Under the following terms:

Attribution — You must give appropriate credit, provide a link to the
license, and indicate if changes were made. You may do so in any
reasonable manner, but not in any way that suggests the licensor endorses
you or your use.
NonCommercial — You may not use the material for commercial
purposes.

• The licensor cannot revoke these freedoms as long as you follow the license
terms.
• You do not have to comply with the license for elements of the material in the
public domain or where your use is permitted by an applicable exception or
limitation.
• No warranties are given. The license may not give you all of the permissions
necessary for your intended use. For example, other rights such as publicity,
privacy, or moral rights may limit how you use the material.
• For other uses you have to contact the author.

Date of publication: July 2013 Revision and translation: April 2017

French books: Calcul d'incertitudes and Probabilités, statistiques et analyses multicritères .

Foreword

This handbook aims to be accessible to the major

public. This work is meant to be used as a pedagogical tool
and as a reference book. The following book represents
personal reflections on the probabilistic nature of measures
in science. In a classical curriculum these aspects are
seldom, if not at all, dealt with. It is important that the
experimental and practical foundations of science are
complementary to the theoretical lectures. There is a
scientific beauty that arises from the interaction between
theory and experience.
While introducing the fundamental principles of
statistics, this book explains how to determine uncertainties
in different experimental situations. Many examples come
from courses and practical work done in preparatory
classes for the engineering schools.

I hope you enjoy reading!

Thanks to the readers who by their questions, comments

and constructive criticisms make it possible to improve the
book.

Thanks to the life and to all those who have come before
me.
Contents

I. RANDOM VARIABLE.........................................................1
A. How to measure a quantity ?..........................................1
B. The center of a distribution............................................1
C. The dispersion of a distribution......................................3
D. Examples of distributions..............................................4
E. Central limit theorem.....................................................7
1) Population and samples.....................................................7
2) The central limit theorem................................................10
3) Student's t-value and uncertainty.....................................12
4) Examples.........................................................................15
F. Gaussian distribution....................................................19
1) Definition of continuous distribution..............................19
2) Bell-shaped density curve................................................20
3) Standard normal distribution...........................................23
G. Hypothesis test............................................................24
H. Chi-squared test...........................................................30
I. The sources of the uncertainties....................................33
J. Exercises.......................................................................37

II. CORRELATION AND INDEPENDENCE.......................47

A. Correlation coefficient..................................................47
B. Propagation of uncertainties formula............................52
1) Propagation of standard deviations formula....................52
2) Uncertainty calculations..................................................53
C. Linear regression..........................................................58
1) Principle and formulas....................................................58
2) Absolute zero measurement............................................62
3) Regression with uncertainties on the data.......................64
 4) Linearization...................................................................67
5) Comparison of methods..................................................68
D. Nonlinear regression....................................................75
1) Principle..........................................................................75
2) Polynomial regression.....................................................77
3) Nonlinear regression.......................................................80
E. Exercises......................................................................86

III. PROBABILITY DISTRIBUTIONS................................101

A. Discrete Random Variables........................................102
1) Binomial distribution....................................................102
2) Geometric distribution..................................................103
3) Poisson distribution.......................................................105
B. Continuous Random Variables...................................107
1) Uniform distribution.....................................................107
2) Exponential distribution................................................109
3) Normal distribution.......................................................110
4) Student's t-distribution..................................................110
5) Chi-squared distribution...............................................113
C. Function of a continuous distribution.........................115
D. Numerical simulation.................................................118
E. Exercises....................................................................121

IV. ESTIMATORS................................................................126
A. Properties of an Estimator.........................................126
1) Bias...............................................................................126
2) Mean Square Error........................................................127
B. Construction of estimators.........................................129
1) Method of Moments.....................................................129
2) Method of Maximum Likelihood.................................133
 C. Interval estimate.........................................................137
D. Exercises...................................................................146

V. Measure with a ruler.........................................................151

VI. Mathematical Tools.........................................................163

VII. Answers to Exercises......................................................168

VIII. Bibliography / Sources / Softwares / Illustrations...........171

IX. TABLES / Index..............................................................176

A. Standard normal distribution......................................176
B. Student's t-values........................................................177
C. Chi-square values.......................................................178
I. RANDOM VARIABLE

A. How to measure a quantity ?

In what follows, X is a random variable and {x i} a

sample of n outcomes.

If we ask how many days are in a week, it is easy to

answer. Now, consider different groups of students who
have measured the thermal capacity of water1 and have
obtained the following values : {5100; 4230; 3750; 4560;
3980} J/K/kg. How can we estimate the water capacity in
this situation? The answer will use a probabilistic approach.

B. The center of a distribution

How does one determine the most representative value
of a sample? There are different ways to define the center
of a distribution. For example, we have the mode, the
median and the mean. The mode is the value which occurs
with the greatest frequency in a data set. The median is the
middle of a distribution that divides the sample into two
parts whereby each half has an equal number of
observations. The most commonly used measure of the

1PHYSIQUE : The amount of energy needed to raise the temperature of

one kilogram of mass by 1°C. That way, the water stores energy
and can then return it by decreasing its temperature. Tables : c water =
4180 Joules per Celsius degree and per kilogram.

1
center is the mean. The mean is the sum of the observed
values divided by the number of observations :

x 1 x 2...x i...x n
∑ xi
i=1
x =
n
also x = 2

For the thermal capacity of water we have :

51004230375045603980
c = =4324 J / K / kg
5

We have considered the arithmetic mean. We could have

used the geometric mean:

x = ∏ x
n
i

For instance for two speeds of 20 m/s and 40 m/s, the

geometric mean is √ 20 m/s⋅40 m /s≃28.3 m/ s whereas
the arithmetic mean is of 30 m/s. The arithmetic mean is
used more often globally due to its conveniently simpler
calculation.

2MATH : To simplify the writing of a sum, the Greek letter sigma is used
as a shorthand and read as "the sum of all x i with i ranging from 1
to n".

2
 C. The dispersion of a distribution
In addition to locating the center of the observed values we
want to evaluate the extent of variation around the center.
Two data sets may have the same mean but may be differ-
ent with respect to variability. There are several ways to
measure the spread of data. Firstly the range is the differ-
ence between the maximum and minimum values in the
sample. The sample range of the variable is very easy to
compute, however it is sensitive to extreme values that can
be unrepresentative.

The sample standard deviation is preferred:


n

∑  x i −x 2
i=1
s=
n−1

It is the most frequently used measure of variability.

For the thermal capacity of water we have :

and s c ≃530 J / K / kg

3
The mean deviation may also be used (see Exercise 1).
Using the standard deviation formula, dividing by n rather
than n-1, will obtain the root mean square deviation (square
root of average square deviation). The choice of the
standard deviation is justified on page 128. Besides, the
values of n are often large and the difference is small.

D. Examples of distributions

Case 1 :

x1 6
x11 11
x12 9 5

x13 10
4
x14 14
frequencies

x15 11 3
x16 8
x17 9 2

x18 12
1
x19 7
x 1
10
8 0
x111 8 7 8 9 10 11 12 13 14 15 16 17 18 19
x112 9 x1
x113 11
x114 14 mean = 10 standard deviation= 2.07
x115 10 mode= 9 range= 7
x116 9 median= 9.5 root mean square deviation= 2.00

4
 Case 2 :
x1 6
x11 15
x12 5
13
x 1
3
12
4
x14 13
frequencies

x15 14 3
x16 13
x17 16 2
x18 19
x19 1
13
x 1
10
14
0
x111 10 7 8 9 10 11 12 13 14 15 16 17 18 19
x112 16 x1
x113 14
x114 15 mean = 14 standard deviation= 2.00
x115 13 mode= 13 range= 9
x116 14 median= 14 root mean square deviation= 1.94

Case 3 :
6
x1
x11 10 5
x12 10
x13 12 4
frequencies

x14 11
3
x15 9
x 1
6
8 2
x17 10
x18 9 1

x19 9
0
x110 11 7 8 9 10 11 12 13 14 15 16 17 18 19
x 1
11
9 x1
x112 11
x113 10
x114 10 mean = 10 standard deviation= 1.03
x115 11 mode= 10 range= 4
x116 10 median= 10 root mean square deviation= 1.00

5
The mean is not always the most represented value (case 1
and 2) and in some cases does not appear at all. In case 3
the histogram is symmetrical and illustrating that the
median and mean are equal.
In the event that some values are represented several times,
we determine the frequency fi for each value xi.
c
We have n=∑ f i , where c gives the number of different
i=1
values of xi.

The mean and standard deviation then become:

√
c c

∑ f i⋅x i c
fi
∑ f i( xi − x̄)2
i=1
=∑
i=1
x = ⋅x s=
n i=1 n i n−1

Sometimes the data can be grouped into class intervals. For

example, if we measure the size of the inhabitants of a city,
we can group all the inhabitants with a size between 160
cm and 170 cm within the same class interval. The number
of observations in this class is the frequency and the middle
of this class interval is the value assigned, here 165 cm (see
Exercise 5).

The more the histogram is concentrated around the center,

the more the standard deviation is small.

6
 E. Central limit theorem

1) Population and samples

Consider a city of one million inhabitants. To survey

the population we can interview a sample of only one thou-
sand people drawn at random. Thanks to the statistical
tools, from this n=1000 individuals sample, we can have
information on the whole population. The larger the sample
size is, the more accurate the results will be. Let x be the
sample mean and s be the sample standard deviation. For
the population we denote μ (Greek letter mu) the mean
and σ (sigma) the standard deviation. The larger the sam-
ple, the more likely x̄ and s are close to μ and σ respec-
tively.
In the case of opinion polls, samples are around one
thousand people. If we measure the size of one thousand
inhabitants selected randomly from the population of a city
of one million people, the average size of this sample is
likely to be close to the average size of the entire popula-
tion but has no reason to be equal.

Let us take the example of a coin toss. The coin is

balanced and the outcomes are heads or tails. In this case
the population is infinite and we can have an infinit number
of measurements. Furthermore, the probabilities are known
and we can determine the population features.

7
When the sample size becomes very large, it tends towards
the population : =lim x 3.
n∞

We introduce here the concept of probability :

fi
p i=lim where pi is the probability of the outcome xi.
n ∞ n

With this formula (using the formula page 6) we find the

population mean formula : =∑ pi⋅x i .
Also if we consider all the events possible, we have
∑ pi=1 (1=100%).

The outcome heads is associated with x0=0, and the out-

come tails with x1=1. The coin is balanced as
p0=p1=1/2=0,5=50% and μ=p0.x0 +p1. x1.

Furthermore: =lim s and with the formula for s page 6

n∞

we obtain σ= √∑ p ⋅(x −μ)

i i
2
(for n large, n-1 is close to
n).

Eventually : =0,5 and =0,5 .

3 MATH : Reads as «μ is equal to the limit of x when n tends to

infinity».

8
Let us flip nine coins and collect a sample :
{0; 1; 1; 0; 1; 1; 1; 0; 0}.
Then we find x ≃0,56 and s≃0,53 .

If this procedure is performed many times, each time we

would have a different result for x .

For example, if two other samples are taken:

{1; 1; 0; 1; 1; 0; 1; 1; 0} then x ≃0,67 and s≃0,50
{0; 1; 0; 0; 0; 0; 1; 0; 0} then x ≃0,22 and s≃0,44
What would be the distribution of these results as a whole?
(Called the sampling distribution)

The values obtained for the samples are generally different

from those of the population, but the larger the sample, the
more likely it is that the values are closer to those of the
population.
Case of a sample with n=50, where x̄≃0,520 and
s≃0,505 :
{00100011111110010111110001001101101000110000011101}

9
 2) The central limit theorem

CENTRAL LIMIT THEOREM :

Within a population we collect random samples of size n.
The mean of the sample x̄ varies around the mean of the
population μ with a standard deviation equal to σ/√n,
where σ is the standard deviation of the population.
As n increases, the sampling distribution of x̄ is
increasingly concentrated around μ and becomes closer and
closer to a Gaussian distribution.

We will describe in due course what a Gaussian distri-

bution, also called normal distribution, is. For the moment
we will simply consider a bell curve. This is a very impor-
tant theorem. Whatever the form of the population distri-
bution, the sampling distribution tends to a Gaussian, and
its dispersion is given by the Central Limit Theorem.

This is illustrated through the following diagrams:

On the left we have the probability p of an event x (popula-

tion distribution).

10
Hypothetically, for a city with a population of one million
inhabitants, p could represent the probability that one have
a given height x. If we could measure the height of all the
inhabitants, we could exactly determine their average
height μ and the standard deviation σ. However, it is prag-
matically difficult, if not impossible, to measure a popula-
tion of that size. Therefore a sample size of only a thousand
inhabitants is taken to reduce the burden of labour. For this
to be as representative of the whole population as possible,
the thousand-person sample is picked randomly.
We obtain a thousand measures of height from x1 to x1000.
From this sample of size n=1000 we calculate a mean x̄
and a standard deviation s. We think that x̄ is close to μ,
but at the same time there is no reason for it to be equal to
μ. We put this value of x̄ on the right side of the figure on
page 10.
We take a new random sample of a thousand people and a
new value for x̄ .
We then repeat this operation a great number of times. We
see on the right the distribution of the samples obtained:

11
 3) Student's t-value and uncertainty

The central limit theorem applies to the limit of large

numbers. In the particular case where the distribution of
the population is normal we can apply it from n small
thanks to the coefficients of Student t.

Prediction interval :
If μ and σ are known, the sampling distribution is also
Gaussian and the expected statistical fluctuations are with a
probability of p% between μ−t ∞ σ / √ n and μ+t ∞ σ / √ n .

The t-values are read on page 177.

Confidence interval :
In the case of the calculation of uncertainties μ and σ are
not known and we estimate them from the sample with x̄
and s. Due to a small statistic, there is widening given by
the Student's t-distribution:

s
 =x ±t⋅
n
The Student's t-value depends on the sample size n and on
the confidence. If the confidence is 95%, we have 95 in 100
chance that μ is between x −t⋅s /  n and x̄+ t⋅s / √ n .

12
We recognize here the notion of measurement uncertainty
 x 4:
s
x= x ± x with  x=t⋅
n
 x is also called absolute uncertainty and Δ x /|x̄| rela-
tive uncertainty.

Let us take again the calorimetry experiment described on

page 1. We want to know the thermal capacity of the water
with a confidence of 95%. As is often the case in experi-
mental sciences, we consider that the data follow a normal
distribution, because by the influence of many independent
factors on the value of the measured quantities, we still ex-
pect, under the central limit theorem, to have Gaussian
fluctuations.

We find for four degrees of freedom (ddl=n-1) a Student's

t of 2.78.
From where : c=c ±t⋅sc /  n=4320±660 J / K /kg with
95% confidence.
Here following the dispersion of the values measured by
the students : Δ c / c̄≃15 % . The calorimetry measure-
ments are usually imprecise. The expected value, known
here, is well within the range :
3660< 4180<4980

In experimental sciences we endeavor to quantify all

natural phenomena. Yet, due to the very nature of the ex-
4MATH : Reads "delta x".

13
perimental approach, the various parameters which make it
possible to describe a experimental situation are not per-
fectly known. We do not have a simple numerical value as-
sociated with each characteristic, but an interval for a given
confidence. Strictly speaking, any experimental value must
associate its uncertainty with its confidence.

Exceptions:
▸ Large number samples: the size of the sample n is large
enough to be able to directly apply the central limit theo-
rem. The sampling distribution is normal regardless of the
distribution of the population. We do not have to worry
about Student's distribution which anyway identifies with a
Gaussian distribution in this case.

▸ Small samples and normality of the data: we apply the

Student law as before.

▸ Small samples with non-normality of data: For example,

we find by computing the skewness and kurtosis of the data
that they do not match a normal distribution. To counter
this, it is necessary to perform a case by case study. For in-
stance, when we have a uniform distribution of the popula-
tion, the prediction interval given on page 12 works from
n=2 (it is shown by using the data of the article Measure
with a ruler p151) . However for a binomial distribution
with parameters n=2 and p=0.24, the 50% prediction inter-
val contains 0% of the values... A more complex case on
page 144 shows for n = 12, in comparison with a numerical
simulation, that the central limit theorem underestimates
the confidence interval.

14
 4) Examples

A large number of random factors will influence the

measurement of a physical quantity; independent factors,
which, whatever their natures will ultimately generate a
Gaussian distribution of observations. Let us consider two
examples, the toss of a coin and the roll of a six-sided die.

Consider the sample space for tossing a fair coin n

times. We count the number of tails. For one toss, we have
two outcomes possible, one with zero tails and one with
one tails.

For two tosses, we have four outcomes possible, one with

zero tails (H H), two with one tails (H T or T H) and one
with two tails (T T). The more tosses that are thrown, the
closer we get to a Gaussian distribution.

one two
toss tosses
n=1

15
We can obtain the probability (the number of tails divided
by the number of possibilities 2n) as a function of the
number of tails. For n = 1 we have the distribution of the
population and following the sampling distributions for
different values of n.

7
6
5
4
3
2
three four 1
tosses tosses 0
0 1 2 3 4

12 25
10 20
8
15
6
10
4
five 2 six 5

tosses 0 tosses 0
0 1 2 3 4 5 0 1 2 3 4 5 6

Similarly for the dice we enumerate the possibilities for

their sum and the distribution also tend towards a Gaussian.
For a single die, the sum simply corresponds to the value of
the die. We have a possibility for each value:

16
 2

one die : 1

0
1 2 3 4 5 6

For two dice, there is only one possibility for the sum to be
two: 1 for the first die and 1 for the second die. For the sum
to be three, there are two possibilities: 1 and 2, or, 2 and 1.
The most likely with two dice is to obtain a sum of 7: (1,6)
(6,1) (2,5) (5,2) (3,4) (4,3).
7

two dice : 1

0
2 3 4 5 6 7 8 9 10 11 12

30

25

20

15

10

5
three dice :
0
3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

17
For four dice, we already recognize the bell curve and the
profile is clearly of the Gaussian type:

160

140

120

100

80

60

40

four 20

dice : 0
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

On this last example, we check the validity of the central

limit theorem.
The population mean is :
μ x =(1+ 2+ 3+ 4 +5 +6 )/ 6=3.5

We verify that the sampling distribution mean is the same:

μx̄ =14 / 6=3.5 .
The population standard deviation is :
 x=  ∑ p ⋅ x −  then
i i
2

σ x= √ 1/6⋅{(1−3.5)2 +(2−3.5)2+(3−3.5)2 +(4−3.5)2 +(5−3.5)2 +(6−3.5)2 }

and σ x ≃1.71

18
also for four dice : σ x̄ =σ x/ √ n =σ x /2≃0.85 . Now, on the
above curve, 40% from the maximum (explanation page
21), we have a deviation close to 3.5 (between 3 and 4),
and an average of 3,5/ 4≃0.88 . It matches.

F. Gaussian distribution

1) Definition of continuous distribution

Some quantities are fundamentally continuous : time,
space, temperature, et cetera. Time is like fluid, it does not
jump from one value to another. A continuous random
variable can take any value in an interval of numbers and
has a continuum of possible values. On the other hand
when we throw a six-sided die, it is impossible to say "I got
2.35!". It is a forbidden value, only integer values from one
to six are allowed.
Thus, some probability distributions are discrete and
others continuous. For a die we have : p1=...=p6=1/6 and
n=6

∑ pi =1 . Now, if we are interested in the height of the

i=1

inhabitants of a city, there is a continuum of possibilities,

the distribution is continuous. We define the probability
density function p(x) with p(x)dx the probability to be be-
tween x and x+dx. Where dx is a small variation, and x+dx
is close to x. The continuous random variable probability
model assigns probabilities to intervals of outcomes rather
than to individual outcomes.

19
So, the probability that the event is realized on the set of
possible values is 100%:
∞

∫ p  x  dx=1
x=−∞

Mean and variance in the continuous case :

+∞ +∞
μ=∫ x⋅p(x)dx V =σ =∫ (x−μ) p (x)dx
2 2
−∞ −∞

2) Bell-shaped density curve

A continuous random variable X following normal distribu-

tion has two parameters: the mean μ and the standard devi-
ation σ. Density function :
2

p  x =
1
e
−
2   
1 x−

 2⋅
In the mathematic tools section page 165 some demonstra-
tions are performed.

5 MATH : The mean is also called E(X), expectation of X. σ2=V(X) is

called variance of the random variable X. Properties :
E(aX+b)=aE(X)+b , V(aX)=a2V(X) and V(X)=E(X2)-E(X)2.

20
We have represented two cases on the following graph:

The total area under the curve is always 1. The probability

concentrated within interval [μ − σ, μ + σ] is 0.68 :

μ+σ

∫
μ−σ
p(x )dx=0.683 ...≃68 %

We evaluate the standard deviation at 60%.pmax :

p (μ±σ)/ p max=1 / √ e≃0.607

21
The probability concentrated within interval [μ − 2σ, μ +
2σ] is 0.95 :

μ +2 σ

∫ p(x )dx =0.954 ...≃95 %

μ−2 σ

The probability concentrated within interval [μ − 3σ, μ +

3σ] is more than 0.99 :

μ +3 σ

∫ p( x )dx =0.997 ...>99 %

μ−3 σ

22
 3) Standard normal distribution

A standard normal distribution Z is normally distributed

with a mean μ = 0 and a standard deviation σ = 1.
The distribution X is transformed into a distribution Z
using the following two transformations : x '=x −

x− 1 −2
z
and z= then : p  z = e
 2 

23
 G. Hypothesis test

Similarly to how to estimate the mean of an unknown

probability distribution, the central limit theorem is also
used for a hypothesis test. With a collected sample we have
used the properties of the sampling distribution to
determine a value and it's uncertainty. For the hypothesis
tests, we proceed in the other direction: the law of
probability is assumed known, so the sampling distribution
is perfectly known and we take a sample to define a
decision criterion allowing us to accept or reject the
hypothesis.
Using the mean we test a hypothesis H0. This is re-
ferred to as the null hypothesis. It is an assumption made
on the probability distribution X. Let μ and σ be the mean
and standard deviation of X. We take from the population a
sample of size n large enough. The sample mean is x̄ . If
x̄ is between μ−t ∞ . σ/ √ n and μ+t ∞ .σ / √n then H0 is
accepted. However if x̄ is outside of those values, the null
hypothesis is rejected (two-tailed test).

24
We consider the coefficient t∞ of a normal distribution for a
p% confidence (or Student's t-value when n → ∞).
We can also use other characteristic intervals of the sam-
pling distribution. In general, the hypothesis H 0 implies a
property A of the sampling distribution. Here, the involve-
ment is not deterministic but statistical and decision-mak-
ing proceeds differently.

Deterministic test case : H 0 ⇒ A

• If we observe A then H0 cannot be rejected.
• If we do not observe A then H0 is rejected6.

Statistical test case :

In p% of cases : H 0 ⇒ A
In (1-p)% of cases : H 0 ⇏ A
• If we observe A then H0 cannot be rejected
with p% confidence.
• If we do not observe A then H0 is rejected, with a
risk to reject H0 when it is true of (1-p)%.

This protocol makes it possible to make a decision, but at

the same time it carries risks of making a mistake. We can
reject the hypothesis when it is true or we can accept the
hypothesis when it is false.

6 The contrapositive of an implication : if P ⇒ Q then Q̄ ⇒ P̄ .

25
 Accept H0 Reject H0

Wrong decision
H0 is true Right decision (error of the first
kind α)

Wrong decision
H0 is false (error of the Right decision
second kind β)

The aim is to minimize the risks linked to α and β. If we

accept H0, then α (the probability of rejecting the null hy-
pothesis when it is true) has to be important, and β (the
probability of accepting the null hypothesis when it is false)
has to be low. If, on the contrary, we reject H0 it is because
α is low and β important.
When H0 is true we are able to compute the risk, and α is
equal to 1-p. However if H0 is false we cannot compute β,
unless an alternative hypothesis H1 is known.
For a standard test we fix 1-p in advance. For example we
can consider the test statistically significant at the threshold
of 5%, and, according to the result found reject or accept
the hypothesis. Another method is to calculate the
probabilities α and β which correspond to the value x̄
found with our sample. Then we measure the credibility of
H0 and we choose whether or not to accept our hypothesis.

For example, let's imagine that we have several dice. All

the dice are unbiased except one which has double the
chances of falling on six. Unfortunately the rigged die is

26
mixed with the others and it does not bear any distinctive
marks. We choose one die for a game night and we want to
distinguish the biased die to be sure that the chosen die is
well balanced.

The die is thrown 92 times :

3151353256243365313441354244652632465436616546
2241154636433165514444241456414316555146362534
For H0 we define a discrete random variable X. When the
outcome is six the value is 1. All other outcomes are
recorded as 0 :
P(X=0)=5/6 and P(X=1)=1/6.

xi x0 = 0 x1 = 1
pi p0 = 5/6 p1 = 1/6

μ= p0 x0 + p1 x 1
2 2
σ ²= p0 ( x 0−μ) + p1 ( x 1−μ)

The mean of X is μ=1/6≃0.167 and the standard devia-

tion is σ=√ 5/6≃0.373 .
In our sample there are sixteen sixes and the sample mean
is x̄=16 /92≃0.174 . Therefore x̄−μ=t ∞ .σ / √ n which
gives:
t ∞ =( x̄−μ) √n /σ≃0.187 .
The right tail of the Gauss distribution for values greater
than 0.187 has an area of 0.43, showing that α≃43 % 7.

7 Here σ is known and n=92 is large enough to use the central limit

27
If H0 is false then H1 is true. If the alternative hypothesis H1
was "the die is biased" (with no details about the way the
die is loaded) we would have conducted a two-tailed test
and for α we would have to consider the two tails of the
distribution. This scenario only requires a one-tailed test: if
H0 were false, the probability of observing six would be
doubled and we would observe greater values.

For H1 we define the random variable Y with P(Y=0)=2/3

and P(Y=1)=1/3. The mean is μ ' =1/3≃0.333 and the
standard deviation is σ ' =√ 2/3≃0.471 .
Then x̄−μ ' =t ∞ ' . σ ' / √n and t ∞ '=(μ '− x̄ ) √n /σ '≃3.24 .
The left tail of this distribution has an area of β≃0.06 % .
We can therefore very comfortably accept the hypothesis
that the chosen die is balanced. In the case of rejection we
would have a 43% chance of making a mistake (we try to
minimize this error first, classically it is only below the
threshold of 5% that one begins to question the null hy-
pothesis). With regard to the alternative hypothesis, there is

theorem.

28
less than a 1 out of 1000 chance that we considered the die
balanced while the die is rigged (we also talk about the
power of the test : η=1-β).
Note that we never calculate the probability that a
hypothesis is true but the probability to reject the
hypothesis while it is true (error of the first kind).

In the legal framework an error of the first kind is made if

an innocent person is convicted and second kind if a guilty
person is acquitted. The jury is asked to prove the guilt
beyond a reasonable doubt and if the person is convicted α
must be sufficiently small [vi]. We try to minimize the
probability to condemn an innocent person. We do not
directly consider the probability of being guilty, the person
on trial is presumed innocent, a defendant is considered not
guilty as long as his or her guilt is not proven (H 0 : "the
defendant is not guilty").

Exercises 7 and 8 treat different cases for this test.

29
 H. Chi-squared test

The Chi-squared test is an another hypothesis test that is

simple to use. It tests a null hypothesis stating that the fre-
quency distribution of events in a sample is consistent with
a theoretical distribution. We consider different disjoint
events that have a total probability of 1.
number of observa- O1 O2 ... Oj ... Oc
tions :

expected E1 E2 ... Ej ... Ec

frequency :

We compute the following sum :

2 O j − E j 2
c

 =∑
j=1 Ej
Next we have a table on page 178 to estimate the probabil-
ity of rejecting the null hypothesis H0 when it is true. Ac-
2
cording to the value of  and the number of degrees of
freedom we determine whether the assumption is accept-
able. The number of degrees of freedom is :
ddl=c−1 (number of categories minus one)
Let us illustrate with the experiments carried out by the
botanist Mendel. He makes crosses between plants. He
crosses peas with pink flowers. His theory implies that he
must obtain 25% of peas with red flowers, 25% of peas
with white flowers and 50% of peas with pink flowers. This
result is derived from the random encounter of gametes.

30
Imagine that he observes one thousand flowers with the fol-
lowing values: 27% white, 24% red and 49% roses. Should
he continue to believe in his hypothesis?

Observed numbers : 270 240 490

Theoretical frequencies : 250 250 500

then
2 2 2
(270− 250) ( 240− 250) (490− 500)
χ 2= + + ≃2.2
250 250 500
and ddl=3−1=2 .

According to the table, there is more than a 30% chance

that the assumption will be rejected when it is true.
We then decide to accept the hypothesis. In general, we
take a critical probability α of 5%, below which it is
envisaged to reject the hypothesis.

31
The test is easily generalized for a table :

Experimental frequencies : Theoretical frequencies :

O 11 O12 ... O 1 j ... O1c E11 E 12 ... E1 j ... E1 c

( )( )
O 21 O22 ... ... ... ... E21 E 22 ... ... ... ...
... ... ... ... ... ... ... ... ... ... ... ...
Oi1 ... ... O i j ... ... Ei 1 ... ... Ei j ... ...
... ... ... ... ... ... ... ... ... ... ... ...
O r 1 ... ... ... ... Or c E r1 ... ... ... ... E rc

The ddl is dependent on the number of columns c and of

rows r :

ddl=(c−1)(r−1)

2
We compute the χ with a similar formula :

2
O i j − Ei j 
 2= ∑
i , j Ei j

Moreover, we use the same table to determine the validity

of the hypothesis.

32
 I. The sources of the uncertainties

A random variable has a small uncertainty if the

measurement is precise, accurate and the acquisition
system has a good resolution.

Accuracy is ensured by the absence of systematic errors.

There could be a bias that makes the measurement inaccu-
rate (even if the dispersion is low). Reading errors, absence
of systematic control and corrections of influential factors,
hypotheses in the modeling, etc. All biases must be identi-
fied and estimated in order to be added to the dispersion, so
the system becomes accurate.
Precision pertains to the repeatability and reproducibility
of the measurements. The values of a precise system have
low variability. The dispersion may be caused by accidental
errors or by a random physical phenomenon (such as ra-
dioactivity). The experimenters by their own work, consci-
entious and according to a well defined and rational proto-
col, can minimize dispersion. The sources can be countless,
but we will try to identify a maximum of sources in order
to evaluate them.
The resolution of a sensor depends on the distance between
the graduation marks, the type of vernier or the number of
digits on the display screen. Sometimes other factors have
to be added to the uncertainty due to discretization. You
have to refer to the technical datasheet, the instruction
guide or contact the manufacturer for a good knowledge of
your measuring tool. Calibration of measuring instruments

33
can also be performed with a high-precision apparatus that
is used as a reference.
The influence of these different sources of uncertainty can
be illustrated by a target and arrows. The center of the tar-
get corresponds to the quantity to be measured and the ar-
rows represent the different measurements. If the arrows as
a whole are not correctly centered, the accuracy is not as-
sured. The tightening of arrows represents precision. The
distance between the circles on the target indicates the res-
olution. The value noted is that of the circle whose arrow is
closest. The experimenter sees the arrows and the circles,
however he does not know where is the center of the target.
He holds the bow and his desire to be closer to the center
of the target shows the quality and rigor of his work.

Measure accurate, precise and with a good resolution :

34
Measure accurate, with a poor precision and a low resolution :

Measure precise but with a bias and a low resolution :

35
 Measure biased, with low precision and resolution :

The full standard deviation will be determined from the de-

viations of each source by adding the squares (due to the
propagation of uncertainties explained in Chapter 2):

=   1  2  3 ...
2 2 2

36
 J. Exercises

Exercise 1 : Ages Answers p168

Students of a class have the following ages : {18; 20;

18; 19; 18; 18; 18; 17; 18; 19; 17; 19; 17; 21; 18}. De-
termine the mode, median, arithmetic mean, geomet-
ric mean, range, standard deviation, root mean
square deviation, and mean deviation8.

Exercise 2 : Card game Answers p168

We play with a 32-card standard deck. We randomly

draw five cards.
a) Determine the probability of a four aces hand.
b) Determine the probability of having a flush (a flush
is a poker hand containing five cards all of the same
suit - hearts, spades, diamonds or clubs).

Exercise 3 : Gravity field

Answers (Complete Edition)

Students measure the Earth's gravitational field

strength g. The students measure the following
values at the laboratory : 6,20 ; 8,35 ; 13,00 ; 8,37 ;
8,54 ; 9,67 ; 9,75 ; 10,66 (m/s²).
a) What comments can be made about these results?
b) Calculate the mean and standard deviation.
c) What is the mean uncertainty (95% confidence)?
Is the result consistent with the expected value?
d) A ninth student performs a new measurement
under the same experimental conditions. Evaluate
the probability that the student will get a result
between 8 and 12 m/s².

Absolute deviation mean= (∑ |xi − x̄|) /n= ( ∑ ( x i − x̄ ) ) / n

√ 2
8

37
Exercise 4 : Elevator Answers (Complete Edition)

The maximum load of an elevator is 300 kg, and the

total mass of the occupants is 280 ± 10 kg at σ. What
is the probability of being overloaded?

Exercise 5 : Assignment Answers (Complete Edition)

The following table represents Students' assignment

grades (ranking on a 20 point scale) :

10 5 13 7 6 9 5 5 10 15 5 3

15 12 11 1 3 13 11 10 2 7 2 8

2 15 4 11 11 5 8 12 10 18 6

a) Calculate the mean and standard deviation.

b) Make a graph with the grades on the x-axis and

the frequencies on the y-axis.

c) Make another diagram with the following class

intervals : [0, 1, 2], [3, 4, 5] ... , [18, 19, 20].
Which bar chart do you prefer?

Exercise 6 : Yahtzee Answers (Complete Edition)

We play this game with five six-sided dice.

1) The five dice are drawn. What is the probability

of having Yahtzee (all five dice the same).

2) What is the probability of having a sum smaller

than ten?

38
 3) We do a series of throws and we get the following
sums: 18, 15, 17, 22, 16, 12, 14, 22, 23, 14, 23, 14,
18, 21, 12, 15, 18, 13, 15, 18, 17, 15, 17, 21, 25, 16,
8, 15, 15, 13.

a) Calculate the mean and standard deviation.

b) What mean do you estimate with 95%
confidence? Is it consistent with the theoretical
value?
c) Make a graph with the values and their
frequencies.
d) If we roll the dice again, what is the
probability of the result being higher than 24?

Exercise 7 : Elastic bands

Answers (Complete Edition)

An elastic bands manufacturer indicates that among

a thousand elastics sold, on average, ten are not
functional. A buyer wants to test the delivered
batches before accepting the delivery. He decides to
refuse the delivery if the number of damaged elastic
bands is too high and wants to have a less than 1%
chance of making a mistake by refuting the manu-
facturer's indication. The buyer picks n elastics
randomly. How many damaged elastics should the
delivery contain for the client to refuse the delivery?
Indicate this number for three different cases: a
sample of 1000, 200 and 50 elastic bands.

Exercise 8 : Testing an insulating panel

Answers (Complete Edition)

A manufacturer specifies a thermal conductivity of

0.039 W/m/K for a insulation board. The value is
certified within ±5%. You want to check if this is
true. To do so, you take ten panels at random and
measure their respective conductivity (mW.m -1.K-1) :

39
39.1 38.8 39.5 39.2 38.9 39.1 39.2 41.1 38.6 39.3

Are the values in agreement with those announced by

the manufacturer (95% confidence on the given
margin is consider)? Could he, according to your
results, announce another margin?

Exercise 9 : Coins Answers (Complete Edition)

We perform a large number of coin tosses to test if

the probabilities of landing heads or tails are equal.
We carry out the experiment with three different
coins. Are they balanced? (Answers with 95%
confidence)

1) 42 tails and 58 heads.

2) 510 tails and 490 heads.
3) 420 tails and 580 heads.

Exercise 10 : Parity Answers (Complete Edition)

Is gender equality respected in both chambers and The

Supreme Court?

Male Female
Parliament 470 107
Senate 272 76
The Supreme Court 10 2

40
Exercise 11 : Births Answers (Complete Edition)

Let us test the following hypothesis : births in

Sweden are distributed uniformly throughout the
year. Suppose we have a random sample of 88 births.
The results are grouped according to seasons of
variable length : 27 births in the spring (April - June),
20 in the summer (July / August), 8 in the fall
(September / October) and 33 in the winter
(November - March).

At a 5% critical level, can the hypothesis be rejected?

Now, we collect a very large sample : 23385, 14978,

14106 and 35804.

What is the conclusion?

41
 Theory

Exercise 12 : Gaussian distributions in the plane

and the space
Answers (Complete Edition)

One-dimensional Gaussian distribution :

Let p  x  be the probability density function of the

standard normal distribution.

1- Calculate and compare the means of x and ∣x∣.

Which is equivalent to the mean distance from the
origin? What do you think about σx and σ∣x∣?

2- Do a numerical calculation of P ∣x∣1 , P ∣x∣2

and P ∣x∣3.

Two-dimensional Gaussian distribution :

Let p (x , y ) be a two-dimensional standard normal

density with p  x , y = p x  p  y  . Where p( x) and
p ( y) are one-dimensional standard normal densities.

Hints:

Let consider a multiple integral of a two-variable

continuous function. If we can separate the variables
and the limits of integration are independent of x and
y:

∬ f x , y dxdy=∬ f  x  f  y dxdy=∫ f  x dx ∫ f  y dy

Converting between polar and Cartesian coordinates :
 2= x 2  y 2 and dxdy=2   d  (rotational symmetry)

42
1- What is the expression of p  x , y  ? Show that
p( x , y) satisfies the two necessary conditions for a
probability distribution.

2- By introducing the polar coordinates verify that the

probability on all the plane is one. You will express
p   define as:

∬ p x , y dx dy=∫ p  d .

p  is the density function with respect to ρ.
p   d  corresponds to the probability of an event
being between ρ and ρ+dρ.

What is the value of the mean ρ of the distance ρ from

the point of origin? What is the standard deviation σρ
for this distribution?

3- Calculate P     , P  2    and P  3   .

Three-dimensional Gaussian distribution :

Let p (x , y , z) be a three-dimensional standard normal

density with p  x , y , z= p x  p  y  p z  . Where p  x  ,
p  y  and p  z  are one-dimensional standard normal

43
densities.

Hints:

Same properties for the multiple integral than in two

dimensions.

Converting between spherical and Cartesian

coordinates : r 2 =x 2 y 2z 2 and dx dy dz=4  r 2 dr
(spherical symmetry)

1- What is the expression of p  x , y , z ? Determine

∭
p  r define as : ∫
p x , y , zdx dy dz= p rdr .
p  r is the density function with respect to r.
p  r dr corresponds to the probability of an event
being between r and r+dr.

2- Verify that the probability on all the space is one.

3- What is the value of the mean r and the standard
deviation σr ?

4- Calculate P R r  , P R2  r  and P R3  r .

5- Compare the three cases.

44
 Gaussian 1D 2D 3D
Distance from
∣x∣ 9
ρ r
the origin

Mean
 2
  
2
2
 2

Standard 1 2 3
deviations
P (σ) 68.3% 1-1/e=63.2% 60.8%
P (2σ) 95.4% 98.2% 99.3%
P (3σ) 99.7% 99.988% 99.9994%

To check your calculations on a spreadsheet you can use the

following functions:

On OpenOffice :
• sum of a selected area * (ex.: B43:B53) :
=SOMME(*)
• value set to the cell B3 : $B$3
• =MOYENNE(*)
• squared value : *^2
• square root : *^(1/2)
• =ECARTYPE(*)
• Student's t-value 95% confidence and n=20 :
=LOI.STUDENT.INVERSE(0,05;19)
• =TEST.KHIDEUX(*;**); * : experimental frequencies
(ex.: B71:E72), ** : theoretical frequencies.

9 MATH : We may be surprised at the difference of the one-dimensional

expression with the absolute value, it is only a question of definitions in
cylindrical and spherical coordinates. For example, ρ∈[0;+∞[ and θ∈[0;2π[,
but we could also take ρ∈]-∞;+∞[ and θ∈[0;π[, then the mean of ρ would be
zero and we would have considered its absolute value.

45
46
II. CORRELATION AND
INDEPENDENCE

In the previous chapter we looked at a singular ran-

dom variable X with n number of outcomes {xi}. Now we
have several random quantities and a new index to distin-
guish them : Xj and its observations {xjk}. Xj is the j th quan-
tity and xjk is the kth observation of this quantity. We are in-
terested in the interactions between these different quanti-
ties.

To illustrate, we consider a sample of four individuals

with three characteristics, of height X1, weight X2 and of
birth month X3. A priori, we expect a correlation between
height and weight : generally taller people also have heavier
body mass (positive correlation). On the other hand, we can
think that birth months have no effect on weight and height
(X3 uncorrelated with X1 and X2).

A. Correlation coefficient
The sample correlation coefficient r is used to identify a
linear relationship between two variables Xi and Xj :

∑ [ x i k − xi  x j k− xj ]
k
r ij =
∑ [ x
k
ik − xi 2]⋅∑ [ x
k
jk − xj 2 ]

47
r varies between -1 and +1. If ∣r∣=1 the variables are
perfectly correlated : r=1 is verified in the case of a
perfect increasing linear relationship, and r=−1 in the
case of a perfect decreasing linear correlation. If r=0 ,
the variables are uncorrelated and independents.

Calculus of r12 ,r13 and r23 :

2
X1 X2 X3 x 1 − x1 x 2− x2 x 3 − x3  x1− x1 
(cm) (kg)

1 160 64 4 -15 -11 -2 225

2 170 66 8 -5 -9 2 25
3 180 84 9 5 9 3 25
4 190 86 3 15 11 -3 225
x 175 75 6 Σ= 500

and :

2 2
 x2 − x2   x3 − x3   x1 − x1   x1 − x1   x 2− x2
. x 2− x2  . x 3− x3 . x 3− x3

121 4 165 30 22
81 4 45 -10 -18
81 9 45 15 27
121 9 165 -45 -33
404 26 420 -10 -2

48
 420
then : r 12= ≃0.93 , r 13≃−0.09 and
√500 √ 404
r 23≃−0.02 .
r12 is close to +1, so we have a strong positive correlation.
r13 and r23 are close to zero and therefore: X3 is independent
of X1 and X2 :

Examples of data sets :

49
Examples 7, 9 and 10 illustrate a strong correlation
between two variables. Yet the correlation coefficient is not
as close to -1 or +1 as we could imagine, it is even zero in
example 9. This is due to the fact that the correlations are
not linear.

There may be saturation phenomena (example 10) or

50
threshold effect (a product may be beneficial at low dose
and harmful at higher doses - example 9). To prevent these
phenomena from misrepresenting the data, it is important
to carefully consider the relevance of the variables chosen
before starting a statistical study.
Another example: if we study the volume V of different
objects according to their size T, we find a positive
correlation. But the correlation will be much stronger
between V and X = T 3 (graphs 7 and 8).

The fact that two variables are correlated does not

necessarily imply a causal relationship between the two
variables (the variation of one variable leads to the
variation of the other). Rather than the variables effecting
each other, the change may be attributable to a common
external cause.
For example, it can be assumed that there is a correlation
between the consumption of sunscreen and of ice cream.
There is obviously no causal link between the two but a
common cause i.e. the weather.
A physics study can show a causality, not a statistical one.

51
 B. Propagation of uncertainties formula

Consider a bucket filled with a million grains of sand.

The mass of a grain is 10 mg with an uncertainty of 1mg.
What is the mass of sand contained in the bucket?

1) Propagation of standard deviations formula

As a general approach, let f be a function of p independent

variables:

f  x1 , x2 , ... , x j ,... , x p 

Each of these random variables is associated a mean value

xj and a standard deviation  j .

What are the values of f and  f ?

Statistics give the answer and demonstrates the propagation

formula of the standard deviations:

10

[ ]
p 2
2 ∂f
σ =∑
f ( ) σ2j
j=1 ∂ xj

10 We obtained the variance formula by replacing σ2 by V.

52
 2) Uncertainty calculations

For the uncertainties (defined on page 13) we also have a

propagation formula:

[  ]
p 2
2 ∂ f 2
 f =∑  xj
j =1 ∂ xj

The uncertainty propagation formula is not as exact as for

standard deviations, but this formula is very practical and
often very close to the exact result.

Considering our bucket: M m1 , m2 ,... , m j ,... , m p 

p

with M =∑ m j
j =1

where we call M the total mass of sand in the bucket, mj

the mass of each grain and p the number of grains.
p
2 2 2
 M = ∑  ∂ M /∂ m j   m j
j=1

∂ M /∂ m j=∂ m1 /∂ m j...∂ m j /∂ m j ...∂m p / ∂ m j

∂ M /∂ m j=0...1...0=1
(Calculations of partial derivatives are explained on page 164)

53
 p

then ΔM =
2
(∑ )
j=1
2
1 Δm
2

and  M 2= p⋅ m 2 with  m= m j whatever j.

Finally : Δ M = √ p⋅Δ m=√ 1000000×0.001 g .

The bucket weighs 10 kilograms with an accuracy to the

gram. The precision on the mass of the bucket is thus
0.01%. Naively, we might have thought that the overall
uncertainty on the bucket mass was the sum of the
uncertainties of each grain. We would then have an
absolute uncertainty of 1 kilogram and a relative of 10%,
which is very different from reality and would ignore the
compensations.

Here the propagation formula is very precise because we

have a very large number of grains. It is even exact, from
the small numbers, if the distribution of the mass of the
grains is Gaussian11.

11MATH : A linear combination of Gaussian quantities is itself Gaussian

(applies here to a sum). And in the propagation of uncertainties
formula, if f and the xi have the same kind of probability
distribution, the formula is exact like this one with the standard
deviations.

54
In practice, there are multiple common ways to calculate
uncertainties' propagation, depending on the situation,
given below:

• For sums or differences we add absolute

uncertainties squared:
p
 f 2=∑  x j 2
j=1

For example if d=x2-x1 with Δx2=Δx1=1cm then

Δ d ≃1.4 cm .

• For products or quotients we add relative

uncertainties squared:
2 2

   
p
f  xj
=∑
f j=1 xj

For example if R=U/I with U and I with a precision of

1% then R is known with a precision of 1.4%.

In more complex cases, the partial derivative calculation

must be performed explicitly.

Alternatively, using a random number generator or

55
uncorrelated packets approximately Gaussian, it is possible
to do a numerical calculation.
The latter method is illustrated in Exercise 2 of this
chapter. A spreadsheet can do the calculations auto-
matically (for example see on sheet 4 of the file
IncertitudesLibres on www.incertitudes.fr).

There are also methods which give general ideas on

uncertainty. They develop a general impression at the risk
of reliability.

For example as we add the uncertainties squared, we can

anticipate that the largest uncertainty will quickly prevail
over the others. Consider the example of R=U/I, if U is
known with an uncertainty of 1% and I of 0,1% then R is
known with an uncertainty of 1,005%≃1%, we can ignore
the uncertainty of I.

For addition and subtraction, it is sometimes considered

that the parameter with the last significant figure the less
precise indicates the precision of the last significant figure
of the result.
Yet on our example calculation of the mass of the sand-
filled bucket, it does not work. Since the mass of a grain is
m = 10 mg but the mass of the bucket M is known to the

56
gram and not to the milligram!

For multiplication and division, it is sometimes considered

that the parameter with the lowest number of significant
figures indicates the result's number of significant digits,
but here too one must be careful.
As a simple illustration, if H=2h with h=5,00m (h known
within a cm), what is the value of H ? According to the rule
below, H would be known with three significant digits:
H=10,0m. H would only be known to 10cm, it goes without
saying that it is closer the cm ...

While these tricks serve to aid the calculations, they are not
without their pitfalls and must used with caution.

57
 C. Linear regression

We have two correlated random variables, X and Y,

and are in need of an approach for modeling the relation-
ship between them. The outcomes generate a cloud of data
in the plane y(x) and we want to determine the best affine
function y=axb that fit the observations. For example,
what is the most appropriate relationship between the
height X and the weight Y in our initial example?
What are the uncertainties Δ a and Δ b ?

1) Principle and formulas

y
^ Mi
^
a
yi
varies
di
^
yi
di = yi - ^
yi = ei
b ^ ei
varies
^

xi x

We choose the least squares method: this method mini-

mizes the sum of squared residuals.
The set of dots is denoted M i  x i , yi  . For xi given, the es-
timated value of y is: y i=a x ib .

58
We have to minimize the following quantity:

∑ d 2 =∑  y i − y i 2
i

We differentiate this quantity with respect to a and b, and

we set the derivatives equal to 0. We then have the best
straight line and can obtain the following equations:

∑  y i−a x i−b x i =0 and ∑  y i−a x i−b=0 .

i i

This is equivalent to

x y− x y
a= and b= y −a x .
x 2− x 2

We call ei the residuals: y i= y^ i +e i .

we find the following different standard deviations 12:

• for the residuals s r=

 ∑ ei 2
i
n−2
sr
• for the slope s a=
√∑ ( x − x̄ )
2
i
i

• for the y-intercept

12 Demonstrations p97.
s b=s r
 ∑ x i2
n ∑  x i− x 
2

59
Then  a=t n−2 s a and  b =t n−2 s b .

tn-2 : Student's t-value for n-2 degrees of freedom.

You are now able to carry out all calculations.
Let's do the calculations for weight in relation to height:
x y=160x64170x66180x84190x86/4
x2 =160 217021802190 2/4
a=(13230−175 x 75)/(30750−1752 )=0.84 and
b=75−0.84 x 175=−72
s r =√[( 64−(0.84 x 160−72))2 +(−4.8)2 +4.82+(−1.6)2 ]/ 2≃5.06

s a≃5.06 / √(160−175)2 +(−5)2 +52 +152≃0.226

Δ a≃2.92 x 0.226≃0.66 with a 90% confidence
s b≃5.06 √ (1602 +1702 + 1802 +1902 )/[ 4 (15 2+ 52+25+ 225)]≃39.7
Δ b≃2.92 x 39.7≃116 with a 90% confidence
then: Height =( 0.84±0.66)Weight−(72±116 )
with a 90% confidence.
Here, the formula is very imprecise, which is unsurprising
given the small number of points and the dispersion of the
data. However, the method of calculation is now explicit
and comprehensible.

In the graph that follows we have:

• In the middle, the interpolated straight line repre-

sents the best balance between the points above and
below this line.

60
• The dotted lines represent the two extreme lines (
y=a min x b max and y=amax x+b min ).

• The first curves represent the estimated values for y.

It is the mean confidence interval of yo correspond-
ing to xo :


2
1  x o− x 
 y o=t n−2 s r 
n ∑  x i −x 2

For example if xo=175 cm we can calculate

yo=75.0±7.4 kg. Also we can obtain an estimation
out of the interval, for example if xo=195 cm we
obtain yo=92±15 kg.

61
 • The outer curves represent a prediction for a new
measurement. Prediction interval for an observation
yo :


2
1  x o− x 
 y o=t n−2 s r  1
n ∑  xi −x 2
For example, if the height equals 175 cm there is a
90% chance of their mass being between 58 and 92
kg (generally 90% of the data are inside the curves
and 10% outside).

2) Absolute zero measurement

We study a gas en-

closed in a rigid container
of constant volume. We
have sensors to measure
its temperature and pres-
sure. Initially the gas is at
ambient temperature and
pressure. Then we im-
merse the whole container
in hot water and measure
the changes over time1 :

1 This experiment was realized on Tuesday 17 October 2006

in Bourges by M. ROUAUD and O. LEROY at the Lycée Alain-
Fournier (France).

62
time 10h 10h 10h 10h not not 12h
15 30 40 55 noted noted
temperature 19.8 52.9 47.8 42.4 36.2 33.5 30.2
ϴ (°C)
pressure 1013 1130 1112 1093 1072 1061 1049
P (hPa)

We assume that the gas obeys the ideal gas law

P V =n RT =n R ϴ−ϴ0 K  . Plotting ϴ(P) we can obtain
a temperature of absolute zero : the y-intercept give ϴ0K.

The regression is good (r=0.99991) but the measured

points are far from absolute zero. By extension we obtain
with a 95% confidence:
Θ0 K =−266.0±4.8 ° C

63
We know that absolute zero is -273.15 °C, therefore this is
not consistent. We can therefore assume that there is a bias
and that we have not considered all the sources of
uncertainties.

3) Regression with uncertainties on the data

The measuring instruments are not perfect and the

manuals indicate the precision for each. The uncertainties
are 1% for the pressure sensor and 1°C for the
thermometer. We thus have an uncertainty on x and y :

M i  x i ± x i , y i± yi 

Now both the dependent and independent variables

have uncertainties which contribute to the weighting
factors. Let wi be the weights. In an experimental situation
the weights are frequently not equal. The linear least-
squared fit requires the minimization of :

1
∑ w i e i2 with w i=
 y i  a  x i 2
2
i

The problem is solved iteratively because the weights

depend on the slope. We initially put an estimated value of
a, then the value of a obtained replaces it until the equality
of the values.

64
We obtain: 0 K =−266±35 ° C with the same con-
fidence on the uncertainties of xi and yi. The value is now
correct. The main sources of uncertainties seem to be
included.
We could also consider the modeling uncertainty induced
by the ideal gas hypothesis, but under the experimental
conditions of this experiment, the model provides a good
approximation. This source of uncertainty is negligible
compared to the others uncertainties considered here. The
use of a real gas model (like Van der Waals equation of
state) would demonstrate this.

65
 Formulas [i] :

S 2 =∑ wi [ y i −a xi b]2
i

∂S 2 ∂S 2
=0 and =0
∂b ∂a

leads to

b =
∑ wi y i ∑ w i xi2 − ∑ wi x i ∑ w i xi yi


and

a =
∑ wi ∑ wi x i y i − ∑ wi x i ∑ w i y i


with
2
 = ∑ wi ∑ wi x2i −  ∑ wi x i 
then

b =
 ∑ wi x2i


and

a =
 ∑ wi


66
 4) Linearization

In many cases we can go back to a linear model (i.e.

y=a xb ). Here are some examples below:

y= x  y ' =a x ' b with

y ' =ln  y  x ' =ln  x  and
b and
x , y ,0  =a =e
 = a  =  b
y= e x x ,0 y ' =ln  y  x ' =x
1 1
y= y'=
 x y

e   x (logistic y
y= y ' =ln   y ' = x
 x distri- 1− y
1e bution)

 (Pareto y ' =ln 1−y  x ' =ln  x 

y=1−  

x
distri-
bution)  =−a =e
−
b
a

y=1−e
−  x 



y ' =ln ln 
1
1− y
  b
(Weibull distribution) −
x ' =ln  x  =a = e a

y=e x no linear model

x
y= no linear model
 x

y= x x 2 no linear model as y(x)

67
 5) Comparison of methods

In all the regression methods we consider, we are interested

in the variability of y to x fixed (the opposite approach
would yield equivalent results).
For a given xi we have a yi and its standard deviation σyi.

a) Summary

1- Simple linear regression

y1=8.3 y1=17.3 y1=18.3 y1=27.3 y1=28.3 y1=37.3 y1=38.3

Case 1 : a=5 b=5 sr=2.34 sa=0.443 and sb=1.98

Simple regression does not mean that the data has no un-
certainties. The uncertainties are unknown and we estimate
them with the data itself. The uncertainties are considered

68
constant whatever regardless of yi. The uncertainty corre-
sponds to the standard deviation of yi with respect to the es-
timated line :

s y =s y =sr =
i

sr
 ∑  yi− yi 2
i

n−2

sa =
∑  x − x
i
i
2 sb =
 x2
∑  xi− x2
sr

2- Regression with constant standard deviation sy:

Case 2 : a=5 b=5 sy=2.55 sa=0.482 sb=2.16

In this case the syi are equal and known : s y =s y and i

sa =
sy

∑ x − x 
i
i
2
sb =
 x2
∑ xi− x 2
sy

69
If the straight line does not pass through the error bars, it
can be assumed that all sources of uncertainty have not
been calculated. It is necessary either to integrate them into
sy, or to apply the previous method.

3- Regression with standard deviation sy:

sy1=4 sy2=3.5 sy3=3 sy4=2.5 sy5=2 sy6=1.5 sy7=1

Case 3 : a=4.67 b=6.51 sa=0.482 sb=2.16
The syi are known. We can apply the propagation formula
of the standard deviations:
2 2
s a =∑
2

i
 
∂a
∂ yi
sy 2
i
and s b =∑
2

i
 
∂b
∂ yi
sy 2 i

The formulas' results are exact and help us to find the ex-
pressions of the previous cases. Also in this case we can

70
use the following estimates:


1 1 1 x2 1
 a=  b= w i=
∑ w i  x −x2
2
∑ w i x 2 −x 2  y2
i

4- Regression with standard deviation sy and sx

sy1=4 sy2=3.5 sy3=3 sy4=2.5 sy5=2 sy6=1.5 sy7=1

sx1=0.1 sx2=0.2 sx3=0.3 sx4=0.4 sx5=0.5 sx6=0.6 sx7=0.7
s1=4.0 s2=3.6 s3=3.4 s4=3.2 s5=3.2 s6=3.4 s7=3.6
Case 4 : a=4.98 b=5.14 sa=0.695 sb=3.16

But if xi is assumed fixed, we transfer the dispersion

on yi : s y Total 2 =si 2=s y 2a 2 s x 2
i i i

Everything happens as if only yi had standard deviations si.

71
Hence the formulas :
2 2
s a 2= ∑
i
  ∂a
s2
∂ yi i
and s b 2= ∑
i
 
∂b
s2
∂ yi i
The derivatives are difficult to calculate (the weights de-
pend on a), but we can easily evaluate them numerically.
Also we commonly use the following estimates:

a =
 ∑ wi

b =
 ∑ wi x2i

w i=
1
 y a 2  x 2
i
2
i

b) Discussion

Case 5 : a=5 b=5 sa=0.696 sb=3.16

Uncertainties in this case can come from measuring instru-
ments. The dispersion of the first case on page 68 can come
from accidental errors linked to the experiment or from a

72
fundamentally random phenomenon.
We would like that the fourth case on page 71 includes all
the sources of uncertainty of the cases 1 and 5 :
s a ≃0.443 and s a ≃0.696 but
1 5
s a ≃0.695
4

Using the conventional formulas, we get the impression that

the dispersion around the regression line is not included. In
order to get the correct dispersion around the regression
line, we perform the direct calculation with the propagation
formula and I propose the small variations method:
2
sa 2 =∑
i  
∂a
s2
∂ y i i and  
∂a
= lim
a
 
∂ y j  y 0  y j
j

Where yi ≠j are kept constant: y j  y j  a a

Δyj is a small variation with respect of yj , if Δyj becomes

smaller,  a stays constant (definition of the derivative).
 yj

We try to find the result for sa 5 : we replace y1=10 with

y1=10.001 and then from a=5 we have after iteration
Δa
a=4.999907 then ≃−0.093 .
Δ y1
We return y1= 10 and repeat the procedure for y2 by re-
placing it with 15.001. We obtain the following results:

73
 j=1 j=2 j=3 j=4 j=5 j=6 j=7
∂a
-0.093 -0.078 -0.049 -0.006 0.041 0.080 0.105
∂ yj

We find s a ≃0.696 , the same result than the previous

5

method.

Let's do the same for sa 4 :

j=1 j=2 j=3 j=4 j=5 j=6 j=7
∂a
-0.096 -0.080 -0.050 -0.007 0.081 0.122 0.147
∂ yj

We then find that s a ≃0.786 . The result is significantly

4

different from the classical estimate and now seems to in-

corporate all the sources of uncertainty.

In the exercise Standard deviation proportional to y on page

98 we study a particular case where we carry out the direct
analytical calculation. The comparison can thus be ex-
tended.

74
 D. Nonlinear regression
We generalize the weighted least squares method in the
nonlinear case. The function is nonlinear with respect to x
and also nonlinear with respect to the parameters. Although
multiple regression has similar developments, it is not dealt
with in this book [x].

1) Principle

We compare y i to the value f ( xi ) given by the function

2
sought: y i −f ( x i) . The weight assigned to ( y i−f ( x i))
is inversely proportional to the variance of y i −f ( x i) .

The quantities x i and y i are independent13 from where:

13 Talking about independence between two variables as we look for a

functional relationship between them may seem illogical. We refer
here to the experimental determination of each quantity which is
independent (in the sense of uncorrelated uncertainties).

75
 V  y i − f  x i =V  y i V  f  x i 
By applying the variance propagation formula again:
V  f x i = f ' x i 2 V  x i 

2 2
into S =∑ w i  y i− f  x i
i

with w i= 1 14

 y  f '  x i2  x 2
2
i i

∂ S2
Then =0 allows to determine the parameters ak of our
∂a k
function (by an analytical computation or a numerical
resolution).
Each time we can return to a system of linear equations of
the form HA=B, with H a square matrix and A the vector
associated with the parameters. Then A=H-1B, where H-1 is
the inverse matrix of H.

The parameters uncertainties are given by the diagonal

terms of the matrix H-1 :
 a 2 =H −1 kk  r 2 where  r 2 is the residual variance with
k

respect to the estimated curve.

When there are no uncertainties on the data, the standard

deviation of the residuals with p parameters are written:

14S2 is also called χ2. If we assume the distributions Gaussian, the

standard deviations can be replaced by the uncertainties using
Student's t-values.

76
 sr =
 ∑  yi − f xi 2
n− p
When wi depends on the parameters we iterate the method
until we can consider the weights to be constant. If we
know the standard deviations of the data, the standard
deviations of the parameters can be computed with the
propagation formula, or they can be estimated using the
same procedure used for linear regression with error bars
on page 66.

2) Polynomial regression

In that case:
m
f  x =a 0a1 xa 2 x 2...a m x m=∑ a i x i
i=0

This function is non-linear with respect to x and linear with

respect to the parameters.

Let us illustrate with the example of Cauchy's equation.

This equation explains the phenomenon of light dispersion.
It is an empirical relationship between the refractive index
and wavelength:

a1 a2
n =a0  2

 4

with the following data:

λ(μm) 0.6157 0.5892 0.5685 0.5152 0.4981

n 1.71276 1.71578 1.71852 1.72716 1.73060

77
 The uncertainties on λ and n are initially neglected. What
are the values and the uncertainties of a0, a1 and a2?

We have the following parabolic regression:

2 2
f (x)=a0 +a1 x+ a2 x with f =n , x=1 / .

S =∑  y i −a0 −a 1 x i −a 2 x i 
2 2 2
and ∂ S 2 /∂ a k =0

{
2
y−a o−a 1 x −a 2 x =0
we obtain: x y −ao x−a1 x 2 −a 2 x3=0
x 2 y−ao x 2−a 1 x3−a 2 x4 =0

then HA=B with:


1 x x2

   
a0 y
H = x x2 x3 , A= a 1 and B= x y .
2
x2 x3 x4 a2 x y

With a spreadsheet we can easily do the calculations:

( )
1 3.3 11 4150 −2530 376

(
H≃ 3.3 11 38
11 38 135 ) −1
, H ≃ −2530
376
1546 −230
−230 34.3
,

() ( )
1.7 1.68129
−1
B≃ 5.7 then A=H B≃ 0.01135 .
19 0.00022

For the uncertainties:

 a k = H  kk t n−3 sr with sr =
−1

 ∑  yi − f xi 2
n−3

78
 sr=1.87⨉10-5 and with 95% confidence tn-3=4.30
then
a0=1.6813±0.0017 , a1=(1.13±0.10)⨉10-2 μm2
and a2=(2.2±1.5)⨉10-4 μm4

Now, if we take the uncertainty Δn=0.00004 with 95%

confidence, we have an uncertainty on y but not on x.
Therefore the weights are constant: w i =w=1/  n 2 . We
obtain the same system of equations and we have the same
results for H, H-1 B and A.
Here, in a similar way to the linear regression with error
bars, we can estimate the dispersion of the residues by
1/ √ wi :

Δ ak=
√ (H −1 )kk
∑ wi
then

a0=1.6813±0.0012 , a1=(1.14±0.07)⨉10-2 μm2

and a2=(2.2±1.0)⨉10-4 μm4

With all the uncertainties, Δn=0.00004, Δλ=0.005μm and

Δx=2Δλ/λ3, the weights depend on the observations. We
nevertheless arrive at the same system of equations by
considering the weights on an iteration constant, and we
compute H, H-1, B, and A with estimated parameters.
To calculate the means, we use the expression of the
following weight:
1
w i=
Δ n +( a1 +2 a2 x i )2 Δ x i2
2

79
We thus obtain a first expression of the parameters. To
iterate, we replace the previously used estimated
parameters with these new parameters. We iterate as much
as necessary to obtain a consistent value. Convergence is
often very rapid:

Iteration a0 a1 a2
Estimated 1.5 0.005 0.0001
1 1.681282848208 0.011350379724 0.000219632215

2 1.681269875466 0.011358254795 0.000218466771

...

Also here we consider:  a k =

  H −1 kk
∑ wi
then

a0=1.6813±0.0013 , a1=(1.14±0.08)⨉10-2 μm2

and a2=(2.2±1.2)⨉10-4 μm4

3) Nonlinear regression

We will start from p parameters ak estimated and use an

iteration method. On each iteration, the weights will be
considered constant and the function will be linearized for
each of n points on the set of parameters.
The function depends on the xi and the parameters ak.

Then we have f i =f ( xi ; a k ) .

80
 n
∂ S2 ∂f
S 2 = ∑ wi  y i − f i  2 and =−2 ∑ wi ( y −f )=0
i ∂a k i=1 ∂a k i i

The first estimated parameters are noted a0 k. Following

parameters will be noted aj k for the jth iteration. We will
carry out a linearization with a small variation δa0 k around
a0 k 15.

Let a =a1, ... ,a k ,... , a p  and 

 a= a 1, ... ,  a k ,... , a p  :
p
∂ f ( x i ; ⃗a)
a 0+ ⃗
f ( xi ; ⃗ a 0)+ ∑ δ a0 l
δ a0 )=f ( x i ; ⃗
l=1
( ∂al ) a 0l

or by reducing the notations :

f i= f 0, i∑
l
 
∂f
∂a l 0,i
 a0l

then ∑ wi ∂∂ af
i k
 y i − f 0, i −∑
l  
∂f
∂ al 0,i
 a 0l =0

and ∑ wi ∂∂ af  y i − f 0, i =∑ w i
∂f ∂f
a
∂ a k ∂ al 0 l .
i k i ,l

i
∂f
We set H k ,l =∑ w i ∂ a
k
   i
∂f
∂ al i
=H l , k ,

∂f
B k =∑ wi  y − f 0,i  and Al = a 0l .
i ∂ ak i

from where again HA=B and A=H-1B. We iterate until

15 MATH : We generalize the notion of derivative by adding the

variations according to all the parameters:
f  x0  ≃ f  x 0  f '  xx 0

81
variation on the parameters is negligible and the values
converge.
2 −1 2
Here also, we use:  a =H kk  r .
k

Let us illustrate with a biological experiment, in which the

relation between the concentration of the substrate [S] and
the reaction rate in an enzymatic reaction is studied from
data reported in the following table [ix] :

i 1 2 3 4 5 6 7
[S] 0.038 0.194 0.425 0.626 1.253 2.500 3.740
v 0.050 0.127 0.094 0.2122 0.2729 0.2665 0.3317

The model can be written as:

x
y= where v=y and [S]=x.
 x

We start with the estimations α0=0.9 and β0=0.2

∂f
=
x
∂ x
,
∂f
∂
=−
x
 x2 , H=
H 11 H 12
H 21 H 22   ,

A=  

and B=
  B1
B2
.

In the absence of given uncertainties on xi and yi :

7 2
2
xi
H 11=∑
i=1
∂f
( ) ( )
∂α α0i
∂f
∂α α0 i
=∑
i
∂f
( )
∂α α 0 ,i
=∑
i
( ),
β0 + x i

82
 0 x 2i 20 x 2i
H 12=H 21=∑ 3 and H 22= ∑ 4 .
i  0 x i  i 0  x i 

xi  0 xi
B1 = ∑  y i − f 0,i  and B 2=−∑ 2  y i − f 0, i 
i  0 x i i  0 x i 

0 x i
with f 0,i =
 0 x i

We can now put everything into a spreadsheet, which

produces:

H 11≃3.81 , H 12=H 21≃−2.89 , H 22≃3.70 ,

B 1≃−2.33 et B 2≃1.86 .

and also

−1
H ≃ (0.649
0.508
0.508
0.668 ) then A=H B≃
−1
(−0.567
0.0602 )
.

From α1=α0 +δα0 and β1=β0 +δβ0 we have the new

estimated parameters:

α 1≃0.9−0.567≃0.333 and β1 ≃0.2+0.06≃0.260

We repeat the calculation from the start, this time using

these values instead of α0 and β0 to iterate. We calculate
new matrices and vectors H, H-1, B and A, and obtain α2

83
and β2.
The results are shown in the following table:
Iterat° α β δα δβ S2

0.9 0.2 -0.57 0.060 1.4454965815

1 0.333 0.26 0.0101 0.166 0.0150720753

2 0.343 0.43 0.0150 0.103 0.0084583228

3 0.358 0.53 0.0040 0.024 0.0078643240

4 0.3614 0.554 0.0004 0.0024 0.0078441826

5 0.36180 0.5561 0.00003 0.00018 0.0078440067

6 0.36183 0.5563 0.000002 0.000013 0.0078440057

−1 1.52 6.34
After enough iterations: H ≃ 6.34 36.2 ( )
Calculate the uncertainties on the parameters:

sr =
 S2
n−2
, Δ α=√ (H )11 t n−2 sr ≃ √1.52.1 .48 .
−1

5 √
0.00784

then Δ α≃0.07 , Δ β≃0.35 .

Eventually: α=0.36±0.07 and β=0.56±0.35

with 80% confidence.

84
The following graph shows the reaction rate as a function of substrate
concentration. The squares are the experimental points, the solid line is
the optimal curve and the dotted lines are the two extreme curves
f  ,  and f  ,  .
max min min max

0.45

0.4

0.35

0.3

0.25

0.2

0.15

0.1

0.05

0
0 0.5 1 1.5 2 2.5 3 3.5 4

To consider the uncertainties on the data we need to add

weights. These are considered constant on each iteration. It
will be necessary to calculate the derivative of f with
respect to x which is present in the expression of the
weight.
For standard deviations on the set of data we can calculate
the standard deviations on the parameters using methods
described on page 68 for the linear regression.

85
 E. Exercises

Exercise 1 : Correlations Answers p168

1- We carry out nine experiments to obtain in each case

three realizations of the quantities X1, X2 and X3:
i= 1 2 3 4 5 6 7 8 9
X1 x11 = -1 -1 -1 0 0 0 1 1 1
X2 x21 = 1 0 -1 1 0 -1 1 0 -1
X3 x31 = -1 0 1 -1 0 1 -1 0 1

a) Determine the arithmetic means and standard

deviations of these three quantities from these
samples.

b) Plot X2 as a function of X1. The same for X3(X1) and

X3(X2).

c) Calculate the correlation coefficients r12, r13 and r23.

What comments can be we made from the results?
2- Same for the following data:
X1 0 1 -1 2 0 -1 0 -2 1
X2 1 2 -1 2 0 -2 -1 -2 1

3- Same for the following data:

X1 -1 2 -2 0 -2 2 1
X2 -1 0 2 -2 0 2 -1

86
Exercise 2 : Volumes Answers p168

We fill four beakers with 100 mL of water each with a

pipette. To test the pipette and know the precise
quantity of water, we weigh to the nearest decigram
and obtain the following results for the different
beakers in mL:

V1={100.1 , 100.0 , 99.9 , 100.0}

1- Calculate the mean and the standard deviation of
V1. Estimate the precision of the pipette with a 95%
confidence.
We now fill two beakers and gather the contents of the
two into one:
V = V1 + V2.
In the same order as V1, we obtain the following
measurements for V2:
V2={100.0 , 100.1 , 100.0 , 99.9}
For example, for the third measurement, V1=99.9 mL
and V2=100.0 mL.
2- Show that V1 and V2 are independent quantities.

3- Calculate the mean of V, its standard deviation and

its uncertainty ∆V with 95% confidence.

4- Could you find this result with the uncertainty

propagation formula?

(To improve the test it would take more measurements, but

the principle remains the same, and the results remain valid
because we have used the Student, considered decorrelated
data and globally Gaussian packages. We should also take
into account the uncertainties on the measures - resolution -
in addition to their dispersion.)

87
Exercise 3 : Trees Answers (Complete Edition)

We want to measure the distance d between two trees.

For this we have a stick of length one meter. From one
tree to the other, we place the stick end to end a
hundred times. For each displacement, we estimate an
uncertainty of 1 cm.
What is the uncertainty estimated on the value of d?

Exercise 4 : The two-position method

Answers (Complete Edition)

We measure the focal length f' of a convex lens using

the two-position method (also called Bessel method or
displacement method).
An illuminated object is set up in front of a lens and a
focused image forms on a screen. The distance D
between the object and the screen is fixed. When
D>4f', there are two lens positions where the image is
sharp. The distance between these two positions is
denoted d. We then have the focal length of the lens by
the relation f'=(D²-d²)/4D. We measure D=2000±10
mm and d=536±20 mm.

What is the uncertainty on f' ?

Exercise 5 : Refractive index

Answers (Complete Edition)

We want to measure the index of refraction n 2 of a

window glass. We perform the experiment of the
refraction of a laser beam. According to Snell's law of
refraction n1.sin(i1)=n2.sin(i2), where ni are the indices
of the materials and ii the angles of incidence and
refraction. We get n1=nair=1, i1=30±1° and i2=20±2°.

Determine n2 with its uncertainty.

88
Exercise 6 : Cauchy's equation
Answers (Complete Edition)

We want to measure the variation of the light

index n as a function of the wavelength λ in a
transparent material (phenomenon of dispersion). To
carry out the experiment we need a prism, a sodium
lamp and a goniometer. According the theory, in the
visible spectrum, the index variation n(λ) follows
Cauchy's equation:
B
n =A
2

The sodium line spectrum is known. For each line of

wavelength λi, the corresponding index n is calculated
using the formula of the prism:

ni=
sin  AD m ,i
2 
sin
A
2 
Dm is the minimal deviation angle. A=60° is the
internal angle of the prism. These two angles are
measured within 2' (1'=arc-minute and 1°=60').

We obtain the following values:

λ(nm) 615.7 589.2 568.5 515.2 498.1

Color red yellow green-yellow green blue-green
Dm 57° 49.5' 58° 9' 58° 28' 59° 26.5' 59° 50'
n 1.71276 1.71568 1.71852 1.72716 1.73060

1- Determine the uncertainty for n (Δn is assumed

constant).

89
2- Using Cauchy's equation find A, B and their
respective uncertainties.
What is the value of the regression coefficient r?

3- We hypothesize that plotting n as a function of 1/λ

or 1/λ3, will produce a better alignment of the points.
We want to verify that the variation in 1/λ² is indeed
the best of the polynomial relations. For this we take
the form:
n =AB . 
Propose a method for determining A, B and α.
We can verify the model because we have α with its
uncertainty.

Exercise 7 : Wall Answers (Complete Edition)

There is a wall with area S=72 m². The outside

temperature is 6°C and the internal temperature is
maintained at 18°C. This wall is 50 cm thick and
consists of ep=40 cm of compressed straw (thermal
conductivity λp=45 mW/K/m) and ee=10 cm of coating
(λe=200 mW/K/m). The values for λ, thicknesses, and
the temperature are rounded to within 10%, the
nearest cm and the nearest half degree respectively.
1- Determine the thermal resistance with its
uncertainty of the straw for this wall (R.λ.S=e)

2- Repeat for the coating.

3- Taking into account that thermal resistances

associate like electrical resistors in series, determine
the total thermal resistance of the wall with its
uncertainty.

4- What should be the minimum heating power of the

house to compensate for the losses by the walls?
(ΔT=R.Φ)

90
Exercise 8 : Insulation and inertia
Answers (Complete Edition)

In a low-energy house, the thermal resistances

e/λ are measured per square meter. The thermal
resistances are 8 m².K/W for the roof, 4 m².K/W for
walls and floor, and 1 m².K/W for door and window
frames. The R-values are known to the nearest 10%.
The house area for the floor, the roof, the walls and the
frames are 36 m², 54 m² , 82 m² and 8 m² respectively.

1- The equivalent resistances of the roof, the walls, the

floor and frames are in parallel. Determine the total
thermal resistance (in K/W) with its uncertainty.
Outdoor and indoor temperatures are constant.

2- What should be the minimum heating power of the

house to keep the indoor temperature constant while
offsetting the losses?

3- We switch off the heating and measure the

temperature over time to obtain the following results:

t in hours 0 1 2 4 5 6 8 9 10
T in °C 18 16 14 12 11 10 9 9 8

Explain why the decay cannot be linear. We consider

an exponential decay: T(t)=a.exp(-t/τ)+b . Calculate b,
a and τ, with their uncertainties.

4- The house is insulated from the outside. The lost

heat flux corresponds to a decrease in the energy
stored in the house. This inertia is due to the thermal
capacity C of the materials (J/K).
a) By reasoning on an infinitesimal time interval dt,
find the differential equation verified by T(t) and the
expression of question 3.

91
b) What is the relationship between τ, R and C?
Determine C and its uncertainty.

In question 3 we could also take into account measurement

uncertainties: time can be considered as perfectly known
and temperature is measured using a thermometer with
graduations all the Celsius degrees.
For simplicity we considered that the outside temperature
remains constant (to account for day/night variations we
would consider sinusoidal variations and a harmonic
approach).

Exercise 9 : Yield Answers (Complete Edition)

Specific quantities of fertilizer are spread on

fields and we obtain the following yields:

Fertilizer 100 200 300 400 500 600 700

(kg/ha)
Yield 41 44 53 63 66 65 78
(Quintal/ha)

1- Determine the regression line that passes through

the scatterplot. Slope, intercept and uncertainties with
a confidence of 95%.

2- For 550 kg/ha of fertilizer, estimate the yield.

3- Repeat the calculation without fertilizer.

4- If a farmer spreads 250 kg/ha of fertilizer, what is

the probability that he will get 40 to 60 quintals of
grain?

92
Exercise 10 : Study of a battery
Answers (Complete Edition)

To determine the open-circuit voltage E and the

internal resistor r, we measure for the battery
different values of U and I with a voltmeter and an
ammeter (U = E - r.I) :
range unit : V
for U : accuracy ±0.05% ±0.003
U (V) 4. 4. 4. 4. 4. 4. 4. 4. 4. 4.
731 731 730 728 724 724 722 721 719 716
range unit : μA unit : mA
for I : accuracy accuracy ±0.2% ±0.0003
±0.2% ±0.03
I 92. 115. 152. 0. 0. 0. 0. 0. 0. 0.
83 45 65 2352 4686 5200 5841 6661 7750 9264

1- Without error bars: determine E±ΔE and r±Δr.

2- Repeat including the uncertainties on U and I

indicated in the instructions of the multimeter
manufacturer.

Exercise 11: Thin lens formula

Answers (Complete Edition)

We want to measure the focal length f' of a

converging lens. At point O, the lens forms a sharp
image at A' of an object at A. We measure OA, OA' and
their uncertainties (are included all the sources of
uncertainty: geometry, focusing and modeling). We
consider that the lens verifies the thin lens formula:
1/OA' + 1/OA = 1/f'.

Determine f' using a linear regression. Table on the

next page.

93
 Experimental Data (mm) :
OA ΔOA OA' ΔOA'

635 5 150 15
530 5 160 17
496 5 164 15
440 5 172 18
350 5 191 20
280 5 214 25
210 5 292 28
150 5 730 102

Theory

Exercise 12 : Answers (Complete Edition)

For simple regression, show that:

n

∑  xi−x 2=n x2 −x 2 
i=1

Exercise 13 : Answers (Complete Edition)

For simple regression, show that we can also write:


2
1 x
sb =s r  .
n ∑  x i −x 2

Exercise 14 : Answers (Complete Edition)

For simple regression, show that:  b= x 2  a .

94
Exercise 15 : Asymptotes Answers (Complete Edition)

For the simple regression, show that, above and below

the regression line, extreme line, confidence and
prediction curves, have the same asymptote when xo
becomes large.

Exercise 16 : Confidence Interval and Prediction

Interval for Linear Regression with error bars
Answers (Complete Edition)

Regression with error bars

1- Give the expressions of x and x² using the wi

weights. Could you find a new expression of Δ with x
and x² (p66) ?

Analogy

2- From the confidence and prediction intervals for

simple regression (p61 and following), use an analogy
to determine the following formulas:

Confidence: Prediction:

 
2 2
1  xo −x  1  x o− x 
 y o= 1 2 2  y o= 1n 2 2
n ∑ w i x −x  n∑ w i x −x

3- Determine the y-distances to the regression line for

extreme lines, confidence curves, and prediction
curves when xo becomes large.
In a previous exercise we showed that for the linear
regression, the asymptotes are the same. By analogy
what should we set so that it is the same in regression
with bars of errors?

95
Show that we then have the following formulas:
Confidence: Prediction:

 
2 2
1  x o −x  1 x o− x 
 y o= 1  y o= 2
∑ w i x 2 −x 2 ∑ w i x 2− x2

The formulas obtained by analogy are thus empirical.

Yet, while they seem experimentally coherent, they
require confirmation by theoretical demonstration.

Confidence and prediction curves for the absolute zero

experiment: T (°C) as a function of P (hPa)

Exercise 17 : Other expressions

Answers (Complete Edition)

For regression with error bars give the expression of

a, b, Δa and Δb as a function of x, y, xy, x² and y².
Compare with simple regression.

96
Exercise 18 : Least squares method
Answers (Complete Edition)

Demonstrate by the least squares method the

expressions of a and b:

1- For simple linear regression.

2- For linear regression with error bars. The Δxi and

Δyi are considered small with respect to xi and yi.

Proof of the expressions of Δa and Δb for the simple

regression:

Method 1 :
x i − x
1- Show that a= ∑ pi yi with p i = .
i ∑ xi− x 2
2- Deduce from this formula of a its variance V(a) 16
.

Method 2 :
Use the propagation of standard deviation formula.
For simple linear regression, is it possible to find a, b,
Δa and Δb using the generalized regression matrix
method?

Exercise 19 : Expectation of a
Answers (Complete Edition)

For linear regression, we denote α and β as the

parameters of the population: E(yi)=α xi+ β.
a and b are the estimated parameters from a sample:
y i =a xi b
Show that we have an unbiased estimator for α, so
E(a)=α.

16 MATH : E(X+Y) = E(X) + E(Y). If X and Y are two independent

variables: V(X+Y) = V(X) + V(Y).

97
Exercise 20 : Standard deviations
proportional to y
Answers (Complete Edition)

We consider the particular theoretical case where the

standard deviations of the linear regression are
proportional to y :  y =k y i . This case, where the
i

relative uncertainties are constant, is experimentally

common. We are interested in the slope a.

∑ y12 ∑ xy −∑ yx2 ∑ 1y
1- Show that: a= 2
2

∑ y12 ∑ xy 2 − ∑ yx2 
∂a
2- Express (long calculation).
∂ yj

3- Calculate sa using the expressions found for the

following two datasets:

xi 1 2 3 4 5 6 7
1 : yi 10 15 20 25 30 35 40
2 : yi 8.286 17.286 18.286 27.286 28.286 37.286 38.286
(We let k=0.1).

Find these results again with the numerical method

(evaluation of the derivatives with the small variations
method).

Compare the values obtained by the classical method.

Exercise 21 : Interpretation of
the expression of wi
Answers (Complete Edition)

Graphically justify the position of a in the expression

of wi.

98
Non-linear regression

Exercise 22 : Decomposition into Gaussians

Answers (Complete Edition)

A factory manufactures nails. With a caliper, we

measure the size of 48 nails:

Two machines manufacture the nails. The nails made

by one of the machines are not exactly the same size.
We assume that the size is distributed according to a
Gaussian distribution. Determine the parameters of
the Gaussian distributions of the nails manufactured
by each machine (maximum, mean and standard
deviation). How many nails have each machine
produced?

To check your correlation calculations you can use the

spreadsheet function of OOo :
COEFFICIENT.CORRELATION(*;**).
Courbes : Insertion>Diagramme...>XY(Dispersion)>etc.
For matrix calculations, inversion of matrices :
INVERSEMAT(*), product of matrices : PRODUITMAT(*;**).

You can use the file IncertitudesLibresOOo32.ods on the

website www.incertitudes.fr to realize regressions. Sheet 2,
simple regression and sheet 3 with error bars.

99
100
III. PROBABILITY DISTRIBUTIONS

We list different laws of discrete and continuous prob-

abilities. We verify they are distributions of probability and
we give their expectation μ=E( X ) and variance
V =E [(X −μ )2 ].
The variance can also be calculated with the formula
2 2
V =E [ X ]−E [ X ] and then we have the standard devia-
tion σ=√ V .

We can calculate other moments that allow us to de-

termine, for example, the symmetry and the flatness of the
probability distribution.

k
Moments: μ k =E( X )

Normalized moments are instructive because they charac-

terize the form of the distribution:

[ ( X−μ
σ )]
k μk
βk−2=E or βk−2=
σk
β1 : Skewness (third standardized moment)
β2 : Kurtosis

We also consider the sum of independent random

variables: Z =X +Y .

101
 A. Discrete Random Variables

1) Binomial distribution
We consider the number of successes in a sequence of
n identical and independent experiments. Each trial has two
outcomes named success and failure ( Bernoulli trial). Each
success occurs with the probability p=P(S) . The two
parameters are n and p and we write B(n,p). We want to
determine the probability to obtain exactly k successes out
of n trials. A path containing k successes has n-k failures
and its probability is pk qn−k , where q=1− p is the
probability of a failure.
Then we have to count the number of paths where k
successes occur, there are different ways of distributing k
successes in a sequence of n trials. n choice for the position
of the first success, n-1 for the second and n+1-k for the
kth success:
n( n−1)...(n+1−k )=n! /(n−k ) ! possibilities17.
After we divide by k! to remove
multiple counts (for example
S1S2F and S2S1F correspond to the
same path). From where:

P( X =k ) = n ( ) pq k n−k

k
n!
with (nk )= (n−k) !k !
(often read aloud as « n choose k »)
n
and ∑ P( X=k)=1
k=0

17 n!, is said « n factorial » with n!=1×2×3. ..×n .

102
 we show that E( X)=np and V (X )=npq .

The sum of two independent binomial laws of the same p is

also a binomial law. The sum of a Bx(n1,p) and a BY(n2,p) is
a Bz(n1+n2,p).
To determine the distribution of Z sum of X and Y
we use the following property for discrete distributions:

P( Z=k )=∑ P([ X=i]∩[Y =k−i])

i

2) Geometric distribution
We consider a random experiment with exactly two
possible outcomes : S="success" and F="failure" ;
p=P(S) and q=1− p=P(F) . We repeat independent
trials until the first success. Let X be a random variable of
the number of trials needed to get one success. We denote
the distribution by G(p).
∞
P( X =k )=q
k−1
p , ∑ P( X=k)=1
k=1

E( X)=1/ p and V ( X )=q/ p2

⦁ Example: We have a box with two white balls and a black

ball inside. The balls are indistinguishable to the touch. We
draw a ball, if we get a black ball we stop, otherwise we
return the ball to the box and we repeat. What is the
probability of drawing the first black ball in the third draw?

103
Answer: P( X=3)=(2/ 3)2 1 /3=4 /27 .

⦁ Application: This distribution can be used like a discrete

model of lifetime. For example, consider a cell which every
day has a chance in ten to die. If it did not die the next day,
its probability of dying the next day did not change: there is
no aging18. Every day the cell can escape death and it can
live forever. Rather than making a decision every day, we
can decide his death every hour with a probability q ' of
survival, like this q ' 24=q and q ' ≃0.996 . Or even a
choice every second, we then have different geometric
distributions witch modelize the same reality with p→0
and q→1 .

From a discrete distribution we reach a continuous one.

The elapsed time since the beginning of the experiment is
t=k Δ t .
Let's look at the cumulative distribution function:

18 The geometric distribution has a remarkable property, which is

known as the memoryless property.

104
P( X ⩽k )=∑ qk−1 p=∑ e ln (1− p).(k−1) p≃∑ p e− pk
We have used a Taylor series, indeed t≫Δ t and k ≫1 .
p
− t
Then P( X ⩽k )→∫ f (t)dt , ∑ p e →∫ λ e−λ t dt so,
Δt

if we look at the limit, the geometric distribution becomes

the exponential one with λ= p/Δ t . Here for our cell λ is
about 1.16⨉10-6 per second.

The sum of two independent geometric random variables

G(p) is a negative binomial distribution BN(2,p), rank for
obtaining two successes.

For a binomial distribution the number of trials is fixed and

we look at the number of successes, in the case of the neg-
ative binomial distribution it is the opposite we are inter-
ested in the number of trials necessary to achieve a number
of successes fixed in advance. BN(r,p) is the probability
distribution of the rank of the r-th success19.
Then G(p) =BN(1,p).

3) Poisson distribution

The Poisson distribution expresses the probability of a

given number of events occurring in a fixed interval. The

19 There is also another definition: number of failures preceding the r-

th success. The relations remain true by redefining the geometric
distribution in the same way.

105
events occur with a known frequency independent of the
time elapsed since the previous events20. If λ is the number
of times the event occurs on average over a given interval
then the probability that the event occurs k times over this
time interval is:

k ∞
P( X =k )= λ e ∑ P( X=k)=1
−λ
,
k! k=0

E( X)=λ and V ( X )=λ

• Example: During the Perseids meteor shower of

August 1911 the hourly rate of shooting stars was
50. What was the probability of seeing exactly 7
meteors in 12 minutes?

Answer:
12 107 −10
λ= 50=10 and P( X =7)= e ≃9 %
60 7!

The sum of two independent Poisson distributions PX(λ1) 

and PY(λ2) is a Poisson distribution PZ(λ1+λ2).

20 or space traveled.

106
 B. Continuous Random Variables

1) Uniform distribution

{
x <a : f (x)=0
1
a⩽x⩽b :f ( x)=
b−a
x >b : f (x)=0

2
a+b ( b−a)
E( X)= and V ( X )=
2 12

To determine the distribution of Z sum of two

continuous independent random variables X and Y we
use a convolution:

+∞
f Z (x)=∫ f X ( y ) f Y ( x− y )dy
−∞

Consider the sum of two independent uniform distributions

U(a,b). The integrand is nonzero if:

107
 a< y <b and a< x − y< b then x−b< y< x−a

If 2a<x<a+b : x-b x-a

^
a b y

If a+b<x<2b : x-b x-a

^
a b y

If 2 a< x <a+ b
x−a
1 x −2 a
then f Z ( x)= ∫ 2
dy=
a (b−a) (b−a)2

{
x <2 a : f Z (x)=0
x−2 a
2 a⩽x <a+b :f Z (x)=
(b−a)2
2 b−x
a+b⩽x⩽2 b :f Z (x)=
(b−a)2
x >2 b : f Z (x)=0

We obtain a triangular distribution:

f
^
^

2a a+b 2b x
108
 2) Exponential distribution
f
1.0

λ=1

0.5

λ=1/2
0.1

0 1.0 2.0 X

{t⩾0: f (t )=λ e−λ t

t<0 : f (t)=0
1
E(T )= λ et V (T )=
1
λ
2

The exponential distribution satisfies the memoryless

property: P(T > b)=PT >a (T >a+ b) . This distribution is
widely used to model waiting times. It is also used to model
lifetime without aging : like the lifetime for a particle
decay.
The mean life E(T ) and the half-life t1/2, P(T > t 1 /2 )=0.5
, are two different times.

109
The distribution of the sum of two independent exponential
distributions is not an exponential distribution.

3) Normal distribution
The normal or Gaussian distribution has previously been
described page 20.
The sum of two independent Gaussian distributions
NX(μ1,σ12) and  NY(μ2,σ22) is the Gaussian distribution
NZ(μ1+μ2, σ12+σ22) .

4) Student's t-distribution
The t-distribution is express with the number of degrees of
freedom k and the gamma function (function described in
the mathematical tools).

For k ≥1 , f k (x)=
1
Γ ( k +12 ) 1
2 k +1
√k π k
Γ ( ) 1+ x
2 ( k)
2

As the number of degrees of freedom grows, the t-distribu-

tion approaches the normal distribution N(0,1) :
2
x
1 −2
lim f k (x)= e
k→+∞ √2 π

k
Variance: V k = if k ≥3 .
k −2

110
 k −2
Kurtosis: βk =3 if k ≥5 .
k−4
Y
∞
8
4
3
2
k=1
+0.3

+0.2

+0.1 0.1

X
1
−4.0 −3.0 −2.0 −1.0 +1.0 +2.0 +3.0 +4.0

In exercises the expressions of the first Students are ex-

+∞
pressed, we show that ∫ f k ( x) dx=1 , we compute the ex-
−∞
pression of the variance and finally we show on examples
that the sum of two independent Students is not a Student.

111
 First Students

k d° V β2 β4 f k( x) f k (0)
1 2 - - - 1 1 ≃0.318
π 2
1+ x
3
2 3 - - - x2 ≃0.354
2 √2 ( )
1 2
1 / 1+
2
3 4 3 - - 2 2 ≃0.366
1/ ( 1+ )
2 x
√3 π 3
4 5 2 - - 2 5 ≃0.375
1/ ( 1+ )
3 x 2
8 4
2 3
5 6 5/3 9 - ≃0.380
≃1.67
8
3√5 π ( )
1/ 1+
x
5
6 7 3/2 6 - 2 7 ≃0.383
16 √ 6 ( 6 )
15 x 2
=1.5 1/ 1+

2 4
7 8 7/5 5 125 ≃0.385
5√7 π (
1/ 1+ )
16 x
=1.4 7
8 9 4/3 9/2 67.5 2 9 ≃0.387
64 √ 2 ( 8 )
35 x 2
≃1.33 =4.5 1/ 1+

2 5
9 10 9/7 21/5 49 ...
105 π (
1/ 1+ )
128 x
≃1.29 =4.2 9
10 11 5/4 4 ... ...
=1.25
11 12 11/9 27/7
≃1.22 ≃3.9
...
2
∞ 1 3 15 1/ √2 π e− x / 2 ≃0.399

112
 5) Chi-squared distribution

Let consider k independent normal distributions

N(0,1) : T1, T2,… and Tk. The sum X k =T 12 +T 22 +...+T k 2
follows a χ2-distribution with k degrees of freedom.

k
E( X k )=∑ E(T i2)=k (V (T )+ E(T )2 )=k
i=1

1 k/ 2−1 −x /2
For k ≥1 and x≥0 , f k (x)= k /2
x e
2 Γ ( k /2 )

Expectation : Ek =k Variance : V k =2 k

Skewness : β1,k =√ 8/ k

Kurtosis : β2,k =3+12 /k

113
 f
k=1
0,5

k=2

0,2
k=3
k=10

0 1 5 10 X

The chi-squared distribution converges to a normal distri-

bution for large k . Normal distribution of expectation k
and variance 2 k .

114
 C. Function of a continuous distribution

We have a random variable Y defined as a function of a

continuous random variable X: Y=φ(X).
We know the law of X and we want to determine the law
of Y. We use the cumulative distribution functions F and
then we consider the derivative of FX to get the probability
density function f.
Cumulative distribution function of X :
x
F X ( x)=P( X ≤x)= ∫ f X (x) dx
−∞

We consider the case where φ(x) is a strictly monotonic

function. Then Y is a continuous distribution.
Cumulative distribution function of Y :
d FY ( y )
FY ( y )=P(Y ≤ y ) and f Y ( y )=
dy
We try to express FY(y) with FX.

1) Case where φ(x)=ln(x): x>0 and Y=ln X

ln X y y
FY ( y )=P(Y ≤ y )=P( ln X≤ y )=P(e ≤e )=P( X≤e )
We have applied the inverse function φ-1(x)=ex.
The exponential function is strictly increasing, so the direc-
tion of the inequality has been preserved.
so FY ( y )=F X (e y ) and f Y ( y )=e y f X (e y )

115
⦁ Example: X is a uniform distribution U(1;2). What prob-
ability distribution is Y?

{
0 x⩽1
y
We have f X (x )= 1 1< x <2 then if e ≤1 , y≤0 and
0 x⩾2
f Y ( y )=0 . We continue like this for the two other cases
and we obtain the distribution of Y:

{
0 y⩽0
f Y ( y )= y
e 0< y < ln 2
0 y ⩾ln 2

2) Case where φ(x)=ax+b: a≠0 and Y=aX+b

If a>0 :
y−b
FY ( y )=P(Y ≤ y )=P(aX + b≤ y)=P (X≤ )
a
A linear function is strictly increasing for a strictly positive,
the direction of the inequality is not changed.
y−b 1 y−b
Then FY ( y )=F X ( ) and f Y ( y )= f X ( )
a a a
If a<0 :
y−b y−b y−b
FY ( y )=P( X≥ )=1−P (X < )=1−FY ( )
a a a

1 y −b 1 y −b
and f Y ( y )=− f X ( ) then f Y ( y )= f X ( )
a a |a| a

116
⦁ Example 1: X is a uniform distribution U(0,1). What
probability distribution is Y?

{
0 x⩽0 y−b
We have f X (x )= 1 0< x <1 , so if ≤0 and
a
0 x⩾1
a> 0 then y≤b and f Y ( y )=0 . We continue like this for
the two other cases and we obtain the distribution of Y:

{
0 y⩽b U (0 ,1)→U ( b , a+b)

f Y ( y )= 1
b< y < a+b
a If φ(x)=(b-a)x+a and a<b:

0 y ⩾a+b U (0 ,1)→U ( a , b)

⦁ Example 2: X is a Gaussian distribution N(0,1). We can

find again a N(  μ,σ) distribution with Y= σX+μ. In gen-
eral, by applying an linear function we obtain a distribution
of the same kind.

3) Case where φ(x)=x2: Y=X2 and y>0

FY ( y )=P( X 2≤ y )=P(−√ y≤X ≤√ y)=F Y ( √ y)−FY (−√ y)

1
and f Y ( y )= ( f X ( √ y )+f X (−√ y)) if y>0 else zero.
2√y

117
4) Case where φ(x)=ex: Y=eX and y>0

FY ( y )=P( e X ≤ y )=P( X≤ln y )=F Y ( ln y)

1
and f Y ( y )= f (ln y) if y>0 else zero.
y X

D. Numerical simulation

We simulate continuous and discrete probability dis-

tributions using computers. For this purpose we use uni-
form distributions created by random number generation
algorithms:
Continuous uniform distributions U(0,1) : Ran# on a
pocket calculator, ALEA() in the LibreOffice spreadsheet,
etc.
Discrete uniform distributions U(i,j) : for example,
rand(i,j) in PHP language. rand(0,999)/1000 simulates a
uniform continuous law discretized to the thousandth.

⦁ Inverse transform sampling : the inverse transformation

method takes uniform samples between 0 and 1 from U.
We express a X distribution as a function of U by inversion
of the cumulative distribution function FX. With F strictly
−1
increasing: X =F (U ) .

118
 −λ x
Case of an exponential distribution: f X ( x )=λ e if x>0
x
else zero, then F X (x)=∫ f X ( x ) dx=1−e−λ x = y if x>0
−∞
(1− y )
else zero. So x=−ln and finally we simplify,
λ
knowing that 1-U and U have the same distribution.

ln U
Simulation of an exponential distribution: X =−
λ

⦁ Simulation of two independent normal N(0;1) distribu-

tions X1 and X2 from two independent uniform U(0,1) dis-
tributions U1 and U2:
X 1= √−2 ln U 1 cos (2 π U 2 )
X 2= √−2 ln U 1 sin (2 π U 2)
This is the Box-Muller transform. For a Gaussian there is
no direct formulation of the cumulative distribution func-
tion F. Thus F-1 has no simple analytic expression.

⦁ Method for any X distribution:

If we have a continuous distribution we obtain a discrete
distribution using class intervals for x. We then have a his-
togram and each bar of height pi is also discretized in units.
We get an outcome of the distribution X drawing randomly
and equiprobably a unit of the histogram. We get by
putting the bars end to end a full range of units. We use a
discrete uniform distribution U(1,N) where N is the total
number of units in the histogram. The generated value is
compared to its position on the range and we get a xi obser-

119
vation.

⦁ There are many other methods that use the different prop-
erties of the probability distributions. For example, by sim-
ulating the Bernoulli distribution, we obtain, by sum, a bi-
nomial distribution which itself allows us to simulate a nor-
mal distribution.

120
 E. Exercises
Exercise 1: Binomial distribution
Answers (Complete Edition)
Check that the binomial distribution defines a
probability distribution. Calculate the expectation and
the variance as a function of the parameters n and p.

Exercise 2: Sum of binomial distributions

Answers (Complete Edition)
Show that the sum of independent binomial
distributions with the same parameters p is itself a
binomial distribution whose parameters will be
determined.

Exercise 3: Geometric distribution

Answers (Complete Edition)
Check that the geometric distribution defines a
probability distribution.
Calculate the expectation and the variance as a
function of the parameter p.
Show that the sum of two independent geometric
distributions with the same parameter p is a negative
binomial distribution NB(2,p).

Exercise 4: First successes

Answers (Complete Edition)
1- Let consider a balanced coin. What is the
probability that the first tail will appear on the fifth
toss? Knowing that the first tail has not yet appeared
on the third throw, what it is the probability that it will
appear for the first time at the eighth toss?
2- Let consider a balanced die. On average after how
many tosses appears the first six? What is the
probability that the first six will appear in the first six
throws?

121
Exercise 5: Poisson distribution
Answers (Complete Edition)

Check that the Poisson distribution defines a

probability distribution.
Calculate the expectation and the variance as a
function of the parameter λ.
Determine the sum of two Poisson distributions.

Exercise 6: Uniform distribution

Answers (Complete Edition)

Calculate the variance for a continuous uniform

distribution U(a,b).

Exercise 7: Exponential distribution

Answers (Complete Edition)

Check that the exponential distribution defines a

probability distribution.
Calculate the expectation and the variance as a
function of the parameter λ.
Determine the sum of two exponential distributions.

Exercise 8: Sum of Gaussians

Answers (Complete Edition)

1- Determine the sum of two standard normal

distributions N(0,1).
2- Determine the sum of two normal distributions.

Exercise 9: First Students

Answers (Complete Edition)
1- Give the expressions of the first Student functions.
2- Give the polynomial degree in denominator, the
variance, the kurtosis and the y-intercept.
3- Give the expression of the Student for k=9,
centered and with a variance equals to 14/5.

122
Exercise 10: Student's t-distribution
Answers (Complete Edition)

Whatever k, show that the Student distribution

corresponds to a probability distribution.
2 − k +1

( )
+∞
x
We can use the integral I k =∫ 1+ 2
dx and
−∞ k
x
carry out an integration by substitution with u= .
√k
Exercise 11: Variance of Student's t-distribution
Answers (Complete Edition)

Determine the variance of a t-distribution.

Exercise 12: Sum of Student's t-distributions

Answers (Complete Edition)

Using an example show that the sum of two

independent t-distributions is not a t-distribution.

Exercise 13: Chi-squared distribution

Answers (Complete Edition)

Give the expressions of the first chi-square density

functions in the cases where k=1, k=2, k=3 and k=10.

Exercise 14: Product of distributions

Answers (Complete Edition)

Let X and Y be two independent distributions.

1. Propose a general method for determining the
probability distribution of Z=XY.

123
2. We now consider the case where X and Y are two
independent continuous uniform distributions U(1,2).
a. Determine the analytic expression of the Z
distribution.
b. Find the shape of probability density function of Z
with a numerical simulation of the product on a
spreadsheet for n=10,000.

Exercise 15: Sum of exponentials

Answers (Complete Edition)

Let Xi be n independent exponential distributions with

the same parameters λ. Let Sn be the distribution of
the sum: Sn=X1+X2+...+Xn. We have too: Mn=Sn/n.
1. Determine the probability distribution S2.
2. Determine the probability distribution Sn.
3. Determine the probability distribution Mn.

Exercise 16: Inverse distribution

Answers (Complete Edition)

Let X be a random variable with strictly positive

support.
1. Determine the probability density function of Y=1/X.
2. a. Find Tn the inverse distribution of Mn defined in
the previous exercise.
b. Find the inverse distribution of the Cauchy
distribution:
1 1
Cauchy distribution X: f x= π for all x.
1+ x 2

124
125
IV. ESTIMATORS

We have a random variable X that follows a known

distribution with parameters that are unknown.
We have a n sample (X1,X2,…,Xn) of the random variable
X from a population. We want to have a method to estimate
at best the different parameters that define our distribution
(inferential statistics). Our parameters are denoted by the
letter θ and we call Tn our estimator of θ.

At first we use the sample to give a point estimate of θ,

that has a high probability of being close to the parameter,
next we provide an interval estimate of θ that has a high
probability of containing θ.

A. Properties of an Estimator

1) Bias
The bias of an estimator Tn is the expectation E(T n−θ) .
Then : bT (θ)=E (T n )−θ
n

The estimator is unbiased if bn =0 so E(T n )=θ .

It is best to have an unbiased estimator if not
asymptotically unbiased: lim bn=0 .
n→+∞

126
 2) Mean Square Error
We can compare different estimators with their mean
square errors. The mean square error of Tn is defined as the
expectation E[(T n−θ)2 ] and we show that:

2
r T (θ)=V (T n)+ b
n

Indeed E[(T n−θ)2 ]=E(T n2)−2θ E (T n)+θ2 (linearity of

the expectation) and eventually after simplifications of
r=V (T n )+ E(T n )2−2θ (b+θ )+θ 2 we find the previous
expression. For an unbiased estimator its mean square error
is equal to its variance. It is useful to choose an estimator
whose mean square error tends to zero with the sample size
and the faster is the convergence the better is the estimator.

⦁ Example 1: X̄ n is the sample average regarding the

sample (X1,X2,…,Xn) defined as:
n
1
T n= X̄ n= ∑ X i
n i=1
Demonstrate that X̄ n is an estimator of the expectation
θ=m=E( X) . Study its properties: bias, convergence.

E(T n )=1/n ∑ E( X i) using linearity of the expectation

then E(T n )=1/n . n m=m and b=0 .

V (T n)=1/n2 ∑ V ( X i ) (variance of a sum with

independent variables) then r=1 /n2 . n σ 2 and r=σ 2 /n .

The sample average is a good estimator of the expectation

of a random variable. The estimator is unbiased and the

127
mean square error tends to zero when n tends to infinity.

⦁ Example 2: We are looking for an estimator of the

variance of a random variable, we propose the following
two estimators:
n n

∑ ( X i− X¯n)2 ∑ ( X i− X¯n)2
S n2= i=1 and Rn2= i=1
n n−1

Which, in your opinion, is the best estimator of σ2 ?

S n2=1/n ∑ ( X i2−2 X i X̄ n + X¯n2) and

S n2=1/n ∑ X i2−2T n ∑ X i /n+T 2n=1/n ∑ X i2−T 2n
2 2 2 2 2 2
then E( S n )=E ( X )−E(T n )=σ +m −V (T n )−E (T n)
2 n−1 2 2
and E( S n2 )=σ 2− σ so E(S n )= σ and b S =− σ
2
2
n n n
n
.

By a quite similar calculation we find b R =0 .

2
n

This quantities are two estimators of the variance because

they are unbiased or asymptotically unbiased.
Rn2 is the best estimator because it has no bias.

128
 B. Construction of estimators

We want to have a general method to find the

appropriate estimators to estimate the parameters of a
distribution.

1) Method of Moments
We identify the moments of the population with those
of the sample. We consider as many moments as we have
parameters starting with the first moment. As we will see
on examples this method provides us with estimators but
does not guarantee us that these are the best in terms of
bias and mean square error.

k *
Theoretical moments: mk =E (X ) , k ∈ℕ

n
1
Sample moments: X nk = ∑Xk
n i=1 i

We can also consider the moments centered or com-

pletely normalized, the approach remains the same. The
first n-sample moment is the sample average and it corre-
sponds to an excellent estimator as shown before. On the
other hand, the centered second moment has a bias, as we
have shown before we should divide by n-1 instead of n to
have a unbiased estimator.

129
⦁ Example 1: We have a checkerboard of 100 squares and
200 seeds. Every second
we randomly place a seed
in one of the squares. At
the end of the experiment
we count the number of
seeds in each square and
we count the number of
squares that contain no
seed, one seed, two seeds
and so on. Let X be the
random variable for the
numbers of seeds per
square. We obtain the following distribution:

k 0 1 2 3 4 5 6 7 8
n 12 28 34 10 8 7 0 1 0

We assume that this distribution follows a Poisson

distribution. Deduce the value of the parameter λ.
The parameter of this distribution is equal to the
expectation therefore according to the theorem of
moments, λ is estimated by the sample average:
n p
1
λ= ∑ x = 1 ∑ n x then λ=2
n i=1 i n k=1 k k
21
.

This result is consistent with our model, indeed let us name

n the number of squares and N the number of seeds. We
have a uniform random distribution. We could, for
example, use two balanced ten faces dice of different colors
for the horizontal and vertical position. The frequency of
21 Using the second moment we will find λ=2.06 .

130
the event per second for each square is 1/n. For a Poisson
distribution at any instant the event can occur, here the
time is discretized, nevertheless the approximation of a
continuous time is correct because one second is a small
duration compared to that of 200 seconds of the
experiment. So we can take λ=N /n . We tend to a
Poisson distribution when the number of squares and seeds
tend to infinity.

⦁ Example 2: We assume that the lifespan of a glass follows

an exponential distribution. We observe in months the
following lifetimes: 7, 5, 37, 35, 17, 9, 6, 13, 4 and 8
months. Determine thee parameter λ of the distribution.
1
For an exponential distribution E(T )= , hence by using
λ
n 10
the first moment: λ= n and λ= ≃0.071 .
141
∑ ti
i=1
We thus have a point estimate of the parameter λ and the
life expectancy is about 14 months.
We will show in an exercise that this estimator T n has a
n
bias: E(T n )= λ and bT = λ .
n−1 n
n−1
This bias and its mean square error tend to zero:
(n+2)
r T =λ 2 .
n
(n−1)(n−2)
We construct from this estimator a new estimator Wn
n−1 n−1
without bias: W n= T n= .
n ∑ Xi
We show that E(W n)=λ and r W =V (W n )=λ2 /(n−2) .
n

This estimator is better than the previous: zero bias and

131
lower risk (mean square error). A new estimate of the
parameter gives λ≃0.064 and the life expectancy is about
16 months.

⦁ Example 3: We consider that the mass of the apples

produced by a tree follow a normal distribution. We
randomly draw and measure the following masses in grams:

158 131 146 158 125 153 166 121

127 123 195 149 124 153 123 129

Determine the mean and standard deviation of the

distribution.
n n
1 1
μ= ∑
n i=1
mi≃142.5 , σ2 +μ 2= ∑ mi2 and σ≃20.2 .
n i=1

⦁ Example 4: The planks produced by a sawmill have a

length which follows a uniform distribution U(a,b).
We measure the lengths in millimeters of 8 planks drawn at
random: 2017, 1987, 2018, 2014, 2003, 1985, 2013 and
1981. Estimate a and b.

a+b 1
We have E( X)= = Xn
2
2 ( b−a)2 (a+ b)2 a 2+ ab+b 2
and E( X )= + = = X n2
12 4 3
1 1 2 2
then a+ b=2 X n and ab=4 X n −3 X n

Eventually a≃1977 and b≃2028

132
 2) Method of Maximum Likelihood

Let X be a discrete or continuous random variable. We

want to estimate a unknown parameter θ from a set of
observations {xi}.
We define a function f(x,θ) such that:

{
Pθ (X =x) for a discrete variable

f (x , θ) = or

f θ ( x) for a continuous variable

We define the likelihood function L. This is a function of

θ, determine by the numbers x1, x2, . . . , xn:
L (x1 , x 2 ,..., x i ,... , x n ,θ)=f ( x 1 ,θ)×f ( x2 ,θ)×...×f ( xn ,θ)

also we can simply write:

n
L ( x1 , x 2 , ... , x i ,... , x n , θ) = ∏ f ( x i , θ)
i=1

The value of the parameter for which the product of the

probabilities, or probability densities, taken at the
different points of the sample, is maximum, is considered
the most probable value. The maximum likelihood
estimate of θ is the value that maximizes the likelihood
function L(θ).
The maximum likelihood principle is simple and the
method is easy to implement:

∂ L ( x i , θ) ∂2 L ( xi , θ)
= 0 and < 0
∂θ ∂θ 2

133
Usually on can find the maximum by differentiating the
likelihood function L(θ). However the calculation of the
derivative may be tedious, it is why we prefer to consider
the logarithm of L(θ). The logarithm is a increasing
function and we can consider the extreme values of
ln(L(θ)) instead of L(θ).

⦁ Example 1: Let us take again the case of a Poisson

distribution and determine the estimator of λ by the
method of maximum likelihood.
x x n n xi
Pλ (X =x)= λ e−λ and L=∏ λ e−λ =e−n λ ∏ λ
i

x! i=1 x i ! i=1 x i !

n n
x
∂ln L x
ln L=−n λ+ ∑ ln λ
i

and =−n+ ∑ i =0
i=1 xi ! ∂λ i=1
λ
n
1
then λ= ∑ x i
n i=1

We find the same estimator as by the previous method.

⦁ Example 2: Let us take again the case of an exponential

distribution and determine the estimator of λ by the
method of maximum likelihood.
n
n −λ ∑ t i
−λt −λ ti
f λ (t)=λ e and L =∏ λ e n
=λ e i=1

i=1

134
 n x
∂ln L n n
ln L=n ln λ−∑ ln λ
i

and = −∑ t i
i =1 xi ! ∂λ λ i=1
n
then λ= n

∑ ti
i=1

The same estimator as by the previous method.

⦁ Example 3: Let us take again the case of a Gaussian

distribution and determine the estimators of μ and σ by the
method of maximum likelihood.
2 2
1 x−μ 1 xi −μ
1 − ( σ ) n
1 − ( σ )
f ( x )= e 2 and L=∏ e 2
√ 2 π⋅σ i=1 √ 2 π σ

n 2
xi −μ
so ln L =−n ln σ−
n
2
ln (2 π)−
1
∑
2 i=1 σ
( )
∂ln L n x i−μ n
1
∂μ ∑ ∑x
= =0 and μ=
i=1 σ2 n i=1 i

n n
∂ln L n 1 1
=− σ + 3 ∑ ( x i−μ) =0 and σ = ∑ ( x i−μ)2
2 2
∂σ σ i=1 n i=1

We obtain the same estimators as by the moment theorem

and we have again an estimator of the variance biased.

135
⦁ Example 4: Let consider now the uniform distribution.

{
1
n
n
if {x i}⊂[a , b]
L ( xi , a , b)=∏ f (xi , a , b)= (b−a)
i=1
0 else

For a fixed, smaller is b bigger is the likelihood, then

b=max({x i }) .
For b fixed, bigger is a bigger is the likelihood, then
a=min({x i}) .

We have here very different estimators than by the method

of moments, for the example of the planks we have the
following estimates: a=1981 and b=2018 .

These estimators are biased, for example b is necessarily

smaller than the theoretical value, but, contrary to the
method of moments we are assured that all the xi belong to
[a, b].

136
 C. Interval estimate

We now have effective tools to determine the values of

the parameters of a distribution. We have determined point
estimates and now we want to determine a confidence inter-
val.
The central limit theorem allows, from a large sample,
to estimate the mean of a probability distribution with a
confidence interval.
But what about the other parameters different from
the mean? For example, what are the uncertainties on the
parameter λ of an exponential distribution, or the bounds
a and b for a uniform distribution?
We consider an unbiased estimator and if the estima-
tor is biased we create a new estimator by removing the
bias.
We use three different methods. The integral method which
requires to determine the full probability distribution of the
estimator. A second method, simpler, provides an inequal-
ity that overestimates our uncertainty, but only requires
knowledge of the variance of the estimator. And finally a
third by numerical simulation.

For the second method we use the Chebyshev's in-

equality. This inequality can be applied to any probability
distribution in which the mean and variance are defined.
We do not have to know the aspects of the estimator distri-
bution, only its variance, but the inequality generally gives a
poor bound:

137
 V (X )
∀ ϵ>0, P(|X−E (X )|≥ϵ) ≤ 2
ϵ

⦁ Example 1: Consider again the checkerboard and the

Poisson distribution with parameter λ. This parameter is
estimated by the mean and we can therefore, in the case of
large numbers, use the central limit theorem:
λ=λm±t ∞ σ / √ n
Let us estimate the variance:

k 0 1 2 3 4 5 6 7 8
k-λm -2 -1 0 1 2 3 4 5 6
2
(k-λm) 4 1 0 1 4 9 16 25 36
n 12 28 34 10 8 7 0 1 0
n
1
σ =
λ
2
∑
n−1 i=1
(x i− x̄)2 , σ λ ≃1.44 and λ=2.0±0.3 with

95% confidence.
Here the sample size is not sufficient for this method but
we have illustrated the general method for large samples.
We probably have underestimated the width of the interval.

⦁ Second example: The waiting time of a train follows a

uniform distribution U(0,a). We observe the following
waiting times 3, 12, 7, 17, 8, 14, 2, 5, 10, 14, 15 and 11
minutes. What is the value of a ?
What is the bias, the variance of the estimator Tn and the
uncertainty on a at 90% confidence?

138
 a) Let us first use the method of maximum likelihood
which gives us the estimator Tn , the distribution of the
maximum of a number n of independent, identically
distributed variables: T n=max( {X i})
a is here estimated at 17 minutes.

The distribution of the maximum is such that:

P(T n≤x)=P([X 1≤x]∩[ X 2≤x ]∩...∩[ X n≤ x])
the variables are independents, then:
n
P(T n≤x)=∏ P( X i≤x )
i=1

Cumulative distribution function of X: F X (x)=P(X ≤x)

n n
x
FT (x)=∏ F X ( x )=
n
i=1
i ()
a
for 0≤x ≤a

FT (x)=0 if x< 0 , FT (x)=1 if x> a

n n

d FT (x ) x n−1
So we obtain the density of Tn : f T (x)= =n n
n

n
dx a

for 0≤x ≤a and f T (x)=0 else.

n

a
Expectation: E(T n )=∫ x f T ( x )dx= nn ∫ x n dx= n a
n
a 0 n+1

n a
The estimator Tn is biased: bT = a−a=−
n
n+1 n+ 1
a is underestimated and the bias is asymptotically zero.

139
 a
Variance: E( X )=∫ x ² f T ( x )dx= nn ∫ x n+1 dx= n a ²
2
n
a 0 n+2
2 2
n n na
V (T n)=E(X 2)−E ( X )2= a ²− 2
a ²= 2
n+2 ( n+ 1) (n+2)( n+1)

It is better to take an unbiased estimator, for this we re-

move the bias and we get Wn:
n+1 a2
W n= T n with bW =0 and V (W n)=
n n
n( n+2)

New estimate of a with Wn : a≃18.4 minutes.

For the uncertainty Δa we will compare three methods.

-> With determine a confidence interval with the Cheby-

shev's inequality:
V (W n )
We apply the inequality to Wn : P(|W n−a|≥ϵ)≤
ϵ2
so P(|W n−a|≤ϵ)=P(−ϵ≤W n−a≤ϵ)

and P(ϵ≥a−W n≥−ϵ)=P(U n+ ϵ≥a≥W n−ϵ)

we set ϵ= √
V (W n)
α and we have:
1−P(W n−ϵ≤a≤W n+ ϵ)≤α
Eventually:

P(W n − √ V (W n)
α ≤ a ≤ W n + √
V (W n )
α ) ≥ 1−α

140
In the 90% confidence case α=0.1 and for our sample

V (W 12 )=
18.42
12×14
≃2.0 and Δ a=
V (W n )
α ≃4.5 . √
Then 13.9≤a≤22.9 and a≃18.4±4.5 minutes.

-> Let us determine a confidence interval with an integral

calculation on the probability density of the estimator.
We determine the probability density of Wn:
n n n
P(W n ≤x)=P ( W n≤ x)=P(T n≤ x)
n+ 1 n+1 n+1

n n x n
n+ 1
FW (x)=F T
n n ( n+ 1
x=)(
n+1 a ) if 0≤x ≤
n
a.

n
n n+ 1
f W (x)=
n ( a(n+1) )
n x n−1 if 0≤x ≤
n
a

and f W ( x)=0 else.

n

We have an asymmetric probability distribution whose

maximum corresponds to the upper bound. For a 90% con-
fidence we remove the 10% of the left distribution tail from
the confidence interval.
Thus we define amax and amin such as:
n+1
Upper bound: amax = a
n

141
 13 2
so amax =
12 ( )
×17≃19.95 and amax ≃20.0 minutes.

amax
Lower bound: ∫ f W ( x)dx=1−α
n
amin

By numerical calculation with a≃18.4 and n=12,

20

∫ fW 12
( x )dx=0.9 with amin ≃16.4 minutes.
amin

In conclusion, a≃18.4 +1.6 minutes

−2.0
with 90% confidence.
The interval found is asymmetrical. The bounds are in-
cluded into those of the Chebyshev's inequality and we
have here a more precise estimate of a.

-> Let us now perform a numerical simulation on a

spreadsheet. The ALEA() function of the spreadsheet pro-
vides real randomly and uniformly distributed between 0
and 1. Then we multiply by the point estimate of a to ob-
tain the distributions Xi=Ui(0,a). We generate 10,000 sam-
ples of size 12. We place the maximum of each of these
samples on a graph and thus have the sampling distribution
of Tn:

142
We thus find the same results as by the preceding method
(file : www.incertitudes.fr/livre/Train.ods).

a) We now use the estimator of the moment theorem:

T n=2 X n . Here, for large n, we can use the central limit
theorem because an affine function of a distribution gives a
new distribution of the same form. We estimate the mean
with the Gaussian law:

x̄=
3+12+ ...+ 11
12
≃9.83 and s=
11√
(3− x̄)2 +...
≃4.88

m=x̄±t ∞ s / √ n=9.8±2.3 with 90% confidence

Then a=19.7±4.6 minutes with 90% confidence.
In this case the value of n is only 12 and the distribution of
the population is not normal, we want nevertheless to show
how one would proceed for n large. As we will see with

143
 numerical simulation, the interval is thus underestimated.
Indeed by carrying out a numerical simulation with this
estimator, a=19.7±5.5 minutes with 90% confidence:

Comparison of the results of the different methods with

90% confidence:
Chebyshev's inequality
13.9 18.4 22.9
Maximum
Likelihood
Integrals and
Simulation 16.4 18.4 20.0

Central Limit Theorem

Method (sample too small) 15.1 19.7 24.3
of
Moments
Simulation
14.2 19.7 25.2

144
In conclusion, the maximum likelihood estimator converges
much faster than that of the moment theorem and we
prefer this first method. The variance converges to 1/n 2
instead of 1/n:

a2 a2
[ V (W n)] ML= n(n+2) and [ V (T n) ]MM = 3 n

70
Convergence of variances
60

V(n) Method of Moments

50

V(n) Maximum Likelihood

40

30

20

10

0
0 5 10 15 20 25 30

145
 D. Exercises

Exercise 1 : Estimators of the mean

Answers (Complete Edition)

Let consider the (X1, X2, X3). X1, X2 and X3 are three
independent variables with the same distribution,
expectation m and variance σ2.
Compare the following three proposed estimators to
estimate the mean m of the sample:
A3=(X1+X2+X3)/3, B3=(X1+2X2+3X3)/6 and
C3=(X1+2X2+X3)/3.

Exercise 2 : Homokinetic Beam

Answers (Complete Edition)

We consider a homokinetic beam of C+ ionized

carbon atoms. We measure the momentum and the
kinetic energy of 100 atoms of the beam. The beam is
considered perfectly unidirectional.
The total momentum magnitude and kinetic energy
are:
100
p=∑ mv i =2.418×10⁻21 kg.m/s and
i=1
100
1
Ek =∑ m v i2 =1.518×10⁻18 J with m=1.993⨉10-26 kg.
i=1 2

Let V be the probability distribution of the speed of

the ions. Determine the average speed vm, its variance
σV2 and the uncertainty Δv with 95% confidence, using
the sample taken and the appropriate estimators.

146
Exercise 3 : Two estimators
Answers (Complete Edition)

Let X be the following discrete random variable:

Values 0 1 2
Probabilities 3θ θ 1 - 4θ

1. What values of θ define a valid probability distribu-

tion?
2. Calculate E(X) and V(X).
3. Let (X1, X2,..., Xn) be a sample of X.
n
1
We have Xn= ∑ X and T n=a X n + b .
n i=1 i
Determine a and b for which Tn is an estimator without
bias for θ.
Determine V(Tn).
4. We now consider the random variable Yi defined for
all i∈⟦0 ; n⟧ by Yi=1 if Xi=1 and Yi=0 otherwise.
n
Let Z n =∑ Y i . Determine E(Zn).
i=1

Zn
Show that U n= is an unbiased estimator of θ. De-
n
termine V(Un).
5. We make estimates of θ with the following realiza-
tions:

Values 0 1 2
Frequencies 31 12 57

Estimate θ. Which estimator do you prefer?

147
Exercise 4 : Ballot boxes Answers (Complete Edition)

Two identical urns contain the same proportion p of

black balls. In the first ballot box we draw with
remplacement a sample of size n1 and we note P1 the
proportion of black balls in this sample. We perform
the same experiment for the second urn.
P1 + P2
We define T= and U=x P1+(1− x)P 2
2
with x ∈ ]0,1[
Show that T and U are two estimators of p.
Which one is the best ?
Determine the value of x to have the optimum
estimator.

Exercise 5 : Continuous variable

Answers (Complete Edition)

We consider the continuous

{
variable X with the following a
a+ 1
if x>1
density: f (x)= x
Where a is the parameter we
want to estimate (a>1). 0 otherwise
1. Check that f defines a valid probability density.
2. Calculate E(X).
3. Determine estimators of a by the method of
maximum likelihood and by the method of moments.
4. Give point estimates of a for the following
observations:
1.16 / 1.80 / 1.04 / 3.40 / 1.22 / 1.06 / 1.35 / 1.10.
5. Can we calculate V(X)?
6. Perform a numerical simulation of the law of X.
What can we guess about the biases and convergences
of the estimators found?

148
Exercise 6 : Linear density
Answers (Complete Edition)

Consider the following continuous random variable X:

{
a x +b if 0≤x≤1
f (x)=
0 otherwise
1. Express b as a function of a such that f defines a
probability distribution.
2. Calculate E(X) and V(X).
3. Determine an estimator Tn of a by the method of
moments. Discuss the properties of this estimator.
4. We draw a sample:
0.81 0.67 0.72 0.41 0.93 0.55 0.28 0.09 0.89

Determine a point estimate of a. How would we get an

interval estimate?

Exercise 7 : Estimators for the exponential law

Answers (Complete Edition)

The exercise of the previous chapter on page 124

provides the expression of the probability density of
the estimator Tn of λ obtained in the course:
nλ
n n λn −
x
f T (x)= n+1
e if x>0 and zero if not
(n−1)! x
n

1. Determine the expectation, bias, variance and the

mean square error of Tn.
n−1
2. Let W n= T n . Determine the expectation, bias,
n
variance and the mean square error of Wn.
3. Which estimator would you recommend for λ?

149
Exercise 8 : Decays Answers (Complete Edition)

The law of probability X of particle decay as a

function of time follows an exponential distribution
with parameter λ. We measure the lifetimes in
microseconds of a sample
2.2 3.32 1.93 0.88 4.36
of ten particles and we
7.1 0.65 0.53 4.69 0.31
want to deduce a point
estimate and an interval estimate of λ. We will use
different methods and comment on them.

1. With the central limit theorem could you estimate

the expectation m of X and its uncertainty with a
confidence of 90% (m : mean lifetime)? With the
propagation of uncertainties formula we then could
find the value of λ with its uncertainty. What do you
think about this estimate of λ?

2. Let Tn be the estimator of λ found during the lesson.

Like we shown previously this one is biased and then
n−1
we use W n= Tn .
n
Determine by an integral calculus the uncertainty on λ
with 90% confidence.

3. Find now this result by numerical simulation.

150
V. Measure with a ruler

Article published in the BUP [ii].

ABSTRACT
The measurement of a physical quantity by an acquisition
system induces because its resolution a discretization error.
We are here concerned with measuring a length with a
graduated ruler. This type of measure leads us to consider a
uniform continuous probability distribution. We then use a
convolution to determine the uncertainty with its confidence
of a sum of lengths. Finally, we generalize to the general
case of the calculation of uncertainties for independent
random variables using the error propagation formula.

INTRODUCTION
We want to measure lengths and evaluate uncertainties
as exactly as possible. Uncertainties about the measured
values and their sums. We have a ruler of 15cm graduated
to the millimeter and two sets of cards. The ruler is
assumed to be perfect and the cards of each game identical.

151
1. MEASURE OF THE LENGTH OF ONE CARD

We place the
graduation zero on
the left edge of the
card. On the right
edge we consider
the graduation clo-
sest to the edge.
The experimenter
does not read
between the gra-
duations. The thickness of the lines
which delimit a graduation is considered negligible
compared with the width of this graduation. We get thus
for the deck 1:

x 1=8.4±0.05 cm .

152
Concerning the cards of the second deck:
x 2=11 .2±0. 05 cm.

We accept a loss of information due to the resolution

δ =1 mm of the ruler. When we use these data later, all
values between x min =x m−δ / 2 and x min =x m−δ / 2 are
equally probable. We have to consider a continuous and
uniform random variable X. x is a realization of X. This
distribution of probability has a range E= x max − x min and
its density f  x  verify:
+∞
∫ f (x)dx=1
−∞

The probability to be between x and xdx is f  x  dx .

The result is necessarily between x min and x max : for
example x 1=8.4±0.05 cm with 100% of confidence, but
x 1=8.4±0.04 cm with a probability of 80%.

153
To characterize the spreading of a distribution we consider
the range E and the standard deviation σ whose definition
for a continuous law is:
2 2
V =σ =∫ (x−x m) f ( x ) dx ,

V is called the variance. For a uniform distribution:

σ=δ/ √ 12≃0.29 δ ,

and we have x=x m±σ with 58% confidence. The

standard deviation is an appropriate quantity to characterize
the width of a distribution. The range is defined by the
extreme values which may be unrepresentative or absurds.

2. LENGTH OF TWO CARDS PUT END TO END

We want to determine the uncertainty on x with

x= x 1 x 2 . If we plot x 2 as a function of x 1 the set of
the possible points forms a square domain. The set of
points such as x is constant is straight line segment of
slope -1 and intercept x : x 2 =−x 1 x . There is only one
case where x= x min then { x1 = x1 min ; x 2 =x 2 min } at point
A in the figure. However on all the segment [CD] we get
x=x m . We understand that the different values of x do
not have the same probability.

154
The probability density f of X is computed from those of
X1 and X2. For a sum of independent random variables the
result is given by a convolution [iii] :

{
x< x min ⇒ f ( x)=0
( x− xmin )
xmin <x<x m ⇒ f ( x)= 2
f ( x)=∫ f 1( y) f 2( x− y) dy ⇒ δ
( xmax − x)
x m< x< x max ⇒ f ( x)= 2
δ
x> x max ⇒ f ( x)=0

155
We then have a triangular probability distribution.
We obtain x=19 .6±0.1cm with 100% confidence, and
x=19 .6±0.055 cm with 80% confidence.

3. ANALOGY WITH THE THROW OF TWO DICE

For each die the six values are equally likely. Here the
law of probability is no longer continuous but discrete. We
launch two dice simultaneously, the sum of the values
obtained is between two and 12. In this case, there is no
equiprobability, a way to get two with a double one, two
ways to get three with one and two or two and one ... to get
seven we have the maximum of possibilities. We thus find a
triangular distribution.

4. LENGTH OF TWO CARDS OF THE SAME DECK

PUT END TO END

156
 The cards of a deck are supposed identical then if the
length of one of them is overestimated, it will be the same
for the second one. In this case the errors are added and
can not be compensated. For two different cards, the first
measure can be underestimated and the second
overestimated, and a compensation can occur. Here it is no
longer the case and when X =X i + X i ' we obtain a
uniform distribution of width 2 δ . Our random variables
are not independent.

For the deck 1:

x 1=8.4±0.04 cm⇒ x =2 x 1 =16 .8±0. 08 cm

with a confidence of 80%.

5. SUM OF N INDEPENDENT LENGTHS

N
We have X =∑ X i . Each length X i follow a
i=1
uniform distribution of width δ . For the sum of nine
independent random variables after iteration of the
calculation we obtain the following curve:

157
In this case we obtain x=x moy±0.11 cm with 80%
confidence. With 100% confidence: x=x moy±0. 45 cm ,
which leads us to consider domains where the probability
of presence of X is really negligible. An uncertainty of
0.45cm seems unnecessary while 99% of the cases were
already present with an uncertainty of 0.22cm.

Working with a confidence of 100% it is like considering

the range, it is additive for a sum of variables. The range is
proportional to N.

158
 80% 95% 99%
N=1 0.40  0.48  0.50 
2 0.55  0.78  0.90 
3 0.66  0.97  1.19 
4 0.75  1.12  1.41 
5 0.84  1.25  1.60 
6 0.92  1.38  1.76 
7 0.99  1.49  1.91 
8 1.06  1.59  2.05 
9 1.12  1.69  2.18 
10 1.2  1.8  2.3 
20 1.7  2.5  3.3 
50 2.6  4.0  5.2 
100 3.7  5.7  7.4 

But this approach does not take into account one thing: the
curve narrows around the mean when N increases. There is
another additive quantity: the variance. The standard
deviation, square root of the variance, is proportional to √N
and takes account of error compensations.

We obtain a bell curve. A statistical theorem, called the

limit central theorem, indicates that for N large the curve
tends to a Gaussian. The range of a Gaussian is infinite for
a finite standard deviation.

We summarize the evolution of the uncertainty on the

sum of N independent lengths measured with the same
resolution δ in a table. In italics, from N=10, these are
numerical simulations carried out on a computer by
generating random numbers.

159
 The results of the measurements are often given with
a confidence of 95%, which corresponds for a Gaussian to
an uncertainty of about 2σ .

6. OTHER APPLICATIONS

A runner wishes to measure his travel time. He has a

watch with a digital display. The watch shows that he starts
at 10 h 52 min and arrives at 11 h 11 min . The display is
at the minute, so he starts between 10 h 52 min 00 s and
10 h 52 min 59 s . Hence the date of departure is within the
interval t 1=10 h 52 min 30 s±30 s . The resolution is one
minute. The duration of the course is  t=t 2−t 1 . The
results remain true for a difference. We have N=2 and
 t=19min±47s with 95% confidence.

Same procedure if students measure an angular difference

with a goniometer. Each measurement is within a minute
of arc so the uncertainty of the angular difference is of 47
arc seconds with a confidence of 95%.

Seven persons are in an elevator. Its maximum load is

500kg. Their individual masses are measured with a
resolution of one kilogram. The total mass is 499kg. What
is the probability of being overloaded?

For N=7 the uncertainty reaches one kilogram with a

confidence of 80%. So there is a one out of ten chance for
the elevator to be overloaded.

160
 In the laboratory many measuring instruments have
digital displays. The resolution is define by the last digit.
But the overall uncertainty is much higher. It is necessary
to consult the operating instructions of each device.

CONCLUSION

The general approach consists in combining laws of

probabilities. The mathematical tool used is a change of
variables, then one or more integrations. For the measure
with a ruler described in this article, it was a sum of two
independent random variables and we obtained a
convolution.
If one wants to do a faster calculation, an analysis of
variance may be enough. We have a random variable X
that depends on N independent random variables X i :
X = f  X 1 , X 2 ,. .. , X i , .. . , X N  . We call σ i the
standard deviation of X i and σ that of X . For finite
σ i and small variations, we have the propagation of
standard deviations formula [iv]:
n
∂f 2 2
σ2 =∑ ( ) σi
i=1 ∂ xi

And, independently of the probability distributions, this

relation between the variances remains true. One can thus
give its result with an uncertainty for 2σ or 3σ .
Is there a similar formula with the confidences? Yes, but it
is approximate, it is the propagation of uncertainties
formula:

161
 n
∂f 2
 f 2=∑    xi 2 ,
i =1 ∂ xi

with x i= x i moy ± x i , f = f moy ± f and a constant

confidence. This formula is very useful and allows a quick
and reasonable calculation of the combined uncertainties.
Moreover, it is exact if the form of the distributions is the
same for X and the X i . For example, if the distributions
X i are Gaussian, any linear combination is also Gaussian.
We thus take account of compensations and avoid using the
n
formula  f =∑ ∣∂ f /∂ x i∣ x i which overestimates
i=1
the uncertainties, sometimes even with such excess that one
loses its physical sense. This last formula does not take into
account any compensation, we have the worst situation,
statistically unlikely. Here, for example, for N=100, one
would have an uncertainty of 50 δ , instead of 5.7 δ in
practice (95% confidence).
In this article we focused on the resolution of an
acquisition system that gives a discretization error. But one
can also consider systematic errors and random errors.
Here the ruler was supposed perfect, that means, accurate
and precise.

162
VI. Mathematical Tools

The mathematical tools used in chapter 1 are studied into

the high school scientific section. The partial derivatives used in
chapter 2 are taught during the first year of university but we
quickly understand the link with the derivatives seen at the high
school. It is at the end of the second chapter, with the use of
matrices for generalized regression, that we immerse ourselves in
university education. My goal is not to review or introduce all
these concepts, only a small recap that can be useful to solve the
exercises proposed.

A - Derivatives

1- Definition : f '  x=lim

 0
 f x − f  x 
 
For example if f(x)=x² :
 x2−x 2 =x 22 x 2 −x 2≃2 x  and f '(x)=2x.

On a graph the derivative corresponds to the slope of the curve at

each point.

2- Rules for basic and combined functions:

function f derivative f '  f

ax a (constant)  a x =∣a∣ x
x x
−1
  x  =∣ x −1∣ x
sin x  cos x  sin x =∣cos  x∣ x

163
 cos x −sin  x   cos x =∣sin  x∣ x
ex ex  e x =e x  x
ln  x  1/ x  ln  x = x /∣x∣
uv u ' v ' (u and v as functions of x)
uv u ' vv ' u (product rule)
u u ' v −v ' u
v 2 (quotient rule)
v
f g  x g '  x  f ' g x  (chain rule)

1 −1 1 −1− 1 1
• =x so  ' =−1 x =− 2 .
x x x
1 1
 x =x 2 then ( √ x )'= 1 x 2 = 1 .
−1
•
2 2√x
• sin  x 2 ' = x 2 ' cos x 2 =2 x cos x2 

B - Partial derivatives

A partial derivative of a function of several variables is its

derivative with respect to one of those variables, with the others
held constant. For example, consider the following function of
three variables:
f(x,y,z) = x² - 2z + xy
We can look at the variations of this function with respect to a
variable while considering the other constants. We proceed then
as for a derivative. So we have:

( ∂∂ xf ) y ,z
=2 x + y , ( ∂∂ fy )
x, z
=x and  
∂f
∂z x,y
=−2

The first expression is said "partial derivative of f with respect to

x" treating y and z like constants. ∂: curly d.

164
C - Taylor series
With the notion of derivative we have studied the first-order
behavior of a function around a point, here we refine to the
higher orders.
For every infinitely differentiable function and for ϵ≪1 we
have the following development in the neighborhood of a point:
2 3 n
f ( x 0+ϵ)=f ( x 0)+ϵ f ' (x 0 )+ ϵ f ' ' ( x0 )+ ϵ f (3)( x 0)+...+ ϵ f ( n)( x 0)+...
2 3! n!
The more we take high order terms the better is the
approximation. For example for f (x )=sin( x ) and x 0 =0 we
3 2
find: sin(ϵ)≃ϵ− ϵ , in the same way cos (ϵ)≃1− ϵ .
3! 2

Also: exp(ϵ)≃1+ ϵ ln (1+ϵ)≃ϵ (1+ϵ)α≃1+α ϵ

D - Integrals
+∞ 2

We show in maths that: ∫ −x

e dx=√ π .
x=−∞
For the standard normal distribution, we can verify that the mean
is zero and that the standard deviation is equal to 1:
x2 +∞
1 − 2 and
p  x =
2 
e
−∞
∫
μ= x⋅p(x)dx=0 because the

integral over a symmetric interval of an odd function is zero.

+∞ x2 +∞ x2
1 −2 1 −
σ =∫ x
2 2
e dx= ∫ (−x)(−x e 2 )dx then
−∞ √2π √2 π −∞
2 2
x +∞ x +∞
− − 2
2
σ √ 2 π=[−x e 2 +∞
−∞ ] −∫ (−1)e 2
dx=0+ √2 ∫ e− x dx
−∞ −∞
so the standard deviation equals one.

165
We have used an integration by parts:
b b
b
∫ u(x )v '(x )dx =[u(x )v (x)] −∫ u' (x )v (x)dx
a
a a

then we did an integration by substitution: x ' =x /  2 .

We can go further by calculating the skewness

+∞ +∞
μ3= ∫ x3⋅p(x)dx , the kurtosis μ 4=∫ x4⋅p(x)dx , and, in
−∞ −∞
+∞
general, the moments of order n μ n = ∫
−∞
x n⋅p( x) dx .
All these moments allow us to characterize a distribution.

For a Gaussian  3=0 (symmetric). If this coefficient is

negative the curve spreads to the left, if it is positive the curve
spreads to the right. For a Gaussian β2 =μ 4 / σ4 =3 . If this
coefficient is less than 3 the curve is more flat than a Gaussian.
For a binomial distribution : β2 =3−8 /n .

Integration by substitution:

−1 −1
Let u=g (x) be a new variable with g a continuous
function strictly monotonic on [a,b] and g the inverse function,
then:
b g−1 (b)

∫ f (x)dx= ∫ −1
f (g(u))g ' (u)du
a g (a)

166
E – Series
n

Binomial formula: (a+b ) =∑ n a b ()

n k n− k

k =0 k

Derivatives of geometric series:

∞
1
if |q|<1 then ∑ q k= 1−q
k =0
∞
1
we take the derivative with respect to q: ∑ k q k− 1= 2
k =1 (1−q )
∞
2
then ∑ k (k −1)q k− 2= 3 …
k =2 (1−q )
And finally we find the negative binomial formula:

∑ k (k −1)...(k−r+ 1)q k− r= r ! r+ 1
∞

k =r (1−q)
1 ∞
k q k −r
so
(1−q)
r+1 = ∑
k =r r
()
x
∞
xk
A definition of the exponential function: e =∑
k =0 k!

+∞
F - Gamma function Γ( x )= ∫ t x− 1 e−t dt
0
This function is an extension of the factorial to real and complex
number (except for 0, -1, -2...)22. We will use it for half-integer
numbers. We demonstrate with an integration by parts:
Γ( x+1)=x Γ( x) . Γ(1)=1 then for n integer
Γ(n+1)=n! . Moreover, Γ(1/ 2)= √ π allows to calculate the
function for the half-integers.

22For example π!≃7.2 .

167
VII. Answers to Exercises

Complete and High Quality Digital And Paper Book

with all the exercises corrected on www.lulu.com
To contact the author: [email protected]

Chapter I : Random Variable

E1 : Ages Exercise on page 37

n=15; mode=18; median=18; mean⋍18.333; geometric mean⋍18.303;
range=4; standard deviation⋍1.113; root mean square deviation⋍1.075;
mean deviation⋍0.844.

E2 : Card Game Exercise on page 37

Number of possible draws: 32x31x30x29x28 divided by the different
ways of arranging 5 cards 5x4x3x2x1=5! permutations then 201 376
hands (32 choose 5).
* Number of possible draws for a four aces hand: only one possiblity
for the 4 aces, and 28 possibilities for the fifth card, so 28 possible
hands (one hand = 120 possible draws).
Hence the probability p=28/201376=139 chances on a million=1 out
of 7192=0,014%.
* For a flush: each color has 8 cards. Number of ways to choose 5
8!
cards among 8 = 5!(8−5)! = (85) 5
= C 8 = 56.

There are 4 colors : 4x56 =224.

Hence the probability p = 224 / 201 376 = 1 out of 899 = 0,11%.

168
Chapitre II : Correlation and Independence

E1 : Correlations Exercise on page 86

1- a) x1=(-1-1-1+0+0+0+1+1+1)/9=0 also x2=0 and x3=0.

 1 = ∑9i =1  xi 1− x12 /9−1=  6x1/8 so
σ 1=σ 2=σ 3=√ 3 /2≃0.87 .
b)

c) r12=0 . r13=0 . r23=-1 . X1 and X2 are not correlated. The same for X1
and X3. X2 and X3 are dependents and totally correlated.
2- a) x1=0 and x2=0. σ 1≃1.22 and σ 2≃1.58 .
c) r12=0.904 . Quantities are positively correlated.
3- a) x1=0 and x2=0. σ 1≃1.73 and σ 2≃1.53 .
c) r12=0 . Quantities completely uncorrelated, do not forget that the
correlations sought here are linear. There is a correlation in the form
of a V.

2- b) 3- c)

3 3

2 2

1 1

0 0
-3 -2 -1-1 0 1 2 3 -3 -2 -1-1 0 1 2 3

-2 -2

-3 -3

169
E2 : Volumes Exercise on page 87

1-
V1=(100.1+100.0+99.9+100.0)/4 so V1 = 100.0 mL.

√
2 2 2 2
σ 1=
0.1 +0 +(−0.1) +0
4−1
=
√ 2
3
⋅0.1 mL then σ 1≃0.082mL

According to the central limit theorem and the distribution of the

population Gaussian: ΔV=t.σ/√n=3.18x0.0816/2
tddl=3; 95%=3.18 so ΔV1 ≃ 0.13 mL and ΔV1 /V1 ≃ 0.13/100
The pipette, withe a confidence of 95%, has a uncertainty of 0.13 mL,
so for 100 mL a percent uncertainty of 0.13%.

2-
V i =V i−V and
∑ [(V 1i−V 1)(V 2i−V 2 )]=∑ [ V^1i V^2i ]=0.1×0+0×0.1+(−0.1)×0+0×0.1=0
i i

∑ [ V 1i−V 1  V 2i −V 2 ]
i
by definition r 12=
∑  V 1 −V 1 ∑  V 2 −V 2
i 2 i 2

i i

so r12=0, the quantities are totally uncorrelated and

therefore independent.

3-
V={200.1 , 200.1 , 199.9 , 199.9}mL so V = 200 mL.

√
2 2 2 2
0.1 +0.1 +(−0.1) +(−0.1) 2
σV= = ⋅0.1mL
4 −1 √3
then σ V ≃0.115 mL and ΔV≃ 0.183 mL , ΔV /V ≃ 0.09%
4-
V(V1,V2) then we have:
2 2
∂V ∂V
 V 2=   V 12    V 22 = V 1 2 V 22
∂V1 V ∂V 2 V
2 1

and Δ V = √ 2. Δ V 1 ≃0.18 Same result as in question 3-.

170
VIII. Bibliography / Sources /
Softwares / Illustrations

Books
[vi] WONNACOTT. Introductory Statistics for Business and
Economics, 1972. 919 p.
SHELDON M. ROSS. A first course in probability.1976. 585 p.

Livres
[iv] PROTASSOV Konstantin. Probabilités et incertitudes dans
l'analyse des données expérimentales. Presses Universitaires de
Grenoble, 1999. 128 p.

[x] JOURNEAUX Roger. Traitement des mesures. Ellipses 2009 377p

[iii] SAPORTA Gilbert. Probabilités, analyse des données et statistique.
Technip, 2006. 622 p.
RONDY SYLVAIN. Maths - Économique et Commerciale option Scientifique
- 2e année. Collection Prépas Sciences, ellipse, 2014. 810 p.

Web
Place http:// in front of the site name. Most files are
copied to the folder <www.incertitudes.fr/livre/>.

ROUAUD Mathieu. Calculs d'incertitudes. <www.

incertitudes.fr/Incertitudes.html>

Online integral calculator: <www.integral-calculator.com>

171
 [v] Nombres, mesures et incertitudes. mai 2010.
<www.educnet.education.fr/rnchimie/recom/mesures_incertitudes.pdf>

[vii] Évaluation des données de mesure — Guide pour

l’expression de l’incertitude de mesure. <www.bipm.org/utils/
common/documents/jcgm/JCGM_100_2008_F.pdf>, 2008. 132p.

BREUIL P, DI BENEDETTO D. Incertitude et étalonnage.2000. 16p.

[ix] Gauss–Newton algorithm - Wikipédia
(26 February 2017)

Articles

[i] Regression line analysis K.S.KRANE et L.SCHECTER

American J. of Physics 50,82 (1982).
[ii] ROUAUD Mathieu. Mesure avec une règle. Bulletin de
l'Union des Physiciens n°913, avril 2009.

Softwares
All the software used are free and open source.
• Word processing and spreadsheet: LibreOffice
• Graphics: Gimp (points), Inkskape (vectors) and Blender
(3D).
• Computation: Scilab (numerical), XCAS (symbolic) et
PHP programs (on server).
• Plotters: KmPlot, TeXgraph.
• Operating System: Ubuntu.

172
Illustrations

3D representations in front of the table of contents and third thumbnail

of the back cover.

A random walker moves in a plane. He throws two coins and looks at

the results: two tails, one tail and one head, one head and one tail or
two heads. The first coin tells him if he has to make a first move
towards the East or the West, the second if he has to make a second
step towards the South or the North. Thus at each time interval Δt he
moves in the plane of two steps. During Δt the distance traveled by the
walker is Δd = 2 p (p length of one step).

At what distance from the starting point is the walker at the instant t?
(after n time intervals: t = n Δt)

For n=1, we draw the following tables:

0.5 1 1 25% 25%

-0.5 1 1
-0.5 0.5 25% 25%

The center of the table is its starting point. On the abscissa x (direction
East-West) the displacement is more or less one step (x=±p , p= Δd /2
and we have fixed Δd=1). Similarly on the y-axis: y=±p. In each
square is indicated the number of possibilities to meet at this place.
The second table indicates the probabilities (1/4, 4=2x2).
In the four possibilities he is at a distance √2/2 from the point of origin.
And, in terms of standard deviation, the characteristic distance is
s= √ (4×1/ √ 2) /(4−1)≃0.816 .
2

For n=2, we draw the following tables:

1 1 2 1 6% 13% 6%
0 2 4 2 13% 25% 13%
-1 1 2 1
6% 13% 6%
-1 0 1

173
For example to be at (x=0 ; y=-1), there are two possible paths:
(PF,PP) and (FP,PP). Standard deviation s2 ≃1.033 .
The walker has a one in four chance to be back to the starting point.

For n=3, we draw the following tables:

1.5 1 3 3 1 2% 5% 5% 2%
0.5 3 9 9 3 5% 14% 14% 5%
-0.5 3 9 9 3 s3 ≃1.234
5% 14% 14% 5%
-1.5 1 3 3 1
2% 5% 5% 2%
-1.5 -0.5 0.5 1.5

For n=4, we draw the following tables:

2 1 4 6 4 1 0.4% 1.6% 2.3% 1.6% 0.4%

1 4 16 24 16 4
1.6% 6.3% 9.4% 6.3% 1.6%
0 6 24 36 24 6
2.3% 9.4% 14% 9.4% 2.3%
-1 4 16 24 16 4
-2 1 4 6 4 1 1.6% 6.3% 9.4% 6.3% 1.6%

-2 -1 0 1 2 0.4% 1.6% 2.3% 1.6% 0.4%

For n=5, we draw the following tables:

2.5 1 5 10 10 5 1 0.1% 0.5% 1.0% 1.0% 0.5% 0.1%

1.5 5 25 50 50 25 5
0.5% 2.4% 4.9% 4.9% 2.4% 0.5%
0.5 10 50 100 100 50 10
1.0% 4.9% 9.8% 9.8% 4.9% 1.0%
-0.5 10 50 100 100 50 10
1.0% 4.9% 9.8% 9.8% 4.9% 1.0%
-1.5 5 25 50 50 25 5
0.5% 2.4% 4.9% 4.9% 2.4% 0.5%
-2.5 1 5 10 10 5 1
0.1% 0.5% 1.0% 1.0% 0.5% 0.1%
-2.5 -1.5 -0.5 0.5 1.5 2.5

174
For n=6, we draw the following tables:

3 1 6 15 20 15 6 1 0.0% 0.1% 0.4% 0.5% 0.4% 0.1% 0.0%

2 6 36 90 120 90 36 6
0.1% 0.9% 2.2% 2.9% 2.2% 0.9% 0.1%
1 15 90 225 300 225 90 15
0.4% 2.2% 5.5% 7.3% 5.5% 2.2% 0.4%
0 20 120 300 400 300 120 20
-1 15 90 225 300 225 90 15
0.5% 2.9% 7.3% 9.8% 7.3% 2.9% 0.5%
-2 6 36 90 120 90 36 6 0.4% 2.2% 5.5% 7.3% 5.5% 2.2% 0.4%
-3 1 6 15 20 15 6 1 0.1% 0.9% 2.2% 2.9% 2.2% 0.9% 0.1%
-3 -2 -1 0 1 2 3 0.0% 0.1% 0.4% 0.5% 0.4% 0.1% 0.0%

Hence the evolution of the quadratic mean distance from the starting
point as a function of time:

t (Δt) 0 1 2 3 4 5 6
√t 0.00 1.00 1.41 1.73 2.00 2.24 2.45
s (2p) 0.000 0.816 1.033 1.234 1.417 1.582 1.732

We plot the curves s as a function of t and s as a function of √t:

2.0 2.0

1.5 1.5

1.0 1.0

0.5 0.5

0.0 0.0
0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0

We find a much better correlation in √t. Indeed we saw directly that the
distance at the origin did not evolve proportionally to time, for n=2 we
are about one unit of the origin, so we should be towards 3 for n=6.
This variation in √t is characteristic of diffusion phenomena and here
finds its analogy with the compensation of errors in √n.

175
IX. TABLES / Index z2
1 −2
f (z)= e
A. Standard normal distribution √2 π
z

F ( z)=∫ f ( z )dz=P (Z ⩽ z) P (Z > z)=1− P( Z⩽ z)

−∞
P (Z ⩽−z)=P (Z > z) Example : P (Z ⩽1.67)≃0.95254
z 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09
0.0 0.50000 0.50399 0.50798 0.51197 0.51595 0.51994 0.52392 0.52790 0.53188 0.53586
0.1 0.53983 0.54380 0.54776 0.55172 0.55567 0.55962 0.56356 0.56749 0.57142 0.57535
0.2 0.57926 0.58317 0.58706 0.59095 0.59483 0.59871 0.60257 0.60642 0.61026 0.61409
0.3 0.61791 0.62172 0.62552 0.62930 0.63307 0.63683 0.64058 0.64431 0.64803 0.65173
0.4 0.65542 0.65910 0.66276 0.66640 0.67003 0.67364 0.67724 0.68082 0.68439 0.68793
0.5 0.69146 0.69497 0.69847 0.70194 0.70540 0.70884 0.71226 0.71566 0.71904 0.72240
0.6 0.72575 0.72907 0.73237 0.73565 0.73891 0.74215 0.74537 0.74857 0.75175 0.75490
0.7 0.75804 0.76115 0.76424 0.76730 0.77035 0.77337 0.77637 0.77935 0.78230 0.78524
0.8 0.78814 0.79103 0.79389 0.79673 0.79955 0.80234 0.80511 0.80785 0.81057 0.81327
0.9 0.81594 0.81859 0.82121 0.82381 0.82639 0.82894 0.83147 0.83398 0.83646 0.83891
1.0 0.84134 0.84375 0.84614 0.84849 0.85083 0.85314 0.85543 0.85769 0.85993 0.86214
1.1 0.86433 0.86650 0.86864 0.87076 0.87286 0.87493 0.87698 0.87900 0.88100 0.88298
1.2 0.88493 0.88686 0.88877 0.89065 0.89251 0.89435 0.89617 0.89796 0.89973 0.90147
1.3 0.90320 0.90490 0.90658 0.90824 0.90988 0.91149 0.91309 0.91466 0.91621 0.91774
1.4 0.91924 0.92073 0.92220 0.92364 0.92507 0.92647 0.92785 0.92922 0.93056 0.93189
1.5 0.93319 0.93448 0.93574 0.93699 0.93822 0.93943 0.94062 0.94179 0.94295 0.94408
1.6 0.94520 0.94630 0.94738 0.94845 0.94950 0.95053 0.95154 0.95254 0.95352 0.95449
1.7 0.95543 0.95637 0.95728 0.95818 0.95907 0.95994 0.96080 0.96164 0.96246 0.96327
1.8 0.96407 0.96485 0.96562 0.96638 0.96712 0.96784 0.96856 0.96926 0.96995 0.97062
1.9 0.97128 0.97193 0.97257 0.97320 0.97381 0.97441 0.97500 0.97558 0.97615 0.97670
2.0 0.97725 0.97778 0.97831 0.97882 0.97932 0.97982 0.98030 0.98077 0.98124 0.98169
2.1 0.98214 0.98257 0.98300 0.98341 0.98382 0.98422 0.98461 0.98500 0.98537 0.98574
2.2 0.98610 0.98645 0.98679 0.98713 0.98745 0.98778 0.98809 0.98840 0.98870 0.98899
2.3 0.98928 0.98956 0.98983 0.99010 0.99036 0.99061 0.99086 0.99111 0.99134 0.99158
2.4 0.99180 0.99202 0.99224 0.99245 0.99266 0.99286 0.99305 0.99324 0.99343 0.99361
2.5 0.99379 0.99396 0.99413 0.99430 0.99446 0.99461 0.99477 0.99492 0.99506 0.99520
2.6 0.99534 0.99547 0.99560 0.99573 0.99585 0.99598 0.99609 0.99621 0.99632 0.99643
2.7 0.99653 0.99664 0.99674 0.99683 0.99693 0.99702 0.99711 0.99720 0.99728 0.99736
2.8 0.99744 0.99752 0.99760 0.99767 0.99774 0.99781 0.99788 0.99795 0.99801 0.99807
2.9 0.99813 0.99819 0.99825 0.99831 0.99836 0.99841 0.99846 0.99851 0.99856 0.99861
3.0 0.99865 0.99869 0.99874 0.99878 0.99882 0.99886 0.99889 0.99893 0.99896 0.99900
3.1 0.99903 0.99906 0.99910 0.99913 0.99916 0.99918 0.99921 0.99924 0.99926 0.99929
3.2 0.99931 0.99934 0.99936 0.99938 0.99940 0.99942 0.99944 0.99946 0.99948 0.99950
3.3 0.99952 0.99953 0.99955 0.99957 0.99958 0.99960 0.99961 0.99962 0.99964 0.99965
3.4 0.99966 0.99968 0.99969 0.99970 0.99971 0.99972 0.99973 0.99974 0.99975 0.99976
3.5 0.99977 0.99978 0.99978 0.99979 0.99980 0.99981 0.99981 0.99982 0.99983 0.99983
3.6 0.99984 0.99985 0.99985 0.99986 0.99986 0.99987 0.99987 0.99988 0.99988 0.99989
3.7 0.99989 0.99990 0.99990 0.99990 0.99991 0.99991 0.99992 0.99992 0.99992 0.99992
3.8 0.99993 0.99993 0.99993 0.99994 0.99994 0.99994 0.99994 0.99995 0.99995 0.99995
3.9 0.99995 0.99995 0.99996 0.99996 0.99996 0.99996 0.99996 0.99996 0.99997 0.99997
4.0 0.99997 0.99997 0.99997 0.99997 0.99997 0.99997 0.99998 0.99998 0.99998 0.99998

176
 B. Student's t-values

Student
value Confidence (%)
t 50 80 90 95 98 99 99.5 99.8 99.9
1 1.00 3.08 6.31 12.7 31.8 63.7 127 318 637
2 0.82 1.89 2.92 4.30 6.96 9.92 14.1 22.3 31.6
3 0.76 1.64 2.35 3.18 4.54 5.84 7.45 10.2 12.9
4 0.74 1.53 2.13 2.78 3.75 4.60 5.60 7.17 8.61
5 0.73 1.48 2.02 2.57 3.36 4.03 4.77 5.89 6.87
Degrees of freedom (sample size minus the number of parameters)

6 0.72 1.44 1.94 2.45 3.14 3.71 4.32 5.21 5.96

7 0.71 1.41 1.89 2.36 3.00 3.50 4.03 4.79 5.41
8 0.71 1.40 1.86 2.31 2.90 3.36 3.83 4.50 5.04
9 0.70 1.38 1.83 2.26 2.82 3.25 3.69 4.30 4.78
10 0.70 1.37 1.81 2.23 2.76 3.17 3.58 4.14 4.59
11 0.70 1.36 1.80 2.20 2.72 3.11 3.50 4.02 4.44
12 0.70 1.36 1.78 2.18 2.68 3.05 3.43 3.93 4.32
13 0.69 1.35 1.77 2.16 2.65 3.01 3.37 3.85 4.22
14 0.69 1.35 1.76 2.14 2.62 2.98 3.33 3.79 4.14
15 0.69 1.34 1.75 2.13 2.60 2.95 3.29 3.73 4.07
16 0.69 1.34 1.75 2.12 2.58 2.92 3.25 3.69 4.01
17 0.69 1.33 1.74 2.11 2.57 2.90 3.22 3.65 3.97
18 0.69 1.33 1.73 2.10 2.55 2.88 3.20 3.61 3.92
19 0.69 1.33 1.73 2.09 2.54 2.86 3.17 3.58 3.88
20 0.69 1.33 1.72 2.09 2.53 2.85 3.15 3.55 3.85
22 0.69 1.32 1.72 2.07 2.51 2.82 3.12 3.50 3.79
24 0.68 1.32 1.71 2.06 2.49 2.80 3.09 3.47 3.75
26 0.68 1.31 1.71 2.06 2.48 2.78 3.07 3.43 3.71
28 0.68 1.31 1.70 2.05 2.47 2.76 3.05 3.41 3.67
30 0.68 1.31 1.70 2.04 2.46 2.75 3.03 3.39 3.65
40 0.68 1.30 1.68 2.02 2.42 2.70 2.97 3.31 3.55
50 0.68 1.30 1.68 2.01 2.40 2.68 2.94 3.26 3.50
60 0.68 1.30 1.67 2.00 2.39 2.66 2.91 3.23 3.46
70 0.68 1.29 1.67 1.99 2.38 2.65 2.90 3.21 3.44
80 0.68 1.29 1.66 1.99 2.37 2.64 2.89 3.20 3.42
90 0.68 1.29 1.66 1.99 2.37 2.63 2.88 3.18 3.40
100 0.68 1.29 1.66 1.98 2.36 2.63 2.87 3.17 3.39
200 0.68 1.29 1.65 1.97 2.35 2.60 2.84 3.13 3.34
300 0.68 1.28 1.65 1.97 2.34 2.59 2.83 3.12 3.32
500 0.67 1.28 1.65 1.96 2.33 2.59 2.82 3.11 3.31
1000 0.67 1.28 1.65 1.96 2.33 2.58 2.81 3.10 3.30
∞ 0.67 1.28 1.64 1.96 2.33 2.58 2.81 3.09 3.29

177
 C. Chi-square values

Probability α to reject the hypothesis while it is true (%)

χ2 99 90 75 50 30 20 10 5 2 1 0,1
1 0.000157 0.0158 0.102 0.455 1.07 1.64 2.71 3.84 5.41 6.63 10.8
2 0.0201 0.211 0.575 1.386 2.41 3.22 4.61 5.99 7.82 9.21 13.8
3 0.115 0.584 1.21 2.366 3.66 4.64 6.25 7.81 9.84 11.3 16.3
4 0.297 1.06 1.92 3.357 4.88 5.99 7.78 9.49 11.7 13.3 18.5
5 0.554 1.61 2.67 4.351 6.06 7.29 9.24 11.1 13.4 15.1 20.5
6 0.872 2.20 3.45 5.348 7.23 8.56 10.6 12.6 15.0 16.8 22.5
7 1.24 2.83 4.25 6.346 8.38 9.80 12.0 14.1 16.6 18.5 24.3
8 1.65 3.49 5.07 7.344 9.52 11.0 13.4 15.5 18.2 20.1 26.1
9 2.09 4.17 5.90 8.343 10.7 12.2 14.7 16.9 19.7 21.7 27.9
10 2.56 4.87 6.74 9.342 11.8 13.4 16.0 18.3 21.2 23.2 29.6
11 3.05 5.58 7.58 10.34 12.9 14.6 17.3 19.7 22.6 24.7 31.3
12
number of degrees of freedom

3.57 6.30 8.44 11.34 14.0 15.8 18.5 21.0 24.1 26.2 32.9
13 4.11 7.04 9.30 12.34 15.1 17.0 19.8 22.4 25.5 27.7 34.5
14 4.66 7.79 10.2 13.34 16.2 18.2 21.1 23.7 26.9 29.1 36.1
15 5.23 8.55 11.0 14.34 17.3 19.3 22.3 25.0 28.3 30.6 37.7
16 5.81 9.31 11.9 15.34 18.4 20.5 23.5 26.3 29.6 32.0 39.3
17 6.41 10.1 12.8 16.34 19.5 21.6 24.8 27.6 31.0 33.4 40.8
18 7.01 10.9 13.7 17.34 20.6 22.8 26.0 28.9 32.3 34.8 42.3
19 7.63 11.7 14.6 18.34 21.7 23.9 27.2 30.1 33.7 36.2 43.8
20 8.26 12.4 15.5 19.34 22.8 25.0 28.4 31.4 35.0 37.6 45.3
21 8.90 13.2 16.3 20.34 23.9 26.2 29.6 32.7 36.3 38.9 46.8
22 9.54 14.0 17.2 21.34 24.9 27.3 30.8 33.9 37.7 40.3 48.3
23 10.2 14.8 18.1 22.34 26.0 28.4 32.0 35.2 39.0 41.6 49.7
24 10.9 15.7 19.0 23.34 27.1 29.6 33.2 36.4 40.3 43.0 51.2
25 11.5 16.5 19.9 24.34 28.2 30.7 34.4 37.7 41.6 44.3 52.6
26 12.2 17.3 20.8 25.34 29.2 31.8 35.6 38.9 42.9 45.6 54.1
27 12.9 18.1 21.7 26.34 30.3 32.9 36.7 40.1 44.1 47.0 55.5
28 13.6 18.9 22.7 27.34 31.4 34.0 37.9 41.3 45.4 48.3 56.9
29 14.3 19.8 23.6 28.34 32.5 35.1 39.1 42.6 46.7 49.6 58.3
30 15.0 20.6 24.5 29.34 33.5 36.3 40.3 43.8 48.0 50.9 59.7
31 15.7 21.4 25.4 30.34 34.6 37.4 41.4 45.0 49.2 52.2 61.1
32 16.4 22.3 26.3 31.34 35.7 38.5 42.6 46.2 50.5 53.5 62.5
33 17.1 23.1 27.2 32.34 36.7 39.6 43.7 47.4 51.7 54.8 63.9
34 17.8 24.0 28.1 33.34 37.8 40.7 44.9 48.6 53.0 56.1 65.2
35 18.5 24.8 29.1 34.34 38.9 41.8 46.1 49.8 54.2 57.3 66.6
36 19.2 25.6 30.0 35.34 39.9 42.9 47.2 51.0 55.5 58.6 68.0
37 20.0 26.5 30.9 36.34 41.0 44.0 48.4 52.2 56.7 59.9 69.3
38 20.7 27.3 31.8 37.34 42.0 45.1 49.5 53.4 58.0 61.2 70.7
39 21.4 28.2 32.7 38.34 43.1 46.2 50.7 54.6 59.2 62.4 72.1
40 22.2 29.1 33.7 39.34 44.2 47.3 51.8 55.8 60.4 63.7 73.4

178
Index
Absolute zero.................................................................................62
Accuracy........................................................................................33
Aging............................................................................................104
Arithmetic mean..............................................................................2
Asymptotes.....................................................................................95
Ballot boxes..................................................................................148
Bernoulli distribution...........................................................102, 120
Bias...............................................................................................126
Binomial distribution....................................................102, 120, 121
Binomial formula.........................................................................167
Box-Muller transform..................................................................119
Cauchy's equation...........................................................................89
Central limit theorem.............................................................10, 138
Chebyshev's inequality.........................................................137, 140
Chi 2...............................................................................................76
Chi-squared distribution.......................................................113, 123
Chi-squared test.............................................................................30
Class interval..............................................................................6, 38
Coefficient de dissymétrie............................................................101
Confidence Interval............................................................12, 61, 95
Convolution..................................................................107, 151, 155
Correlation coefficient....................................................................47
Cumulative distribution function..........................................104, 115
Decay...........................................................................................150
Decomposition into Gaussians.......................................................99
Degrees of freedom........................................................................13
Derivatives of geometric series....................................................167
Diffusion phenomena....................................................................175
Discretization error......................................................................151
Error of the first kind.....................................................................26
Error of the second kind................................................................26
Estimator......................................................................................126
Expectation.....................................................................................20
Exponential distribution...............................................109, 122, 131

179
Frequency.........................................................................................6
Function of a continuous distribution...........................................115
Gamma function...........................................................................167
Gaussian distribution................................................................10, 19
Gaussian distribution 3D................................................................42
Geometric distribution.........................................................103, 121
Geometric mean...............................................................................2
Homokinetic Beam......................................................................146
Hypothesis test...............................................................................24
Integral.........................................................................................165
Integration by parts......................................................................166
Integration by substitution............................................................166
Interval estimate...........................................................................137
Inverse distribution.......................................................................124
Inverse transform sampling..........................................................118
Inverse transformation method.....................................................118
Kurtosis........................................................................101, 113, 166
Least squares method...............................................................58, 75
Likelihood....................................................................................133
Line spectrum.................................................................................89
Linear density...............................................................................149
Linear regression............................................................................58
Linearization..................................................................................67
Mean deviation...........................................................................4, 37
Mean Square Error.......................................................................127
Memoryless property....................................................................104
Method of Maximum Likelihood.................................................133
Method of Moments.....................................................................129
Moment................................................................................101, 166
Negative binomial distribution.....................................................121
Nonlinear regression.................................................................75, 80
Normal distribution......................................................................110
Numerical simulation...................................................................118
Parabolic regression.......................................................................78
Poisson distribution..............................................................105, 122
Polynomial regression.....................................................................77

180
Power of the test.............................................................................29
Precision.........................................................................................33
Prediction.......................................................................................62
Prediction Interval..............................................................12, 62, 95
Prism..............................................................................................89
Probability density function............................................................19
Product of distributions................................................................123
Propagation of standard deviations formula.................................161
Propagation of uncertainties formula.....................................52, 161
Random errors..............................................................................162
Range.......................................................................................3, 153
Refractive index..............................................................................88
Repeatability..................................................................................33
Reproducibility...............................................................................33
Residual....................................................................................59, 76
Resolution..............................................................................33, 151
Sample standard deviation................................................................3
Sampling distribution.....................................................................10
Skewness......................................................................101, 113, 166
Small variations method...........................................................73, 98
Student's t-distribution.........................................................110, 122
Student's t-value.............................................................................60
Sum of binomial distributions......................................................121
Sum of exponentials.....................................................................124
Sum of Gaussians.........................................................................122
Sum of independent random variables.........................................101
Sum of Student's t-distributions...................................................123
Taylor series.........................................................................105, 165
Thermal conductivity......................................................................90
Triangular distribution..................................................................156
Uncertainty.....................................................................................13
Uncertainty calculations.................................................................53
Uniform distribution....................................................107, 122, 153
Variance.................................................................20, 101, 154, 159
Waiting time.................................................................................138

181