UNCERTAINTIES-I

CE 766 L ECTURES 13

Riddhi Singh
Figure from: Wilby and Desai (2010), Weather Email: [email protected]
In this module, we will learn about:

• Sources of uncertainties in decision analysis for

watershed management
• Quantification methods: Monte Carlo, fuzzy methods
• Decision making in the presence of uncertainty:
overview of methods

2
SOURCES OF UNCERTAINTY

3
There are various sources of uncertainty in a
typical management problem
• R: relationships or models that relate actions to impacts may
have uncertainties associated with them
– The system we are analysing may change with time
• L: levers or actions may have uncertainties related to their
implementation, even if an optimal management strategy is
identified
• M: sometimes, we may be uncertain about our relative
preferences between multiple conflicting objectives, or we
may be uncertain how our measures will change with time

4
Generally, uncertainty is linked to risk

Risk involves an ‘exposure to a chance of injury or loss’ (Random House, 1966)

Image: http://www.ukrainianweb.com/images/chernobyl/chernobyl_zone.jpg 5
Risk is associated with both natural and man made
disasters

Image: http://www.nhc.noaa.gov/gifs/1992andrew2.gif 6
Risk is the potential of losing something of value.

The ISO 31000 (2009) / ISO Guide 73:2002 definition of risk is the 'effect of
uncertainty on objectives'.

http://en.wikipedia.org/wiki/Risk
Image http://accelerateddevelopment.ca/blog_images/RiskAndUncertainty.png 7
Uncertainty can enter into our analysis in
several ways
• Statistical variation (random error in measurement)
• Subjective judgement (systematic)
• Linguistic imprecision
• Inherent randomness (process randomness)
• Disagreement (conflicting information)
• Approximation

Morgan and Henrion, 1990

8
Uncertainty classification

Aleatory Epistemic
Random Systematic
Uncertainty inherent in Uncertainty due to incomplete
stochastic phenomenon knowledge or incomplete
measurements about a
phenomenon
Cannot be reduced with time May increase or decrease with
time
Can be represented by Monte Fuzzy logic
Carlo method using specified
probability distribution
9
Example: sources of uncertainty in hydrologic modelling
(after Beven, 2012):

1. Initial and boundary conditions including model inputs

2. Model structure
3. Model parameter estimates
4. Uncertainties that have been overlooked: known
omissions and unknown unknowns
5. Data uncertainties (rainfall, PET, temperature etc.)

10
Example. Uncertainty sources in assessing impact
of climate change in water resources
Emission scenarios
(future CO2
concentration)

Legend
Return flows
Withdrawal General
AG Agriculture circulation
H/I Household/Industrial
E Energy model
Ec Ecological Services
ET Evapo-transpiration
P Precipitation
SW Surface Water
GWs Shallow Ground Water
GWd Deep Ground Water Downscaling

Hydrologic
model

Indicators of
hydrologic
response
11
Can we explore all possibilities?

Emission Scenarios Uncertainty about

(Future CO2 future economy
concentration)

Parametric Uncertainty General Model structural

Circulation uncertainty
Initial conditions Model

Local Circulation Uncertainty in

Patterns – Local relating global
Climate processes to local
processes

Hydrologic Model structural

Parametric Uncertainty
Model uncertainty

Indicators of
Hydrologic
Response 12
UNCERTAINTY QUANTIFICATION

13
Tackling uncertainty: many ways

1. Analytical approach: error propagation in relatively

simple models
2. Scenario analysis
3. Probability based methods: Monte Carlo analysis for
complex models
– Assessing confidence/prediction intervals: Bootstrap
4. Fuzzy set theory

Developing a structured approach to identify which

method to use when is crucial!
14
 We will cover two methods that are a part of
this tree

Beven, 2008 15
 Image: http://en.wikipedia.org/wiki/File:Pi_30K.gif

MONTE-CARLO METHODS

16
Recall: random variables
A random variable is a variable whose value
depends on the outcome of an event X1 =5
X2 =1
X3 =6
Flip the dice 5 times X4 =1
X5 =4
Note down the values X6 = ?
on the upper face. .
.
.

Random variable X = outcome of an event

Event = throwing the dice
17
Image source: http://www.clker.com/cliparts/6/4/1/6/12456404511653839274badaman_dice.svg.med.png
Though you cannot predict the exact value at the next
throw, you can associate a probability with each value

P(X=1) = 1/6
P(X=2) = 1/6
P(X=3) = 1/6
P(X=4) = 1/6
P(X=5) = 1/6
P(X=6) = 1/6

What did we assume in estimating these probabilities?

Image source: http://www.clker.com/cliparts/6/4/1/6/12456404511653839274badaman_dice.svg.med.png 18

In this way, a Probability Mass Function (or PMF) can be
developed for any random variable
PMF (X)

1/6

0
1 2 3 4 5 6
Random variable X

In order to develop the PMF, you have to know or assume the

underlying processes leading to the event (in this case, a fair dice)
Image source: http://www.clker.com/cliparts/6/4/1/6/12456404511653839274badaman_dice.svg.med.png 19
For continuous random variables, PMF becomes PDF
(probability density function)

PDF (X)

1 2 3 4 5 6
Random variable X

Example: a random variable that can take any value between 1 and 6
with equal probability
20
Examples of probability distributions- Normal,
Lognormal, Beta, Poisson, Binomial, and many more…

NORMAL Poisson

b
P(a  X  b)   P ( x)dx P( X  a)  pa
a

X can be a continuous variable – taking any value between feasible ranges.

Or, it can be a discrete variable where it can take only specific values. 21
CDF or cumulative distribution function is the area
under the PDF curve

PDF CDF

Area = F(X)

F(x)   P( )d


b

P(a  X  b)   P(x)dx  F(b)  F(a)

22
a
Table of commonly available continuous distributions

Distribution PDF CDF

Normal or Gaussian 1 
( x   )2
1  x   
2 2 
2   
e 1 erf
 2   2 
(ln x   )2
Lognormal 1  1 1  ln x   
e 2
2
 erf  
x 2 2 2   2 

(   )  1 (   )
Beta
x
x (1  x)  1
( )(  ) 
( )(  ) 0
x 1 (1  x)  1

  1 
Student’s-t 
Too long…!
 1
 2 
 2  1  x  2
 
23
   
    
2
Monte Carlo methods (or Monte Carlo
experiments) are a broad class of
computational algorithms that rely on repeated
random sampling to obtain numerical results;
typically one runs simulations many times over
in order to obtain the distribution of an
unknown probabilistic entity.

Source: http://en.wikipedia.org/wiki/Monte_Carlo_method24
The basic idea of Monte-Carlo is to relate the true statistic of a
random variable with its sample statistic

N (0,1)

Y  f ( x) Y  f ( x)
Mean of true
population
 ? ˆ n Mean of samples

25
The basic idea of Monte-Carlo is to relate the true statistic of a
random variable with its sample statistic

N (0,1)

Y  f ( x) Y  f ( x)

1 n
  f ( x) p ( x)dx, E (Y )  ˆ n   f ( xi ) p ( xi )

n i 1
1 n
  E (Y )  ˆ n   Yi
n i 1
26
Monte-Carlo works because of the strong and
weak laws of large numbers


P lim ˆ n    0  1
n 


lim P  ˆ n       0
n 

This law states that as the number of samples increases the absolute error
between the sample mean and the true population mean goes to zero

27
RANDOM SAMPLING

28
Random sampling is the process of drawing random samples from a
given distribution.

You assume the distribution of the variable, then select a set

of number to represent the distribution

29
Thankfully, MATLAB has inbuilt functions to easily generate random
samples from a distribution!

NORMAL LOGNORMAL

>> normal_x = random('normal',0,1,100,1);

>> histfit(normal_x,sqrt(length(normal_x)), 'normal’)
>> lognorm_x = random(’logn',0,1,100,1);
>> histfit(lognorm_x,sqrt(length(lognorm_x)), 'lognormal')
30
Plotting CDFs in MATLAB is equally easy

PDF CDF
x

F(x)   P( )d



P(a  X  b)   P(x)dx  F(b)  F(a)

a

>> x=random('normal',0,1,100,1);
>> histfit(x,sqrt(length(x)), 'normal')
31
>> cdfplot(x,'black',2)
We learnt how to generate random samples for a single variable, but
doing the same for multiple variables requires specification of MORE than
the distributions of the variables

Variable name
Distribution Range
(MATLAB name)

Variable A
Uniform [0, 1]
(varA)

Variable B
(varB) Uniform [0, 1]

For simplicity, let us consider only 2 variables.

We can easily (given we have enough computing resources) extend the methods for 2
variables for any number of variables.
32
Plotting one variable against other will help:

This is UNIFORM RANDOM SAMPLING (URS)

We simply sampled each variable according to its distribution and combined
them together to form a matrix.
33
Easy as it may look, it is not the most efficient way to sample the space
spanned by the two parameters

You may end up loosing out on the extremes or important regions of the
space. 34
Sampling techniques that are an improvement over the standard URS:

1. Grid sampling: divide each variable’s range into ‘N’

segments, select one value from the middle of each
segment. Generate each combination of possible values
of VarA and VarB leading to NxN samples

2. Latin hypercube sampling: Grid sampling can be very

computationally expensive. LHS tries to combine the
space coverage features of grid sampling with
computational efficiency of URS.

35
Sampling techniques that are an improvement over the standard URS:

0.8
Variable B
(varB)

0.6

0.4

0.2

0 0.2 0.4 0.6 0.8 1

Variable A
(varA)

This exhaustive sampling ensures that all regions of the space are covered.
But...
36
Exponential growth in number of samples required with variables for
exhaustive grid sampling:
Number of Number of samples per Total number of
variables variable required samples
2 10 10x10 = 100
3 10 1000
4 10 10000
5 10 10^5
6 10 10^6
7 10 10^7
8 10 10^8

Clearly, this type of sampling is limited by computational power. Even for

relatively small number of variables, if we want a fine coverage of the space, a
large number of parameters will be needed!

37
Stratified sampling attempts to overcome the limitations of URS (not
sampling certain regions of the space) by implementing an intelligent
sampling scheme.

0.8
Variable B
(varB)

0.6

0.4

0.2

0 0.2 0.4 0.6 0.8 1

Variable A
(varA)

Break up the space into separate regions and select points randomly from each
region 38
Latin Hypercube Sampling (LHS) is a typical type of stratified sampling

0.8
Variable B
(varB)

0.6

0.4

0.2

0 0.2 0.4 0.6 0.8 1

Variable A
(varA)

Each row or column corresponding to a cell has only 1 entry.

39
How do we sample distributions other than uniform?

1. Method 1: Sample using uniform distribution, assign weights

to each parameter set based on their joint probability

2. Method II: Sample such that more number of samples are

selected from the region of space with higher probability

40
Sampling a joint normal distribution using Method 1/II

41