[1]

Monte Carlo
Sampling Methods

Jasmina L. Vujic
Nuclear Engineering Department
University of California, Berkeley
Email: [email protected]
phone: (510) 643-8085
fax: (510) 643-9685

UCBNE, J. Vujic
 [2]

Monte Carlo
Monte Carlo is a computational technique based on
constructing a random process for a problem and
carrying out a NUMERICAL EXPERIMENT by N-fold
sampling from a random sequence of numbers with a
PRESCRIBED probability distribution.

x - random variable
N

∑x
1
x̂ = ---- i
N
i=1

X̂ - the estimated or sample mean of x

x - the expectation or true mean value of x

If a problem can be given a PROBABILISTIC

interpretation, then it can be modeled using RANDOM
NUMBERS.
UCBNE, J. Vujic
 [3]

Monte Carlo
• Monte Carlo techniques came from the complicated diffusion
problems that were encountered in the early work on atomic energy.
• 1772 Compte de Bufon - earliest documented use of random
sampling to solve a mathematical problem.
• 1786 Laplace suggested that π could be evaluated by random
sampling.
• Lord Kelvin used random sampling to aid in evaluating time
integrals associated with the kinetic theory of gases.
• Enrico Fermi was among the first to apply random sampling
methods to study neutron moderation in Rome.
• 1947 Fermi, John von Neuman, Stan Frankel, Nicholas Metropolis,
Stan Ulam and others developed computer-oriented Monte Carlo
methods at Los Alamos to trace neutrons through fissionable
materials

UCBNE, J. Vujic
 Monte Carlo
[4]

Monte Carlo methods can be used to solve:

a) The problems that are stochastic (probabilistic) by

nature:
- particle transport,
- telephone and other communication systems,
- population studies based on the statistics of
survival and reproduction.

b) The problems that are deterministic by nature:

- the evaluation of integrals,
- solving the systems of algebraic equations,
- solving partial differential equations.

UCBNE, J. Vujic
 Monte Carlo
[5]

Monte Carlo methods are divided into:

a) ANALOG, where the natural laws are PRESERVED

- the game played is the analog of the physical
problem of interest
(i.e., the history of each particle is simulated
exactly),

b) NON-ANALOG, where in order to reduce required

computational time the strict analog simulation of
particle histories is abounded (i.e., we CHEAT!)
Variance-reduction techniques:
- Absorption suppression
- History termination and Russian Roulette
- Splitting and Russian Roulette
- Forced Collisions
- Source Biasing
UCBNE, J. Vujic
 Example 1: Evaluation of Integrals
[6]

fmax
f(x)
y1 • reject

• accept
a x1 b
b
I =
∫ f( x )dx - area uder the function f(x),R = ( b – a )f
a
max - area of rectangle

I
P = ---- - is a probability that a random point lies under f(x), thus I = RP
R

Step 1: Choose a random point (x1,y1): x1 = a + ( b – a )ξ 1 and y1 = f max ξ 2

Step 2: Check if y1 ≤ f ( x1 ) - accept the point, if y1 < f ( x1 ) - reject the point

Step 3: Repeat this process N times, Ni - the number of accepted points

Ni Ni
Step 4: Determine P = ------ and the value of integral I = R ------
N N

UCBNE, J. Vujic
 Major Components of Monte Carlo
[7]

Major Components of a Monte Carlo Algorithm

• Probability distribution functions (pdf’s) - the physical (or
mathematical) system must be described by a set of pdf’s.
• Random number generator - a source of random numbers uniformly
distributed on the unit interval must be available.
• Sampling rule - a prescription for sampling from the specified pdf,
assuming the availability of random numbers on the unit interval.
• Scoring (or tallying) - the outcomes must be accumulated into
overall tallies or scores for the quantities of interest.
• Error estimation - an estimate of the statistical error (variance) as a
function of the number of trials and other quantities must be
determined.
• Variance reduction techniques - methods for reducing the varinace
in the estimated solution to reduce the computational time for Monte
Carlo simulation.
• Parallelization and vectorization - efficient use of advanced
computer architectures.

UCBNE, J. Vujic
 Probability Distribution Functions
[8]

Probability Distribution Functions

Random Variable, x, - a variable that takes on particular val-

ues with a frequency that is determined by some underlying
probability distribution.

Continuous Probability Distribution

P{a ≤ x ≤ b}

Discrete Probability Distribution

P { x = xi } = pi

UCBNE, J. Vujic
 Probability Distribution Functions
[9]

PDFs and CDFs (continuous)

Probability Density Function (PDF) - continuous

• f(x), f ( x )dx = P { x ≤ x' ≤ x + dx }
∞
• 0 ≤ f(x) , ∫ f ( x )dx = 1
–∞
b f(x)
• Probability{ a ≤ x ≤ b } =
∫ f( x )dx
a
x→
Cumulative Distribution Function (CDF) - continuous
x

• F(x) =
∫ f( x' )dx' = P{ x' ≤ x } 1
–∞

• 0 ≤ F(x) ≤ 1 F(x)
d
• 0≤ F(x) = f(x)
dx 0
b x→
•
∫ f( x' )dx' = P{ a ≤ x ≤ b } = F( b ) – F( a )
a
UCBNE, J. Vujic
 Probability Distribution Functions
[10]

PDFs and CDFs (discrete)

Probability Density Function (PDF) - discrete

• f(xi), f ( x i ) = p i δ ( x – x i )

• 0 ≤ f ( xi ) p3

• ∑ f( x )( ∆x ) = 1 or ∑ p
i
i i
i
i = 1 f(x)
p1
p2

x1 x2 x3
Cumulative Distribution Function (CDF) - discrete
F(x)
• F(x) = ∑ p = ∑ f( x )∆x
xi < x
i
xi < x
i i p1+p2+p3 1

p1+p2
• 0 ≤ F(x) ≤ 1 p1
•
x1 x2 x3

UCBNE, J. Vujic
 Monte Carlo & Random Sampling
[11]

Sampling from a given discrete distribution

Given f ( x i ) = p i and ∑p
i
i = 1 , i = 1, 2, ..., N

0 p1 p1+p2 p1+p2+p3 p1+p2+...+pN=1

and 0 ≤ ξ ≤ 1 , then P ( x = x k ) = p k = P ( ξ ∈ d k ) or

k–1 k

∑p ≤ ξ < ∑p
i i
i=1 i=1

UCBNE, J. Vujic
 Monte Carlo & Random Sampling
[12]

Sampling from a given continuous distribution

If f(x) and F(x) represent PDF and CDF od a random variable x, and if ξ is a random
number distributed uniformly on [0,1] with PDF g( ξ )=1, and if x is such that

F(x) = ξ

than for each ξ there is a corresponding x, and the variable x is distribute according
to the probability density function f(x).

Proof:

For each ξ in ( ξ , ξ +∆ ξ ), there is x in (x,x+∆x). Assume that PDF for x is q(x). Show
that q(x) = f(x):

q(x)∆x = g( ξ )∆ ξ =∆ ξ = ( ξ +∆ ξ )- ξ = F(x+∆x)- F(x)

q(x) = [F(x+∆x)- F(x)]/∆x = f(x)

Thus, if x = F-1( ξ ), then x is distributed according to f(x).

UCBNE, J. Vujic
 Monte Carlo & Random Sampling
[13]

Monte Carlo Codes

Categories of Random Sampling

• Random number generator uniform PDF on [0,1]
• Sampling from analytic PDF’s normal, exponential, Maxwellian, .....
• Sampling from tabulated PDF’s angular PDF’s, spectrum, cross sect

For Monte Carlo Codes...

• Random numbers, ξ , are produced by the R.N. generator on [0,1]
• Non-uniform random variates are produced from the ξ ’s by
— direct inversion of CDFs
— rejection methods
— transformations
— composition (mixtures)
— sums, products, ratios, .....
— table lookup + interpolation
— lots (!) of other tricks .....
• < 10% of total cpu time (typical)

UCBNE, J. Vujic
 Random Sampling Methods
[14]

Random Number Generator

Pseudo-Random Numbers
• Not strictly "random", but good enough 1
— pass statistical tests f(x)
— reproducible sequence
0 1
• Uniform PDF on [0,1]
• Must be easy to compute, must have a large period 1

Multiplicative congruential method F(x)

• Algorithm
S0 = initial seed, odd integer, < M 0 1

Sk = G • Sk-1 mod M, k = 1, 2, .....

ξ k = Sk / M

• Typical (vim, mcnp):

Sk = 519 • Sk-1 mod 248
ξ k = Sk / 248
period = 246 ≈ 7.0 x 1013
UCBNE, J. Vujic
 Random Sampling Methods
[15]

Direct Sampling (Direct Inversion of CDFs)

Direct Solution of
–1
x̂ ← F ( ξ )

Sampling Procedure: 1
• Generate ξ ξ
• Determine x̂ such that F ( x̂ ) = ξ F(x)

0
x→ x̂
Advantages
• Straightforward mathematics & coding
• "High-level" approach

Disadvantages
• Often involves complicated functions
• In some cases, F(x) cannot be inverted (e.g., Klein-Nishina)

UCBNE, J. Vujic
 Random Sampling Methods
[16]

Rejection Sampling
Used when the inverse of CDF is costly ot impossible to find.
Select a bounding function, g(x), such that
• c ⋅ g ( x ) ≥ f ( x ) for all x
• g(x) is an easy-to-sample PDF

Sampling Procedure:
–1
• sample x̂ from g(x): x̂ ← g ( ξ 1 )
cg ( x )
• test: ξ 2 ⋅ cg ( x̂ ) ≤ f ( x̂ )

if true accept x̂ , done • reject

if false reject x̂ , try again f(x)
6

• accept
Advantages
x→
• Simple computer operations
Disadvantages
• "Low-level" approach, sometimes hard to understand

UCBNE, J. Vujic
 Random Sampling Methods
[17]

Table Look-Up
Used when f(x) is given in a form of a histogram

f1 fi
f(x) f2

x1 x2 xi-1 xi
F(x)
ξ

x1 x2 xi-1 x xi

Then by linear interpolation

( x – x i – 1 )F i + ( x i – x )F i – 1 [ ( x i – x i – 1 )ξ – x i F i – 1 + x i – 1 F i ]
F ( x ) = --------------------------------------------------------------------- , x = ----------------------------------------------------------------------------------
xi – xi – 1 Fi – Fi – 1

UCBNE, J. Vujic
 Random Sampling Methods
[18]

Sampling Multidimensional Random Variables

If the random quantity is a function of two or more random variables that are
independent, the joint PDF and CDF can be written as

f ( x, y ) = f 1 ( x )f 2 ( y )

F ( x, y ) = F 1 ( x )F 2 ( y )

EXAMPLE: Sampling the direction of isotropically scattered particle in 3D

Ω = Ω ( θ, ϕ ) = Ω x i + Ω y j + Ω z k = v + w + u ,
dΩ sin θdθdϕ – d ( cos θ )dϕ – dµdϕ
-------- = -------------------------- = -------------------------------- = ------------------
4π 4π 4π 4π

1 1
f ( Ω ) = f 1 ( µ )f 2 ( ϕ ) = --- ------
2 2π
µ
∫
1
F1 ( µ ) = f 1 ( µ' ) dµ' = --- ( µ + 1 ) = ξ 1 or µ = 2ξ 1 – 1
–1 2
ϕ ϕ
F1 ( ϕ ) =
0 ∫f 2 ( ϕ' ) dϕ' = ------ = ξ 2 , or ϕ = 2πξ 2
2π

UCBNE, J. Vujic
 Random Sampling Methods
[19]

Probability Density Function Direct Sampling Method

Linear: f(x) = 2x , 0<x<1 x← ξ
(L1, L2)
–x
Exponential: f(x) = e , 0<x x ← – log ξ
(E)
2D Isotropic: 1 , u ← cos 2πξ 1
f ( ρ ) = ----- ρ = ( u, v )
(C) 2π v ← sin 2πξ 1

3D Isotropic: u ← 2 ξ1 – 1
1
(I1, I2) f ( Ω ) = ------ , Ω = ( u, v, w ) 2
4π v ← 1 – u cos 2πξ 2
2
w ← 1 – u sin 2πξ 2

Maxwellian: x π
f ( x ) = ----------- --- e –x / T, 0<x x ← T ( – log ξ 1 – log ξ 2 cos2 --ξ 3)
2
(M1, M2, M3)
T π T
2

Watt – a b ⁄ 4 –x ⁄ a w ← a ( – log ξ 1 – log ξ 2 cos2 π-- ξ 3)

f ( x ) = ---------------- 0<x
2e 2
Spectrum: e sinh bx ,
(W1, W2, W3) πa b3
a b
2
2
x ← w + --------- + ( 2 ξ 4 – 1 ) a bw
4

Normal: 1  x – µ
2
– - -----------
- x ← µ + σ – 2 log ξ 1 cos 2πξ 2
1
------------- e  σ 
(N1, N2)
f(x) =
2

σ 2π
UCBNE, J. Vujic

← ξ
 Random Sampling Examples
[20]

Example — 2D Isotropic
1 ,
f ( ρ ) = ----- ρ = ( u, v )
2π

Rejection (old vim)

SUBROUTINE AZIRN_VIM( S, C )
IMPLICIT DOUBLE PRECISION (A-H, O-Z)
100 R1=2.*RANF() - 1.
R1SQ=R1*R1
R2=RANF()
R2SQ=R2*R2
RSQ=R1SQ+R2SQ s
IF(1.-RSQ)100,105,105
105 S=2.*R1*R2/RSQ
C=(R2SQ-R1SQ)/RSQ
RETURN c
END

Direct (racer, new vim)

subroutine azirn_new( s, c )
implicit double precision (a-h,o-z)
parameter ( twopi = 2.*3.14159265 )
phi = twopi*ranf()
c = cos(phi)
s = sin(phi)
return
end
UCBNE, J. Vujic
 Random Sampling Examples
[21]

Example — Watt Spectrum

– a b ⁄ 4 –x ⁄ a
2e
( x ) = ---------------- e sinh bx , 0<x
πa b 3

Rejection (mcnp)
• Based on Algorithm R12 from 3rd Monte Carlo Sampler, Everett & Cashwell

• Define K = 1 + ab ⁄ 8 , L = a { K + ( K 2 – 1 )1 / 2 } , M = L ⁄ a – 1

• Set x ← – log ξ 1 , y ← – log ξ 2

• If { y – M ( x + 1 ) } 2 ≤ bLx , accept: return (Lx)

otherwise, reject

Direct (new vim)

• Sample from Maxwellian in C-of-M, transform to lab
w ← a ( – log ξ 1 – log ξ 2 cos2 π-- ξ 3)
2
2
a b 2 (assume isotropic emission from fission fragment
x ← w + --------- + ( 2ξ 4 – 1 ) a bw
4 moving with constant velocity in C-of-M)
• Unpublished sampling scheme, based on original Watt spectrum derivation
UCBNE, J. Vujic
 Random Sampling Examples
[22]

Example — Linear PDF

f(x) = 2x , 0≤x≤1

Rejection
(strictly — this is not "rejection", but has the same flavor)

if ξ 1 ≥ ξ 2 , then x̂ ← ξ 1
else x̂ ← ξ 2

or
x̂ ← max ( ξ 1, ξ 2 )

or
x̂ ← ξ 1 – ξ 2

Direct Inversion

F( x ) = x2 , 0≤x≤1

← ξ

UCBNE, J. Vujic
 Random Sampling Examples
[23]

Example — Collision Type Sampling

Assume (for photon interactions):

µ tot = µ cs + µ fe + µ pp

Define
µ cs µ fe µ pp
p 1 = --------
- , p = --------- , and p = ---------
µ tot 2 µ tot 3 µ tot

3
with
∑p i = 1.
i=1

Then
0 p1 p1+p2 p1+p2+p3=1

Collision event: Photoefect

p1 < ξ < p1 + p2

UCBNE, J. Vujic
 [24]

How Do We Model A Complicated Physical System?

a) Need to know the physics of the system

b) Need to derive equations describing physical
processes
c) Need to generate material specific data bases
(cross sections for particle interactions, kerma-
factors, Q-factors)
d) Need to “translate” equations into a computer
program (code)
e) Need to "describe" geometrical configuration of
the system to computer
f) Need an adequate computer system

g) Need a physicist smart enough to do the job and

dumb enough to want to do it.

UCBNE, J. Vujic
 Transport Equation
[25]

Time-Dependent Particle Transport Equation

(Boltzmann Transport Equation):

1∂
--- ψ ( r, E, Ω, t ) + Ω • ∇ψ ( r, E, Ω, t ) + Σ t ( E )ψ ( r, E, Ω, t ) =
v∂t

∞ 
∫
0 4π
s ∫
 dE' Σ ( E' → E, Ω' → Ω )Ψ ( r, E', Ω', t ) dΩ' +
 


∞ 
χ(E) 
4π ∫ 0
∫
------------  dE' υΣ f ( E' )Ψ ( r, E', Ω', t ) dΩ' +
4π 

1
------Q ( r, E, t )
4π

UCBNE, J. Vujic