Performance of Computer System

Random Number Generation

Dr. Nima Afraz

[email protected]
http://nima.ie/

School of Computer Science,

University College Dublin

Week-4

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 1 / 28
 Random Arrival - Bank Example

1
People vector created by macrovector - www.freepik.com
Dr. Nima Afraz [email protected] Performance of(School
http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 2 / 28
 Random Number

Why?
Simulation must generate random values for variables in a specified
random distribution, such as normal, exponential distribution
How? Two steps:
random number generation: generate a sequence of uniform
Full-period random numbers in [0,1]
random variate generation: transform a uniform random sequence
to produce a sequence with the desired distribution
▶ A random variate generation algorithm is any program that halts and
exits with a real number x. This x is called a random variate.

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 3 / 28
 Generate random numbers with a given distribution
Given a random number generator (1 ≦ x ≦ 11) with a uniform
distribution the probability density function (PDF) is:

Integral of p(x) – P(x): cumulative distribution function (CDF).

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 4 / 28
 Generate random numbers with a given distribution
PDF

CDF:

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 5 / 28
 How Random Number Generators Work

Most commonly use recurrence relation

xn = f (xn−1 , xn−2 ...)
recurrence is a function of last 1 (or a few numbers), e.g.
xn = (5xn−1 + 1) mod 16
Example: For x0= 5, first 32 numbers are
10, 3, 0, 1, 6, 15, 12, 13, 2, 11, 8, 9, 14, 7, 4, 5, 10, 3, 0, 1, 6, 15,
12, 13, 2, 11, 8, 9, 14, 7, 4, 5
xn are integers in [0, 16) !
Dividing by 16, get random numbers in interval [0, 1]

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 6 / 28
 Properties of pseudo-random number sequences

Function f is deterministic
Given the seed, we can tell with a probability of 1 what the numbers
in the sequence would be.
Reproducibility (often desirable)
Be able to repeat a simulation experiment
They pass statistical tests for randomness
Statistics Test
Contains no recognisable patterns or regularities
▶ Dice roll

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 7 / 28
 Pseudo-random and fully random numbers

Pseudo-random generator generated sequence is not truly random,

because it is completely determined by an initial value
Careful mathematical analysis is required to have any confidence that
a Pseudo-random generates numbers that are sufficiently close to
random to suit the intended use
The digits in the decimal expansions of irrational number s such as π,
e (the natural-logarithm base), or the square roots of numbers that
are not perfect squares are believed by some mathematicians to be
truly random.

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 8 / 28
 Random Number Sequences

Some generators do not repeat the initial part of a sequence

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 9 / 28
 Desired Properties of a Good Generator

Efficiently computable
Since simulations typically require several thousand random numbers
in each run, the processor time required to generate these numbers
should be small.
Period should be large
A small period may cause the random-number sequence to recycle,
resulting in a repeated event sequence. This may limit the useful
length of simulation runs.
Successive values should be
The correlation between successive numbers should be small.
Correlation, if significant, indicates dependence
Independent
Uniformly distributed

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 10 / 28
 Linear-Congruential Generators

1951: D.H. Lehmer found that residues of successive powers of a number

have good randomness

Many of the currently used random-number generators are a generalisation

of the Lehmer’s proposal

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 11 / 28
 Mixed Linear-Congruential Generators

Lehmer’s generator: multiplicative LCG

Modern generalization: mixed LCG
▶ xn = (axn−1 + b) mod m
▶ a, b, m > 0
Result: xn are integers in [0, m − 1]
Popular because
▶ Analysed easily
▶ Certain guarantees can be made about their properties
Question: The choice of a, b, and m affects the sequence?

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 12 / 28
 Linear-Congruential Generators: Examples
Desirable: Long Cycle, short tail, low M (less resources needed to
generate)

xn = (axn−1 + b) mod M

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 13 / 28
 Properties of LCGs
Choice of a, b, m affects
▶ Period
▶ Autocorrelation
What is autocorrelation?
▶ To detect non-randomness in data.
▶ To identify an appropriate time series model if the data are not random.
▶ Autocorrelation is a correlation coefficient. However, instead of
correlation between two different variables, the correlation is between
two values of the same variable at times xi and xi + k.
▶ Range of values from +1 to -1. A value of 0 indicates that there is no
association
▶ Given measurements, y1 , y2 , ..., yn at time x1 , x2 , ..., xn , the lag k
autocorrelation function is defined as
PN−k
i=1 (Yi −Ȳ )(Yi+k −Ȳ )
rk = PN 2
i=1 (Yi −Ȳ )

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 14 / 28
 Properties of LCGs

Observations about LCGs

▶ period can never be more than m –> modulus m should be large
▶ m = 2k yields efficient implementation by truncation
If b is non-zero, obtain full period of m if and only if
▶ m and b are relatively prime – no common factors
▶ every prime that is a factor of m is also a factor of a − 1
▶ if m is a multiple of 4, a − 1 must be too
All of these conditions are met if
▶ m = 2k, for some integer k
▶ a = 4c + 1, for some integer c
▶ b is an odd integer

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 15 / 28
 LCGs – Good Practice

In practice, we often want random integers in a small range, say

between 0 and 9. Typically we do so by using a large M in a linear
congruential random number generator, and then use arithmetic to
bring them down into a small range.
For example: a = 11, b = 37, c = 1, m = 100
▶ Range 100: 1 48 65 52 9 36 33 0 37 44 21 68 85 72 29 56 53 20 57 64
41 88 5 92 49 76 73 40 77 84 61 8 25 12 69 96 93 60 97 4 81 28 45 32
89 16 13 80 17 24 1 48 65 52 9 36 33 0 37 44 21 68 85 72 29 56 53 20
57 64 41 88 5 92 49 76 73 40 77 84 61 8 25 12 69 96 93 60 97 4 81 28
45 32 89 16 13 80 17 24
▶ Most left digit: 0 4 6 5 0 3 3 0 3 4 2 6 8 7 2 5 5 2 5 6 4 8 0 9 4 7 7 4
7860216996908243811812
▶ Most right digit: 1 8 5 2 9 6 3 0 7 4 1 8 5 2 9 6 3 0 7 4 1 8 5 2 9 6 3
0 7 4 1....

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 16 / 28
 Properties of LCGs – Continue...

Full-period generator = one with period m

▶ Not all are equally good
▶ Lower Autocorrelation between adjacent elements is better

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 17 / 28
 Example: Two Candidate LCGs

Examples

Both must be full period generators

▶ m = 2k, for some integer k
▶ a = 4c + 1, for some integer c
▶ b is an odd integer
Autocorrelation for the first one is 0.25, for the second one is 2−18

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 18 / 28
 Multiplicative LCGs

More efficient than mixed LCGs: no addition

Two classes: m = 2k and m ! = 2k

xn = an mod m

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 19 / 28
 Multiplicative LCG with m = 2k

Most efficient LCG: mod = truncation

Not full-period: maximum possible period for m = 2k is 2k−2
▶ only possible if multiplier a = 8i + / − 3 and x0 (initial seed) is odd
▶ consider
xn = 5n mod 25
xn = 7n mod 25
If 2k−2 period suffices, may use multiplicative LCG for efficiency !

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 20 / 28
 Multiplicative LCG with m ! = 2k

Avoid small period of LCG when m ! = 2k : use prime modulus

Full period generator with proper choice of a, let a to be a primitive
root of the modulus m
▶ when a is primitive root of m, v= an mod m, v has different values for
n = 1, 2, ..., m-2
Consider
▶ xn = 3n mod 31:
1, 3, 9, 27, 19, 26, 16, 17, 20, 29, 25, 13, 8, 24, 10, 30, 28, 22, 4, 12,
5, 15, 14, 11, 2, 6, 18, 23, 7, 21, 1...
▶ xn = 5n mod 31:
1, 5, 25, 1...
Note: 53 mod 31 = 125 mod 31 = 1

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 21 / 28
 Primitive Root Table

For example, 31: the primitive roots are 3, 11, 12, 13, 17, 21, 22, 24
Dr. Nima Afraz [email protected] Performance of(School
http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 22 / 28
 Randomness Test

Goal: To ensure that the random number generator produces a random

stream.
Goodness of Fit Test: such as Chi-square, t-test
Autocorrelation
Ljung-Piece-Box – Autocorrelation
Durbin Watson –Autocorrelation
Wald-Wolfowitz

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 23 / 28
 Randomness Test

Plot histograms —Uniform distribution

Plot quantile-quantile plot — Linear relationship with Uniform

distribution

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 24 / 28
 Randomness Test

Chi-square Goodness of Fit Test:

Chi-Square goodness of fit test is a non-parametric test that is used
to find out how the observed value of a given phenomena is
significantly different from the expected value.
Can be used for any distribution
Compare observed frequencies with theoretical
X has a chi-square distribution with k-1 degrees of freedom

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 25 / 28
 Chi-square Test

1000 random numbers with x0 = 1

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 26 / 28
 Chi-square Test
Check table

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 27 / 28
 Chi-square Test

1000 random numbers with x0 = 1 chi-square[0.9;9]=14.68

Dr. Nima Afraz [email protected] Performance of(School

http://nima.ie/ Computer
of Computer
System Science, UniversityWeek-4
College Dublin) 28 / 28