See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/228970502

Random Number Generator Recommendation

Article · January 1997

CITATIONS READS
6 577

2 authors, including:

P.J. Burns
Colorado State University
48 PUBLICATIONS   574 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Building energy Analysis View project

All content following this page was uploaded by P.J. Burns on 21 March 2015.

The user has requested enhancement of the downloaded file.

 Random Number Generator Recommendation
by
Charles N. Zeeb and Patrick J. Burns
Colorado State University
Department of Mechanical Engineering
Fort Collins, CO 80523
[email protected] and [email protected]
(970) 491-7479 and (970) 491-5778

The generation of uniform pseudo-random numbers between 0 and 1 is important in

many numerical simulations. The purpose of this report is to explore the best genera-
tor(s) of such random numbers in terms of statistical properties and speed. While
attempting to find the best generator in general, the specific goal of this report is to find
the best generator for Latin hypercube sampling [Iman and Shortencarier, 1984]. This
report starts by giving some background on uniform random numbers and the three
basic types of generators: linear congruential, lagged Fibonacci, and combined. Next,
some general coding issues are covered. Then some random number generators, the
MCNP linear congruential generator and some implementations of the addition lagged
Fibonacci generators, are tested in several ways to assess the quality of the random num-
bers they generate. This is followed by timing results are given for several different gen-
erators. Finally, conclusions and recommendations are given.

1 Uniform Random Numbers and Random Bits

For most applications, the type of random sequence desired is Rn, a sequence of ran-
dom real numbers uniformly distributed between 0 and 1. Due to the finite accuracy to
which a computer can represent a real number, what is actually generated is a series of
integers, Xn, uniformly distributed between 0 and m, the modulus. The random num-
bers, Rn, are then generated by the following formula:

R n = Xn ⁄ m (1)

m is usually 2N where N is often related to the number of binary bits in the integer vari-
able type used to store Xn.

For some applications, not only is it important for the sequence Rn to be random, but
also for the sequences of each individual bit (binary digit) representing Xn to be random.
This is referred to as the bitwise behavior of the random number generator (RNG).

2 Basic Generators
There are three general classes of pseudo-random number generators in popular use:
linear congruential generators, lagged Fibonacci generators, and combination genera-

1
tors. All three will be discussed briefly below. There are several good overviews of this
topic. The classic reference is Knuth [1981]. A particularly brief overview is given by
Marsaglia [1985]. More complete overviews are given by Anderson [1990] and Brent
[1992].

2.1 Linear Congruential Generators

Linear congruential generators, also referred to as multiplicative generators, are very

popular. They are also the oldest in common use, having been proposed by D. H. Leh-
mer in 1949 [Knuth, 1981]. Besides Knuth [1981], a particularly good source of informa-
tion is Park and Miller [1988]. A linear congruential generator is determined by the
following four integer values [Knuth, 1981]:

m the modulus m>0

a the multiplier 0≤a<m
c the increment 0 ≤ c< m
X0 the starting value 0 ≤ X0 < m
The sequence is given by:

X n + 1 = ( aX n + c )modm n≥0 (1)

where mod is the integer remainder function (the integer remainder of the value in
parenthesis divided by m.) In the notation of Burns and Pryor [1995], these generators
are designated as LCG (a, c, m, X0). Multiplicative congruential generators are divided
into three broad classes [Anderson, 1990]:

1. m = 2N, c > 0
2. m = 2N, c = 0
3. m = prime, c = 0.

While class 2 is the fastest and easiest to implement, Class 3 is the only class for which
the individual low order bits are random. Choice of the values of m, a, and c are critical.
While some choices give good results, others give horrible results [Knuth, 1981; Park
and Miller, 1988; Brent, 1992].

There are three particular problems with LCG’s. One serious flaw is that plotting n-
tuples of consecutive random numbers generated by the LCG display an obvious lattice
structure. An example of this is given in Figure 1 which shows the result of plotting the
consecutive pairs of random numbers (Rn, Rn+1) over the full period of the RNG
LCG(16807, 0, 231 - 1, 1), where we “zoom in” on only the portion of the unit square in
the (0,0.0010) X (0.001,0) corner or 1/1,000,000 of the total area. This generator is the
“minimal standard” suggested by Park and Miller [1988]. “Although the row spacing for
this generator is obviously not optimal, this particular one is popular for its universal
portability and its satisfactory performance in many applications” [Burns and Pryor,

2
 0.0010

0.0008

0.0006
Rn+1

0.0004

0.0002

0.0000
0.0000 0.0002 0.0004 0.0006 0.0008 0.0010
Rn

Figure 1 Lattice Structure for LCG (16807, 0, 231 - 1, 1)

1995]. More details about this particular LCG generator are given by Press et al. [1992].
Other examples of the lattice structure of linear congruential generators are given by
Anderson [1990] and Burns and Pryor [1995].

Another problem is that the periods of LCG generators are often too short. The period
of the LCG depends on m. The maximum period is m for class 1; 2N-2 for class 2; and m -
1 for class 3 [Anderson, 1990]. For easy implementation on 32 byte machines, m is usu-
ally set to 231 or less. (The maximum value of a signed 32 bit integer is 231 - 1 ≅ 2 billion).
Many problems require many more random numbers than this. Furthermore, these
types of generators are known to have long-range correlations, especially for intervals
which are a power of 2 [Coddington, 1992].

Still, the general consensus is that while LCG generators are not the best random
number generators, they are still very good, provided the constants are chosen carefully.
Furthermore, there has been more theoretical work done on these types of generators
than any other type, so for problems where the above two limitations do not matter,
LCG generators are often the generator of choice. Knuth [1981] considers it “the “nicest”
and “simplest” random number generator for the machine language of most comput-
ers.” Marsaglia [1985] states that “They are the most widely used RNG’s, and they work

3
remarkably well for most purposes.” It should be noted however that in that very same
paper, the LCG fail three of the eight stringent tests in his DIEHARD test battery, partic-
ularly those designed to show lattice effects. Anderson [1990] states: “Multiplicative con-
gruential generators are adequate to good for many applications. They are not
acceptable (because of the lattice effect) for high-dimensional work. They can be very
good if speed is the major consideration. Prime moduli are best. However moduli of the
form 2n are faster on binary computers when the code is written in assembly language.”
Coddington [1992] states: “LCG’s have the advantage that we have a relatively good
(although still limited) theoretical understanding of their randomness properties. They
are known to be defective, but their defects are fairly well understood (for example, the
lattice structure of an LCG can be determined analytically using the spectral test), and in
practice they work quite well.”

The LHS program [Iman and Shortencarier, 1984] currently uses a LCG(519, 0, 248, X0)
[Hendricks, 1991]. This generator is also called the MCNP generator in this report
because of its purported implementation in the Los Alamos Monte Carlo code, MCNP.
Particularly noteworthy is the large value of m (248). The LCG has a maximum period of
246 ≅ 7.04X1012 if the initial seed is odd [Anderson, 1990; Hendricks, 1991]. The price
paid for this large period is that it takes a very slow and complicated algorithm to imple-
ment this LCG on 32 bit machines. This will be shown clearly in the time trials below.
The implementation used in the LHSTEST program [Romero, 1997] is designed only for
32 bit machines. The same algorithm is used even if 48 bit integers are available.

It should also be noted that there is an error in the implementation of the generator
that limits its portability. The generator code assumes that the values of variables are
saved between subroutine calls and does not use the FORTRAN SAVE statement to
ensure this. On machines/compilers where this is not true, such as the Macintosh Absoft
FORTRAN compiler and at least some versions of the Intel Paragon, each call of the LCG
generates the same value.

Furthermore, the version of the generator used by LHSTEST is older than the one dis-
cussed in Hendricks’ article. It uses a different random number stride than the one pro-
posed in that article.

2.2 Lagged Fibonacci Generators

2.2.1 Definition

From [Anderson, 1990]:

“The general form of such generators is represented by

LF [r, s, m, op; x{0…r-1}].

4
 Here, 0<s<r are the lags, m is a base, op is a binary operation, and x{0… r-1} is a
sequence of r initial values. For n ≥ r, the sequence is defined by

x n = x n – r op x n – s (1)

The possible operations here are

+ addition mod m,
- subtraction mod m,
* multiplication mod m,
⊕ exclusive or if m is a power of 2.∆”

Often, m is taken as 2N. Popular values of (r, s) are (17,5), (55,24), (127, 97), (607, 273),
and (1,279, 418). Good candidates for values of r and s are primitive trinomials modulo 2
[Knuth, 1981]. Lists of such values can be found in several sources [Knuth, 1981; Brent,
1992; Pryor et al., 1994].

Lagged Fibonacci generators are also called generalized Fibonacci generators [Brent,
1992]. Both names refer to the similarity between the lagged Fibonacci generating
sequence and the Fibonacci number recurrence formula.

The addition and subtraction generators are widely used and are discussed in greater
detail below.

The multiplication generator has been shown to have very good properties. It was the
only generator besides some combination generators to pass all of Marsaglia’s [1985]
DIEHARD tests. It also did well on Coddington’s [1992] Ising model Monte Carlo tests.
Still, multiplication generators have not been tested extensively as the addition-subtrac-
tion Fibonacci generators, and there is little theory about them. For these reasons, multi-
plication Fibonacci generators are not widely used.

The exclusive-or generator has been widely studied [Marsaglia, 1985; Marsaglia and
Tsay, 1985; Aluru et al., 1992] and is sometimes referred to as generalized feedback shift
register generator. This type of generator is known to be very poor. Although for large
lags of 607 or greater it will pass all of Marsaglia’s [1985] DIEHARD tests, it fails all but
the lattice test if the lags are 55 or shorter.

Lagged Fibonacci generators must be initialized (seeded) by another random number

generator. While all except the ⊕ LF generators are noted for producing good series of
random numbers, Altman [1988] notes that the bitwise random behavior of the LF gen-
erators depends on the generator used to seed the LF. In general, it appears that if the
generator seeding the LF has good bitwise behavior, so will the LF.

5
2.2.2 Addition and Subtraction Lagged Fibonacci Generators

The theory and properties of the addition and subtraction lagged Fibonacci genera-
tors (hereafter referred to as ASLF) are essentially the same. If m = 2N, the maximum
period of these generators is 2N-1(2r -1) [Anderson, 1990; Brent, 1992]. The only require-
ment for obtaining the maximum period is that at least one seed must be odd. Still, one
should refrain from using more than 2r - 1 of the numbers generated by these generators
because for a generator with period, p, Xn and Xn+p/2k differ by at most k bits (0 < k < N)
[Brent, 1992].

These generators did rather well on Marsaglia’s [1985] DIEHARD tests. For genera-
tors of length 17 and 55, only one test, the birthday spacing test, was failed. For genera-
tors of length 607 or 1279, all tests were passed. Preliminary tests by Brent [1992] suggest
that the birthday spacing test is passed by all ASLF’s over 100 in length.

It should be noted that while all the ASLF generators passed the DIEHARD m-tuple
test for Marsaglia [1985], several failed that test for Altman [1988]. The m-tuple test is
used to determine the bitwise randomness of a generator. Altman applied the m-tuple
test to ASLF’s seeded by two generators, one generator that passed the m-tuple test and
one that did not. All the ASLF’s passed the m-tuple test that had been seeded by the
passing generator. The 17 seed ASLF was the only ASLF to pass when seeded by the fail-
ing generator. All the longer lag ASLF’s failed when seeded by the bad generator, with
the worst results for the longest lags. Since all of Marsaglia’s ASLF’s passed the m-tuple
test, we must assume that he seeded them with a generator with good bitwise proper-
ties. This suggests that for lags of 55 or less, an ASLF with good bitwise properties will
still fail the birthday spacing test.

Knuth [1981] covers both types of LF generators but calls them additive and subtrac-
tive generators. His suggested values of r and s are (55, 24). He says m can be any even
number and suggests m = 2N for the additive generator and m = 1,000,000,000 for the
subtractive generator. He gives both high marks. About the additive generator, he notes
“This generator is usually faster than the previous methods we have been discussing,
since it does not require any multiplication. Besides its speed, it has the longest period
we have seen yet; and it has consistently produced reliable results, in extensive tests
since its invention in 1958.” Furthermore, he states “Therefore it may very well prove the
very best source of random numbers for practical purposes. The only reason it is difficult
to recommend sequence (7) wholeheartedly is that there is still very little theory to prove
that it does or does not have desirable randomness properties...” The subtractive
method is discussed in his summary and is described as “a “portable” random generator
that is efficiently describable in any higher-level programming language” that is a possi-
ble replacement for the linear congruential generator. He notes that: “Although a great
deal is known about linear congruential sequences like (1), very little has been proved
about the randomness properties of the subtractive method. Both approaches seem to be
reliable in practice.”

6
 As can be seen above, Knuth likes the addition and subtraction LF’s but worries
about the lack of study of them. A similar complaint about the lack of a firm theoretical
basis is voiced by Coddington [1992]. This is largely unjustified today because in recent
years, there have been many studies and much theory about ASLF’s [Marsaglia and
Tsay, 1985; Brent, 1992; Brent, 1994; Pryor et al., 1994; Mascagni et al., 1995], and they are
now much better understood. It should also be noted that the Numerical Recipes’ [Press
et al., 1992] “back up” generator, RAN3, is derived from Knuth’s subtractive method
and has had no problems of which we have heard.

Anderson [1990] notes “lagged Fibonacci generators using +, -, and * are good and,
with long lags, can be magnificent. With reasonable coding care, they can be quite fast.
However, Altman (1988) indicates initialization is critical.”

Pryor and his coworkers have noted that the ASLF is particularly well suited for
implementation on parallel platforms [Pryor et al., 1994; Mascagni et al., 1995]. We have
recently created a parallel version of our Monte Carlo code, MONT3D using PVM [Zeeb
et al., 1999]. For this code, we worked with Pryor on an addition lagged Fibonacci gener-
ator which is described below.

2.2.3 The CCS Addition Lagged Fibonacci Generator

We call this generator the Center for Computing Sciences (CCS) generator because
that is where much of the work on it was done [Pryor et al., 1994; Mascagni et al., 1995].
The CCS generator is the standard addition lagged Fibonacci generator described above
with a few additions that are particularly useful for parallel programming. To be porta-
ble to almost any architecture, m is set to 231. r and s can easily be set by the programmer
to any reasonable values.

In parallel programming, it is beneficial to make sure that the random number

sequences used by different processes are uncorrelated. Pryor and his coworkers have
shown that if m is equal to 2N, there are 2(r-1)(N-1) full period cycles of length (2r - 1)2N-1
called equivalence classes (EC). They have also determined how to specify a “canonical
form” of the initial seeds for each EC. By presetting certain bits in the array of seeds, it is
possible to guarantee that each set of seeds specified is part of a different full period
cycle or EC. The CCS addition lagged Fibonacci generator implements these “canonical
form” seeds.

Even though the LF requires an array of seeds, it is beneficial to allow the user to ini-
tialize the generator using one positive integer value like any other generator. On a 32 bit
machine, the maximum allowed single seed value is 231-1. While there are 230(r-1) full
length periods from which to choose, one must also keep in mind that, as Altman (1988)
has shown, improper seeding of the LF generator will lead to poor bitwise behavior. The
CCS generator initializes the generator from a single 32 bit seed using a binary shift reg-
ister [Burns and Pryor, 1999]. This is known to be a good generator, and experience has
shown that this appears to give good initial fills for ASLF’s [Pryor, 1997].

7
 There are a couple of issues about canonical seeds that ought to be addressed. First, it
has been noted that when all the initial seeds are set to low values, then when generating
a sequence of uniform random numbers between 0 and 1, there is a “flat spot” at the
beginning of the sequence in which a whole sequence of random numbers much less
than 0.5 are generated [Pryor et al., 1994; Burns and Pryor, 1995; Burns and Pryor, 1999].
It has been proven that such “flat spots” are expected statistically and do not show a
defect in the generator [Pryor et al., 1994]. Pryor and his coworkers’ original method for
seeding the generator in parallel used low value seed sequences first and several of the
processes would start with these “flat spots” [Mascagni et al., 1995]. A better seeding
method without these defects has been proposed [Pryor et al., 1994]. Since the CCS gen-
erator obtains its seeds from a binary shift register, the chance of “flat spots” in the
beginning of the sequence is very low.

A second problem with the CCS generator has been noted by Burns and Pryor [Burns
and Pryor, 1995; Burns and Pryor, 1999]. Due to the way the canonical seeds are defined,
the last bit of each random number is simply not random. The last bit exhibits the same
sequence no matter which canonical form is used. This is the trade off required to force
the generator to use unique EC’s. It should be noted that none of the other bits exhibit
this defect. In fact, using the binary shift register, all other bits probably show very good
random behavior. The nonrandom last bit can be fixed several ways including using a
separate generator for the last bit or shifting off the least significant bit. Still, it should be
noted that while the last bit is significant when generating random integers, it is not
important when generating random numbers between 0 and 1. Therefore, in the CCS
generator, nothing is done about the last bit, and the random number is generated from
the unmodified 31 bit result.

As mentioned above, the CCS generator, allows the programmer to specify any value
for r and s (called l and k by Pryor and his coworkers.) To specify the canonical form for
each r-s pair requires two other constants, L and S. Several have been tabulated [Pryor et
al., 1994]. Although there is a large number of r-s pairs, one or two ought to be chosen for
general use. To reduce storage space, one of the shorter sequences should be looked at.
The (17, 5) pair has been shown to give incorrect results in the program LAYMC [Zeeb
and Romero, 1999] (more detail below) so it and smaller sequences should be rejected.
As mentioned above, the (55, 24) ASLG has been recommended by both Knuth [1981]
and Press et al. [1992], and has been shown to have very good behavior. Still, this gener-
ator is known to fail Marsaglia’s birthday spacings test, so another sequence should be
tested also. It is known that the (607, 273) generator passes the birthday spacing tests.
Therefore in the tests below, two generators will be tested: (r = 55, s = 24, L = 1, S = 11)
and (r = 607, s = 273, L = 1, S = 105). The L and S values for the (607, 273) pair are unpub-
lished and have been obtained from Pryor [Pryor, 1997]. It takes about three hours on the
Cray-2 to determine L and S for this case.

The CCS generator is very portable. Tests have shown that it compiles on Sun, DEC,
SGI, Hewlett Packard, IBM RS/6000, Microsoft Windows (Microsoft FORTRAN) and
Macintosh (Absoft FORTRAN) platforms in FORTRAN 77. The FORTRAN 77 version
can also be compiled using the NAG Fortran 90 compiler under Linux.

8
2.3 Combination Generators

As stated by Knuth [1981]: “There will always be people who feel that the linear con-
gruential methods, additive methods, etc. are all too simple to give sufficiently random
sequences; and it may never be possible to prove that their skepticism is unjustified
(although we believe it is), so it is pretty useless to argue the point. There are reasonably
efficient methods for combining two sequences into a third one that should be haphaz-
ard enough to satisfy all but the most hardened skeptic.”

Indeed, the opinions on combination generators are rather mixed. Marsaglia is a

strong proponent of these types of generators, giving some theoretical support and some
generators in one paper [Marsaglia, 1985], and presenting many more in another [Mar-
saglia and Zaman, 1994]. Brent [1992], on the other hand, states, “Combination genera-
tors are not recommended because they are slow - usually two to three times slower
than the component generators. Also, they are not guaranteed to have good statistical
properties.” In a similar vein, Coddington [1992] states: “Although these mixed genera-
tors perform well in empirical tests, there is little theoretical understanding of their
behavior, and it is quite possible that mixing two generators may introduce new defects
of which we are unaware. A good single generator may therefore be preferable to a
mixed generator.”

A particularly well-known combination generator is Numerical Recipes’ RAN2 [Press

et al., 1992]. This is their “$1,000.00 generator” for which they will give $1,000.00 to the
first reader who finds a “statistical test that RAN2 fails in a non-trivial way, excluding
the ordinary limitations of a machine’s floating point representation.” This generator
combines two linear congruential generators and further randomizes by shuffling. Mar-
saglia and Zaman [1994] are somewhat disappointed with the RAN2 generator. While
passing all of Marsaglia’s DIEHARD tests, they present several other combination gen-
erators which appear to have random properties as least as good, are much faster, and
have longer periods. Since the RAN2 generator is so well documented, no tests except
timing tests will be done in this report for it.

3 Coding Considerations
Since random number generators are often used repeatedly, it is beneficial to code the
random number generator as efficiently as possible. Marsaglia and Zaman [1994] note
that RAN2 requires a value IDUM be passed to it whose only purpose is to initialize the
generator when IDUM is negative. They state: “The subroutine must deal with both an
argument idum and the returned value for ran2. Inefficient. The modern trend for ran-
dom number generators is to call them with an empty argument list, such as u=ran2().
The problem of initializing is handled by an entry point in the subroutine, and the
default entry values are put into the subroutine to make its use fool proof.”

While we agree that an empty argument list is beneficial, we find that putting an
entry point in the subroutine leads to confusing code which is hard to read. Instead, we
believe that there should be two separate subroutines/functions: an initialization rou-

9
tine that can have as many values passed to it as needed and a separate empty argument
list function to generate the actual random numbers. By separating the two functions,
there is no need to check if this call of the random number generator is to initialize it,
and an unneeded IF statement can be left out of the random number generating func-
tion.

4 Tests of Randomness
While it is known that RAN2 has been extensively tested, to our knowledge, the CCS
generator and the MCNP linear congruential generator have not been tested as well. In
this section, several tests were done to verify the MCNP generator and the CCS genera-
tor with 55 and 607 seeds. The tests done involve the average, the standard deviation,
the chi-square test and the Kolmogorov-Smirnov test [Knuth, 1981]. Tests involving
actual Monte Carlo simulations will be covered in the next section.

4.1 Tests of Average and Standard Deviation

The first four tests are based on the average and standard deviation. The results for
each are obtained in a two stage process. First, 1,000 random sequences are computed
and the averages and standard deviations for 10,000, 100,000, and 1,000,000 numbers in
each sequence are determined. Each sequence is initialized from the current time. The
formula for the time-generated initial seed, iseed0, is:

5 5 9 14 19 25
iseed0 = mod ( iyear, 2 ) + 2 imonth + 2 iday + 2 ihour + 2 imin + 2 isec (1)

where iyear ranges from 0 to 99, imonth ranges from 1 to 12, iday ranges from 1 to 31,
ihour ranges from 0 to 23 and imin and isec both range from 0 to 59.

The next step is to calculate for these 1,000 sequences of 10,000, 100,000, and 1,000,000
numbers the average and standard deviation for their averages and standard deviations.
For uniformly distributed random numbers, the average should be 0.5 and the standard
deviation should be 1/12.

The error in the average of the average and average of the standard deviation is
shown in Figures 2 and 3. The results are for the MCNP linear congruential generator
and the 55 and 607 seed CCS generators. As can be seen from the first figure, for the
average of the average, all three generators fluctuate as the number of random numbers
is increased. For the average of the standard deviation, the MCNP does have a lower
error than the other two generators, but this is probably not significant.

The standard deviation for the average and the standard deviation are shown in Fig-
ures 4 and 5. As can be seen from the figures, the standard deviations are essentially the
same for all three generators.

10
 10-3
MCNP
LF 55

Error in the Average of the Average

LF 607

10-4

10-5

-6
1010,000 100,000 1,000,000
Number of Random Numbers per Trial
Figure 2 Error in the Average of the Average for 1,000 Trials

10-4
Error in the Average of the Standard Deviation

MCNP
LF 55
LF 607

10-5

10-6

-7
1010,000 100,000 1,000,000
Number of Random Numbers per Trial
Figure 3 Error in the Average if the Standard Deviation for 1,000 Trials

11
 10-4
MCNP
LF 55

Standard Deviation of the Average

LF 607
10-5

10-6

10-7

-8
1010,000 100,000 1,000,000
Number of Random Numbers Per Trial
Figure 4 Standard Deviation of the Average for 1,000 Trials

10-6
Standard Deviation of the Standard Deviation

MCNP
LF 55
LF 607

10-7

10-8

-9
1010,000 100,000 1,000,000
Number of Random Numbers per Trial
Figure 5 Standard Deviation of the Standard Deviation for 1,000 Trials

12
4.2 The “Chi-Square” Test

The chi-square (χ2) test is listed by Knuth [1981] as a good statistical test of a random
number generator. The basic idea behind the test presented here is as follows. Divide the
range between 0 and 1 into 100 uniform bins. For a uniform distribution, the probability
of a random number being in any particular bin is 0.01. Next, we generate a million ran-
dom numbers and tabulate into which bins they fall. Then the χ2 statistic, V, can be cal-
culated:

k k 2 100
( Y s – n ps ) 1 Y s
∑ ∑  -----
- – n = 1 ×10 ∑ ( 100Y s – 1000000 )
–6 2
V = - = ---
------------------------ (1)
n ps n ps 
s=1 s=1 s=1

where k is the total number of bins (100), Ys is the number of values in bin s, ps is the
probability of a random number in bin s (0.01), and n is the total number of random
numbers (1,000,000). nps is the total number of random numbers expected in each bin.
For a truly random sequence, most bins will not have the expected number in them, but,
on the other hand, not many will have values much greater or less than the expected
numbers. V is expected to follow a χ2 distribution that depends on the number of
degrees of freedom. The number of degrees of freedom is one less than the total number
of bins since the number in the last bin can be calculated if the numbers in all the other
bins are known.

If the χ2 test is repeated several times, it is expected that X% of the results will have a
V value less than a certain number. Tables giving V as a function of the number of
degrees of freedom and χ exist [Knuth, 1981]. Obtaining just one or even a few V values
is not helpful unless the V are at the very edges of the χ2 distribution, proving the ran-
dom number generator defective. For this report, the following was done instead. A
thousand χ2 tests on a million random numbers were done using time generated seeds
from eqn. (1). The distribution of these one thousand V’s (F(V)) were plotted and com-
pared to the theoretical χ2 distribution. Just as the bins are not expected to match the
expected numbers exactly, this one thousand point distribution was not expected to
match the theoretical χ2 distribution exactly either. Still, by viewing the results graphi-
cally, it should be clear if there is any serious deviation from the theoretical χ2 distribu-
tion. In Figures 6 to 8 below, three χ2 distributions with a thousand points per
distribution and a million random numbers per point are plotted using each generator.
These are compared to the theoretical distribution. The theoretical distribution is not as
smooth as the other distributions because only seven points are plotted for it: 0.01, 0.05,
0.25, 0.50, 0.75, 0.95, and 0.99. The error in these theoretical values is on the order of 1/n
or 1X10-6 [Knuth, 1981]. As can be seen by the graph all nine curves match the theoretical
curve well. Therefore, all the generators pass this test. This is a good test of the global
behavior of the random number generators.

13
 1.0

0.8

F(V) 0.6

Theoretical

0.4

0.2

0.0
50 70 90 110 130 150 170
V
Figure 6 χ2 Distribution for the MCNP Generator

1.0

0.8

0.6
F(V)

Theoretical

0.4

0.2

0.0
50 70 90 110 130 150 170
V
Figure 7 χ2 Distribution for the CCS LF 55 Generator

14
 1.0

0.8

F(V) 0.6

Theoretical

0.4

0.2

0.0
50 70 90 110 130 150 170
V
Figure 8 χ2 Distribution for the CCS LF 607 Generator

4.3 The Kolmogorov-Smirnov Test

The χ2 test is designed to test quantities that fall into a finite number of categories.
Random numbers are expected to have continuous distributions. A good test for contin-
uous distributions is the Kolmogorov-Smirnov (K-S) test [Knuth, 1981]. For a distribu-
tion that has a cumulative distribution function, F, with no jumps (i.e. the distribution is
continuous), the values K+n and K-n are defined as:

+ max
Kn = n (F ( x) – F ( x ))) (1)
–∞ < x < ∞ n

- max
Kn = n (F ( x) – Fn( x)) (1)
–∞ < x < ∞

where n is the number of values being tested, Fn is the cumulative distribution function
calculated from these values and F is the theoretical value of this cumulative distribution
function. For a uniform distribution, F(x) = x.

Knuth [1981] has devised a test based on the K-S test that is a good test of local and
global random behavior. The test was implemented for this report as follows. Generate a
million random numbers and partition sequentially in the order generated into one

15
thousand sequences of one thousand numbers. Perform the K-S test on these one thou-
sand sequences to obtain one thousand K+1000. Now, the K+1000’s will have a cumulative
distribution function of their own. This function is closely approximated by:

2
F ∞ ( x ) = 1 – exp(-x ) (1)

While the 1,000 K+1000 values could be used in another K-S test, this will only give one
value to determine if the generator is good or bad. It is more enlightening to graph the
cumulative distribution of the 1,000 K+1000 values and to observe visually how well it
matches the theoretical distribution given in eqn. (1). In a fashion similar to the χ2 tests
done above, three K-S distributions are plotted using each generator in Figures 9 to 11
below. Once again, the plot of the theoretical curve is interpolated from only seven val-
ues: 0.01, 0.05, 0.25, 0.50, 0.75, 0.95, and 0.99. As before, all the curves match the theoreti-
cal curve well, so all three generators pass.

1.0

0.8

0.6
F(K+1000)

Theoretical

0.4

0.2

0.0
0.0 0.2 0.4 0.6 0.8 1.0 + 1.2 1.4 1.6 1.8 2.0 2.2
K 1000
Figure 9 K-S Distribution for the MCNP Generator

16
 1.0

0.8

F(K+1000) 0.6

Theoretical

0.4

0.2

0.0
0.0 0.2 0.4 0.6 0.8 1.0 + 1.2 1.4 1.6 1.8 2.0 2.2
K 1000
Figure 10 K-S Distribution for the CCS LF 55 Generator
1.0

0.8

0.6
F(K+1000)

Theoretical

0.4

0.2

0.0
0.0 0.2 0.4 0.6 0.8 1.0 + 1.2 1.4 1.6 1.8 2.0 2.2
K 1000
Figure 11 K-S Distribution for the CCS LF 607 Generator

17
5 Monte Carlo Tests
Just as important as whether a generator passes all statistical tests is how well it
works in actual simulations. In testing the code LAYMC [Zeeb and Romero, 1999], Zeeb
tested several random number generators in the code. While the two Numerical Recipes
generators (the combination generator, RAN2, and the 55 seed addition lagged
Fibonacci generator, RAN3) passed all the tests, another 17 seed addition lagged
Fibonacci generator, RANA, failed a van de Hulst test problem [van de Hulst, 1980].
Since RANA is an earlier version of the CCS generator, it was decided to test the new
CCS generators with this test problem. As a further test, another test problem by Giov-
anelli [1955] was also done. The results are given in the next subsection. Coddington
[1992] also performed some Monte Carlo tests of random number generators. His results
are covered in the second subsection.

5.1 LAYMC Tests

A full description of the van de Hulst and Giovanelli problems is given in the LAYMC
manual [Zeeb and Romero, 1999]. For the van de Hulst problem, ten trials of ten million
photons per trial were performed instead of ten trials of one million as was done in the
original investigation. For the Giovanelli problem, ten trials of one million photons each
were done for each generator, just as in the original investigation.

The “integrated” reflectance and total transmittance results for the van de Hulst prob-
lem and the total reflectance results for the Giovanelli problem are shown below in
Tables 1 to 3. The values presented are the averages over the ten trials. The other metrics listed
are the same as those given in the LAYMC manual. “‘Stan Err’ is the standard error (= the
standard deviation of the results over the ten trials divided by the square root of the number of tri-
als); ‘Max Diff’ is the maximum difference between the individual trial results and the average of
the ten trials; ‘% Diff’ is this result as a percent of the average value; ‘Error’ gives the error in the
average value compared to an the essentially exact analytic result; and ‘% Error’ gives this value
as a percentage of the “exact” result” [Zeeb and Romero, 1999]. Due to the earlier problems with
the 17 seed LF generator, RANA, results were generated for the MCNP generator, and 55, 607,
and 17 seed CCS generators. Two different algorithms were used for each problem. Z-R is the
Zeeb-Romero algorithm and pMCML is the psuedoMCML algorithm of Zeeb and Romero. Both
algorithms were done in the “MCML [Wang and Jacques, 1993] compatible mode.”.

The tests were done using default seeds. 19,895,660 for the MCNP generators and
1,672,217,120 for the LF generators. Although not recorded, each test was also repeated
using time generated seeds. Comparing the default seed to the time generated seed
results, it was found that the MCNP, LF 55, and LF 607 results all vary negligibly around
the same answer. As before, the LF 17 results show a bias. This is most clearly seen by
comparing the Max Diff and Error results for each case. If the simulations are converging
to the proper answer, then the absolute error in the average of the ten trials should be
less than the Max Diff relative error listed above. For every LF 17 result for the van de
Hulst problem and for the LF 17 pMCML simulation of the Giovanelli problem, this is
not the case. This shows a bias of up to 1% for the LF 17 generator.

18
 Table 1 Total Reflectance Results for the van de Hulst Problem

Case Ri Stan Error Max Diff % Diff Error % Error

van de Hulst 0.09739

MCNP, Z-R 0.097394 3.05X10-5 1.48X10-4 0.152% 3.97X10-6 4.08X10-3%
MCNP, pMCML 0.097398 2.86X10-5 1.50X10-4 0.154% 8.11X10-6 8.33X10-3%
LF 17, Z-R 0.098359 3.88X10-5 2.06X10-4 0.210% 9.69X10-4 0.995%
LF 17, pMCML 0.096843 2.97X10-5 1.47X10-4 0.153% -5.47X10-4 -0.562%
LF 55, Z-R 0.097415 2.14X10-5 1.30X10-4 0.134% 2.53X10-5 0.0260%
LF 55, pMCML 0.097452 3.24X10-5 1.59X10-4 0.163% 6.17X10-5 0.0633%
LF 607, Z-R 0.097410 2.40X10-5 1.26X10-4 0.129% 2.03X10-5 0.0209%
LF 607, pMCML 0.097391 2.26X10-5 1.23X10-4 0.126% 1.47X10-6 1.51X10-3%

Table 2 Total Transmittance Results for the van de Hulst Problem

Case Tt Stan Error Max Diff % Diff Error % Error

van de Hulst 0.6609

6
MCNP, Z-R 0.6609 5.06X10-5 2.54X10-4 0.0384% -2.57X10-5 -3.89X10-3%
3
MCMP, pMCML 0.6609 4.31X10-5 2.32X10-4 0.0351% -1.71X10-5 -2.58X10-3%
4
LF 17, Z-R 0.6613 4.72X10-5 2.56X10-4 0.0387% 3.92X10-4 0.0594%
5
LF 17, pMCML 0.6619 4.08X10-5 2.41X10-4 0.0364% 9.56X10-4 0.145%
2
LF 55, Z-R 0.6609 4.63X10-5 3.18X10-4 0.0481% -5.48X10-5 -8.29X10-3%
1
LF 55, pMCML 0.6608 4.65X10-5 2.82X10-4 0.0426% -9.04X10-5 -0.0137%
7
LF 607, Z-R 0.6610 5.27X10-5 2.66X10-4 0.0403% 5.87X10-5 8.88X10-3%
2
LF 607, pMCML 0.6609 2.01X10-5 1.04X10-4 0.0157% 2.10X10-5 3.18X10-3%
8

19
 Table 3 Total Reflectance Results for the Giovanelli

Case Rt Stan Error Max Diff % Diff Error % Error

Giovanelli 0.2600
MCNP, Z-R 0.2600 1.01X10-4 6.51X10-4 0.250% 3.26X10-5 0.0125%
3
MCNP, pMCML 0.2597 5.67X10-5 4.24X10-4 0.163% -2.67X10-4 -0.103%
3
LF 17, Z-R 0.2596 1.42X10-4 8.73X10-4 0.336% -3.42X10-4 -0.132%
6
LF 17, pMCML 0.2587 9.31X10-5 4.45X10-4 0.172% -1.22X10-3 -0.468%
8
LF 55, Z-R 0.2599 1.17X10-4 5.92X10-4 0.228% 3.15X10-5 0.0121%
7
LF 55, pMCML 0.2600 9.66X10-5 6.97X10-4 0.268% 6.88X10-6 2.65e-3%
1
LF 607, Z-R 0.2601 9.86X10-5 5.57X10-4 0.214% 1.70X10-4 0.0653%
7
LF 607, pMCML 0.2599 6.72X10-5 4.08X10-4 0.157% -7.73X10-5 -0.0297%
2

As mentioned before, when more than 2r - 1 numbers are generated by an ASLF gen-
erator, there are correlations between the bits of the numbers [Brent, 1992]. This probably
explains the failure of the LF 17 generator. 217 is only about 131,000. Since the simula-
tions required from ten millions to billions of random numbers, the correlations caused
the biased results. Since 255 is greater than 1016 and 2607 is greater than 10182, this will
not be a problem for these two generators. It should be noted that this bias in the gener-
ator does not appear very frequently. Two other LAYMC test problems show no bias
[Zeeb and Romero, 1999]. Also, the 17 seed addition LF generator, RANA, tested in the
LAYMC manual is used by a previous version of the Monte Carlo radiative heat transfer
program MONT3D [Maltby et al., 1994]. Several tests of MONT3D have been performed
with no bias having been found. It has been suggested that if the physics of a simulation
are complex enough, it may lessen the impact of defects in a random number generator.
Both the van de Hulst and the Giovanelli problems are rather simple. Perhaps the
defects of the ASLF 17 generator only appear in certain simple problems.

20
5.2 Ising Model

Coddington [1992] tests many random number generators on the two-dimensional

Ising model. He gives the results for three different Monte Carlo algorithms, SW, Wolff,
and Metropolis. He finds that even 1,279 seed ASLF’s give bad results for some of these
algorithms. The only ASLF to pass all the tests has 4,423 seeds. RAN2, on the other hand,
passes all the tests with no problems.

The importance of this result is not known yet. While covering many random number
generators, Coddington says little about the Ising model. We have not had a chance to
research the problem in any detail yet but know it is about modeling spin states. There
are only two spin states, so this could be the ultimate “simple” problem. Considering
how badly many of the generators he tested failed, we think these results say more
about the Monte Carlo algorithms than the random number generators they are testing.

6 Timing
To determine how fast the generators are, timings to initialize the random number
generator and to generate one hundred million random numbers were performed. The
tests were done on a 36 MHz Sparc-10. The code was compiled using the -O and -dalign
options. Each random number generator was in a format compatible with the program
LAYMC [Zeeb and Romero, 1999]. For each generator, the initialization is done in a sub-
routine RANINIT which takes two arguments, and the random numbers are generated
by the function RANF which takes no arguments. Considering the discussion about cod-
ing considerations in Section 2, it would have been beneficial to see what effect adding
an argument to RANF and combining RANF and RANINIT into one subroutine, but
that coding has not been done yet.

The program was run three times for each generator. Times varied by 0.6% or less.
The minimum run times for each generator are listed below in Table 4.

Table 4 Timing Results

time
Generator
(seconds)

MCNP1 298.47
MCNP2 248.30
LF 55 61.36
LF 607 61.44
RAN2 481.19
RAN3 75.31

21
 MCNP1 is the version of the MCNP generator used in the LHSTEST program
[Romero, 1997], RANF (RNUM1 in the program LHSTEST) is a wrapper function that
takes no arguments and calls the initialization subroutine, RMCNPI, and the random
number generating function, RMCNP, every time it is called. Both subroutines/func-
tions take ISEED as an argument. If the generator has already been initialized, RMCNPI
goes through one IF statement and returns. This is very inefficient, requiring an extra IF
statement, RETURN, and a function and subroutine call each with one argument for
each random number generated. MCNP2 is a version of the generator we have “cleaned
up.” RANF in this case is just the function RMCNP and RANINIT is essentially just the
subroutine RMCNPI. All the unneeded statements used to generate a random number
for MCNP1 described above have been removed. Furthermore, the lack of portability
due to the missing SAVE statement has been fixed, and certain unused variables have
been removed. As can be seen above, this reduces the run time of the generator by
almost 17%.

LF 55 and LF 607 are the 55 and 607 seed CCS generators. They are by far the fastest
generators. In fact, they are slightly four times faster than MCNP2. The only time differ-
ence caused by increasing the number of seeds from 55 to 607 is 0.08 seconds. This is all
initialization time. No matter how many seeds, all CCS generators take the same time to
generate a random number.

RAN2 and RAN3 are the generators from Numerical Recipes [Press et al., 1992]. As
mentioned above, they have been modified so that generator initialization and random
number generation are in separate functions/subroutines. Also, the generating function
takes no arguments. RAN2 which combines two linear congruential generators and does
shuffling, is the slowest generator. It is almost 8 times slower than the LF generators.
Still, since it only takes about eight minutes for even RAN2 to generate one hundred mil-
lion random numbers on a slow machine, the speed of RAN2 may not be an issue. RAN3
is a 55 seed addition LF. It is 25% slower than the CCS LF’s because it uses an IF and a
possible add instruction where the CCS uses a bitwise and. Obviously, very few opera-
tions are done in an addition LF. It should be noted that Press et al. [1992] state that
RAN3 is only 3 and 1/3 times faster than RAN2. In our tests, it is over 6 and 1/3 times
faster. The changes made to RAN2 and RAN3 listed above are not enough to explain this
discrepancy.

7 Recommendations
Knuth [1981] notes: “The most prudent policy for a person to follow is to run each
Monte Carlo program at least twice using quite different sources of random numbers,
before taking the answers of the program seriously; this not only will give an indication
of the stability of the results, it will also guard against the danger of trusting in a genera-
tor with hidden deficiencies. (Every random number generator will fail in at least one
application.)” After all this research, we concur and think that two very different ran-
dom number generators should be used in case one random number generator proves
unable to properly generate one or more of the distributions used by the program. It
should be realized that since all computer-generated random sequences are generated in

22
a deterministic way, they are actually pseudo-random numbers that only appear to be
random. While many RNG’s produce numbers that are “as good as” random for most
situations, for each generator, there will always be one or more tests it will fail. After
looking over the choices, we suggest that LHSTEST be implemented so that it can run
with either a version of the CCS generator or a combination generator. Exactly which
two generators to use is discussed below.

We do not suggest a linear congruential generator for one reason. Although it passes
all the tests we did, it is known to be bad for higher dimensional problems. This is illus-
trated in Figure 1. We have heard that if there is any correlation between pairs of random
numbers, they will not model a normal distribution correctly.

The addition LF has proven to be very good for 55 or more seeds. The most significant
deficiency is that it fails the Ising model test [Coddington, 1992] but we do believe that is
important. The CCS addition LF has proven to be fast and as good as any other addition
LF. Furthermore, it is the only generator that is easy to implement in parallel. The prob-
lem here is determining which length to use. Knuth [1981] and Numerical Recipes’
RAN3 have shown that a 55 seed LF appears to work well, but it is known to fail Marsa-
glia’s birthday spacings test [1985]. For this reason, we prefer the 607 seed LF which has
passed all of Marsaglia’s tests. It requires 609 words of storage (including the two
“taps”) but is no slower than a shorter LF. It should be noted that if restart from a previ-
ous state is added as an option to the code, the entire state of the random number gener-
ator, the seeds and the two taps, will have to be written to the disk on a regular basis.
Since Brent [1992] thinks that addition LF’s with r > 100 pass the birthday spacings test,
a possible third option is (r = 127, s = 97, L = 1, S = 21) [Pryor et al, 1994]. In fact, the 127
seed CCS generator is the one used by the current version of our Monte Carlo radiative
heat transfer code, MONT3D [Zeeb et al., 1999], which does include a restart option.

For combination generators, the one to choose is probably RAN2. As far as we are
aware, it has not failed a test yet. It is much slower than the other generators, but consid-
ering how fast it generates one hundred million random numbers on even a slow 36
MHz Sparc-10, this is probably not an issue. If it is an issue, one of the newer combina-
tion generators by Marsaglia and Zaman [1994] should probably be chosen. Even these
will probably be three times slower than the CCS LF’s. It should be noted that there is
almost no theory to support these combination generators. All proofs of their ability
have been empirical only.

Therefore our current suggestion is that all problems with LHSTEST be done twice;
once with the 607 CCS generator and once with RAN2.

8 References
Altman, N. S., 1988. “Bit-wise Behavior of Random Number Generators,” SIAM J Sci.
Stat. Comput., Vol. 9 No. 5, pp. 941-949.

23
Aluru, S., Prabju, G. M., and Gustafson, J., 1992. “A Random Number Generator for Par-
allel Computers,” Parallel Computing, Vol. 18, pp. 839-847.

Anderson, S.L., 1990. “Random Number Generators on Vector Supercomputers and

Other Advanced Architectures,” SIAM Review, Vol. 32 No. 2, pp. 221-251.

Brent, R. P., 1992. “Uniform Random Number Generators for Supercomputers, “Proceed-
ings Fifth Australian Supercomputing Conference, SASC Organizing Committee, 1992, pp.
95-104. (unpublished).

Brent, R. P., 1994. “On the Periods of Generalized Fibonacci Recurrences,” Math Comput.,
Vol. 63, pp. 389-401.

Burns, P. J. and Pryor, D. V., 1995. “Random Numbers,” Available as a WWW document.,
URL = http://csep.acns.colostate.edu:80/csep/RN/RN.html

Burns, P. J. and Pryor, D. V., 1999. “Surface Radiative Transport at Large Scale via Monte
Carlo,” Vol. 9 of Annual Review of Heat Transfer, Begell House, New York, NY. (in press).

Coddington, P. D., 1992., “Analysis of Random Number Generators Using Monte Carlo
Simulation,” International Journal of Modern Physics C, Vol. 0 No. 0, pp. 1-13.

Giovanelli, R. G., 1955. “Reflection by Semi-Infinite Diffusers,” Optica Acta, Vol. 2, pp.
153-162.

Hendricks, J. S., 1991. “Effects of Changing the Random Number Stride in Monte Carlo
Calculations,” Nuclear Science and Engineering, Vol. 109, pp. 86-91.

Iman R. L. and Shortencarier, M. J., 1984. “A FORTRAN 77 Program and User’s Guide
for the Generation of Latin Hypercube and Random Samples to Use with Computer
Models,” Sandia National Laboratories report SAND83-2365 (RG).

Knuth, D. E., 1981. Seminumerical Algorithms, 2nd Ed., Vol. 2 of The Art Of Computer Pro-
gramming, Addison-Wesley, Reading, MA.

Maltby, J. D., Zeeb, C. N., Dolaghan, J., and Burns, P. J., 1994. “User’s Manual for
MONT2D - Version 2.6 and MONT3D - Version 2.3,” Department of Mechanical Engi-
neering, Colorado State University, Fort Collins, CO.

24
Marsaglia, G., 1985. “A Current View of Random Number Generators”, Computing Sci-
ence and Statistics: Proceedings of the XVIth Symposium on the Interface, pp. 3-10.

Marsaglia, G. and Tsay, L.-H., 1985. “Matrices and the Structure of Random Number
Sequences”, Computing Science and Statistics: Linear Alg. and Applic., Vol. 67 pp. 147-156.

Marsaglia, G. and Zaman, A., 1994. “Some Portable Very-Long Period Random Number
Generators”, Computers in Physics, Vol. 8 No. 1, pp. 117-121.

Mascagni, M., Cuccaro, S., Pryor, D., and Robinson, M., 1995. “A Fast, High Quality, and
Reproducible Parallel Lagged-Fibonacci Pseudorandom Number Generator,” Journal of
Computational Physics, Vol. 119, pp. 211-219.

Park, S. K. and Miller, K. W., 1988. “Random Number Generators: Good Ones are Hard
to Find,” Communications of the ACM, Vol. 31. No. 10, pp. 1192-1201

Press, W. H., Teukolsky, S. A., Vetterling, W. T., and Flannery, B. P., 1992. Numerical Reci-
pes in FORTRAN: The Art of Scientific Computing, 2nd Ed., Cambridge University Press,
New York, NY.

Pryor, D. V., Cuccaro, S. A., Mascagni, M., and Robinson, M. L., 1994. “Implementation
of a Portable and Reproducible Parallel Pseudorandom Number Generator,” Supercom-
puting ‘94 Proceedings, IEEE Computer Society Press, Los Alamitos, CA, pp. 311-319.

Pryor, D. V., 1997. Personal Communication, Center for Computing Sciences, Bowie,
MD.

Romero, V. J., 1997. Personal Communication, Sandia National Laboratories, Albuquer-

que, NM.

van de Hulst, H. C., 1980. Multiple Light Scattering, Volume II, Academic Press, New York,
NY.

Wang, L. and Jacques, S. L., 1993. “Monte Carlo Modeling of Light Transport in Multi-layered
Tissues in Standard C”, University of Texas M. D. Anderson Cancer Center, Houston,
Texas.

Zeeb, C. N., Burns, P. J., Branner, K., and Dolaghan, J., 1999. “User’s Manual for
MONT3D - Version 2.4,” Department of Mechanical Engineering, Colorado State Uni-

25
 versity, Fort Collins, CO. Available as a WWW document, URL =http://
www.colostate.edu/~pburns/monte/manual.html.

Zeeb, C. N. and Romero, V. J. 1999. “LAYMC: A Monte Carlo Code for Modeling Photon
Transport in Layered Participating Media — Theory and User Manual Version 2A”, to be
published as a Sandia National Laboratories report (unlimited distribution), (In review;
expected 99).

26

View publication stats