App12 CDF PDF Rand
App12 CDF PDF Rand
The functions described in this appendix are used in more than one procedure. They
are grouped into the following categories:
• Special Functions. Gamma function, beta function, incomplete gamma function
(ratio), incomplete beta function (ratio), and standard normal function
Notation
The following notation is used throughout this appendix unless otherwise stated:
1
2 Appendix 12
Special Functions
Gamma Function
Γa =05 I
0
∞
x a −1e − x dx a>0
Properties
• 05
Γ 1 =1
• Γ01 / 25 = π
• Γ0 a5 = 0a − 15Γ0 a − 15 a >1
Note. Since Γ(a) can be very large even for a moderate value of a, the (natural)
logarithm of Γ(a) is computed instead.
References. The ln(Γ(a)) function: CACM 291 (1966) and AS 245 (1989).
Beta Function
B( a, b) =
I 1
0
0 5
x a −1 1 − x b −1
dx a > 0, b > 0
Properties
• 0 5
B a,1 = 1 / a
• B3 12 , 12 8 = π
• B0b, a5 = B0a, b5
Appendix 12 3
• 0 5 0 50
B a, b = b − 1 B a + 1, b − 1 / a5
• B0a, b5 = 0 a + b5B0 a + 1, b5 / a
0 5 3 1 0 56 2 0 57 2 0 578
B x, y = exp ln Γ x + ln Γ y − ln Γ x + y
IG( x; a) =
I
0
x 1 a −1 − t
Γ ( a)
t e dt x≥0
x p = IG −1 ( p; a) ⇔ p = IG( x p ; a) 0 ≤ p ≤1
for a > 0
Properties
• IG( x;1) = 1 − e − x
1
IG( x; a) = x a e − x + IG( x; a + 1)
Γ (a + 1)
0 5
Note. IG -1 1, a = ∞ .
References. AS 32 (1970), AS 147 (1980), and AS 239 (1988).
IB( x; a, b) =
I x 1
0 B( a, b )
t a −1 (1 − t ) b −1 dt 0 ≤ x ≤1
x p = IB −1 ( p; a, b) ⇔ p = IB( x p ; a, b) 0 ≤ p ≤1
for a > 0 and b > 0
4 Appendix 12
Properties
• IB( x; a,1) = x a
• 3
Using the transformation t = sin 2 θ , we get IB x; 12 , 12 = π2 sin −1 x 8
• 0
IB( x; a, b) = 1 − IB 1 − x; b, a 5
• Using integration by parts, we get, for b > 1,
0
IB x; a, b =5 ΓΓ00aa5Γ+0bb55 x 01 − x5
a b −1
0
+ IB x; a + 1, b − 1 5
• Using the fact that
d a
dx
0 5
x 1− x b
0 5
= ax a −1 1 − x b −1
0 5 0 5
− a + b xa 1− x b −1
we have
0
IB x; a, b =5 Γ0Γa0+a1+5Γb05b5 x 01 − x5
a b
0
+ IB x; a + 1, b 5
References. AS 63 (1973); Inverse: AS 64 (1973), AS 109 (1977).
05
Φ x =
I x 1 −u2 / 2
−∞ 2π
e du −∞ < x < ∞
05
x p = Φ −1 p ⇔ Φ x = p 05
For Φ −1 , the Abramowitz and Stegun method is used.
References. AS 66 (1973); Inverse: AS 111 (1977) and AS 241 (1988). See Patel
and Read (1982) for related distributions.
Appendix 12 5
Continuous Distributions
Beta 0 x; a, b5. 0 ≤ x ≤ 1, a > 0 and b > 0
0 5 B0a1, b5 x 01 − x5
f x = a −1 b −1
F0 x 5 = IB0 x; a, b5
x = IB 0 p; a, b5
p
−1
0 ≤ p ≤1
Random Number
Special case
1. Generate U from Uniform(0,1).
2. 0
If a = 1, set X = 1 − 1 − U 1/ b . 5
3. If b = 1, set X = U 1/ a
.
4. If both a = 1 and b = 1, set X = U .
Algorithm BN due to Ahrens and Dieter (1974) for a > 1 and b > 1
1. 05
Set e = a − 1, f = b − 1, c = e + f , g = c ln c , u = e / c, and s = 0.5 / c .
2. Generate Y from N(0,1) and set X = sY + u .
3. If X < 0 or X > 1, go to step 2.
4. Generate U from Uniform(0,1).
5. 05 4 0 5 10 5 6 9
If ln U ≤ e ln X / e + f ln 1 − X / f + g + 0.5Y 2 , finish; otherwise go to
step 2.
05 x−a
2 −1
f x =
1
πb
1+
b
05 1 1 x−a
F x = + tan −1
2 π b
xp = a + b tan2π 0 p − 1 / 257 0 ≤ p ≤1
Random Number
05
f x = a/ 2
1 0 5
x a / 2 −1e − x / 2
2 Γ a/2 0 5
F0 x 5 = IG3 ; 8x a
2 2
x p =2 IG −1 p; a 0≤ p≤1
2
Random Number
Generate X from the Gamma(a/ 2, 1/2) distribution.
05
f x = ae − ax
F0 x 5 = 1 − e − ax
x p = − ln01 − p5
1
0 ≤ p ≤1
a
Random Number
−0 a + b 5 / 2
0 5 B0a / 21, b / 25 ab x 0a/ 25−1 1 + ab x
a/2
f x =
F0 x 5 = IB
ax ; a , b
b + ax 2 2
−1
1 6
xp =
b IB p; a / 2, b / 2
1 6
0≤ p≤1
a 1 − IB −1 p; a / 2, b / 2
Random Number
Using the chi-square distribution
1. Generate Y and Z independently from chi-square(a) and chi-square(b),
respectively.
2. 0 50 5
Set X = Y / a / Z / b .
References. CDF: CACM 332 (1968). ICDF: use inverse of incomplete beta
function.
8 Appendix 12
0 5 Γb0a5 x e
a
a −1 − bx
f x =
F0 x 5 = IG1bx; a6
x = IG 1 p; a6
1 −1
p 0 ≤ p ≤1
b
Random Number
Special case
If a = 1 and b > 0, generate X from an exponential distribution with parameter b.
Algorithm GS due to Ahrens and Dieter (1974) for 0 < a < 1 and b = 1
1. 0 5
Generate U from Uniform(0,1). Set c = e + a / e , where e = exp 1 . 05
2. Set P = cU . If P > 1 , go to step 4.
3. 0P ≤ 15 Set X = P1/ a . Generate V from Uniform(0,1). If V > exp0− x 5 , go to
step 1; otherwise finish.
4. 10 5 6
( P > 1) Set X = − ln c − P / a . If X < 0 , go to step 1; otherwise go to step 5.
5. Generate V from Uniform(0,1). If V > X a −1 , go to step 1; otherwise finish.
References. CDF: AS 32 (1970) and AS 239 (1988); ICDF: Use the relationship
between gamma and chi-square distributions. RV: Ahrens and Dieter (1974),
Fishman (1976), and Tadikamalla (1978).
Appendix 12 9
05
f x =
1 − x −a / b
2b
e
% 0 5
0 5 K&1 −e e0 5
x −a / b
1 x≤a
F x = 2
K' 1
2
a− x / b
x>a
x =&
%Ka + b ln02 p5 0≤ p≤ 1
Note. The Laplace distribution is also known as the double exponential distribution.
Random Number
Inverse CDF algorithm
1. Generate Y and U independently from Exponential(1/ b) and Uniform(0,1),
respectively.
2. If U ≥ 12 , set X = a + Y ; otherwise set X = a − Y .
0 5 b1 e 0 5 41 + e 0 5 9
f x = − x−a /b − x −a / b
−2
F0 x 5 =
1
1+ e 0 5
− x −a / b
x = a + b ln
p 0 ≤ p ≤ 1
p
1− p
Random Number
Inverse CDF algorithm
1. Generate U from Uniform(0,1).
2. 1 0
Set X = a + b ln U / 1 − U . 56
10 Appendix 12
−1 ln 0 x / a 56 4 9
05
2
1 / 2b 2
f x = e
xb 2π
x p = ae
05
bΦ −1 p
0 ≤ p ≤1
Random Number
Inverse CDF algorithm
1. Generate Z from N(0,1).
2. Set X = a exp bZ . 0 5
Normal 0 x; a, b5. − ∞ < x < ∞, − ∞ < a < ∞, and b > 0
0 5 b 12π e 0
f x =
5 4 9
− x − a 2 / 2 b2
F0 x 5 = Φ
x − a
b
x = a + bΦ 0 p5
p
−1
0 ≤ p ≤1
Random Number
Kinderman and Ramage (1976) method
1. Generate as X = a + bz , where z is an N(0,1) random number.
References. CDF: AS 66 (1973); ICDF: AS 111 (1977) and AS 241 (1988); RV:
CACM 488 (1974) and Kinderman and Ramage (1976).
Appendix 12 11
0 5 ba ax
b +1
f x =
F0 x 5 = 1 −
a b
x
x = a01 − p5
−1/ b
p 0 ≤ p ≤1
Random Number
Inverse CDF
1. Generate U from Uniform(0,1).
2. 0
Set X = a 1 − U −1/ b . 5
Student’s t 0 x; a5. − ∞ < x < ∞ and a > 0
0 5
− a +1 / 2
05
f x =
1
0
a B a / 2,1 / 2
1+
x2
a 5
%K 1 IB a ; a , 1
F0 x 5 = &
K2 a + x 2 2 2
x≤0
%K a 1
K− a1 / IB 2 p; 2 , 2 − 1
−1
0≤ p≤ 1
=&
2
Random Number
Special case
If a = 1, generate X from a Cauchy (0, 1) distribution.
0 5 b −1 a
f x =
F0 x 5 =
x−a
b−a
x = a + 0b − a 5 p
p 0 ≤ p ≤1
Random Number
0 5 ba ax e 0 5
b −1
− x/a b
f x =
F0 x 5 = 1 − e 0 5
− x/a b
Random Number
Discrete Distributions
Bernoulli 0 x; a5. x = 0,1 and 0 ≤ a ≤ 1
0 5 0 5
Prob X = x = a x 1 − a 1− x
%1 − a x = 0
Prob0 X ≤ x 5 = &
'1 x = 1
Random Number
Special case
If a = 0 , X = 0 . If a = 1, X = 1.
0
Prob X = x =5 ax b 01 − b5
x a− x
%1 − IB0b; x + 1, a − x 5
Prob0 X ≤ x 5 = &
x = 0,1,..., a − 1
'1 x=a
14 Appendix 12
Random Number
Special case
If b = 0, X = 0. If b = 1, X = a.
0 5 0 5
Prob X = x = a 1 − a x −1
Prob0 X ≤ x 5 = 1 − 01 − a5 x
Random Number
Special case
If a = 1, X = 1.
c a − c
Prob0 X = x 5 =
x b − x
a
b
Prob0 X ≤ x 5 = ∑ Prob0 X = k 5
x
k = max( 0,b + c − a )
Random Number
Special case
If b = a, X = c. If c = a, X = b .
Direct algorithm
1. (Initialization) X = 0, g = c, h = b, t = a .
2. Do the following loop exactly b times:
Begin Loop
i. Generate U from Uniform(0,1).
ii. 0 5
If U ≤ g / t , set X = X + 1, g = g − 1, else h = h − 1 .
iii. If g = 0 , go to step 3.
iv. If h = 0 , set X = X + b − i , where i (from 1 to b) is the loop index. Go to
step 3.
v. Set t = t − 1.
End Loop
3. Finish.
References. CDF: AS 152 (1989), AS R77 (1989), and AS R86 (1991).
16 Appendix 12
0 5 ax −− 11 b 01 − b5
Prob X = x = a x −a
Prob0 X ≤ x 5 = IB0b; a, x − a + 15
Random Number
Special case
If b = 1, X = a .
Direct algorithm
1. 1
Generate G from Gamma a, b / 1 − b . 0 56
2. If G = 0 , go to step 1. Otherwise generate P from Poisson (G).
3. Compute X = P + a .
0 5 ax ! e
x
−a
Prob X = x =
Prob0 X ≤ x 5 = 1 − IG0 a; x + 15
Random Number
0 5 ∑ 1j ! 2c
∞ j
e −c / 2
0 5
x a + j −1 1 − x b −1
f x =
j =0
0 5
B a + j; b
0 5 ∑ 1j ! 2c
∞
0 5
j
F x = e − c / 2 IB x; a = j, b
j=0
0 5 ∑ 1j ! 2c
∞ j
x a / 2 + j −1e − x / 2
f x = e −c / 2
j =0
0
2 a/2+ j Γ a / 2 + j 5
1 c x ; a + j
∞
F0 x 5 = ∑
j
j! 2 2 2
e − c / 2 IG
j=0
0 a / b5 10 5 6
0 5 ∑ 1j ! 2c 1 + a x
∞ j a/ 2+ j − a +b / 2+ j
e −c / 2
b
a / 2 + j −1
B0a / 2 + j, b / 25
f x = x
j =0
0 5 ∑ 1j ! 2c ax ; a + j, b
∞ j
F x =
j=0
e −c / 2 IB
b + ax 2 2
The noncentral F random variable is generated as the ratio between two
independent chi-square random variables. The numerator is a noncentral chi-square
random variable (with parameters a and c) divided by a. The denominator is a
central chi-square random variable (with parameter b) divided by b.
Note. c is the noncentrality parameter. If c = 0 , F(x) is the (central) F distribution
function.
References. CDF: Abramowitz and Stegun (1965, Chapter 26).
∞
20 5 7 x 1 + x 0 5
2 − a + j +1 / 2
0 5 ∑ 1j ! 3c 2 8 Γ a + j +1 / 2 j
Γ0 a / 25Γ 01 / 25 a 0 5 a
j − c2 / 2
f x = e
j +1 / 2
j =0
%K 1 1 ∞
20 5 7 IB a ; a , j + 1
Γ j +1 / 2
KK 2 ∑ j ! 3−c 2 8 e
j −c2 / 2
Γ01 / 25 a+ x 2 2 2
x≤0
F0 x 5 = & j=0
KK1 − 1 ∑ 1 3c 2 8 e Γ 20 j + 15 / 27 a a j + 1
∞
IB
a + x ; 2 , 2
j −c 2 / 2
Γ 01 / 25
x>0
K' 2 j !j =0
2
Special case
05
F 0 = 1− Φ c 05
The noncentral Student’s t random variable is generated as the ratio between two
independent random variables. The numerator is a normal random variable with
mean c and variance 1. The denominator is a central chi-square random variable
(with parameter a) divided by a.
Appendix 12 19
Algorithm Index
AS 3: Cooper (1968a)
AS 5: Cooper (1968b)
AS 27: Taylor (1970)
AS 32: Bhattacharjee (1970)
AS 63: Majumder and Bhattacharjee (1973a)
AS 64: Majumder and Bhattacharjee (1973b)
AS 66: Hill (1973)
AS 76: Young and Minder (1974)
AS 91: Best and Roberts (1975)
AS 109: Cran, Martin, and Thomas (1977)
AS 111: Beasley and Springer (1977)
AS 134: Atkinson and Whittaker (1979)
AS 147: Lau (1980)
AS 152: Lund (1980)
AS 170: Narula and Desu (1981)
AS 226: Lenth (1987)
AS 231: Farebrother (1987)
AS 239: Shea (1988)
AS 241: Wichura (1988)
AS 243: Lenth (1989)
AS 245: Macleod (1989)
AS 275: Ding (1992)
AS R85: Shea (1991)
CACM 291: Pike and Hill (1966)
20 Appendix 12
References
Abramowitz, M., and Stegun, I. A. eds. 1970. Handbook of mathematical functions.
New York: Dover Publications.
Ahrens, J. H., and Dieter, U. 1974. Computer methods for sampling from gamma,
beta, Poisson and binomial distributions. Computing, 12: 223–246.
Atkinson, A. C., and Whittaker, J. 1979. Algorithm AS 134: The generation of beta
random variables with one parameter greater than and one parameter less than
1. Applied Statistics, 28: 90–93.
Beasley, J. D., and Springer, S. G. 1977. Algorithm AS 111: The percentage points
of the normal distribution. Applied Statistics, 26: 118–121.
Best, D. J., and Roberts, D. E. 1975. Algorithm AS 91: The percentage points of
the c2 distribution. Applied Statistics, 24: 385–388.
Box, G. E. P., and Muller, M. E. 1958. A note on the generation of random normal
deviates. Annals of Mathematical Statistics, 29: 610–611.
Bratley, P., Fox, B. L., and Schrage, L. E. 1987. A guide to simulation. New York:
Springer-Verlag.
Cran, G. W., Martin, K. J., and Thomas, G. E. 1977. Algorithm AS 109: A remark
on algorithms: AS 63 and AS 64 (replacing AS 64). Applied Statistics, 26: 111–
114.
Hill, I. D. 1973. Algorithm AS 66: The normal integral. Applied Statistics, 22: 424.
Kennedy, W. J., and Gentle, J. E. 1980. Statistical computing. New York: Marcel
Dekker.
Lau, C. L. 1980. Algorithm AS 147: A simple series for the incomplete gamma
integral. Applied Statistics, 29: 113–114.
Macleod, A. J. 1989. Algorithm AS 245: A robust and reliable algorithm for the
logarithm of the gamma function. Applied Statistics, 38: 397–402.
Mudholkar, G. S., Chaubrey, Y. P., and Lin, C. 1976a. Approximations for the
doubly noncentral F-distribution. Communications in Statistics, Part A, 5: 49–
53.
Mudholkar, G. S., Chaubrey, Y. P., and Lin, C. 1976b. Some Approximations for
the noncentral F-distribution. Technometrics, 18:351–358.
Appendix 12 23
Patel, J. K., and Read, C. B. 1982. Handbook of the normal distribution. New York:
Marcel Dekker.
Pike, M. C., and Hill, I. D. 1966. Algorithm 291: Logarithm of gamma function.
Communications of the ACM, 9: 684.
Von Neumann, J. 1951. Various techniques used in connection with random digits.
National Bureau of Standards Applied Mathematics, Series 12: 36–38.