Chapter 3: ‘Åê){

Contents

1 Methods for Generating Random Variables 1

1.1 Generating Uniform(0,1) random number . . . . . . . . . . 2
1.2 Random Generators of Common Probability Distribution
in R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.1 The Inverse Transform Method . . . . . . . . . . . . 8
1.2.2 The Acceptance-Rejection Method . . . . . . . . . . 18
1.2.3 Transformation Methods . . . . . . . . . . . . . . . . 22
1.2.4 Sums and Mixtures . . . . . . . . . . . . . . . . . . . 25
1.3 Multivariate Distribution . . . . . . . . . . . . . . . . . . . 33
1.3.1 Multivariate Normal Distribution . . . . . . . . . . . 33
1.3.2 Mixtures of Multivariate Normals . . . . . . . . . . . 40
1.3.3 Wishart Distribution . . . . . . . . . . . . . . . . . . 42
1.3.4 Uniform Distribution on the d−Sphere . . . . . . . . 43
1.4 Stochastic Process . . . . . . . . . . . . . . . . . . . . . . . 46

Previous Next First Last Back Forward 1

Chapter 1
Methods for Generating Random
Variables

ÚOOŽ¥˜‡Ä:¯KÒ´lA½VÇ©Ù¥)‘ÅCþ(‘Åê).
3OŽÅþ^êƐ{)(þ!) ‘Å괍U옽OŽ{ )
ê, §‚äkaqu(þ!) ‘ÅCþÕáÄS5Ÿ, ù êQ,
´ì(½Ž{), B،U´ý‘Åê, Ïd~r^êƐ{)
‘Åꡏ–‘Åê.
3{üœ/, l˜‡koN¥)˜‡‘Å*ÿ, ÒI‡˜«llÑ
þ! oN¥)‘Å*ÿ{. l , ˜‡Ü·þ!(–)‘Åê)ì´
lŠþ¤I‡, lÙ¦VÇ©Ù¥)‘Åêѝ6uþ!‘Åê)ì.
• –‘Åê،Uý‘Å; ´, ‡–‘ÅêU ÏL‘Åê˜XÚ

Previous Next First Last Back Forward 1

 Ou, ·‚ÒŒ±r§Šý‘Åê ˜% ¦^.
• é)–‘Åê?1ˆ«ÚObuyÙÎÜÕá5^‡ ©Ù
‡¦©Ù;
• ˜‡Ðþ!©ÙU(0,1) –‘Åêu)ì(Ž{) AäI‡e¡n
‡5Ÿ:
– )SAäkþ!oN‘ÅÚO5Ÿ: X©Ùþ!
5, Ä‘Å5, SmÕá5;
– )S„ݯ, Ó^S3, äkŒ­E5;

– )S±Ïv .

1.1 Generating Uniform(0,1) random number

‚5Ó{u)ì:
Xn+1 = (aXn + c) (mod) m

Previous Next First Last Back Forward 2

ùp:
(1) m: = 2 − 1(32 OŽÅŒ±†L«Œƒê).
31

(2) a: ¦ê, ‡%ÀJ

(3) c: Oþ(Œ±)=0.
(4) X : ЩŠ(«f).
0

• Xn ∈ {0, 1, . . . , m − 1}, Un = Xn /m.

•¦^Ûꊏ«f.
• “êÚ+nØkÏuaÀJ.

• F")¤ì±ÏéŒ.

• ؇)õ†m/1000‡ê.

3R¥, ¦^ÏÌK5 ).Random.seed½öRNGkind'u"Žþ

Previous Next First Last Back Forward 3
!‘Åê )ì[&E. ˆ«ØÓa.‘Åê)ì9Ù5ŸŒ±ë
˜ êŠOŽ¡].
1.2 Random Generators of Common Probability
Distribution in R

3 R ¥þ!–‘Åê)ì´runif. )–þ!‘Åꐪ
)  þݏ •þ
↑Example

)  þݏ •þ
runif{n} # 0 1 n

)01þnxmÝ
runif{n,a,b} # a b n
matrix(runif(n*m),nrow=n,ncol=m) #
↓Example

~^˜VÇ©ÙVǟþ¼ê(pmf)½öVǗݼê(pdf), \
È©Ù¼ê(cdf), © ê¼ê(quantile function)±9‘Åê )ì¼ê
3R¥®²8¤, 'Xé‘©Ù, Œ±ëwÏ©
↑Code
Previous Next First Last Back Forward 4
dbinom(x, size, prob, log = FALSE)
pbinom(q, size, prob, lower.tail = TRUE, log.p = FALSE)
qbinom(p, size, prob, lower.tail = TRUE, log.p = FALSE)
rbinom(n, size, prob)
↓Code

R ¥~„˜©Ù¼ê

Previous Next First Last Back Forward 5

©Ù cdf ‘Åê)ì ëê
‘©Ù
beta pbeta rbeta shape1, shape2

χ ©Ù
pbinom rbinom size, prob
2

ê©Ù
pchisq rchisq df

F©Ù
pexp rexp rate
pf rf df1,df2

AÛ©Ù
gamma pgamma rgamma shape,rate or scale

éê©Ù
pgeom rgeom prob

K‘©Ù
plnorm rlnorm meanlog,sdlog

©Ù
pnbinom rnbinom size, prob

Poisson©Ù
pnorm rnorm mean,sd

t©Ù
ppois rpois lambda

þ!©Ù
pt rt df
punif runif min,max

Previous Next First Last Back Forward 6

 ¦^sample¼êl˜‡klÑoN¥Ä:
Œ±¢ylkoN¥k˜£½öؘ£ü«Äª?1Ä.
sample

↑Example

•M1
#

i1 ˜‡˜†
sample(0:1,size=10,replace=TRUE)
# a-z

õ‘©ÙÄ
sample(letters)
#
x<-sample(1:3,size=100,replace=TRUE,prob=c(0.2,0.3,0.5))
#> table(x)
#x
# 1 2 3
#12 30 58
↓Example

Previous Next First Last Back Forward 7

1.2.1 The Inverse Transform Method

o ëY.|Ü
)¤‘Åê_C†{´Äu±eÙ½n
Theorem 1 (Probability Integral Transformation). eX ëY.‘ÅC
þ, Ùcdf F , K U = F
X X (X) ∼ U (0, 1).

Proof. ½Â

−1
FX (u) = inf{x : FX (x) = u}, 0 < u < 1.

e‘ÅCþU ∼ U (0, 1), Ké¤kx ∈ R, k

−1
P (FX (U ) ≤ x) = P (inf{t : FX (t) = U } ≤ x)

= P (U ≤ FX (x)) = FU (FX (x)) = FX (x).

ÏdF −1
X (U )ڑÅCþX Ó©Ù.
Previous Next First Last Back Forward 8
 l , e‡)X ˜‡‘Å*ÿx, Œ±

1. Ñ_C†¼ê F (u). −1
X

2. lþ!©ÙU (0, 1)¥)˜‡‘Åêu, -x = F −1

X (u).

ù«{‡¦F _¼ê‡N´¦Ñ.
~: ¦^_C†{)ëY.—Ýf (x) = 3x I(0 < x < 1)‘Å*ÿ 2

Š.
d?F X (x) = x3 , (0 < x < 1), Ïd F −1
X (u) = u1/3 , Ïd
↑Code
n<-1000
u<-runif(n)
x<-u^(1/3)
hist(x,prob=TRUE,main=expression(f(x)==3*x^2))
y<-seq(0,1,0.01)

Previous Next First Last Back Forward 9

lines(y,3*y^2)
↓Code

±ã¥êÆÎÒL«õSNëwÏ© ?plotmath.
~: ¦^_C†{)ê©Ù‘Åê.
X ∼ Exp(λ), Ïdéx > 0, F (x) = 1 − e
X , l F (u) = −λx −1
X

− log(1 − u). 5¿U Ú1 − U Ó©Ù, Ïd)ëê´λê©ÙÝ

1
λ
n ‘Åê·-
↑Code
-log(runif(n))/lmbda ↓Code

3R¥Œ±¦^rexp5).

Previous Next First Last Back Forward 10

 o lÑ.|Ü
X ˜lÑ.‘ÅCþ, ٌUŠP
· · · < xi−1 < xi < xi+1 < · · ·

½Â
−1
FX (u) = inf{x : FX (x) ≥ u}

K_C†F −1
X (u) = xi , Ù¥F X (xi−1 ) < u ≤ FX (xi ). l )˜‡‘Å
êxª

1. lþ!©ÙU (0, 1)¥)˜‡‘Åêu

2. x = x , eF (x
i ) < u ≤ F (x ).
X i−1 X i

Previous Next First Last Back Forward 11

 é, ©Ù5`, OŽF (x ) < u ≤ F (x )ŒU'(J. 3lÑ.
X i−1 X i

|ÜA^_C†{)‘ÅêØӐªŒ±ëDevroye1nÙ.
~: ¦^_C†{)0-1©Ù‘Åê.
džF (0) = f (0) = 1 − p±9F (1) = 1. Ïdeu > 1 − p,
X X X

KF (u) = 1; eu ≤ 1 − p, K F (u) = 0. Ïd
−1
X
−1
X

↑Code
n<-1000
p<-0.4
u<-runif(n)

n؊
x<-as.integer(u>1-p)

n؊
mean(x) # p
var(x) # p(1-p)
↓Code

3R¥,·‚¦^rbinom(n,1,p)5)0-1©Ùn˜‡‘Å*ÿ•þ.
~: ¦^_C†{)AÛ©Ù‘Åê.
duA۩٩ټêF (x) = 1 − q , x = 1, 2, · · · . Ïdz‡‘Å
X
x

Previous Next First Last Back Forward 12

ê чOŽ
1 − q x−1 < u ≤ 1 − q x .

ù‡Øªdux − 1 < log(1 − u)/log(q) ≤ x, Ù)x = dlog(1 −

u)/log(q)e, ùpdteL«Øutê(ceiling). Ïd

↑Code
n<-1000
p<-0.4
u<-runif(n)
x<-ceiling(log(1-u)/log(1-p))
↓Code

5¿U Ú1 − U Ó©Ù, ±9log(1 − U )/log(q)êVǏ0, l þã

“è ˜ÚŒ±d
↑Code
k<-floor(log(u)/log(1-p))+1 ↓Code

Previous Next First Last Back Forward 13

 3R¥,·‚¦^rgeom(n,p)5)ëêpA۩ُn˜‡‘
Å*ÿ•þ.
~: ¦^_C†{)Poisson©Ù‘Åê.
éPoisson©Ù5`, ¦)F (x − 1) < u ≤ F (x)'åA۩ٜ/ž‡E
,õ. )Poisson©Ù Đ{´ÏLXe4í'X)Ú;©Ù¼
ê:
λf (x)
f (x + 1) = ; F (x + 1) = F (x) + f (x + 1).
x+1
l Œ±ÏL;e©Ù¼êS¥Ï¦F (x − 1) < u ≤ F (x)). R¥
Œ±ÏLrpois(n,lambda)5)Poisson©Ùn˜|*ÿ•þ.
~: ¦^_C†{)Xeéê©Ù‘Åê.
aθx
f (x) = P (X = x) = , x = 1, 2, · · ·
x

Ù¥0 < θ < 1±9a = (−log(1 − θ)) −1 .

Previous Next First Last Back Forward 14

 w,, 4í'X
θx
f (x + 1) = f (x), x = 1, 2, · · ·
x+1

nØþù, f (x)Œ±ÏLXþ4í'X¦Š, ´ééŒxŠù«OŽ

{°ÝØ , ª ¬—f (x) = 0, F (x) < 1. Ïd·‚ÏLf (x)Lˆ
ªexp(log(a) + xlog(θ) − log(x))5 OŽf (x)Š. ¦)ت, òF (x)
Š;e5, ЩŒ±ÀJ˜‡ŒêN , ˜•þF (x), x = 1, · · · , N , , 3
k7‡žÿ2OŒN . , , éA½u¦)F (x − 1) < u ≤ F (x), OŽ
÷vF (x − 1) < ux‡ê. F •þž, F < u ˜Ü6•þ, ي i

TRUE½öFALSE, u > F Ú=x − 1. ¤±

i

↑Code
rlogarithmic<-function(n,theta){
#return a random logarithmic(theta) sample size n
u<-runif(n)
#set the initial length of cdf vector
N<-ceiling(-16/log10(theta))

Previous Next First Last Back Forward 15

 k<-1:N
a<--1/log(1-theta)
fk<-exp(log(a)+k*log(theta)-log(k))
Fk<-cumsum(fk)
x<-integer(n)
for(i in 1:n){
x[i]<-as.integer(sum(u[i]>Fk))
while(x[i]==N){
logf<-log(a)+(N+1)*log(theta)-log(N+1)
fk<-c(fk,exp(logf))
Fk<-c(Fk,Fk[N]+fk[N+1])
N<-N+1
x[i]<-as.integer(sum(u[i]>Fk))
}
}
x+1
}
↓Code

l ) logarithmic(0.5) ©Ù‘Åê
Previous Next First Last Back Forward 16
 ↑Example
n<-1000
theta<-0.5

OŽéê©Ù©ÙÆ5é'
x<-rlogarithmic(n,theta)
#
k<-sort(unique(x))

IO
p<--1/log(1-theta)*theta^k/k
se<-sqrt(p*(1-p)/n) #
↓Example

e¡(JL²ªÇ©ÙÎÜnØ©Ù:
↑Result

> round(rbind(table(x)/n,p,se),3)
1 2 3 4 5 6 7 8 9
0.720 0.185 0.058 0.018 0.012 0.003 0.002 0.001 0.001
p 0.721 0.180 0.060 0.023 0.009 0.004 0.002 0.001 0.000
se 0.014 0.012 0.008 0.005 0.003 0.002 0.001 0.001 0.001
↓Result

Previous Next First Last Back Forward 17

1.2.2 The Acceptance-Rejection Method

bX ÚY ´‘ÅCþ, ÙVǼê©Of Úg. ÷v

f (t)
≤ c, ∀ t s.t. f (t) > 0
g(t)

KÀ{(Acceptance-Rejection Method)Œ±^5)¤X ‘Åê.

阇Œ±B)¤‘Åê‘ÅCþY , ÙVǼ
êg÷vf (t)/g(t) ≤ c, ∀ t s.t. f (t) > 0.
1.

2. lg ¥)˜‡‘Åêy.

3. lþ!©ÙU (0, 1)¥)˜‡‘Åêu.

4. eu < f (y)/(cg(y)), KÉx = y, ÄKáýy. ­E2-

4, †–)‰½‡êx.

Previous Next First Last Back Forward 18

 3lÑ.|Ü,
h f (Y ) i X f (y) 1
P (A) = E[P (A|Y )] = E = g(y) = .
cg(Y ) y
cg(y) c

éz‡¦f (k) > 0kk

P (A|k)g(k) [f (k)/(cg(k))]g(k)
P (k|A) = = = f (k).
P (A) 1/c

éëY.‘ÅCþ|Ü, =Iy²P (Y ≤ y|U ≤ f (Y )

cg(Y )
) = FX (y). ¯¢þ
f (Y )
f (Y ) P (U ≤ cg(Y )
,Y ≤ y)
P (Y ≤ y|U ≤ ) =
cg(Y ) 1/c
f (Y )
Z y P (U ≤ cg(Y )
|Y = ω ≤ y)
= g(ω)dω
−∞ 1/c
Z y
f (ω)
= c g(ω)dω
−∞ cg(ω)
= FX (y)

À{A::
Previous Next First Last Back Forward 19
 • `:´OŽžmؑXxO\;
• ":´Ï¦˜‡N´)‘Åê %Cf éЩÙg '(J, Ïd˜
‡ØÐg¬—ÉVÇé$,E¤ $1žmL.
~: ¦^À{lXe©Ù)‘Åê.
f (x) = 6x(1 − x), 0<x<1

ld©Ù¥)n‡‘ÅêI‡Ì‚oê6uÉVÇ1/cŒ.
g(x)U (0, 1)—Ý, Kc = 6, l lg¥)˜‡‘ÅêxÉ, ؚ
f (x)
= x(1 − x) > u
cg(x)

Ïd,²þ5wI‡cnĝ, =2cn‡‘ÅêI‡).
↑Code
n<-1000
j<-k<-0

Previous Next First Last Back Forward 20

y<-numeric(n)
while(k<n){
u<-runif(1)

l ¥)˜‡‘Åê
j<-j+1
x<-runif(1) # g
if(x*(1-x)>u) {
k<-k+1
y[k]<-x
}
}
#>j
#[1] 6153
↓Code

3ùg[¥, I‡6153ĝ5)n = 1000‡I‡‘Åê. 'Ù²

© êÚnØ© ê
↑Example

p<-seq(.1,.9,.1)
Qhat<-quantile(y,p)

Previous Next First Last Back Forward 21

Q<-qbeta(p,2,2)
se<-sqrt(p*(1-p)/(n*dbeta(Q,2,2)))
round(rbind(Qhat,Q,se),3)
↓Example

—Ý‚C0žI‡Œþ­E5O© ê.
1.2.3 Transformation Methods

Øc¡0 { , NõC†a.Œ±^5)¤‘Åê. 'X

1. eZ ∼ N (0, 1), KZ ∼ χ (1).
2 2

2. eU ∼ χ (m) ±9 V ∼ χ (n), KF =
2 2 U/m
∼ F (m, n).
V /n

3. eZ ∼ N (0, 1) ±9 V ∼ χ (n) ƒpÕá, KT = √

2 ∼ t(n).Z
V /n

4. eU, V ∼ U (0, 1) ƒpÕá, KZ = −2logU cos(2πV )†Z

√
1 2 =
−2logV sin(2πU ) ƒpÕáIO‘ÅCþ.
√

Previous Next First Last Back Forward 22

'X
• eU ∼ Gamma(r, λ), V ∼ Gamma(s, λ), ƒpÕá, K U
U +V
∼
Beta(r, s).

~: ¦^C†{l)éê©Ù‘Åê.
5¿eU, V ∼ U (0, 1) ƒpÕá, K
 
logV
X = 1+
log(1 − (1 − θ)U )

Ñlëêθéê©Ù. l
1. lU (0, 1)¥)u

2. lU (0, 1)¥)v

3. x = b1 + log(v)/log(1 − (1 − θ) u )c

Previous Next First Last Back Forward 23

 ¢y“è
↑Code
n<-1000
theta<-0.5
u<-runif(n)
v<-runif(n)

OŽéê©ÙVÇ
x<-floor(1+log(v)/log(1-(1-theta)^u))
k<-1:max(x)#
p<--1/log(1-theta)*theta^k/k
se<-sqrt(p*(1-p)/n)
p.hat<-tabulate(x)/n
round(rbind(p.hat,p,se),3)
↓Code

ƒ'u_C†{, C†{kÇ(Ÿo?)
↑Code
rlogarithmic<-function(n,theta){
stopifnot(all(theta>0 & theta <1))
th<-rep(theta,length=n)
u<-runif(n)

Previous Next First Last Back Forward 24

 v<-runif(n)
x<-floor(1+log(v)/log(1-(1-theta)^u))
return(x)
}
↓Code

1.2.4 Sums and Mixtures

‘ÅCþÚ½ö·Ü´˜«AÏa.C†.
Úòȃ'©Ùk
• χ (ν): ν ‡ iid IO©Ù²òÈ.
2

• K‘©Ù NegBin(r, p): r ‡ iid Geom(p) òÈ.

• Gamma(r, λ): r ‡Õá Exp(λ) òÈ.

~: ¦^‘ÅCþÚ)χ ©Ù‘Åê. 2

Previous Next First Last Back Forward 25

 ν ‡IO‘ÅCþ²Úχ (ν)©Ù‘ÅCþ, Ïd2

↑Code
n<-1000
nu<-2

{
X<-matrix(rnorm(n*nu),n,nu)^2
# 1

{
y<-rowSum(X)
# 2
y<-apply(X,MARGIN=1,FUN=sum)
↓Code

é‘ÅCþ·Ü, ·‚kw½Â:

Previous Next First Last Back Forward 26

¡ ˜ ‡ ‘ Å C þ  l Ñ · Ü, X J Ù © Ù  , ‘ Å C
þX , X , · · · ©Ù\:
1 2

Definition
X
FX (x) = θi FXi (x).

θi > 0
P
­.
θi = 1

éëY.‘ÅCþ, aqk

¡˜‡‘ÅCþëY·Ü, XJ٩ُ,‡©ÙxX|Y = y\

: Z
FX (x) = FX|Y =y (x)fY (y)dy. Definition

Ù¥R f Y (y)dy = 1.

Previous Next First Last Back Forward 27

 ~: )Xe·Ü©Ù‘Åê.
FX (x) = pFX1 (x) + (1 − p)FX2 (x)

= p Pr(X1 ≤ x) + (1 − p) Pr(X2 ≤ x)

= Pr(B = 1) ∗ Pr{B ∗ X1 + (1 − B) ∗ X2 ≤ x|B = 1}

+ Pr(B = 0) ∗ Pr{B ∗ X1 + (1 − B) ∗ X2 ≤ x|B = 0}

= Pr{B ∗ X1 + (1 − B) ∗ X2 ≤ x},

ùp B ∼ Binomial(p).
)d‘Åê{w,
1. )˜‡êk ∈ {1, 2}, ùpP (1) = p, P (2) = 1 − p.

2. ek = 1, KlF ¥)x , ¿-x = x .

X1 1 1

3. ek = 2, KlF ¥)x , ¿-x = x .

X2 2 2

Previous Next First Last Back Forward 28

 ~: Γ©Ù·Ü.
5
X
FX = θj FXj
i=1

Ù¥X j ∼ Gamma(r = 3, λj = 1/j) ƒpÕá, θ j = j/15, j = 1, · · · , 5.

↑Code
n<-1000
k<-sample(1:5,size=n,replace=TRUE,prob=(1:5)/15)
rate<-1/k

x·Ü—ݱ9©þ—Ý
x<-rgamma(n,shape=3,rate=rate)
#
plot(density(x),xlim=c(0,40),ylim=c(0,.3),lwd=3,xlab="x",main="")
for(i in 1:5)
lines(density(rgamma(n,3,1/i)))
↓Code

Previous Next First Last Back Forward 29

 ~: Γ—Ý·Ü.
5
X
f (x) = θj fj (x), x > 0
j=1

Ù¥f Gamma(3, λ )—Ý. xd·Ü—Ýã/, I‡OŽÙŠ

j j

↑Code
f<-function(x,lambda,theta){
sum(dgamma(x,3,lambda)*theta)
}
↓Code

ùpdgamma(x, 3, lambda) ∗ theta´•þ(θ f (x), · · · , θ f (x)).

1 1 5 5

↑Code

¦^ ¼êI‡
x<-seq(0,8,length=200)
dim(x)<-length(x) # apply
p<-c(.1,.2,.2,.3,.2)

OŽ·Ü—Ý3 ?Š
lambda<-c(1,1.5,2,2.5,3)
# x
y<-apply(x,1,f,lambda=lambda,theta=p)

Previous Next First Last Back Forward 30

plot(x,y,type="l",ylim=c(0,.85),lwd=3,ylab="Density")
for(j in 1:5){
y<-apply(x,1,dgamma,shape=3,rate=lambda[j])
lines(x,y)
}
↓Code

Previous Next First Last Back Forward 31

 ~: P oisson − Gamma·Ü.
eX|Λ = λ ∼ P ois(λ), Λ ∼ Gamma(r, β), K X ∼ N B(r, p = β/(1 +
β)). ~`²˜‡Poisson-Gamma·Ü,¿ÚK‘ ©Ùƒ'.

↑Code
#Poisson-Gamma Mixture
n<-1000
r<-4

´‘Å
beta<-3
lambda<-rgamma(n,r,beta) #lambda
x<-rpois(n,lambda)
#compare with negative binomial
mix<-tabulate(x+1)/n
negbin<-round(dnbinom(0:max(x),r,beta/(1+beta)),3)
se<-sqrt(negbin*(1-negbin)/n)
round(rbind(mix,negbin,se),3)
↓Code

Previous Next First Last Back Forward 32

1.3 Multivariate Distribution
1.3.1 Multivariate Normal Distribution

˜‡‘Å•þX = (X , · · · , X )Ñld©ÙN (µ, Σ), XJÙéܗÝ

1 d d

k/ª
1
f (x) = (2π)−d/2 |Σ|−1/2 exp{− (x − µ)0 Σ−1 (x − µ)}, x ∈ Rd .
2

)N (µ, Σ)˜‡‘Åê, ˜„ÏLüÚ¢y:

d

1. )i.i.d.IO©Ù‘ÅCþZ , · · · , Z . 1 d

2. òZ = (Z , · · · , Z )C†X = (X , · · · , X )¦ÙÏ"µ,
1 d 1 d

Σ:
X = CZ + µ,

ùpCC = Σ. ©)ӌ±´Ì©){(eigen), Choleski©)(chol)½

0

öÛɊ©)(svd).
Previous Next First Last Back Forward 33
 ˜„جé‘Å•þ˜‡?1˜g‚5C†, ²~´‡é¤
k|¤Ý ?1C†. bZ = (Z )´ ˜‡n × dÝ , Ù ij

¥Z iid N (0, 1). KZ 1´n‡d IO‘ÅCþ*ÿ. KdžI‡

ij

?1C†´
X = ZQ + JµT ,

Ù¥Q Q = Σ, J = J = (1, · · · , 1) . KX 1´N (µ, Σ)‘Åê. l

T
n×1
T
d

o(Xe
1. )˜‡kIO©Ù‘Åê¤n × dÝ Z.

2. OŽ©)Σ = Q Q. T

3. A^C† X = ZQ + Jµ . T

Ù¥X = ZQ + Jµ 3R¥Œ±Xe¢y
T

↑Code

Previous Next First Last Back Forward 34

Z<-matrix(rnorm(n*d),nrow=n,ncol=d)
X<-Z%*%Q+matrix(mu,n,d,byrow=TRUE)
↓Code

)¤N (µ, Σ)‘ÅêÌ©){ du

d

Σ1/2 = P Λ1/2 P T

¤±
↑Code
mu <- c(0, 0)
Sigma <- matrix(c(1, .9, .9, 1), nrow = 2, ncol = 2)
rmvn.eigen <-
function(n, mu, Sigma) {
# generate n random vectors from MVN(mu, Sigma)
# dimension is inferred from mu and Sigma
d <- length(mu)
ev <- eigen(Sigma, symmetric = TRUE)
lambda <- ev$values

Previous Next First Last Back Forward 35

 V <- ev$vectors
R <- V %*% diag(sqrt(lambda)) %*% t(V)
Z <- matrix(rnorm(n*d), nrow = n, ncol = d)
X <- Z %*% R + matrix(mu, n, d, byrow = TRUE)
X
}
# generate the sample
X <- rmvn.eigen(1000, mu, Sigma)
plot(X, xlab = "x", ylab = "y", pch = 20)
print(colMeans(X))
print(cor(X))
↓Code

) ¤N (µ, Σ)‘ Å ê  Û É Š © )  { ŠâÛɊ©)´Σ

d
1/2 =
U D V . ¤±
1/2 T

↑Code
rmvn.svd <-
function(n, mu, Sigma) {
# generate n random vectors from MVN(mu, Sigma)
# dimension is inferred from mu and Sigma

Previous Next First Last Back Forward 36

 d <- length(mu)
S <- svd(Sigma)
R <- S$u %*% diag(sqrt(S$d)) %*% t(S$v) #sq. root Sigma
Z <- matrix(rnorm(n*d), nrow=n, ncol=d)
X <- Z %*% R + matrix(mu, n, d, byrow=TRUE)
X
}
↓Code

)¤N (µ, Σ)‘ÅêCholeski©){

d

↑Code
rmvn.Choleski <-
function(n, mu, Sigma) {
# generate n random vectors from MVN(mu, Sigma)
# dimension is inferred from mu and Sigma
d <- length(mu)
Q <- chol(Sigma) # Choleski factorization of Sigma
Z <- matrix(rnorm(n*d), nrow=n, ncol=d)
X <- Z %*% Q + matrix(mu, n, d, byrow=TRUE)
X
}

Previous Next First Last Back Forward 37

 ↓Code

'ˆ«)¤ì5U
·‚®²?Ø )‘ÅêØӐ{, @o=«{ÐQ? ˜«Ä
Œ±žmE,5, O¡ȱ´Šâ[8´O˜‡½õ‡ë
ê, ½ö´Oþ , ?1 µ(3± ‘§¥ò9).
ùp·‚¦^system.time5µˆ‡)¤ìžmE,5.
↑Code
library(MASS)
library(mvtnorm)
n <- 100 #sample size
d <- 30 #dimension
N <- 2000 #iterations
mu <- numeric(d)

set.seed(100)
system.time(for (i in 1:N)

Previous Next First Last Back Forward 38

 rmvn.eigen(n, mu, cov(matrix(rnorm(n*d), n, d))))
set.seed(100)
system.time(for (i in 1:N)
rmvn.svd(n, mu, cov(matrix(rnorm(n*d), n, d))))
set.seed(100)
system.time(for (i in 1:N)
rmvn.Choleski(n, mu, cov(matrix(rnorm(n*d), n, d))))
set.seed(100)
system.time(for (i in 1:N)
mvrnorm(n, mu, cov(matrix(rnorm(n*d), n, d))))
set.seed(100)
system.time(for (i in 1:N)
rmvnorm(n, mu, cov(matrix(rnorm(n*d), n, d))))
set.seed(100)
system.time(for (i in 1:N)
cov(matrix(rnorm(n*d), n, d)))

detach(package:MASS)
detach(package:mvtnorm)
↓Code

Previous Next First Last Back Forward 39

3õ©Ù‘Åê)¤L§¥, ŒÜ©´óŠ´©) Ý . ùp¦
^  Ý ´ü  Ý , Ïd, ‘Å) Ý Σ
3zĝž´Cz, ´Σš~Cü .  3Ә‡ Ý e
'ˆ«ØӐ{, zg$1 Ñò‘Åê«f¡E.  ˜g$1==´
)  Ý , ±Úožm'.
1.3.2 Mixtures of Multivariate Normals

õ·Ü
pNd (µ1 , Σ1 ) + (1 − p)Nd (µ2 , Σ2 )

)d·Ü©Ùn‡‘Å•þ:
↑Code
library(MASS) #for mvrnorm
loc.mix <- function(n, p, mu1, mu2, Sigma) {
#generate sample from BVN location mixture
n1 <- rbinom(1, size = n, prob = p)

Previous Next First Last Back Forward 40

 n2 <- n - n1
x1 <- mvrnorm(n1, mu = mu1, Sigma)
x2 <- mvrnorm(n2, mu = mu2, Sigma)
X <- rbind(x1, x2) #combine the samples
return(X[sample(1:n), ]) #mix them
}
↓Code

^d§S)1000‡4‘©Ù‘Å•þ:
↑Example

x <- loc.mix(1000, .5, rep(0, 4), 2:5, Sigma = diag(4))

r <- range(x) * 1.2
par(mfrow = c(2, 2))
for (i in 1:4)
hist(x[ , i], xlim = r, ylim = c(0, .3), freq = FALSE,
main = "", breaks = seq(-5, 10, .5))
↓Example

Previous Next First Last Back Forward 41

1.3.3 Wishart Distribution

eM = X X, X lN (0, Σ)¥Än × d‘ÅÝ , KM ÑlWishart©

T
d

ÙW (Σ, n). d = 1ž, W (σ , n) = σ χ (n).

d 1
2 2 2

w,, lWishart©Ù¥)‘ÅꌱÏLXeª:
1. lN (0, Σ)¥)¤X
d

2. -W = X X. T

ù«{´«$ǐ{. ù´du7L)nd‡‘Åê5û½d(d +
1)/2‡ƒŠ. ˜«Ǎp{´ÄuBartlett©): -T = (T ) ij

˜d × denÝ , Ùƒ÷v
1. Tij i.i.d ∼ N (0, 1), i > j.
p
2. Tii ∼ χ2 (n − i + 1), i = 1, · · · , d.

Previous Next First Last Back Forward 42

KÝ A = T T ÑlW (I , n). Ïd)W (Σ, n)Œ±ÏLΣCholeski©
T
d d d

)Σ = LL , KLAL ∼ W (Σ, n).

T T
d

1.3.4 Uniform Distribution on the d−Sphere

¥¡=8Ü{x ∈ R : kxk = 1}. 3d−¥¡þþ!©Ù‘Å•þkƒ

d− d 2

ÓŒU•. )d‘ÅꌱÄuXe5Ÿ:
eX , · · · , X i.i.d N (0, 1), K(U , · · · , U )ÑlR ¥ü ¥¡þþ!
1 d 1 d
d

©Ù, Ù¥
Xj
Uj = , j = 1, · · · , d. (1.1)
kxk
Ž{Xe: éz‡u , ­E i

1. lN (0, 1)¥)‘Åêx , · · · , x i1 id

2. OŽîªkxk = (x + · · · + x ) 2
i1
2 1/2 .
id

3. -u = x /kxk, u = (u , · · · , u
ij ij i i1 id ).

Previous Next First Last Back Forward 43

 3R¥Œ±ÏLXeªJpÇ:
1. )nd‡‘Åê¤n × d‘Ý M , Ù1i1éAu1i‡‘Åê
•þ.
2. éz˜1OŽ(1.1),rdn‡Š3•þL¥.

3. éz‡êM [i, j]رL[i], U .

¢y“èXe
↑Code
runif.sphere <- function(n, d) {
# return a random sample uniformly distributed
# on the unit sphere in R ^d
M <- matrix(rnorm(n*d), nrow = n, ncol = d)
L <- apply(M, MARGIN = 1,
FUN = function(x){sqrt(sum(x*x))})
D <- diag(1 / L)
U <- D %*% M

Previous Next First Last Back Forward 44

 U
}
↓Code

xÑ200‡2‘ ±þþ!©Ù‘ÅêÑ:ã:
↑Example

X <- runif.sphere(200, 2)
par(pty = "s")
plot(X, xlab = bquote(x[1]), ylab = bquote(x[2]))
par(pty = "m")
↓Example

Previous Next First Last Back Forward 45

1.4 Stochastic Process

àgPoissonL§
ÏL[5ïÄOêL§, ·‚Œ±) L§˜‡kžm´».
˜‡OêL§{N (t), t ≥ 0}„Ç´λàgPoissonL§, X
J
1. N (t)kÕáOþ5
(λt)n −λt
2. P (N (s + t) − N (s) = n) = n!
e .

éàgPoissonL§, ¯‡5m T , T , · · · ´i.i.dëêλ 

1 2

ê©Ù‘ÅCþ. Ïd, )àgPoissonL§{)5m žm

T , · · · , T , · · · , K1n‡žm5žS = T + · · · + T , S
1 n n 1 n

Previous Next First Last Back Forward 46

{Tn }∞
n=1½ö {S } =Œ±L«L§˜‡´».∞
n n=1

, ˜«ª´|^S , · · · , S |N (t) = nÓ©ÙulU (0, t)¥Än‡

1 n

{ügSÚOþ. L§3‰½žt?Gu3[0, t]Sˆê

8,min{k : S > t} − 1. = eS ´ˆžm‡Lt, KN (t) = n − 1.
k n

[˜‡àgPoissonL§3[0, t ]«mþGŽ{Xe 0

1. ˜S = 0
1

2. é÷vS ≤ t  j = 1, 2, . . .,
j 0

(a) )T ∼ exp(λ). j

(b) -S = T + · · · + T .
j 1 j

3. N (t0 ) = minj (Sj > t0) − 1.

'X, ‡)„Ǐ2àgPoissonL§3[0, 3]SG,

↑Code
lambda <- 2
t0 <- 3

Previous Next First Last Back Forward 47

 Tn <- rexp(100, lambda) #interarrival times
Sn <- cumsum(Tn) #arrival times
n <- min(which(Sn > t0))-1 #arrivals in [0, t0]
↓Code

¦^, ˜«ªÒ´
↑Code
lambda <- 2
t0 <- 3
upper <- 100
pp <- numeric(10000)
for (i in 1:10000) {
N <- rpois(1, lambda * upper)
Un <- runif(N, 0, upper) #unordered arrival times
Sn <- sort(Un) #arrival times
n <- min(which(Sn > t0)) #arrivals+1 in [0, t0]
pp[i] <- n - 1 #arrivals in [0, t0]
} ↓Code

Previous Next First Last Back Forward 48

 dPoisson©Ù5Ÿ, N (3)Ï"ڐ ÑAT´6:
c(mean(pp), var(pp))

šàg Poisson L§
˜‡OêL§{N (t), t ≥ 0}rÝ´λ(t)PoissonL§, XJ
1. N (t)kÕáOþ5

, Ù
(m̃(s,t))n −m̃(s,t)
2. P (N (s + t) − N (s) = n) = e
¥m̃(s, t) = R λ(y)dy.
n!
s+t
s

XJ˜‡šàgPoissonL§rݼêk., =λ(t) ≤ λ < ∞,, KŒ± 0

ÏL l˜‡àgPoissonL§Ä). ù´ÏlrݏλàgPoissonL
§¥ Ä, ¦3žtu)¯‡±λ(t)/λÉ(Oê),  Oê
L§ÒÑlrݏλ(t)šàg PoissonL§. )[0, t ]šàgPoissonL 0 [ Ÿo?]
§Œ±ÏLdŽ{¢y. k阇λ < ∞, ¦λ(t) ≤ λ , ∀0 ≤ t ≤ t .
0 0 0

Previous Next First Last Back Forward 49

, l rݏλ àgPoissonL§¥)¯‡5ž{S }, , ±V
0 j

Çλ(S )/λ Éz‡S . Ž{Xe

j 0 j

1. ˜S = 0 1

2. é÷vS ≤ t  j = 1, 2, . . .,
j 0

(a) )T ∼ exp(λ ), -S = T + · · · + T .

j 0 j 1 j

(b) )þ!‘ÅêU ∼ U (0, 1). j

(c) eU ≤ λ(S )/λ , KÉS , -I = 1, ÄK-I = 0.

j j 0 j j j

3. ˆ£ˆžm {S : I = 1}. j j

'Xλ(t) = 3cos (t), Kλ(t) ≤ 3 = λ . Ïd1j ‡ˆžmU ≤

2
0 j

3cos (S )/3 = cos (S )ž É.

2
j
2
j

↑Example

lambda <- 3
upper <- 100

Previous Next First Last Back Forward 50

 N <- rpois(1, lambda * upper)
Tn <- rexp(N, lambda)
Sn <- cumsum(Tn)
Un <- runif(N)
keep <- (Un <= cos(Sn)^2) #indicator, as logical vector
Sn[keep]
round(Sn[keep], 4)
↓Example

L§3žt?G=
sum(Sn[keep]<=t)

#L§
˜‡OêL§{N (t), t ≥ 0}#L§, XJ ˆžmm
T , T , . . .´i.i.d.
1 2

Previous Next First Last Back Forward 51

 ~: bˆžmm ÑlëêpAÛ©Ù, =ˆžmm šK
ê, Ïd S = T + · · · + T ÑlK‘©Ù,ëêjÚp, Ïd
j 1 j

↑Example

t0 <- 5
Tn <- rgeom(100, prob = .2) #interarrival times
Sn <- cumsum(Tn) #arrival times
n <- min(which(Sn > t0)) #arrivals+1 in [0, t0]
↓Example

N (t0 ) ©ÙŒ±ÏL­EXþL§5O
↑Example

Nt0 <- replicate(1000, expr = {

Sn <- cumsum(rgeom(100, prob = .2))
min(which(Sn > t0)) - 1

Previous Next First Last Back Forward 52

 })
table(Nt0)/1000
Nt0
↓Example

þŠ¼êEN (t)Œ±ÏLXeO
↑Example

t0 <- seq(0.1, 30, .1)

mt <- numeric(length(t0))
for (i in 1:length(t0)) {
mt[i] <- mean(replicate(1000,
{
Sn <- cumsum(rgeom(100, prob = .2))
min(which(Sn > t0[i])) - 1
}))

Previous Next First Last Back Forward 53

}
plot(t0, mt, type = "l", xlab = "t", ylab = "mean")
abline(0, .25)
↓Example

é¡‘ÅiÄ
X , X , · · · ´˜i.i.d‘ÅCþ
P , ÷vPr(X
Pr(X = −1) = 1/2, -S = X , ±9S K
1 2 i = 1) =
n

L§{S , n ≥ 0}¡é¡‘ÅiÄ.
i n i=1 i 0 = 0,
n

)顑ÅiÄL§é{ü
n <- 400
incr <- sample(c(-1, 1), size = n, replace = TRUE)
S <- as.integer(c(0, cumsum(incr)))

Previous Next First Last Back Forward 54

 plot(0:n, S, type = "l", main = "", xlab = "i")

eЩS = 0, L§3žm[1, 400]Sˆ£0gêwhich(S==0).

0

XJI‡´L§S 3žmnžG, Ø´{¤Š, é'Œn, Œ

n

±ÏLXeª5 pÇ):
bS = 0L§ЩG, eL§3žmnƒcˆ£0, K3
0

)S žÿ, ·‚Œ± ÑL§3C˜gˆ£0žƒcžm.

n

-T L§Ägˆ£0ž, K)S , Œ±{zd¯Kk)–ž n

mT , †–ožm(T ƒÚ)Äg‡Ln, , lnžƒcC˜gˆ£0

žm©, )OþX , , ò§‚ƒ\.
i

T ©Ù

n−1 1 Γ(2n − 1)
p2n = P (T = 2n) = C2n−2 = , n ≥ 1.
n22n−1 n22n−1 Γ2 (n)
P (T = 2n + 1) = 0, n ≥ 0.

KŽ{Xe:
Previous Next First Last Back Forward 55
-W 1jgˆ£Ð© ˜I‡–žm,
j

1. ˜W = 0 1

2. é÷vW ≤ n j = 1, 2, . . .,
j

(a) lžm©Ù¥)T , †–Ägˆ£0. j

(b) -W = T + · · · + T .
j 1 j

3. -t = W − T
0 j j

4. ˜s = 0. 1

5. )lt + 1žnOþ. éi = 1, 2, . . . , n − t
0 0

(a) )‘ÅOþx ∼ P (X = ±1) = 1/2. i

(b) ˜s = x + · · · + x .i 1 i

(c) es = 0, ˜Oêìi = 1.
i

6. ˆ£s . i

Previous Next First Last Back Forward 56

 lT ©Ù¥pÇ)‘Åê{ë„ Devroye, p.754. ùp·‚
¦^˜‡{ü´1ÇØp{. 5¿
1
p2n = P (X = n − 1), X ∼ B(2n − 2, 1/2)
2n

Ïd†OŽ
↑Example

#compute the probabilities directly

n <- 1:10000
p2n <- exp(lgamma(2*n-1)
- log(n) - (2*n-1)*log(2) - 2*lgamma(n))
↓Example

½ö¦^‘©Ù±9_C†{OŽ
↑Example

Previous Next First Last Back Forward 57

P2n <- (.5/n) * dbinom(n-1, size = 2*n-2, prob = 0.5)
pP2n <- cumsum(P2n)
#|^_C†{)˜‡T
# u<-runif(1)
# Tj<-2*(1+sum(u>pP2n))
↓Example

y3b‰½n,·‚‡OŽ3(0, n]S ˜gˆ£0žm, K

↑Example

#given n compute the time of the last return to 0 in (0,n]

n <- 200
sumT <- 0
while (sumT <= n) {
u <- runif(1)

Previous Next First Last Back Forward 58

 s <- sum(u > pP2n)
if (s == length(pP2n))
warning("T is truncated")
Tj <- 2 * (1 + s)
#print(c(Tj, sumT))
sumT <- sumT + Tj
}
sumT - Tj
↓Example

Previous Next First Last Back Forward 59