MTL 106 (Introduction to Probability Theory and Stochastic Processes)

Tutorial Sheet No. 9 (CTMC)

 
−3 2 1
1. Consider a time-homogeneous CTMC {X(t), t ≥ 0} with the infinitesimal generator matrix Q =  1 −2 1 
1 4 −5
and initial distribution (0, 0, 1). Find P (τ > t) where τ denotes the first transition time of the Markov chain.
2. In orbit, there are two communication satellites. Each satellite has an exponential lifetime with a mean of µ1 .
A replacement is sent up if one fails. The preparation and sending up of a replacement takes an exponential
amount of time, with mean λ1 . Let Xt represent the number of satellites in orbit at the given moment.
Consider {Xt , t ≥ 0} as a CTMC with a state space {0, 1, 2}. Show that the infinitesimal generator matrix
is given by  
−λ λ 0
Q= µ −(λ + µ) λ .
0 2µ −2µ
Write down the Kolmogorov forward and backward equations for the above process.

5
3. Consider a CTMC {X(t), t ≥ 0} with state space {0, 1, 2} and

-2
 
−1 1 0

4
Q =  0 −1 1  .
1 0 −1

(a) Prove that its transition probability matrix is

02
r2
 
1 1 1
1 2
P(t) = 1 1 1 + R(t)
3 3
1 1 1
te

where  √ √ √ 
3
cos( 23 √− 2π cos( √23 + 2π
es

cos(
√ 2
) 3 ) 3 )
3 3
R(t) = cos( 2 + 2π 3 ) cos( ) cos( 23 √− 2π
3 )
.
 
√ √ 2
3 2π
cos( 23 + 2π 3
em

cos( 2 − 3 ) 3 ) cos( 2 )

(b) Show that the limiting distribution of P (t) as t → ∞ is π = (1/3, 1/3, 1/3) and
2 −3t/2 4
IS

X
e ≤ sup | pij (t) − πj |≤ e−3t/2 .
3 i
j
3

4. Let  
−3 1 2
Q= 2 −4 2 .
2 1 −3
Prove that its transition probability matrix is

2 + 3e−5t 1 − e−5t 2 − 2e−5t

 
1
P(t) = 2 − 2e−5t 1 + 4e−5t 2 − 2e−5t  .
5
2 − 2e−5t 1 − e−5t 2 + 3e−5t

hence, the limiting distribution of P (t) is π = (2/5, 1/5, 2/5) and

X 8 −5t
supi | pij (t) − πj |= e .
j
5

1
 5. The birth-death process is called a pure death process if λi =0 for all i. Suppose µi = iµ, i = 1, 2, 3, . . . and
initially X0 = n. Show that Xt has B(n, p) distribution with p = e−µt .
6. Consider birth-death process with λi =0 for all i. Suppose µi = µ, i = 1, 2, 3, . . .. Find the value of Pij (t) for
this process.

7. The birth-death process is called a birth process if µn =0 for every n. Suppose λn = nλ, n = 1, 2, 3, . . .. If
X0 = i show that
 
i−1
P (Xt = n|X0 = i) = Pin (t) = e−nλt (1 − e−λt )i−n , i ≥ n.
i−n

8. Consider a birth-death process with birth rate λi = (i + 1)λ, i ≥ 0 and µi = iµ, i ≥ 0. Find the expected
time to go from state 0 to 1.
9. Prove that the irreducible birth-death process is transient iff
∞
X µ1 . . . µn
< ∞, (1)

5
λ . . . λn
n=1 1

-2
positive recurrent iff
∞
λ1 . . . λn

4
X
< ∞, (2)
µ
n=1 1
. . . µn

and null recurrent iff neither (1) nor (2) holds.

02
r2
10. Consider a birth-death process with state space S = {0, 1, 2} and birth rates λ0 = µ2 = λ. Find P (X(t) =
n|X(0) = 0), n = 0, 1, 2.
11. Consider a birth-death process {X(t), t ≥ 0} with state space S = {0, 1, . . .} and birth and death rates of the
te

form λn = nλ and µn = nµ, n = P0, 1, . . . where λ and µ are non- negative constants. Assume Pii (j) = 1 for
∞
fixed i. Take m(t) = E(X(t)) = k=0 kPik (t). Prove that m(t) = ie(λ−µ)t .
es

12. Suppose there are n identical machines operating independently and serviced by a single repair crew. If a
machine breaks down while another is being repaired it must wait its turn before repairs can start. Assume
em

each machine has an operating time exponential with mean µ1 and a repair time exponential with mean λ1 .
Let X(t) = no. of machines in operating condition at time t. Model X(t) as a Markov chain with state
space E = {0, 1, 2, . . . , N }. Determine rate matrix Λ and the forward Kolmogorov equations. Determine the
IS

limiting equilibrium probability distribution of the process.

13. Every Wednesday night, free vision screenings are provided by the eye clinic at City Hospital. Three oph-
thalmologists are on call. The real-time is found to be spread with potential distribution around the average
test time of 20 minutes. Patients are accepted on a first-come, first-served basis and clients arrive according
to a Poisson process with a mean of 6/hr. The hospital planners are interested in knowing:
(a) What is the mean number of people waiting?
(b) What is the total amount of time a patient spends at the clinic?
(c) What is the average percentage idle time of each of the doctors?