Scribd
Upload
11 views

HW9Solutions PDF

This document contains solutions to homework problems for the course "Stochastic Manufacturing and Service Models" taught in Fall 2015. Problem 2 asks to calculate the limiting distribution of a discrete-time Markov chain (DTMC) with 4 states and a specified transition matrix. The solution finds the limiting distribution by first identifying recurrent and transient states, then setting up and solving equations for the stationary probabilities of recurrent states and probabilities of ending in each communication class from transient states. The limiting distribution is calculated to be a mixed distribution with probabilities [0.5, 0.5, 0.33, 0.33] across the 4 states.

Uploaded by

David Lee
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
11 views

HW9Solutions PDF

This document contains solutions to homework problems for the course "Stochastic Manufacturing and Service Models" taught in Fall 2015. Problem 2 asks to calculate the limiting distribution of a discrete-time Markov chain (DTMC) with 4 states and a specified transition matrix. The solution finds the limiting distribution by first identifying recurrent and transient states, then setting up and solving equations for the stationary probabilities of recurrent states and probabilities of ending in each communication class from transient states. The limiting distribution is calculated to be a mixed distribution with probabilities [0.5, 0.5, 0.33, 0.33] across the 4 states.

Uploaded by

David Lee
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

ISyE 3232

Z. She , YL. Chang

Stochastic Manufacturing and Service Models

Fall 2015

Solutions to Homework 9
October 23, 2015
1. Please refer to the lecture note.
2. Let X = {Xn : n = 0, 1, . . .} be a DTMC on state space {1, 2, 3, 4} with transition matrix

1/2 1/2 0
0
1/2 1/2 0
0

P =
1/4 1/4 1/4 1/4
0
0
0
1
calculate limn P n .
0.5

0.5

0.5

2
0.5

0.25

0.25

0.25

0.25

Step 1: Identify classes:


{1,2}, recurrent class
{3}, transient class
{4}, recurrent class (absorption)
Step 2: Work on rows of P for recurrent classes

=
2
3
4

1
1
1
?
0

2
2
2
?
0

3
0
0
?
0

4
0

0
.
?
4

where
(1

2 ) = (1



1/2 1/2
2)
1/2 1/2
1 + 2 = 1

And clearly, 4 = 1.
Step 3: Work on rows of P for transient classes

(1

2 ) = (0.5

0.5)

=
2
3
4

1
0.5
0.5

2
0.5
0.5

f3,{1,2} 1
0

f3,{1,2} 2
0

3
4
0
0

0
0
.
0 f3,4
0
1

where:
f3,{1,2} = P (X {1, 2} | X0 = 3)
f3,4 = P (X = 4 | X0 = 3)
Step 4: Solve the f3,{1,2} , f3,4 and complete the matrix.

f3,{1,2} = P (X {1, 2} | X0 = 3) =

4
X

P (X {1, 2}, X1 = j | X0 = 3)

j=1

4
X

P (X {1, 2} | X1 = j, X0 = 3)P (X1 = j | X0 = 3)

j=1

4
X

P (X {1, 2} | X1 = j)P (X1 = j | X0 = 3)

j=1

= P (X {1, 2} | X1 = 1)P31 + P (X {1, 2} | X1 = 2)P32


+ P (X {1, 2} | X1 = 3)P33 + P (X {1, 2} | X1 = 4)P34
= 1 1/4 + 1 1/4 + f3,{1,2} 1/4 + 0 1/4
Then,
f3,{1,2} = 0.5 + 0.25 f3,{1,2}

f3,{1,2} 0.67

f3,{1,2} 1 = f3,{1,2} 2 0.33


f3,4 = 1 f3,{1,2} 0.33
Finally, P is

1
1 0.5

2 0.5

3 0.33
4
0

2
3
4
0.5 0
0

0.5 0
0
.
0.33 0 0.33
0
0
1

You might also like