Scribd
Upload
120 views

Assignment 3

This document contains instructions for Assignment 3 of the STAT211 Random Processes course, due on May 20th, 2016. It includes 3 questions about Markov chains: 1) Describes a Markov chain modeling the state of a piece of equipment and asks to find transition matrices, explain periodicity, find state distributions, and determine if limiting/stationary distributions exist. 2) Defines a 3-state Markov chain and asks to compute transition probabilities, and determine if limiting/stationary distributions exist using theorems from lectures. 3) Describes an Ehrenfest chain modeling particle movement between partitions and asks to draw the transition diagram, find the first time step distribution, and use detailed balance to find

Uploaded by

marcus_boniface
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
120 views

Assignment 3

This document contains instructions for Assignment 3 of the STAT211 Random Processes course, due on May 20th, 2016. It includes 3 questions about Markov chains: 1) Describes a Markov chain modeling the state of a piece of equipment and asks to find transition matrices, explain periodicity, find state distributions, and determine if limiting/stationary distributions exist. 2) Defines a 3-state Markov chain and asks to compute transition probabilities, and determine if limiting/stationary distributions exist using theorems from lectures. 3) Describes an Ehrenfest chain modeling particle movement between partitions and asks to draw the transition diagram, find the first time step distribution, and use detailed balance to find

Uploaded by

marcus_boniface
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

STAT211 Random Processes

Assignment 3, due on May 20th, 2016 at 2pm


Q1. Suppose a piece of equipment can be in one of three states:
1 - in use;
2 - under repair; and
3 - idle awaiting more work.
We observe this equipment only after it changes state.
Let Xn be a random variables for the state the equipment has changed
into at timestep n.
Assume that if the equipment is under repair, then the next change
will be to in use. If there is no work then the repair crew will be move
to a different job, so will not repair the equipment and thus the MC
will never go from under repair to idle awaiting more work.
If the equipment is idle then it cannot breakdown, so it can only change
to being running.
If the equipment is in use then it can change to being either: i) idle
or ii) breakdown so under repair with equal chance.
(a) Write down the transition matrix and draw the transition diagram.
(b) Find the two-step and three-step transition matrices. Further,
write down the (2n) and (2n + 1) transition matrices (i.e. for even
an odd steps).
(c) Use these multi-step transition matrices to explain why this MC
is periodic and define the period.
(d) Suppose the initial state distribution is 0T = [a b c], where a, b, c >
0 and a + b + c = 1. Find the (2n) and (2n + 1) state distributions.
(e) Use the result from (d) to explain whether the limiting distribution exists and if it is unique. You should also make reference to
appropriate theorems in the lecture notes, if there are any.

(f) Find the stationary distribution of the MC.


(g) Using the long term behaviour of the chain to explain why the
stationary distribution exists and is unique, but the limiting distribution cannot exist.
Q2.

Suppose that we have a Markov chain with state space S =


{0, 1, 2} and transition matrix given by:

1/4 3/4 0
P = 1/3 1/3 1/3
0 1/4 3/4
(a) Compute P (X 0 = 0,X1 = 1, X2 = 1) given the initial state distribution 0T = 41 12 14
(b) Use the theorems in the lecture notes to explain whether the stationary distribution exists and is unique, and do the same for the
limiting distribution. Make statements as to explain why the conditions underlying the theorem conditions are true in this case.
For example, just stating the MC is irreducible is not sufficient
explanation.
(c) Find the stationary distribution for the MC.

Q3. Consider an Ehrenfest chain with 6 particles, and the following initial
status:

(a) Write down the transition matrix and draw the transition diagram.
(b) If the chain starts with 2 particles in the left partition, write down
the state distribution at the first time step.
(c) Find the stationary distribution using the detailed balance condition.

You might also like