Scribd
Upload
194 views

IE 452 Assignment 2

The document contains 8 questions related to Markov chains. Question 1 asks to find the expected value of X3 given the transition probability matrix and initial state probabilities. Question 2 asks to model a gambler's fortune as a Markov chain by classifying states. Question 3 asks to classify states in a given transition matrix as transient or recurrent.

Uploaded by

kvntor
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
194 views

IE 452 Assignment 2

The document contains 8 questions related to Markov chains. Question 1 asks to find the expected value of X3 given the transition probability matrix and initial state probabilities. Question 2 asks to model a gambler's fortune as a Markov chain by classifying states. Question 3 asks to classify states in a given transition matrix as transient or recurrent.

Uploaded by

kvntor
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

IE 452 Assignment 2, Due March, 21 2019

1. A Markov chain {Xn , n ≥ 0} with states 0, 1, 2, has the transition probability matrix
 
1/2 1/3 1/6
 
P =  0 1/3 2/3  .
 
 
1/2 0 1/2

If P {X0 = 0} = P {X0 = 1} = 14, find E[X3 ].

2. A gambler bets at each play one third of his present fortune rounded to the next full
$-amount. For example, if his fortune is $4, he bets $2. At each play he wins twice the
amount of his bet or he loses it. If he is broken he stops playing and if he owns a fortune of
$N, he stops as well. Model the gambler’s fortune by a Markov chain. Classify the states
and find absorbing states.

3. Consider the matrix  


0 0 1/3 0 1/3 1/3
 
0 3/4 0 0 0 1/4
 
 
 
1 0 0 0 0 0
 
 
P =

.


 0 0 0 0 0 1 

 

 0 0 0 0 3/4 1/4 

0 1/6 0 0 0 5/6
Which states are transient and which are recurrent?

4. Three white and three black balls are distributed in two urns in such a way that each
contains three balls. We say that the system is in state i, if the first urn contains i white
balls. At each step, we draw one ball from each urn and place the ball drawn from the
first urn into the second, and conversely with the ball drawn from the second urn. Let Xn
denote the state of the system after the nth step. Explain why Xn is a Markov chain and
calculate its transition probability. Classify the states, determine transient and recurrent
states, and decide whether the chain is irreducible or aperiodic.

1
5. A taxi driver provides service to two zones A and B in a city. Fares picked up in zone
A will have destination in zone A with probability 0.6 or in zone B with probability 0.4.
Fares picked up in zone B will have destination in A with probability 0.3 or in zone B with
probability 0.7. The driver’s expected profit for a trip entirely in zone A is $6, for a trip
entirely in zone B it is $8; and for a trip between the two zones it is $12. Find the drivers
average profit per trip. What is the steady state of this Markov chain?

6. In a good weather year the number of storms is Poisson distributed with mean 1; in a bad
year it is Poisson distributed with mean 3. Suppose that any year’s weather conditions
depends on past years only through the previous year’s condition. Suppose that a good
year is equally likely to be followed by either a good or a bad year, and that a bad year is
twice as likely to be followed by a bad year as by a good year. Suppose that last year—call
it year 0—was a good year.

(a) Find the expected total number of storms in the next two years (that is, in years 1
and 2).

(b) Find the probability there are no storms in year 3.

(c) Find the long-run average number of storms per year.

7. Every time that the team wins a game, it wins its next game with probability 0.8; every
time it loses a game, it wins its next game with probability 0.3. If the team wins a game,
then it has dinner together with probability 0.7, whereas if the team loses then it has
dinner together with probability 0.2. What proportion of games result in a team dinner?

8. A manufacturing setup consists of two distinct machines, each producing one component
per hour. Each component is tested instantly and is identified as defective or non-defective.
Let α1 = 0.8 be the probability that a component produced by machine 1 is non-defective,
and α2 = 0.9 be the probability that a component produced by machine 2 is non-defective.
The defective components are discarded and the non-defective components are stored in
two separate bins, one for each machine. When a component is present in each bin,
the two are instantly assembled together and shipped out. Each bin can hold at most
2 components. When a bin is full the corresponding machine is turned off. It is turned
on again when the bin has space for at least one component. Assume that successive
components are independent. Model this system by a DTMC. Suppose initially both bins

2
are empty. Compute the expected time (in hours) until one of the machines is turned off.

You might also like