Scribd
Upload
48 views

SIE 321 Probabilistic Models in OR Homework 7: Problem 1

This document contains 3 problems about modeling probabilistic systems using Markov chains. Problem 1 models a machine that can break down with probability 0.1 each day and asks to (a) formulate the Markov chain model and (b) calculate probabilities of uptime. Problem 2 models a gambler's ruin problem where a gambler bets $1 each round until going broke or reaching $3, and asks to (a) construct the transition matrix and (b) calculate win probabilities. Problem 3 models a mouse maze and asks to (a) formulate the transition matrix, (b) calculate the probability of reaching intersection 5 in 4 runs, and (c) calculate the expected number of runs to reach intersection 5 for the first time

Uploaded by

Kali Jasso
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
48 views

SIE 321 Probabilistic Models in OR Homework 7: Problem 1

This document contains 3 problems about modeling probabilistic systems using Markov chains. Problem 1 models a machine that can break down with probability 0.1 each day and asks to (a) formulate the Markov chain model and (b) calculate probabilities of uptime. Problem 2 models a gambler's ruin problem where a gambler bets $1 each round until going broke or reaching $3, and asks to (a) construct the transition matrix and (b) calculate win probabilities. Problem 3 models a mouse maze and asks to (a) formulate the transition matrix, (b) calculate the probability of reaching intersection 5 in 4 runs, and (c) calculate the expected number of runs to reach intersection 5 for the first time

Uploaded by

Kali Jasso
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

SIE 321 Probabilistic Models in OR Homework 7

Problem 1. A manufacturer has a machine that, when operational at the beginning of a day, has a
probability of 0.1 of breaking down sometime during the day. When this happens, the
repair is done the next day and completed at the end of that day.

(a) Formulate the evolution of the status of the machine as a Markov chain by identifying
three possible states at the end of each day, and then constructing the (one-step)
transition matrix.
(b) Find the probability that the machine will be operational for 5 straight days before
breaking down. Compare this with the probability that a currently operational ma-
chine will be broken down on day 6.
(c) Find the expected first passage time µi j from state i to state j, for all i and j. Use
these results to identify the expected number of full days that the machine will remain
operational before the next breakdown after a repair is completed.

Problem 2. Consider the following gamblers ruin problem. A gambler bets $1 on each play of a game.
Each time, he has a probability p of winning and probability q = 1 − p of losing the dollar
bet. He will continue to play until he goes broke or nets a fortune of T dollars. Let X n
denote the number of dollars possessed by the gambler after the n-th play of the game.
Then
¨
X n + 1 with probability p
X n+1 = for 0 < X n < T
X n − 1 with probability 1 − p
X n+1 = X n for X n = 0 or X n = T

Defined in such a way X n is a Markov chain. The gambler starts with X 0 dollars, where
0 < X0 < T .

(a) Construct the (one-step) transition matrix


(b) Let T = 3 and p = 0.55. Find the probabilities of winning T dollars in three or four
rounds of play when the initial capital of the gambler is 1, . . . , T − 1 dollars.

1
Problem 3. A mouse maze consists of the paths shown below. At any intersection, the mouse is equally
likely to select any of the available paths.

(a) Formulate the (one-step) transition matrix


(b) Determine the probability that, starting at intersection 1, the mouse will reach inter-
section 5 for the first time in 4 runs (1 run = a run between two intersections)?
(c) What is the expected number of runs needed for the mouse to reach intersection 5
from intersection 1 for the first time?

You might also like