Scribd
Upload
50 views

Assignment 1

This document contains three assignments related to stochastic processes and Markov chains. The first assignment describes a Markov chain with four student financial states and their transition probabilities, and asks about the probability of reaching different states over time. The second assignment involves finding the limiting distribution and future state probabilities of another Markov chain. The third assignment models a queuing system as a continuous-time Markov chain and asks to derive properties of the holding times, draw the jump chain, and provide the generator matrix and transition rate diagram.

Uploaded by

CONTACTS CONTACTS
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
50 views

Assignment 1

This document contains three assignments related to stochastic processes and Markov chains. The first assignment describes a Markov chain with four student financial states and their transition probabilities, and asks about the probability of reaching different states over time. The second assignment involves finding the limiting distribution and future state probabilities of another Markov chain. The third assignment models a queuing system as a continuous-time Markov chain and asks to derive properties of the holding times, draw the jump chain, and provide the generator matrix and transition rate diagram.

Uploaded by

CONTACTS CONTACTS
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

UNIVERSITY OF RWANDA

Course: STOCHASTIC PROCESSES Assignment 1


Date: August 2, 2023

1. Assume that a student can be in 1 of 4 states: Rich, Average, Poor, In Debt.


Assume the following transition probabilities:
If a student is Rich, in the next time step the student will be Average with probability 0.75
If a student is Rich, in the next time step the student will be Poor with probability 0.2
If a student is Rich, in the next time step the student will be In Debt with probability 0.05

If a student is Average, in the next time step the student will be Rich with probability 0.05
If a student is Average, in the next time step the student will be Average with probability 0.2
If a student is Average, in the next time step the student will be In Debt with probability 0.45

If a student is Poor, in the next time step the student will be Average with probability 0.4
If a student is Poor, in the next time step the student will be Poor with probability 0.3
If a student is Poor, in the next time step the student will be In Debt with probability 0.2

If a student is In Debt, in the next time step the student will be Average with probability 0.15
If a student is In Debt, in the next time step the student will be Poor with probability 0.3
If a student is In Debt, in the next time step the student will be In Debt with probability 0.55

Model the above as a discrete Markov chain

(a) Draw the corresponding Markov chain and obtain the corresponding stochastic matrix.
(b) If a student starts at the “Average” state, what will be the probability of them getting ot
state “Rich” after
i) 1 time step?
ii) 2 time steps?
iii) 3 time steps?

2. Let Sn be a Markov Chain with state space S = {1, 2, 3} and transition matrix
 
0.1 0.4 0.5
P = 0.3
 0.5 0.2
0.4 0.3 0.3

a) find the limiting distribution of P


b) Let the initial distribution of X0 be π0 = (0.4, 0.3, 0.2), this means: P (X0 = 1) = 0.4,
P (X0 = 2) = .3, P (X0 = 3) = .2 Find the distribution of all the states 3 days later.
3. Suppose that customers arrive according to a Poisson process with rate λ at a service center
that has a single server. Customers are served one at a time in order of arrival. Service times are
assumed to be i.i.d. Exponential(µ) random variables and independent of the arrival process.
Customers leave the system after being served. Our goal in this problem is to model the above
system as a continuous- time Markov chain. Let X(t) be the number of customers in the system
at time t , so the state space is S = {0, 1, 2, · · · } . Assume i > 0 . If the system is in state i at
time t , then the next state would either be i + 1 (if a new customers arrive) or state i − 1 (if
a customer leaves).

a. Suppose that the system is in state 0 , so there are no customers in the system and the
next transition will be to state 1 . Let T0 be the time until the next 0 transition. Show
that T0 ∼ Exponential(λ) .
b. Suppose that the system is currently in state i , where i > 0 until the next transition.
Show that Ti ∼ Exponential(λ + µ)
c. Suppose that the system is at state i . Find the probability that the next transition will
be to state i + 1 .
d. Draw the jump chain, and provide the holding time parameters λi .
e. Find the Generator matrix.
f. Draw the transition rate diagram.

You might also like