Scribd
Upload
13 views25 pages

Marko

Uploaded by

Mubashir Unnissa
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
13 views25 pages

Marko

Uploaded by

Mubashir Unnissa
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 25

Markov Chains

o In this section, we will study a concept that utilizes a


mathematical model that combines probability and matrices to
analyze what is called a stochastic process, which consists of a
sequence of trials satisfying certain conditions.
o The sequence of trials is called a Markov Chain which is named
after a Russian mathematician called Andrei Markov (1856-1922).

1
Andrei Markov (1856-1922)

 Markov is particularly remembered


for his study of Markov chains,
sequences of random variables in
which the future variable is
determined by the present
variable but is independent of the
way in which the present state
arose from its predecessors. This
work launched the theory of
stochastic processes

2
Example
An employee tried not to be late for his office too
often. If he is late one day, he is 80% sure to be on
time next time. If he is on time, then next time there
is 20% chance of being late. In the long run how
often he is late for the office

Stochastic Process: Change of behaviour of factor


state over the time

3
Stochastic Processes

4
Transition Probabilities

5
Notation

6
Markov Property

7
Transition Probability Matrix (TPM)

8
Transition probability matrix: Rows indicate the current state and
column indicate the transition . For example, given the current state of
A, the probability of going to the next state A is s. Given the current
state A’, the probability of going from this state to A is r. Notice that
the rows sum to 1. We will call this matrix P.

9
Notation

10
Notation

11
Initial State distribution matrix:

A A'
S0  t 1  t 

12
First and second state matrices:

 If we multiply the Initial state matrix by the transition


matrix, we obtain the first state matrix.

S1  So P

 If the first state matrix is multiplied by the transition


matrix we obtain the second state matrix:

2
S2  S1 P  So P P  S0 P

13
Kth – State matrix

 If this process is repeated we will obtain the following


expression:
 The entry in the ith row and jth column indicates the
probability of the system moving from the ith state to the
jth state in k observations or trials.

S k  S k  1 P  S0 P k

14
Example

15
Solution:

16
17
18
19
Problem 2

20
21
22
An example: An insurance company classifies drivers as low-risk if they are accident-
free for one year. Past records indicate that 98% of the drivers in the low-risk category (L)
will remain in that category the next year, and 78% of the drivers who are not in the low-
risk category ( L’) one year will be in the low-risk category the next year.

1. Find the transition matrix, P

L L'
L  0.98 0.02 
P
L '  0.78 0.22 

1. If 90% of the drivers in the community are in the low-risk category


this year, what is the probability that a driver chosen at random
from the community will be in the low-risk category the next year?
The year after next ? (answer 0.96, 0.972 from matrices)
L L' S2  S1 P  So P P S0 P 2
S0  0.90 0.10
S1  So P L L'
L L'
S2  0.972 0.028
S1  0.96 0.04
23
Finding the kth State matrix

k
 Use the formula
k S S P
0
to find the 4th state
matrix for the previous problem.

S4  S0 P 4

4
 0.98 0.02 
  0.90 0.10
=
 0.78 0.22
 

 .97488 0.02512
 after four states, the percentage of low-risk drivers has
increased to .97488

24
25

You might also like