Scribd
Upload
56 views

Understanding Limiting Distribution of Markov Chains

This document summarizes the limiting distribution of Markov chains. It states that for an irreducible, aperiodic and positive recurrent Markov chain with k states and stationary distribution π, the limiting distribution is always π, regardless of the initial distribution. This is because the n-step transition matrix Pn converges to P∞ = π as n approaches infinity, due to the property that limn→∞ Pijn = πj for all states i and j. Therefore, the probability of being in any state i in the long run is simply the stationary distribution πi.

Uploaded by

蔡于飛
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
56 views

Understanding Limiting Distribution of Markov Chains

This document summarizes the limiting distribution of Markov chains. It states that for an irreducible, aperiodic and positive recurrent Markov chain with k states and stationary distribution π, the limiting distribution is always π, regardless of the initial distribution. This is because the n-step transition matrix Pn converges to P∞ = π as n approaches infinity, due to the property that limn→∞ Pijn = πj for all states i and j. Therefore, the probability of being in any state i in the long run is simply the stationary distribution πi.

Uploaded by

蔡于飛
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Understanding Limiting Distribution of Markov

Chains
Cai Yufei ([email protected])
May 12, 2022

Suppose there is an irreducible, aperiodic and positive recurrent Markov


chain {Xn } with k states. Assume
 the stationary distribution of the Markov
chain is π = π1 π2 ... πk .
Then we have known that limn→∞ Pijn = πj for i, j = 1, 2, ..., k.
The n-step transition matrix is given by
 n n n

P11 P12 . . . P1k
P21 n n n
P22 . . . P2k
Pn =  .
 
. .. .
 .. .. . .. 
n n n
Pk1 Pk2 ... Pkk
.
Then    
π1 π2 ... πk π
π1 π2 ... πk  π 
 
P∞ = . ..  =  .. 

.. ..
 .. . . .  .
π1 π2 ... πk π
. 
Now assume the initial state X0 has the distribution a = a1 a2 ... ak
Pk
where ai ≥ 0, i=1 ai = 1 and ai = P (X0 = i) for i = 1, 2, ..., k.
Then aP is the distribution of X1 with P (X1 = i) = (aP )i , i = 1, 2, ..., k,
aP 2 is the distribution of X2 with P (X2 = i) = (aP 2 )i , i = 1, 2, ..., k, . . .
∀n ∈ {1, 2, ...}, aP n is the distribution of Xn with P (Xn = i) = (aP n )i ,
i = 1, 2, ..., k.
As a result, aP ∞ is the distribution of Xn when n is sufficiently large.
(Actually we can use ϵ − N language to describe it, but we do not need.)
Now, we want to figure out what aP ∞ is exactly and whether aP ∞ depends
on a or not.

1
Actually,
 
π
π 
aP ∞ = a .  (1)
 
 .. 
π
 
π
π 
 
= a1 a2 ... ak  .  (2)
 .. 
π
k
X
= ai π (3)
i=1
Xk
=( ai )π (4)
i=1
=π (5)

.
So we can conclude that no matter what initial distribution from which a
Markov chain starts, the limiting distribution of the r.v. Xn will never change
and will always be the stationary distribution π when the Markov chain is
irreducible, aperiodic and positive recurrent.
In other words, when n is sufficiently large, P (Xn = i) ≈ (aP ∞ )i = πi (or
limn→∞ P (Xn = i) = (aP ∞ )i = πi ) for i = 1, 2, ..., k.
This conclusion depends on the fact that limn→∞ Pijn = πj .

You might also like