 Introduction

 Concepts of Markov Modelling

 Stochastic Transitional Probability Matrix
 Time Dependent Probability Evaluation
 Limiting State Probability Evaluation
 Absorbing States
 Applications

2
 A Markov chain is a random process endowed with the
Markov property “the next state depends only on the
current state and not in the past”
 This property is called memoryless

 In addition, the process must be stationary, sometimes

called homogeneous, for the approach to be applicable
 A system which is in a certain state at each “step” ,with
the state changing randomly between steps.
 Generally it’s impossible to predict the exact state of the
system in the future .
 The statistical properties of the system can be predicted.

3
 The Markov approach is applicable to those systems whose
behavior can be described by a probability distribution that
is characterized by a constant hazard rate
 Space is normally represented only as a discrete function,
whereas time may either be discrete or continuous
 This discrete or continuous random variation is known as a
stochastic process

4
suppose A = Normal operation of power system
A` = Outage of the power system, now the transition diagram
will be like that:

Next A ` A
0.9 0.1 0.3 state

A 0.9 0.1
A ` A =P
` A 0.7 0.3

0.7
Current
state

Transition Transition probability matrix

Diagram
A `A
= S0 0.2 0.8
Initial state distribution matrix
 Let’s do some calculation: A `A
= S0 0.2 0.8
0.9 A
A
0.2 0.1 `A
Start Next A `A
A state
0.8 0.7
A 0.9 0.1
`A
=P
0.3 `A `A 0.7 0.3
Probability of using system
A after 1wk Current
P(system A after 1wk) state
)0.7(*)0.8(+)0.9(*)0.2( =
0.18+0.56 = A `A
0.74 = “one week later” = S1 0.74 0.26
6
 0.9 0.1
= S0 .P 0.2 0.8 = 0.74 0.26 S1 =
0.7 0.3
First state - matrix

?Now what will we have 2 wk(s) later

A `A
0.9 0.1 0.152
= S2 0.74 0.26 = 0.848
0.7 0.3

A `A
0.9 0.1
= S3 0.848 0.152 = 0.8696 0.1304 3 wk(s) later
0.7 0.3
7
 The transient behavior is very dependent on the initial
conditions
 The limiting values of the state probabilities are totally
independent of the initial conditions
 A system or process for which the limiting values of state
probabilities are independent of the initial conditions is
known as ergodic
 For a system to be ergodic it is essential that every state of a
system can be reached from all other states of the system
either directly or indirectly
 If this is not possible, once entered cannot be left, the
relevant states are known as absorbing states
 The tree diagram method is totally impractical for large
systems or large number of time intervals
8
 The matrix should represent the probabilities of making a
transition from one state to another in a single step or
time interval
 This matrix is known as the stochastic transitional
probability matrix
 It should be noted that the summation of the probabilities
in each row of the matrix must be unity
 It is used to evaluate the transient behavior of a system

 Gives all the state probabilities of the system after two

time intervals

1
9
 In order to illustrate the evaluation of the transient behavior
of a system using the stochastic transitional probability
matrix

n
 Pij: represents the probability that the system will be in
state j after n time intervals given that it started in state I
 When the initial conditions are not known, then the matrix
n
p can be pre-multiplied by the initial probability vector

1
9
 If the system starts in state 1 the probability vector is:

 If the system is equally likely to start in state 1 or state 2,

then this initial probability vector becomes

 It is evident that the state probabilities can be readily

evaluated at any time interval
 If this process is continued sequentially, the transient
behavior can be deduced and limiting state probabilities
can be derived
1
9
 If the transient behavior is also required, it may be
sensible to use steady-state probability technique
 If on the other hand, only the limiting state probabilities
are required, matrix multiplication can be tedious and
time-consuming
 A very efficient alternative method is: once the limiting
state probabilities have been reached, any further
multiplication by the matrix does not change the values of
the limiting state probabilities

Where:
α: limiting probability vector and
P: the stochastic transitional probability matrix
12
 Some states of a system may be absorbing states (once
entered, cannot be left until the system starts a new
mission)
 These can be identified as the catastrophic failure event

 In such cases, one requirement of the reliability analysis is

to evaluate the average number of time intervals
 This principle can also be applied to repairable systems

 The expected number of time interval is:

Where:
Q: multi-probability elements, I: Identity matrix
Therefore,
13
 Markov Chain refinement

14
 Markov Chain refinement

15
 This modeling approach has found applications in various
fields, in electrical power engineering, such as:

Reliability Analysis

Maintenance Planning

Load Forecasting

Energy Storage Systems

Fault Diagnosis

Voltage Stability Analysis

16