Scribd
Upload
62 views36 pages

ch6 Probability and Statistics With Reliability, Queiung Theory and Computer Science Application

The document summarizes key concepts about stochastic processes from Chapter 6 of the textbook. It defines a stochastic process as a family of random variables indexed by time. Stochastic processes can be classified based on whether the state and time are discrete or continuous. Examples of processes discussed include the Poisson process, renewal counting process, and non-homogeneous Poisson process. The document also covers availability analysis of systems using stochastic processes to model times between failures and repairs.

Uploaded by

Monika Ksh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
62 views36 pages

ch6 Probability and Statistics With Reliability, Queiung Theory and Computer Science Application

The document summarizes key concepts about stochastic processes from Chapter 6 of the textbook. It defines a stochastic process as a family of random variables indexed by time. Stochastic processes can be classified based on whether the state and time are discrete or continuous. Examples of processes discussed include the Poisson process, renewal counting process, and non-homogeneous Poisson process. The document also covers availability analysis of systems using stochastic processes to model times between failures and repairs.

Uploaded by

Monika Ksh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 36

Probability and Statistics with

Reliability, Queuing and


Computer Science Applications:
Chapter 6 on Stochastic Processes

Kishor S. Trivedi
Visiting Professor
Dept. of Computer Science and
Engineering
Indian Institute of Technology, Kanpur

What is a Stochastic Process?

Stochastic Process: is a family of random


variables {X(t) | t T} (T is an index set; it
may be discrete or continuous)
Values assumed by X(t) are called states.
State space (I): set of all possible states
Sometimes called a random process or a
chance process

Stochastic Process Characterization

At a fixed time t=t1, we have a random variable


X(t1). Similarly, we have
X(t2), .., X(tk).
X(t1) can be characterized by its distribution
function,

We can also consider the joint distribution


function,

Discrete and continuous cases:


States X(t) (i.e. time t) may be
discrete/continuous
State space I may be discrete/continuous

Classification of Stochastic Processes

Four classes of stochastic processes:

discrete-state process chain


discrete-time process stochastic sequence {Xn |
n T} (e.g., probing a system every 10 ms.)

Example: a Queuing System


Queue (waiting station)
Random arrivals
Inter arrival time
distribution fn. FY

m
servers

Service time
distribution fn. FS

Interarrival times Y1, Y2, (common dist. Fn. FY)

Service times: S1, S2,

(iid with a common cdf FS)

Notation for a queuing system: FY /FS/m

Some interarrival/service time distributions types are:


M: Memoryless (i.e., EXP)
D: Deterministic
E : k-stage Erlang etc.
k
H : k-stage Hyper exponential distribution
k
G: General distribution
GI: General independent inter arrival times

Discrete/Continuous Stochastic
Processes
Nk: Number of jobs waiting in the system at the time of k th
jobs departure Stochastic process {Nk| k=1,2,}:

Discrete time, discrete state


Nk

Discret
e

Discrete

Continuous Time, Discrete Space


X(t): Number of jobs in the system at time t. {X(t) | t T}
forms a continuous-time, discrete-state stochastic process,
with,
X(t)

Discrete

Continuo

Discrete Time, Continuous Space


Wk: waiting time for the kth job. Then {Wk | k T} forms a
Discrete-time, Continuous-state stochastic process, where,
Wk

Continuo
us

Discrete

Continuous Time, Continuous Space

Y(t): total service time for all jobs in the system at time t. Y(t) forms a continuous-time,
continuous-state stochastic process, Where,

Y(t)

Further Classification

(1st order distribution)

(2nd order distribution)

Similarly, we can define nth order distribution:

Formidable task to provide nth order distribution


for all n.

Further Classification (contd.)

Can the nth order distribution be simplified?


Yes. Under some simplifying assumptions:

Independence

As example, we have the Renewal Process

Discrete time independent process {Xn | n=1,2,} (X1, X2, ..


are iid, non-negative rvs), e.g., repair/replacement after a
failure.

Markov process introduces a limited form of dependence

Markov Process

Stochastic proc. {X(t) | t T} is Markov if for any t 0 < t1<


< tn< t, the conditional distribution satisfies the Markov
property:

Markov Process

We will only deal with discrete state Markov


processes i.e., Markov chains
In some situations, a Markov chain may also exhibit
time-homogeneity
Future of process (probabilistically) determined by
its current state, independent of how it reached this
particular state; but in a non homogeneous case,
current time can also determine the future.
For a homogeneous Markov chain current time is
also not needed to determine the future.
Let Y: time spent in a given state in a hom. CTMC

Homogeneous CTMC-Sojourn
time

Since Y, the sojourn time, has the memoryless prop.

This result says that for a homogeneous continuous time Markov chain,
sojourn time in a state follows EXP( ) distribution (not true for non-hom
CTMC)
Hom. DTMC sojourn time dist. Is geometric.
Semi-Markov process is one in which the sojourn time in a state is generally
distributed.

Bernoulli Process

A sequence of iid Bernoulli rvs, {Yi | i=1,2,3,..}, Yi =1 or 0

{Yi} forms a Bernoulli Process, an example of a renewal


process.
Define another stochastic process , {Sn | n=1,2,3,..},
where Sn = Y1 + Y2 ++ Yn (i.e. Sn :sequence of partial
sums)
S = S
n
n-1+ Yn (recursive form)

P[Sn = k | Sn-1= k] = P[Yn = 0] = (1-p) and,

P[Sn = k | Sn-1= k-1] = P[Yn = 1] = p

{Sn |n=1,2,3,..}, forms a Binomial process, an


example of a homogeneous DTMC

Renewal Counting Process

Renewal counting process: # of


renewals (repairs, replacements,
arrivals) by time t: a continuous time
process:
If time interval between two renewals
follows EXP distribution, then Poisson
Process

Note:
For a fixed t, N(t) is a random
variable (in this case a discrete
random variable known as the Poisson
random variable)
The family {N(t), t 0} is a stochastic
process, in this case, the
homogeneous Poisson process
{N(t), t 0} is a homogeneous CTMC
as well

Poisson Process

A continuous time, discrete state process.


N(t): no. of events occurring in time (0, t]. Events may be,
1.
# of packets arriving at a router port
2.
# of incoming telephone calls at a switch
3.
# of jobs arriving at file/compute server
4.
Number of component failures
Events occurs successively and that intervals between
these successive events are iid rvs, each following EXP( )
1.
2.

: arrival rate (1/ : average time between arrivals)


: failure rate (1/ : average time between failures)

Poisson Process (contd.)

1.

N(t) forms a Poisson process provided:


1.
N(0) = 0
2.
Events within non-overlapping intervals are
independent
3.
In a very small interval h, only one event may occur
(prob. p(h))
Letting, pn(t) = P[N(t)=n],

For a Poisson process, interarrival times follow EXP( )


(memoryless) distribution.
E[N(t)] = Var[N(t)] = t ; What about E[N(t)/t], as t
infinity?

Merged Multiple Poisson Process


Streams

Consider the system,


+

Proof: Using z-transform. Letting, = t,

Decomposing a Poisson Stream

Decompose a Poisson process using a prob. switch

N arrivals decomposed into {N1, N2, .., Nk}; N= N1+N2, ..,


+Nk

Cond. pmf

Since,

The uncond. pmf

Generalizing the Poisson


Process
Poisson Process

Non-Homogeneous Poisson
Process (NHPP)

Non-Homogeneous Poisson
Process (NHPP)
If the expected number of events per unit time, , changes
with age (time), we have a non-homogeneous Poisson model.
We assume that:

1.
If 0 t, the pmf of N(t) is given by:

P N t k m t / k!e m t
k

k 0, 1, 2, ...

where m(t) 0 is the expected number of events in the


time period [0, t]

are

2.

Counts of events in non-overlapping time periods

mutually independent.
m(t) : the mean value function. (x) :the time-dependent
rate of occurrence of events or time-dependent failure rate

m(t ) 0 (x) dx
t

NHPP(cont.)

Generalizing Poisson
Process
Poisson Process

Non-Homogeneous Poisson
Process (NHPP)

Renewal Counting
Process

Renewal Counting Process

Poisson process EXP( ) distributed interarrival times.


What if the EXP( ) assumption is removed renewal proc.
Renewal proc. : {Xi | i=1,2,} (Xis are iid non-EXP rvs)
Xi : time gap between the occurrence of (i-1) st and ith event

Sk = X1 + X2 + .. + Xk time to occurrence of the kth


event.

N(t)- Renewal counting process is a discrete-state,


continuous-time stochastic process. N(t) denotes no. of
renewals in the interval (0, t].

Renewal Counting Processes (contd.)

Sn

For N(t), what is P(N(t) = n)?


tn

More arrivals
possible

Renewal Counting Process


Expectation

Let, m(t) = E[N(t)]. Then, m(t) = mean


no. of arrivals in time (0,t]. m(t) is
called the renewal function.

Renewal Density Function

Renewal density function:

For example, if the renewal interval X is


EXP(), then

d(t) = , t >= 0 and m(t) = t , t >= 0.


P[N(t)=n] = e t ( t)n/n! i.e Poisson pmf
Fn(t) will turn out to be n-stage Erlang

Alternating Renewal
Process
I(t)

Operating

Restoration

0
Time

Where:
Failure times T , T , are mutually independent with a
1
2
common distribution function W
Restoration times D , D , are mutually independent with
1
2
a common distribution function G
The sequences {T } and {D } are independent
n
n

Availability Analysis

Availability: is defined is the ability of a system to


provide the desired service.
If no
repair/replacement,Availability(t)=Reliability(t)
If repairs are possible, then above is pessimistic.
MTBF

T1

D1

T2

D2

T3

D3

T4

D4 .

MTBF = E[Di+Ti+1] = E[Ti+Di]=E[Xi]=MTTF+MTTR

Availability Analysis (contd.)


renewal

Repair is completed with in this interval

Two mutually exclusive situations:


1. System does not fail before time t A(t)
= R(t)
2. System fails, but the repair is completed
before time t
Therefore, A(t) = sum of these two
probabilities

Availability Expression

dA(x) : Incremental availability


Repair is completed with in this interval

x x+dx

Renewed life time >= (t-x)

dA(x) = Prob(that after renewal, life time is >


(t-x) & that the renewal occurs in the interval
(x,x+dx])

Availability Expression (contd.)

A(t) can also be expressed in the Laplace domain.

Since, R(t) = 1-W(t) or LR(s) = 1/s LW(s) = 1/s


Lw(s)/s

What happens when t becomes very large?

However,

Availability, MTTF and MTTR

Steady state availability A is:

Taking the expression of sLA(s) and


taking the limit via LHospital rule and
using the moment generating property
of the LT, we get the required result for
the steady-state
A=MTTF/(MTTF+MTTR)

Availability Example

Assuming EXP( ) density fn for g(t) and


w(t)

Generalizing Poisson
Process
Poisson Process
Renewal Counting
Process

Compound Poisson
Process
Non-Homogeneous Poisson
Process (NHPP)

Non-Homogeneous
Continuous Time
Markov Chain

Bernoulli Process
Homogeneous
Continuous Time
Markov Chain
Homogeneous
Discrete Time
Markov Chain

Semi-Markov
Process
Markov Regenerative
Process

You might also like