SYSTEM PERFORMANCE EVALUATION

CHAPTER 4 Stochatic Processes

WEEK 4: Mathematical Modeling of Queues

 SYSTEM PERFORMANCE EVALUATION

CHAPTER 4 Stochatic Processes

WEEK 4: Mathematical Modeling of Queues

© 2023
Nguyen, VM. Man ‡
Tran, Van Hoai and Nguyen, Phuong Duy †
‡ Faculty of Science - Mahidol University
† Faculty of Computer Science & Engineering - HCMC University of Technology
CHAPTER 4: Stochatic Processes
WEEK 4: Probabilistic Modeling of Queues

1. Queuing System: Specifications with Random (Variables and) Processes

2. The M/M/1 queue - Background
3. The M/M/1 queue - Characterization via Performance Metrics
4. The generic G/G/m queue
5. Stochastic process and Markov process
6. Poisson Process- Variation with Application
Contents

Stochatic Processes
WEEK 4: Probabilistic Modeling of Queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii

Chapter 4 Probabilistic Models in Queueing Theory 1

4.1 Queuing System with Random (Variables and) Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
4.2 The M/M/1 queue- Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.3 The M/M/1 queue - Characterization via Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.4 The generic 𝐺/𝐺/𝑚 queue and Its Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.5 Stochastic process and Markov process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.6 Poisson Process- Variation with Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.7 The Birth and Death processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.8 CHAPTER CONCLUSION- HOMEWORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 Chapter 4

Probabilistic Models in Queueing Theory

 2

CONTENT BRIEF of WEEK 4:

1. Queuing System with Random (Variables and) Processes

MODELING dynamic in time by both Parameters and Random variables
The Little’s Law for stable queue
2. The M/M/1 queue and Evaluating Its Performance
The M/M/1 queue - Full description (M stands for Exponential system)
Poisson arrival process and Exponential inter-arrival times- One phenomenon.
3. The generic G/G/m queue (m stands for a natural nember)
=⇒ ASSIGNMENT 1: Quantify major processes and PERFORMANCE METRICS of queues
4. Stochastic process and Markov process in Section 4.5
5. Poisson Process- Variation with Application
6. Birth-death process (to WEEK 5 on various queues)

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 3

4.1 Queuing System with Random (Variables and) Processes

4.1.1 Key Processes- MODELING by Parameters and Random variables

How to describe jobs going through a queuing system 𝑄?

Need parameters and random variables to describe processes of 𝑄 .

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.1. Queuing System with Random (Variables and) Processes 4

The main stages and components were depicted in Figure 4.1.

1. The figure looks simple, but in fact Queuing Systems (QS) is critical for Design Optimization.
2. Since knowledge of queues can directly be used to solve the optimization problems of system
design and capacity planning,
as determining the number of servers, the number of queues, the system architecture,
an optimum queuing discipline, a schedule for service...
SPE needs both Discrete & Continuous random variables?

Key discrete variables

𝑛𝑞 : Number of jobs waiting,

𝑛𝑠: Number of jobs receiving service.
𝑛: Number of jobs in the system (including jobs currently receiving service
as well as those waiting in the queue).

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 5

Figure 4.1: Schematic queue

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 6

Key continuous variables?

1. 𝜏 = 𝐼: Inter-arrival time = time between two successive arrivals.
2. 𝜆: Average arrival rate = 1/E[𝜏 ], may be a function of the state of the system (as number
of jobs 𝑛 in the system.)
3. 𝑆 = 𝑠: Service time per job
=⇒ 𝜇 = 1/E[𝑆] = Average service rate per server
4. Total service rate for 𝑚 servers obviously is 𝑚𝜇.

5. 𝑤: Waiting time is the Time between arrival and beginning of service.

6. 𝑟 = 𝑤 + 𝑠: Response time or the time in the system (system time) of a job

time waiting + time receiving service.

Parameters for describing performance of a queue

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 7

NOTATIONS Parameters of Indicator

a queuing system (Use for)

E[𝐴(𝑡)]
𝜆𝐴 = 𝜆 = = arrival rate Arriving process
𝑡
1
E[𝐼] = = = mean inter-arrival time Arriving process
𝜆
𝜇 = service rate Service process
1
E[𝑆] = = mean service time Service process
𝜇
𝜆𝐴
𝜌 = = utilization, system’s
𝜆𝑆
(or arrival-to-service ratio) performance

■ EXAMPLE 4.1 (Printing service with timing).

Jobs are sent to a printer at random times, according to a Poisson process of arrivals, with a rate

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.1. Queuing System with Random (Variables and) Processes 8

of 12 jobs per hour. The time it takes to print a job is an Exponential r. variable, independent of

the arrival time, with the average of 2 minutes per job.

(a) A job is sent to a printer at noon. When is it expected to be printed?

(b) How often does the total number of jobs in a queue and currently printed exceed 2?
𝜆𝐴 = 𝜆 12 𝑗𝑜𝑏𝑠/ℎ𝑜𝑢𝑟 = 12 𝑗𝑜𝑏𝑠/60 𝑚𝑖𝑛 = 1/5 Arriving process
1
E[𝐼] = = 5 minutes for one job mean arrival time
𝜆

𝜇 = service rate (jobs / minute) Service process

E[𝑆] = 1/𝜇 ( minutes / job) = mean service time

GUIDANCE for solving. The problem’s inputs give us a summary in the above data table.

(a) When is it expected to be printed? A job is sent to a printer at noon, i.e. 12:00.
1
The mean service time E[𝑆] = = 2 minutes, hence that job should be printed on 12:02.
𝜇
(b) Require the fact that arrivals follow Poisson process, studied next. ■

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 9

4.1.2 Performance measures (indicators) of a queue in statistical equilibrium

AIM: We now determine performance measures from the description of the system.

Six important performance measures of a stable system use random variables below.

Random variables (values) of a queue Indicator

𝑆=𝑠 service time per one customer mean E[𝑆] = 1/𝜆𝑆 = 1/𝜇

𝑊 =𝑤 waiting time of one customer mean E[𝑊 ] =?

𝑅=𝑟 whole time in system of one customer mean E[𝑅] =?

𝑋𝑠 (𝑡) = 𝑛𝑠 = number of jobs receiving service at time 𝑡

𝑋𝑤 (𝑡) = 𝑛𝑞 = number of jobs waiting in a queue at time 𝑡

𝑋(𝑡) = 𝑛 = 𝑋𝑠 (𝑡) + 𝑋𝑤 (𝑡) = 𝑛𝑠 + 𝑛𝑞 ,

the total number of jobs in the system at time 𝑡

Most of the time, our goal is to find the distribution of 𝑋(𝑡), the total number of jobs in the
system. The other characteristics of a queuing system will be assessed from that. As a result, we
shall obtain a complete performance evaluation of that queue.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 10

Figure 4.2: Schematic queue

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 11

The system is then said to be in statistical equilibrium. Such a system ( in statistical

a
equilibrium) is called a stable system.
a
Methods of removing transient behavior to obtain stable quantities of a stable system will be discussed later.

The most useful performance indicators of a given queuing system are:

1. The number of jobs in the system at time point 𝑡 is denoted by 𝑋(𝑡).

Probability of exactly 𝑛 jobs in the system at 𝑡 is

𝑝𝑛(𝑡) = P[𝑋(𝑡) = 𝑛], 𝑛 ∈ Range(𝑋(𝑡)) (4.1)

2. Probability of exactly 𝑛 jobs in the system in statistical equilibrium is given by

𝑝𝑛 = P[ there are 𝑛 jobs in the system in the long-run] = lim P[𝑋(𝑡) = 𝑛].
𝑡→∞

It is referred to as the steady-state probability of exactly 𝑛 jobs in the system.

3. The service demand is described via the arrival rate 𝜆𝐴 = 𝜆 ∈ R+.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 12

4. The service (processing) rate 𝜇 of the server.

5. Utilization 𝜌 in 𝐴/𝑆/1 (single server) is the fraction of the time in which the server is busy
arrival rate 𝜆
𝜌= = . (4.2)
service rate 𝜇

6. Departure rate (throughput):

When the service is completed, the job leaves the system. The departure rate is defined as
the mean number of jobs whose processing is completed in a single unit of time.

For a queuing system in statistical equilibrium, the departure rate is equal to the arrival rate,
hence the throughput is
𝜆 = 𝜌 · 𝜇. (4.3)

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 13

4.1.3 The Little’s Law for stable queue

Little’s Law is shortly based on a black-box view of the system. 1

Little’s law hence can be used for a system or any part of the system.

Average number in queue = arrival rate × average waiting time

Average number in service = arrival rate × average service time

The Little’s Law mathematically says

E[𝑋] = 𝜆𝐴 E[𝑅] (4.4)

relateing expectations of the number of jobs, the response time, and the arrival rate.

It is valid for any stationary queuing system.

1 It was obtained by John D. C. Little who is currently an Institute Professor at Massachusetts Institute of Technology in the United States.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 14

Meaning Little’s law requires no assumptions about arrival or service time distribution, the size
of population, or limits on the system. Here we view

E[𝑋] = lim 𝑋(𝑡), E[𝑅] = lim 𝑅𝑛.

𝑡−→∞ 𝑛−→∞

The Complete Little’s Law

generally applied into the system’s components - the queue and the servers. Thus, we can
immediately deduce the equations for
the mean number of waiting jobs in queue E[𝑋𝑤 ], and
the mean number of jobs receiving service E[𝑋𝑠]. We have
⎧
⎪
⎪
⎪
⎪
⎪ 𝜆𝐴 E[𝑅] = E[𝑋],
⎪
⎨
𝜆𝐴 E[𝑊 ] = E[𝑋𝑤 ], (4.5)
⎪
⎪
⎪
⎪
⎪
⎩𝜆𝐴 E[𝑆] = E[𝑋𝑠].
⎪

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 15

DISCUSSION.

It is important to note a few things.

First, E[𝑅] must be interpreted as the mean time in the system for customers that actually “enter”
the system (and not include customers that were turned away).

Second, the average rate of departure from the system (if the system is stable) is also 𝜆𝐴.
This is called conservation of customers, whereby customers are neither created nor destroyed;
therefore the average customer entering rate equals average customer departure rate (if the
system is stable).
We call this Rate-equality Principle for queuing systems.
Third, Little’s law requires no assumptions about arrival or service time distribution, the size of
population, or limits on the system.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 16

■ EXAMPLE 4.2.

(1) (Generic system).

If the average system time (staying in a system of a customer) is 2 hours, and customers arrive
at a rate of 3 per hour then on average, there are E[𝑋] = 𝜆𝐴 E[𝑅] = 3 · 2 = 6 customers in the
system during time length 120 minutes.

(2) (Queue in a bank). You walk into a bank at 10:00. Being there, you count a total of
10 customers and assume that this is the typical, average number. You also notice that on the
average, customers walk in every 2 minutes.

When should you expect to finish services and leave the bank?

GUIDANCE for solving. We have E[𝑋] = 10 and 𝜇𝐴 = 2 min. By the Little’s Law,
E[𝑋]
𝜆𝐴 E[𝑅] = E[𝑋] =⇒ E[𝑅] = = E[𝑋] 𝜇𝐴 = 20 𝑚𝑖𝑛
𝜆𝐴
So, your expected response time is 20 minutes, and you would leave at 10:20. ■

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.1. Queuing System with Random (Variables and) Processes 17

PRACTICE With Numerical Computation.

Consider problem
A monitor on a disk server showed that the average time to satisfy an I/O request was 100
milliseconds.
The I/O rate was about 100 requests per second.
What was the average number of requests at the disk server?

HINT: Using Little’s law, the average number of requests in the disk server

= Arrival rate × system time = 100 (requests/second) × (0.1 seconds) = 10 requests.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.2. The M/M/1 queue- Background 18

QUIZ 1 (An airport runway for arrivals only (2 points)).

Arriving aircraft join a single queue for the runway

Exponentially distributed service time with a rate 𝜇 = 27 arrivals/hour.
Poisson arrivals with a rate 𝜆 = 20 arrivals/hour.

Compute

Average Time in the airport runway system 𝐸[𝑅] =?

The Average Number of air-crafts in the runway system 𝐸[𝑋]?

The Mean Waiting time for the runway

The Average Number of air-crafts waiting for the runway
𝐸[𝑋𝑤 ] = ...

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.2. The M/M/1 queue- Background 19

4.2 The M/M/1 queue- Background

We firstly need to describe probabilistic processes of M/M/1 queue.

We say that an arrival process 𝑁 (𝑡) at a queue is a Poisson process

if the rate of arriving is a constant 𝜆 per an unit of time.

That means within an unit of time

in average there is 𝜆 jobs enter the queue.
NOTE: from LECTURE 3 (WEEK 3) at Section 3.1.2. Constant 𝜆 could receive rational values.

How to compute the pdf of Poisson arrivals at a given M/M/1 queue?

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.2. The M/M/1 queue- Background 20

Knowledge box 4.1 (We compute the pdf of the Poisson arrivals by Theorem 3.1.).

If 𝑁 (𝑡) is a Poisson process, then its pmf is given by

(𝜆𝑡)𝑛 −𝜆𝑡
P[𝑁 (𝑡) = 𝑛] = 𝑒 𝑛 = 0, 1, 2, ... (4.6)
𝑛!
So 𝑁 (𝑡) ∼ Pois(𝜆𝑡), it is distributed as a Poisson variable with mean 𝜆𝑡.
We also have the identity

P[𝑁 (𝑠 + 𝑡) − 𝑁 (𝑠) = 𝑛] = P[𝑁 (𝑡) = 𝑛]

by the stationarity of the increments, so this distribution completely characterizes the entire
process. ■

Exponential systems
We use the term exponential systems to refer to queue systems in which the inter-arrival
time and service time both have exponential distributions.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.2. The M/M/1 queue- Background 21

4.2.1 Exponential systems in SPE- The M/M/1 queue with Key results

The simplest exponential system is referred to as the M/M/l queue. The M/M/1 briefly is a
system where i) it is equipped with a single server,

ii) the service process is exponential (denoted by the middle M), and

iii) the arrival process is Poisson, so denoted by the first M.

This implies that the inter-arrival times between events are exponential, see Theorem 4.1.

We precisely determine the M/M/l system with wellknown notation below.

Let 𝐴𝑛 denote the time when the 𝑛-th customer arrives, and thereby

𝐼𝑛 = 𝐴𝑛+1 − 𝐴𝑛, an interarrival time (between the 𝑛𝑡ℎ to the (𝑛 + 1)𝑠𝑡 event),

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.2. The M/M/1 queue- Background 22

𝑆𝑛+1 be the service time for the 𝑛 + 1-st customer,

⏞ ⏟
𝐴𝑟𝑟𝑖𝑣𝑎𝑙 𝑡𝑖𝑚𝑒𝑠 0‖ − − − − − −An − − − −An+1 − − − − −− 𝐴𝑛+2 − −− >

In = An − An−1 ↑ 𝑆𝑛+1 ↑

Configuration 1: A single-station (server) queueing system

Figure 4.3: Queueing systems with single queue

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.2. The M/M/1 queue- Background 23

Definition 4.1 (Full description of M/M/1)

We summarize discussed points so far in the followings. ♣

1. The arrival process is a Poisson process 𝑁 (𝑡) with rate 𝜆 𝑡.

Job 𝑛 comes in the queue at time point 𝐴𝑛. The variable 𝐼𝑛 = 𝐴𝑛 − 𝐴𝑛−1 is the
inter-arrival times between the (𝑛 − 1)𝑠𝑡 and the 𝑛𝑡ℎ arrivals, for 𝑛 = 1, 2, 3, . . ..
Then {𝐼𝑛} are independent random variables, and exponentially distributed, i.e.
{𝐼𝑛} ∼ 𝐼 = Exp(𝜆) have the same cdf 𝐹 (𝑡) = P[𝐼𝑛 ≤ 𝑡] = P[𝐼 ≤ 𝑡] = 1 − 𝑒−𝜆𝑡.
2. The service times {𝑆𝑛} ∼ 𝒮 = Exp(𝜇) are i.i.d., with the same cdf

𝐺(𝑡) = P[𝑆𝑛 ≤ 𝑡] = P[𝒮 ≤ 𝑡] = 1 − 𝑒−𝜇𝑡.

3. The arrival process and the service process are independent of each other. ■

In addition, the server adopt a work-conservation policy; that is, the server is never idle when
there are customers in the system.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.2. The M/M/1 queue- Background 24

Theorem 4.1 ( Poisson arrival process gives exponential inter-arrival times.)

♡

If the arrival times 𝐴𝑛 form a Poisson process with arrival rate 𝜆, then the inter-arrival times
{𝐼𝑛 = 𝐴𝑛 − 𝐴𝑛−1 : 1 ≤ 𝑛 < ∞} are independent and identically exponentially distributed, i.e.
they follow a common exponential variable with the above parameter 𝜆.

Mathematically, the i.i.d. series {𝐼𝑛} ∼ 𝐼 = Exp(𝜆) has the common cdf

𝐹 (𝑡) = P[𝐼𝑛 ≤ 𝑡] = P[𝐼 ≤ 𝑡] = 1 − 𝑒−𝜆𝑡. (4.7)

We also get P[𝐼 > 𝑡] = 1 − 𝐹 (𝑡) = 𝑒−𝜆𝑡, and their mean is

1
E[𝐼] = .
𝜆

The inverse of Theorem 4.1 is true.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.3. The M/M/1 queue - Characterization via Performance Metrics 25

We claim that Exponential inter-arrival times imply Poisson arrival process.

Theorem 4.2 ♡

If the inter-arrival times {𝐼𝑛} are iid exponentially distributed with an arrival rate of 𝜆, with
a cdf 𝐹 (𝑡) = P[𝐼𝑛 ≤ 𝑡] = P[𝐼 ≤ 𝑡] = 1 − 𝑒−𝜆𝑡, a mean E[𝐼𝑛] = 1/𝜆 and pdf 𝑓𝐼 (𝑡) = 𝜆 𝑒−𝜆𝑡,

then the number of arrivals 𝑁 (𝑡) in the time interval [0, 𝑡) forms a Poisson process with
parameter 𝜆 𝑡.

Knowledge box 4.2. We have the following conclusion.

An arriving process follows Poisson random variable Pois(𝜆) if and only if the time differences
of consecutive arrivals follows exponential random variable Exp(𝜆).

𝑁 (𝑡) ∼ Pois(𝜆𝑡) ⇔ 𝐼𝑛 = 𝐴𝑛 − 𝐴𝑛−1 ∼ Exp(𝜆).

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.3. The M/M/1 queue - Characterization via Performance Metrics 26

4.3 The M/M/1 queue - Characterization via Performance Metrics

SUMMARY: Briefly in the M/M/1 system

the arrival process is Poisson and

the service process is exponential. By this we mean that

both the inter-arrival times between customers {𝐼𝑛 : 1 ≤ 𝑛 < ∞} is exponential, and

the service times {𝑆𝑛 : 1 ≤ 𝑛 < ∞} is exponential. Precisely,

Arrival process is a Poisson process or Poisson stream. Moreover, by Knowledge box 4.1
the number of arrivals 𝑁 (ℎ) = 𝑘 over a given interval (𝑡, 𝑡 + ℎ) has a Poisson distribution
(𝜆 ℎ)𝑘 𝑒−𝜆 ℎ
P[𝑁 (ℎ) = 𝑘] =
𝑘!
Inter-arrival time 𝐼𝑛 = sequence of IID (identically and independently distributed) and exponen-
tial r.v.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.3. The M/M/1 queue - Characterization via Performance Metrics 27

■ EXAMPLE 4.3.

Suppose that people immigrate to a country at a Poisson rate of 𝜆 = 1.5 persons per day.

1. What is the expected time until the 100𝑡ℎ immigrant arrives?

2. What is the probability that the elapsed time between the 100𝑡ℎ immigrant and the next immi-
grant’s arrival exceeds 2 days?

GUIDANCE for solving. Due to Equation 4.7 we see that the inter-arrival times are i.i.d. random

variables 𝐼1 , 𝐼2 , . . . , 𝐼100 ∼ 𝑌 ∼ Exp(𝜆). Hence

1. The expected (mean) time E[𝑇 ] until the 100𝑡ℎ immigrant arrives: the time 𝑇 from the beginning
100
∑︁
until the 100𝑡ℎ arrival is 𝑇 = 𝐼𝑗 , so the corresponding E[𝑇 ] (known the expectation sum is
𝑗=1
additive)- is E[𝑇 ] = E(𝐼1 + . . . + 𝐼100) = 100 * E(𝑌 ) = 100 * (1/1.5) = 66.7(𝑑𝑎𝑦𝑠).
2. The probability that the elapsed time between the 100𝑡ℎ immigrant and the next immigrant’s
arrival exceeds 2 days: 𝑃 (𝐼101 > 2) = 𝑒−1.5.(2) = 𝑒−3 = 0.4979.■

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.3. The M/M/1 queue - Characterization via Performance Metrics 28

NOTE: About the role of time unit and rate in modeling time-dependence systems
in mass service theory (Queuing Theory).

Name of distribution Poisson Exponential

Notation 𝑋 ∼ Pois(𝜆) 𝑋 ∼ Exp(𝛽)

Parameter 𝜆 𝛽
Usage/Meaning Number of events Time between two events

in a time interval

Values- Range discrete, 𝑥 ∈ N = {0, 1, 2, ...} continuous, R

Mean E[𝑋] = 𝜆 E[𝑋] = 1/𝛽 and

Variance V[𝑋] = 𝜆 V[𝑋] = 1/𝛽 2

(𝜆)𝑛 −𝜆
Probability pdf: P[𝑋 = 𝑛] = 𝑒 cdf: 𝐹 (𝑡) = P[𝑋 ≤ 𝑡] = 1 − 𝑒−𝛽𝑡
𝑛!
distribution or the survival function

𝑆(𝑡) = P[𝑋 > 𝑡] = 𝑒−𝛽𝑡 .

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.3. The M/M/1 queue - Characterization via Performance Metrics 29

QUIZ 2 ((2 points) in max 15 minutes).

A) Use Poisson: What is the probability the trains at a station depart less than two trips in 20

minutes?
B) Use exponential: Assume that 𝛽 = 𝜆 as above.

If use time unit in minute now, what is the probability we complete a homework between 5
and 8 minutes, assuming the rate of completing 1 homework is 𝛽?

GUIDANCE for solving.

A) Denote 𝑋 the number of trains departing from the station. What is 𝑋? what is 𝜆? Shall we
compute P[𝑋 < 2]?

B) We should suitably utilize essential differences between Poisson and exponential distributions.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.3. The M/M/1 queue - Characterization via Performance Metrics 30

4.3.1 Evaluating performance of simple queue with one server

We study the mean number of jobs and few key relevant performance indexes of simple queue
with one server (being either exponential system or not).

Let the transient-state 𝑋(𝑡) be the number of jobs in the system at time 𝑡, and

the steady-state 𝑋 be the number of jobs when 𝑡 is large.

Theorem 4.3
We have the following results ♡

𝑛
(︀ )︀
(𝐴) 𝑝𝑛 = P[𝑋 = 𝑛] = 𝜌 1−𝜌
𝜌
(𝐵) E[𝑋] = , (4.8)
1−𝜌
𝜌
(𝐶) V[𝑋] = ,
(1 − 𝜌)2

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.3. The M/M/1 queue - Characterization via Performance Metrics 31

PROOF: We utilize basic parameters

𝜆, = arrival rate and 𝜇 = service rate

1
𝑆 = service time of each job , with mean E[𝑆] = ,
𝜇
𝜆
𝜌= = load, utilization, or arrival-to-service ratio.
𝜇

The mean number of jobs E[𝑋] in system

To obtain (4.8).(B) and (4.8).(C) we should realize that the distribution of 𝑋(𝑡) is Shifted
Geometric, because 𝑌 = 𝑋 +1 has the standard Geometric distribution with parameter 𝑝 = 1−𝜌,
for 𝑦 ≥ 1 then P[𝑌 = 𝑦] = P[𝑋 = 𝑦 − 1 = 𝑛] = 𝑝𝑦−1 = 𝜌𝑦−1(1 − 𝜌) = 𝑝(1 − 𝑝)𝑦−1.

* The utilization of the server (the probability of having at least one job in the system), actually
𝜆
is the arrival-to-service ratio 𝜌 = = 1 − 𝑝0 = P[𝑋 > 0]
𝜇
=⇒ P[“ server is busy ′′] = 𝜌 =⇒ P[“ server is idle ′′] = 1 − 𝜌
SYSTEM PERFORMANCE EVALUATION Methods and Algorithms
4.3. The M/M/1 queue - Characterization via Performance Metrics 32

* Clearly, the mean and the variance respectively are

⎧
⎪ 1 1 𝜌
⎨E[𝑋] = E[𝑌 ] − 1 = − 1 = −1=
⎪
⎪
𝑝 1−𝜌 1−𝜌 (4.9)
⎪ 1−𝑝 𝜌
⎪
⎪ V[𝑋] = V[𝑌 − 1] = V[𝑌 ] = = .■
𝑝2 (1 − 𝜌)2
⎩

TAKE AWAY REMARK

We can also say that 𝜌 is the proportion of time when the server (system) is put to work.

In other words, 𝜌 shows how much the server is utilized.

The system is functional if 𝜌 < 1. Our derivation of the distribution of 𝑋 is only possible
when 𝜌 < 1, otherwise the geometric series used there diverges.
If 𝜌 ≥ 1, the system gets overloaded. Arrivals are too frequent comparing with the service
rate, and the system cannot manage the incoming flow of jobs. 2
2 The number of jobs in the system will accumulate in this case (unless, of course, it has a limited capacity).

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.3. The M/M/1 queue - Characterization via Performance Metrics 33

4.3.2 Steady-states Analysis for the M/M/1 queue

What if your simple queue really becomes an exponential system like the 𝑀/𝑀/1 system?

Waiting time 𝑊

When a new job arrives, it finds the system with 𝑋 jobs in it. While these 𝑋 jobs are being
served, the new job awaits its turn in a queue. Thus, its waiting time 𝑊 = 𝑆1 + 𝑆2 + · · · + 𝑆𝑋
just consists of service times of 𝑋 earlier jobs, Recall that

(a) service times in M/M/1 systems are exponential, and

(b) this distribution has a memoryless property.

We conclude that the new job has expected waiting time 3 :

E[𝑆] 𝜌
E[𝑊 ] = E[𝑆1 + · · · + 𝑆𝑋 ] = E[𝑆] E[𝑋] = (4.10)
1−𝜌
3 actually we must use the fact that service times are independent of the number of jobs in the system at that time

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.3. The M/M/1 queue - Characterization via Performance Metrics 34

Response time 𝑅: the time a job spends in the system, from its arrival until its departure.

Response time 𝑅 consists of waiting time (if any) and service time. The expected response time
can then be computed as
E[𝑆] 𝜌 E[𝑆]
E[𝑅] = E[𝑊 ] + E[𝑆] = + E[𝑆] = (4.11)
1−𝜌 1−𝜌

Queue- a waiting room of length 𝑋𝑤 is the number of waiting jobs, 𝑋𝑤 = 𝑋 − 𝑋𝑠.

The number of jobs 𝑋𝑠 getting service at any time is 0 or 1.

It is therefore a Bernoulli variable with parameter P[ server is busy ] = 𝜌.
The expected queue length (use (4.9)) is
𝜌 𝜌2
𝐿𝑤 = E[𝑋𝑤 ] = E[𝑋] − E[𝑋𝑠] = −𝜌=
1−𝜌 1−𝜌
What is the probability (chance or likelihood) of idle system? It is measured via system load 𝜌.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.3. The M/M/1 queue - Characterization via Performance Metrics 35

Relationship between 𝑆, 𝑊 and 𝑋, 𝑋𝑤

namely between E[𝑆], E[𝑊 ] and E[𝑋] = 𝐿, E[𝑋𝑤 ] = 𝐿𝑤 .

ASSUMPTION: The system is functional (𝜌 < 1), so all the jobs go through the entire system.

Thus, each component is subject to the same arrival rate 𝜆 = 𝜆𝐴. With 𝜇 is the service rate,
𝜌
E[𝑋] = 𝐿 = , and
1−𝜌
𝜌2
E[𝑋𝑤 ] = 𝐿𝑤 = , here are the main performance characteristics:
1−𝜌
⎧
⎪ E[𝑆] 1
⎪
⎪ E[𝑆] = 1/𝜇 =⇒ E[𝑅] = = ,
1 − 𝜌 𝜇 (1 − 𝜌)
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎨
(4.12)
⎪ E[𝑆] 𝜌 𝜌
⎪
⎪ E[𝑊 ] = = ,
⎪
⎪
⎪
⎪ 1 − 𝜌 𝜇 (1 − 𝜌)
𝜆
⎪
⎪
⎩ P[ server is busy ] = 𝜌 = =⇒ P[ server is idle ] = 1 − 𝜌.
⎪
⎪
𝜇
The Little’s Law certainly applies to the 𝑀/𝑀/1 queue and its components, the queue and the

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.3. The M/M/1 queue - Characterization via Performance Metrics 36

server, showing the second look of relation between 𝑅, 𝑆, 𝑊 and 𝐿, 𝐿𝑤 :

𝜆 E[𝑅] = E[𝑋] = 𝐿 =⇒ E[𝑅] = 𝐿/𝜆

𝜆 E[𝑆] = E[𝑋𝑠], =⇒ E[𝑋𝑠] = 𝜆/𝜇 = 𝜌 = 𝐿 − 𝐿𝑤 (4.13)

𝜆 E[𝑊 ] = E[𝑋𝑤 ] = 𝐿𝑤 =⇒ E[𝑊 ] = 𝐿𝑤 /𝜆.

■

LAB WORK 1 (Message transmission with a single channel ).

Messages arrive to a communication center at random times with an average of 5 messages per
minute. They are transmitted through a single channel in the order they were received.

In average, the transmission takes 10 seconds to transmit a message. Conditions of an M/M/1

queue are satisfied.
Compute the main performance characteristics for this center.

GUIDANCE for solving. Need to five 5 performance metrics as 𝜌, E[𝑋] . . . , E[𝑅]

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.4. The generic 𝐺/𝐺/𝑚 queue and Its Performance 37

The arrival rate 𝜆𝐴 = 5𝑚𝑖𝑛−1 and the expected service time E[𝑆] = 10 seconds or (1/6) min are
given. Then, the utilization is ?

This also represents the proportion of time when the channel is busy and the probability of a
non-zero waiting time. The average number of messages stored in the system at any time

E[𝑋] =? = 5, =⇒ E[𝑋𝑤 ] =?

the average number of messages are waiting, and E[𝑋𝑠] = 𝜌 = 0.83 ones are being transmitted.

When a message arrives to the center, its waiting time 𝑊 (until its transmission begins) and
response time 𝑅 have respectively means

?E[𝑊 ] =?; E[𝑅] =?

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.4. The generic 𝐺/𝐺/𝑚 queue and Its Performance 38

4.4
The generic 𝐺/𝐺/𝑚 queue

QUESTION.

Are all queues exponential like 𝑀/𝑀/1 queue? NO, we have G| G| 𝑚 queue where neither
arrivals are Poisson nor service times are exponential!

General G → Results valid for all distributions

For the generic G| G| 𝑚 queue, with 𝑚 servers, 𝑚𝜇 is the overall service rate, then the utilization
𝜆
ratio 𝜌 = . Rules for all queues firsty applied to 𝐺/𝐺/𝑚 queues are the followings.
𝑚𝜇

(1) Stability Condition: 𝜌 < 1 ⇐⇒ 𝜆 < 𝑚 𝜇.

Finite-population and infinite-buffer systems are always stable.
SYSTEM PERFORMANCE EVALUATION Methods and Algorithms
 4.4. The generic 𝐺/𝐺/𝑚 queue and Its Performance 39

The throughput of G| G| 𝑚 is 𝜆 = 𝜌 · (𝑚 𝜇).

(2) Number in System versus Number in Queue:
The number of jobs 𝑋 = 𝑋𝑤 + 𝑋𝑠, where 𝑋, 𝑋𝑤 , and 𝑋𝑠 are random variables;
If the service rate is independent of the number in the queue, then

Cov(𝑋𝑤 , 𝑋𝑠 ) = 0 =⇒ V[𝑋] = V[𝑋𝑤 ] + V[𝑋𝑠 ].

The expected number of jobs in the queue (the queue length) 𝑄 = E[𝑋𝑤 ].

(3) The expected ( average) number of jobs 𝐿 = E[𝑋] in a stable system:

Write the number of jobs in a stable system by variable 𝑋, we have two ways to find 𝐿:

(I) If we know distribution of 𝑋 or its densities 𝑝𝑘 then the expected number of jobs
∞
∑︁
𝐿 = E[𝑋] = 𝑘 𝑝𝑘 = E[𝑋𝑤 ] + E[𝑋𝑠] (4.14)
𝑘=1

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.4. The generic 𝐺/𝐺/𝑚 queue and Its Performance 40

(II) Use Little’s law as mentioned above.

♦ Directly 𝐿 = E[𝑋] = E[𝑋𝑤 ] + E[𝑋𝑠] = 𝑄 + E[𝑋𝑠] = 𝑄 + 𝜌?

♦ Number versus Time: if jobs are not lost due to insufficient buffers, then
the Mean number of jobs in the system = Arrival rate × Mean response time,

𝐿 = E[𝑋] = 𝜆 E[𝑅]

♦ The Mean number of jobs in the queue = Arrival rate × Mean waiting time,

E[𝑋𝑤 ] = 𝜆 E[𝑊 ].

(4) Response Time 𝑅 ( not Time in Queue 𝑊 ?)

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.4. The generic 𝐺/𝐺/𝑚 queue and Its Performance 41

The Response Time 𝑅 = 𝑊 + 𝑆 is the total time that a job spends in the queuing system
1
𝑅 = 𝑊 + 𝑆 =⇒ E[𝑅] = E[𝑊 ] + E[𝑆] = E[𝑊 ] + . (4.15)
𝜇

(5) Waiting Time 𝑊 = 𝑅 − 𝑆:

the time that a job spends in a queue waiting to be serviced.
Since 𝑊, 𝑅 and 𝑆 are random numbers, and service rate is 𝜇 then their means are
1
E[𝑊 ] = E[𝑅] − E[𝑆] = E[𝑅] − . (4.16)
𝜇

(6) If the service time 𝑆 is independent of the number of jobs 𝑋𝑤 in the queue,
then 𝑆 also independent of waiting time 𝑊 , thereore

Cov(𝑊, 𝑆) = 0 =⇒ V[𝑅] = V[𝑊 ] + V[𝑆].

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.4. The generic 𝐺/𝐺/𝑚 queue and Its Performance 42

ASSIGNMENT 1.

GUIDANCE for solving.

Each team has 3-4 students, time due 23:00 Friday of WEEK 7, max score 8 pts]

Write a program (self choosing either Python or similar language) to quantify major processes
and compute key PERFORMANCE METRICS of the general 𝐴/𝑆/𝑚/𝐵/𝐾/𝑆𝐷 queue, with
𝑚 = 1 server and the two processes of arrival 𝐴 and service 𝑆 is exponential, or can be generic
G.
◇ Describe in the report specific probability distributions of the G| G| 𝑚 queue before you go,
where G is a wellknown distribution (𝑀, 𝐸, or 𝐻... as follows).
◁ The more distributions you employ exactly for arrival process and the service process the higer
score you obtain.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.4. The generic 𝐺/𝐺/𝑚 queue and Its Performance 43

HINT: If the program can express only M| M| 1 the team get only 8 pts.

You could use term and notation reminded below for the general 𝐴/𝑆/𝑚/𝐵/𝐾/𝑆𝐷.

Common Distributions

𝑀 : Exponential,
𝐸𝑘 : Erlang with parameter 𝑘,
𝐻𝑘 : Hyper-exponential with parameter 𝑘,
𝐷: Deterministic → constant,
𝐺: General → All.
Memoryless:
Remembering the past history does not help.

1. Arrival process 𝐴 in 𝐴/𝑆/𝑚/𝐵/𝐾/𝑆𝐷

Arrival times: 𝑡1, 𝑡2, . . . , 𝑡𝑗 .

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.4. The generic 𝐺/𝐺/𝑚 queue and Its Performance 44

Interarrival times: 𝐼𝑗 = 𝜏𝑗 = 𝑡𝑗 − 𝑡𝑗−1.

𝜏𝑗 form a sequence of Independent and Identically Distributed (IID) random variables.
Exponential + IID interarrival times → Poisson arrivals.
Notation of probability distribution
𝑀 = Memoryless = Poisson,
𝐸 = Erlang,
𝐻 = Hyper-exponential,
2. Service time distribution 𝑆 in 𝐴/𝑆/𝑚/𝐵/𝐾/𝑆𝐷
𝑆= Time each job/car/student/task/customer ... spends at the terminal.
Service times are IID.
Distribution: 𝑀, 𝐸, or 𝐻.
Device = Service center = Queue.
Buffer = Waiting positions.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.4. The generic 𝐺/𝐺/𝑚 queue and Its Performance 45

3. Service Disciplines SD
First-Come-First-Served (FCFS);
Last-Come-First-Served (LCFS);
Last-Come-First-Served with Preempt and Resume (LCFS-PR);
Round-Robin (RR) with a fixed quantum.
SIRO (Service-In-Random-Order):
The job to be served next is selected at random.
Small Quantum → Processor Sharing (PS) (**)
Shortest Processing Time first (SPT);

■ EXAMPLE 4.4 ( 𝑀/𝑀/3/20/1500/𝐹 𝐶𝐹 𝑆 ).

Time between successive arrivals is exponentially distributed.

Service times are exponentially distributed.
Three servers,
SYSTEM PERFORMANCE EVALUATION Methods and Algorithms
4.5. Stochastic process and Markov process 46

20 Buffers = 3 service + 17 waiting,

After 20, all arriving jobs are lost,
Total of 1500 jobs that can be serviced.
Service discipline is FCFS.
Defaults:
Infinite buffer capacity,
Infinite population size,
FCFS service discipline.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.5. Stochastic process and Markov process 47

4.5 Stochastic process and Markov process

Definition 4.2 (Stochastic Process)

A stochastic process (SP) is a series of random variables, depends on time 𝑡, or other index.♣

It is therefore a function of two arguments, 𝑋(𝑡, 𝑤), where

parameter 𝑡 ∈ 𝑇 is time, with 𝑇 being a set of possible times, usually [0, ∞), the set of reals
R = (−∞, ∞), the naturals N = {0, 1, 2, . . .}, or Z = {. . . , −3, −2, −1, 0, 1, 2, . . .};
𝑤 ∈ Ω, an outcome of an experiment, in sample space Ω.

A SP can be visualized by a map 𝑋 : 𝑇 × Ω −→ 𝒮, with (𝑡, 𝑤) ↦→ 𝑋(𝑡, 𝑤) = 𝑠.

Values, states or realizations 𝑠 = 𝑋(𝑡, 𝑤) called states make the state space

𝒮 := {𝑠 : 𝑠 = 𝑋(𝑡, 𝑤) for certain 𝑡 ∈ 𝑇 ∧ 𝑤 ∈ Ω}

of the process, 𝑋(𝑡, 𝑤) = 𝑠 says the process is in state 𝑠 at time 𝑡, measured at 𝑤.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.5. Stochastic process and Markov process 48

Time index set T

T could be discrete or continuous
Hence, a stochastic process is just a math-

Stochastic process as a map

ematical model of a probabilistic experiment

X:TxΩ S
that evolves in time and generates a sequence of

(t, w) X(t, w)= s numerical values.

The state space S could If we skip the spatial dimension 𝑤 ∈ Ω,

be discrete or
continuous
The sample space Ω a stochastic process 𝑋 is just a sequence (a
family, usually infinite) of random variables, as

Figure 4.4: Visualization of a stochastic process 𝑋 functions denoted by 𝑋𝑡 or 𝑋(𝑡) of only tempo-
ral parameter 𝑡 ∈ 𝑇 .

E.g., if skip the location (place) we may define

𝑛(𝑡) = 𝑋(𝑡) = the number of jobs waiting for CPU of a computer system at time 𝑡.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.5. Stochastic process and Markov process 49

TAKE AWAY Knowledge Box

Each random variable 𝑛(𝑡) (or 𝑋(𝑡)) is defined by a probability distribution
and Stochastic process is denoted {𝑛(𝑡)}.
Types of stochastic processes
Discrete or Continuous State Processes
𝑛(𝑡) = number of cars, is a discrete-state process
𝑤(𝑡) = the service time a car in washing room, is a continuous-state process

Poisson Process (studied in Lecture 3)

Markov Process and Birth-death Process

Stochastic processes can be generally subdivided into four distinct categories depending on
whether 𝑡 ∈ 𝑇 or 𝑋𝑡 ∈ 𝒮 are discrete or continuous.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.5. Stochastic process and Markov process 50

Time set 𝑇 State space 𝒮

Discrete Continuous

Discrete 1. Discrete process 4. Discrete time - continuous state process

(as naturals N)
Continuous 2. Continuous time- 3. Continuous process
(as reals R) discrete state process

QUIZ 3.

In a printer shop, let

1. 𝑋(𝑛, 𝑤) = the amount of time required to print the 𝑛-th job.

2. 𝑌 (𝑛, 𝑤) be the number of pages of the 𝑛-th printing job.

1. Describe components of process X = {𝑋(𝑛, 𝑤)}. Is it discrete-time, continuous-state SP?

2. Describe components of process Y = {𝑌 (𝑛, 𝑤)}. What class of SP does it belong to?

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.5. Stochastic process and Markov process 51

Classification of stochastic processes

1. Discrete processes: both 𝒮 and 𝑇 are discrete, as Discrete Time Markov chains.
Examples: random walk model consisting of positions 𝑋𝑡 of an object (drunkand) at hourly time point 𝑡 during 24 hours, whose directional distance

from a particular point 0 is measured in integer units. Here 𝑇 = {0, 1, 2, . . . , 24}.

2. Continuous time discrete state processes: the state space 𝒮 is discrete and the time index
set 𝑇 of 𝑡 is continuous, as the reals R = (−∞, ∞) or its intervals.
Poisson process- the number of clients 𝑋(𝑡) entered a bank from the time 0 til 𝑡 > 0;
the number of infant births 𝑋𝑡 in a given population during time period [0, 𝑡]... 𝑋(𝑡) follows a Poisson distribution with mean E[𝑋(𝑡)] = 𝜆𝑡 (𝜆
- the arrive rate). Here the time index 𝑇 = R+ = [0, ∞) and the state space is {0, 1, 2, . . . , } = N.
Continuous time Markov chain- CTMC: the state space 𝒮 is finite.

3. Continuous process: both state space 𝒮 and time index set 𝑇 are continuous,
such as Markov process and diffusion process (Brownian motion). Example: 𝑋𝑡 is population density at time 𝑡 ∈ 𝑇 = R+ , the state space of 𝑋𝑡 is R+ .
4. Discrete time continuous state process: the time index 𝑇 is discrete, and the state space 𝒮 is
continuous– the so-called TIME SERIES such as
monthly fluctuations of the inflation rate of Vietnam, Thailand or India
daily fluctuations of a stock market.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.5. Stochastic process and Markov process 52

Relationship among stochastic processes

A Markov process - a process in which future states are independent of the past and depend
only on the present, so easier to analyze, not have to remember past trajectory.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.5. Stochastic process and Markov process 53

E.g., Markov decision process in learning is visualized as

.
A Markov chain = A discrete-state Markov process {𝑋(𝑡) = 𝑋𝑛}

♣ OBSERVATION 4.1. Some notable remarks:

Knowing current (present) state is sufficient

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.5. Stochastic process and Markov process 54

Not necessary to know how long the process has been in the current state.
=⇒ State time (as service time) has a memoryless distribution.
E.g., Only exponential distribution 𝑋 ∼ Exp(𝜆) gives 𝑃 𝑟{𝑋 > 𝑠 + 𝑡|𝑋 > 𝑡} =
𝑃 𝑟{𝑋 > 𝑠 + 𝑡 and 𝑋 > 𝑡} 𝑃 𝑟{𝑋 > 𝑠 + 𝑡} 𝑒−𝜆(𝑠+𝑡)
= = = −𝜆𝑡 = 𝑒−𝜆𝑠 (4.17)
𝑃 𝑟{𝑋 > 𝑡} 𝑃 𝑟{𝑋 > 𝑠} 𝑒

The time spent by a job in such a queue is a Markov process,

and the number of jobs in the queue is a Markov chain.

Definition 4.3 (Markov process)

Stochastic process 𝑋(𝑡) is Markov process if for any sequence real numbers 𝑡1 < 𝑡2 < . . . <
𝑡𝑛 < 𝑡, and any sets 𝐴; 𝐴1, 𝐴2, . . . , 𝐴𝑛 (𝐴𝑖 ⊂ R)

P[𝑋(𝑡) ∈ 𝐴|𝑋(𝑡1) ∈ 𝐴1, · · · , 𝑋(𝑡𝑛) ∈ 𝐴𝑛] = P[𝑋(𝑡) ∈ 𝐴|𝑋(𝑡𝑛) ∈ 𝐴𝑛]. ♣

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.5. Stochastic process and Markov process 55

Stationary or time homogeneous Markov processes

A Markov process (by Definition 4.3) 𝑀 = {𝑋(𝑡)}𝑡≥0 = (𝒮, P(𝑡)) is called

either a Markov jump process or continuous-time Markov chain (CTMC)

if the state space 𝒮 is a finite or countable set respectively.

The CTMC , for all 𝑖, 𝑗 ∈ 𝒮 and 𝑠, 𝑡 > 0 has the Markov property:

P[𝑋(𝑠 + 𝑡) = 𝑗|𝑋(𝑠) = 𝑖 and 𝑋(𝑢) for 𝑢 < 𝑠] = P[X(s + t) = j|X(s) = i], (4.18)

𝑡𝑖𝑚𝑒 : − − − − − − 𝑢 − − − − − s − − − − − s + t − − − − − − >

𝑠𝑡𝑎𝑡𝑒 : − − −𝑋(𝑢) − − − − X(s) = i − − − −X(s + t) = j − − − −

That means given the evolution of the process up to any current time 𝑠, the future value and its
probabilistic description depend only on the current state at time 𝑠.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.5. Stochastic process and Markov process 56

The state transition probability is the conditional probability

𝑝𝑖,𝑗 (𝑠, 𝑡) = P[𝑋(𝑠 + 𝑡) = 𝑗 |𝑋(𝑠) = 𝑖],

that the process is in state 𝑗 at time 𝑠 + 𝑡 given that

the process was in state 𝑖 at the previous time 𝑠, for all 𝑖, 𝑗 ∈ 𝒮.

Definition 4.4 (Stationary transition probability - Markov jump process)

♣

(1) The transition probability 𝑝𝑖,𝑗 (𝑠, 𝑡) for a fix pair of states (𝑖, 𝑗) are said to be stationary, or

equivalently be unaffected by a shift in the time origin if we have the identity

𝑝𝑖,𝑗 (𝑠, 𝑡) := P[𝑋(𝑠 + 𝑡) = 𝑗 |𝑋(𝑠) = 𝑖] = 𝑝𝑖,𝑗 (𝑡) (4.19)

depends only on the length 𝑡 of the time interval [𝑠, 𝑠 + 𝑡], whatever the time point 𝑠 is.

(2) A Markov “jump” process 𝑀 = {𝑋(𝑡)}𝑡≥0 = (𝒮, 𝑝, 𝑃 (𝑡)) is called time homogeneous or

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.6. Poisson Process- Variation with Application 57

stationary if all state transition probabilities of 𝑀 are stationary. ■

For such homogeneous Markovian “jump” process we have known two distinct concepts:
Transition probabilities 𝑝𝑖,𝑗 (𝑡) = P[𝑋(𝑠 + 𝑡) = 𝑗 |𝑋(𝑠) = 𝑖], and
(4.20)
Marginal probabilities 𝑝(𝑢) = [𝑝𝑗 (𝑢)]𝑗∈𝒮 where 𝑝𝑗 (𝑢) = P[𝑋(𝑢) = 𝑗].
Here
𝑝𝑖,𝑗 (𝑡) is transition probability from state 𝑖 to state 𝑗 after a time gap 𝑡,
𝑝𝑗 (𝑢) is the ‘state’ probability that the process is in state 𝑗 at time point 𝑢.

NOTE: This “jump” process (DTMC) is used to characterize system workloads in Chapter 8.

Both Markov process and Poisson Process with Its Variation play useful roles in WEEEK 5
about Birth and Death processes.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.6. Poisson Process- Variation with Application 58

4.6
Poisson Process- Variation with Application

Poisson Process generally has a few more special properties below.

(1) Merging (Sum): 𝜆 = 𝑘𝑖=1 𝜆𝑖.

∑︀

Proposition 4.1 (Sum of 𝑘 = 2 independent Poisson processes)

♠

Suppose that {𝑁1(𝑡)} and {𝑁2(𝑡)} are two independent Poisson processes with rates 𝜆1 and
𝜆2 respectively. Precisely, 𝑁1(𝑡) ∼ Pois(𝜆1 𝑡), 𝑁2(𝑡) ∼ Pois(𝜆2 𝑡). [They are the respective
cumulative numbers of distinct arrival types in the time interval [0, 𝑡).]

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.6. Poisson Process- Variation with Application 59

Let 𝑁 (𝑡) = 𝑁1(𝑡) + 𝑁2(𝑡), the combined process of a cumulative number of arrivals until
time 𝑡. Then {𝑁 (𝑡)} is a Poisson process with arrival-rate parameter 𝜆1 + 𝜆2 ( the sum of
(︀ )︀
the individual arrival rates). 𝑁 (𝑡) ∼ Pois (𝜆1 + 𝜆2)𝑡 .
This result is extended to the Superposition Process of many independent Poisson processes.
Theorem 4.4 (Superposition Process)
♡

Assume that {𝑁1(𝑡)}𝑡≥0 , {𝑁2(𝑡)}𝑡≥0 , . . . , {𝑁𝑚(𝑡)}𝑡≥0 are 𝑘 = 𝑚 independent Poisson processes

with respective parameters 𝜆1, 𝜆2, . . . , 𝜆𝑚.

Superposition Poisson Process

Let 𝑁 (𝑡) = 𝑁1(𝑡) + 𝑁2(𝑡) + · · · + 𝑁𝑚(𝑡) for 𝑡 ≥ 0. Then,
{𝑁 (𝑡)}𝑡≥0 is a Poisson process with parameter 𝜆 = 𝜆1 + 𝜆2 + . . . + 𝜆𝑚.
This process is called a superposition Poisson process of the {𝑁𝑘 (𝑡)}𝑡≥0, for 𝑘 =
1, . . . , 𝑚.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.6. Poisson Process- Variation with Application 60

(2) Splitting (Thinning):

If the probability of a job going to 𝑖th substream is 𝑝𝑖, each substream is also Poisson with a
mean rate of 𝑝𝑖𝜆.

Thinning of a Poisson process

Proposition 4.2
Let {𝑁 (𝑡), 𝑡 ≥ 0} represent Poisson arrivals with rate 𝜆. ♠

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.6. Poisson Process- Variation with Application 61

Moreover, each arrival can be of type 1 with probability 𝑝 and be of

type 2 with probability 1 − 𝑝 independently of all other arrivals.

Let 𝑁𝑖(𝑡) (𝑖 = 1, 2) be the number of type 𝑖 arrivals up to time 𝑡.

Then {𝑁1(𝑡), 𝑡 ≥ 0} and {𝑁2(𝑡), 𝑡 ≥ 0} are independent Poisson processes
with rates 𝜆𝑝 and 𝜆(1 − 𝑝) respectively. ■

(3) If the arrivals to a single server with exponential service time are Poisson with mean rate 𝜆,
then the departures are also Poisson with the same rate 𝜆 provided 𝜆 < 𝜇.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.7. The Birth and Death processes 62

(4) If the arrivals to a service with 𝑚 service centers are Poisson with a mean rate 𝜆,
then the departures also constitute a Poisson stream with the same rate 𝜆,
∑︀
provided that 𝜆 < 𝑖 𝜇𝑖 . Here, the servers are assumed to have exponentially distributed

service times.

We next present key powerful principles of birth and death processes for analyzing properly
finite fapacity queuing systems...

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.7. The Birth and Death processes 63

4.7 The Birth and Death processes

Briefly a Birth-death process is a discrete-state Markov process in which the transitions are
restricted to neighboring states. Birth and death (BD) processes mathematically are obtained
when we generalize the Poisson process in two ways, by:

1. Letting the value of the arrival rate 𝜆 depend on the current state 𝑛;
2. Including departures, which allows the process to instantaneously decrease its value by one
unit (at a rate that will also be a function of 𝑛). Hence, process in state 𝑛 can change only to
state 𝑛 + 1 or 𝑛 − 1.

These generalizations can be used to describe not only customers entering and leaving a store,
but also populations that increase or decrease in size due to the birth of a new member or the
death of an old one. Birth and death processes formally are a special type of continuous-time
Markov chains (CTMC or Markov jump model).

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.7. The Birth and Death processes 64

Definition 4.5 (Consider a continuous-time Markov chain with states 0, 1, 2, . . .)

♣

If 𝑝𝑖𝑗 = 0 whenever 𝑗 ̸= 𝑖 − 1 or 𝑗 ̸= 𝑖 + 1, then the Markov chain is called a birth and death
process. Thus, a birth and death process is a CTMC in which transitions from state 𝑖 can only
go to either state 𝑖 − 1 or 𝑖 + 1.

That is, a transition either causes an increase in state by one or a decrease in state by one.

A birth is said to occur when the state increases by one, and

a death is said to occur when the state decreases by one.
For example, a queue with a single server and individual arrivals (not bulk arrivals) satisfies that
An arrival (birth): state changed by +1.
A departure (death): state changed by -1.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.7. The Birth and Death processes 65

Figure 4.5: State-transition-rate diagram for birth and death process.

Reminder:

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.7. The Birth and Death processes 66

For all 𝑖 ̸= 𝑗 ∈ 𝒮, the transition rate of the process when the process makes a transition
from state 𝑖 to state 𝑗, denoted by 𝑞𝑖,𝑗 , is defined by
𝑝𝑖,𝑗 (ℎ)
𝑞𝑖,𝑗 = 𝑝′𝑖,𝑗 (0) = lim for 𝑖 ̸= 𝑗. (4.21)
ℎ→0 ℎ
The transition rates 𝑞𝑖,𝑗 are also known as instantaneous transition rates, transition inten-
sities, or forces of transition.

Definition 4.6 ♣

If employ the (instantaneous) transition rate 𝑞𝑖,𝑗 given in Equation 4.21 of the continuoustime
Markov chain, then the process is said to be a birth and death process when

𝑞𝑖,𝑗 = 0 𝑖𝑓 |𝑗 − 𝑖| > 1.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.7. The Birth and Death processes 67

4.7.1 Description of a birth and death process

To describe the process we define the birth and death rates from each state 𝑖 ∈ 𝒮:
𝜆𝑖 = 𝑞𝑖,𝑖+1 = 𝑣𝑖 𝑝𝑖,𝑖+1
(4.22)
𝜇𝑖 = 𝑞𝑖,𝑖−1 = 𝑣𝑖 𝑝𝑖,𝑖−1
Thus, 𝜆𝑖 is the rate at which a birth occurs, and 𝜇𝑖 is the rate at which a death occurs, both when
the process is in state 𝑖.
The rate of transition out of state 𝑖 is the sum of these two rates 𝜆𝑖 + 𝜇𝑖 = 𝑣𝑖.
The state-transition-rate diagram of a B & D process is shown in Figure 4.6.
Figure 4.6 shows a state-transition-rate diagram as opposed to a state-transition diagram;
because it shows the rate at which the process moves from state to state,
and not the probability of moving from one state to another.

Note that 𝜇0 = 0, because there can be no death when the process is in empty state.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.7. The Birth and Death processes 68

Figure 4.6: State-transition-rate diagram for birth and death process.

4.7.2 Transient Analysis of Birth and Death Processes

The generic transition rate matrix, with

∑︁
𝑞𝑖 := 𝑞𝑖,𝑖 = − 𝑞𝑖,𝑗 < 0, for 𝑖 ∈ 𝒮
𝑖̸=𝑗∈𝒮

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.7. The Birth and Death processes 69

was

⎡ ⎤
⎢ 𝑞1 𝑞1,2 𝑞1,3 . . . 𝑞1,𝑠 ⎥
⎢ ⎥
⎢ 𝑞2,1 𝑞2 𝑞2,3 . . . 𝑞2,𝑠 ⎥
⎢ ⎥
⎢ ⎥
Q = ⎢ 𝑞3,1 𝑞3,2 𝑞3 . . . 𝑞3,𝑠 ⎥ . (4.23)
⎢ ⎥
⎢ ⎥
⎢ . ... ... . . . ... ⎥
⎢ .. ⎥
⎢ ⎥
⎣ ⎦
𝑞𝑠,1 𝑞𝑠,2 𝑞𝑠,3 . . . 𝑞𝑠

For a BD process with parameters given by diagram 4.6, Q takes a special form
⎡ ⎤
⎢ −𝜆0 𝜆0 0 ... 0... 0 ⎥
⎢ ⎥
⎢ 𝜇1 −(𝜆1 + 𝜇1) 𝜆1 0... 0... 0 ⎥
Q=⎢ ⎥. (4.24)
⎢ ⎥
⎢ 0
⎢ 𝜇2 −(𝜆2 + 𝜇2) 𝜆2 0 . . . 0 ⎥ ⎥
⎣ ⎦
... ... ... ... .
.. ...

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.7. The Birth and Death processes 70

The Kolmogorov’s forward differential equations say that

𝑑
𝑝(𝑡) = 𝑝(𝑡) Q, (4.25)
𝑑𝑡
where 𝑝(𝑡) = [𝑝0(𝑡), 𝑝1(𝑡), 𝑝2(𝑡), · · · , 𝑝𝑖(𝑡), 𝑝𝑖+1(𝑡), · · · ] is the vector of state distribution.

Transient analysis of this birth and death (B & D) process is done by studying the following
system of linear differential equations (LDS):
𝑑
𝑝0(𝑡) = −𝜆0 𝑝0(𝑡) + 𝜇1 𝑝1(𝑡)
𝑑𝑡
𝑑
𝑝1(𝑡) = 𝜆0 𝑝0(𝑡) − (𝜆1 + 𝜇1)𝑝1(𝑡) + 𝜇2 𝑝2(𝑡)
𝑑𝑡 (4.26)
......
𝑑
𝑝𝑖(𝑡) = 𝜆𝑖−1 𝑝𝑖−1(𝑡) − (𝜆𝑖 + 𝜇𝑖)𝑝𝑖(𝑡) + 𝜇𝑖+1 𝑝𝑖+1(𝑡)
𝑑𝑡
Thus, a B & D process is described by a system, called Local Balance Equations

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.7. The Birth and Death processes 71

𝑑𝑝0 (𝑡)
= −𝜆0 𝑝0 (𝑡) + 𝜇1 𝑝1 (𝑡)
𝑑𝑡 (4.27)
𝑑𝑝𝑖 (𝑡)
= −(𝜆𝑖 + 𝜇𝑖 )𝑝𝑖 (𝑡) + 𝜇𝑖+1 𝑝𝑖+1 (𝑡) + 𝜆𝑖−1 𝑝𝑖−1 (𝑡), for 𝑖 > 0
𝑑𝑡
𝑑𝑝𝑖(𝑡)
where the left hand side is the total rate of transition changing of state 𝑖, the + indicates
𝑑𝑡
the moving in rate and the − indicates the moving out rate.

𝑑𝑝𝑖(𝑡)
Now in the steady state, we have lim𝑡→∞ = 0, and lim𝑡→∞ 𝑝𝑖(𝑡) = 𝑝𝑖 exists, ∀ 𝑖 =
𝑑𝑡
0, 1, 2, . . . then the system (4.27) becomes the ‘local balance conditions’
⎧
⎪ 𝜆0
⎪
⎪ 𝜆 𝑝
0 0 = 𝜇 𝑝
1 1 =⇒ 𝑝 1 = 𝑝0
⎪
⎪
⎪
⎨ 𝜇 1

(𝜆𝑖 + 𝜇𝑖)𝑝𝑖 = 𝜇𝑖+1 𝑝𝑖+1 + 𝜆𝑖−1 𝑝𝑖−1 for 𝑖 = 1, 2, . . . (4.28)

⎪
⎪
⎪
⎪
⎪ ∑︀
𝑖 𝑝𝑖 =1
⎪
⎩

Recursively using this system yields the general result

𝜆𝑖𝑝𝑖 = 𝜇𝑖+1𝑝𝑖+1 ∀𝑖 = 0, 1, . . .

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.7. The Birth and Death processes 72

This result states that when the process is in the steady state, the rate at which it makes a transition
from state 𝑖 to state 𝑖 + 1, which we refer to the rate of flow from state 𝑖 to state 𝑖 + 1, is equal to
the rate of flow from state 𝑖 + 1 to state 𝑖.

This property (4.28) is referred to as the local balance equation or condition because it balances
(or equates) the rate at which the process enters state 𝑖 with the rate at which it leaves state 𝑖.
Direct application of the property allows us to solve for the steady-state probabilities of the birth
and death process recursively as follows:

∏︀𝑖
𝜆𝑖 𝑗=0 𝜆𝑗
𝑝𝑖+1 = 𝑝𝑖 = . . . = ∏︀𝑖+1 𝑝0 (4.29)
𝜇𝑖+1 𝜇
𝑗=1 𝑗

When 𝜆𝑖 = 𝜆 and 𝜇𝑖 = 𝜇 for all 𝑖, [under what distribution?] we get

[︂ ∞ (︂ )︂𝑖 ]︂−1
∑︁ 𝜆
𝑝0 = 1 + (4.30)
𝑖=1
𝜇

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.7. The Birth and Death processes 73

𝜆
The sum converges if and only if < 1, equivalent to 𝜆 < 𝜇. Under this condition, rewriting
𝜇
𝑛 := 𝑖 we obtain the solutions

⎧
⎪ 𝜆
⎨𝑝 0 =1−
⎪
⎪
(︂ 𝜇 )︂(︁ )︁ (4.31)
⎪ 𝜆 𝜆 𝑛
⎩𝑝 𝑛
⎪
⎪ = 1− , for 𝑛 = 1, 2, . . .
𝜇 𝜇
CONCLUSION
𝜆
Clearly, this probability exists only if 𝜆 < 𝜇 or < 1.
𝜇
𝜆
The value 𝑙 = is said to be the load of the system. 4 If the customers arrive at a rate 𝜆 > 𝜇,
𝜇
the service rate, then the system will overload and the queue will become infinite.
4 This is easily understood if we think about the system’s evolution.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.7. The Birth and Death processes 74

∑︀∞
We indeed have known 𝑖=0 𝑝𝑖 = 1 ⇒ 𝑝0(1 + 𝜌1 + 𝜌2 + ...) = 1
1 𝑛
⇒ 𝑝0 = = 1 − 𝜌 ⇒ 𝑝 𝑛 = 𝜌 (1 − 𝜌)
1 + 𝜌1 + 𝜌2 + ...

■ EXAMPLE 4.5.
A machine is operational for an exponentially distributed time with mean 1/𝜆 before breaking
down. When it breaks down, it takes a time that is exponentially distributed with mean 1/𝜇 to
repair it.

What is the fraction of time that the machine is operational (or available)?
SYSTEM PERFORMANCE EVALUATION Methods and Algorithms
4.7. The Birth and Death processes 75

Solution:

This is a two-state birth and death process. Let 𝑈 denote the up state and 𝐷 the down state.

Let 𝑝𝑈 denote the steady-state probability that the process is in the operational state, and

let 𝑝𝐷 denote the steady-state probability that the process is in the down state.

Then the balance equations become

𝜆𝑝𝑈 = 𝜇 𝑝𝐷
(4.32)
𝑝𝑈 + 𝑝𝐷 = 1 ⇒ 𝑝𝐷 = 1 − 𝑝𝑈
Hence, the fraction of time that the machine is operational is just 𝑝𝑈 = 𝜇/(𝜆 + 𝜇).

SUMMARY See application of this section in part II) on MODELING problems- in Section 4.8

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.8. CHAPTER CONCLUSION- HOMEWORK 76

of CHAPTER CONCLUSION- HOMEWORK.

4.8 CHAPTER CONCLUSION- HOMEWORK

4.8.1 M/M/1 - Basic Analysis and Computation

Let 𝑋 denote the total number of jobs in M/M/1 system, and

𝜆, 𝜇 = arrival rate and service rate, respectivelly,

𝑆 = service time of each job, (4.33)

1
=⇒ E[𝑆] = = mean service time.
𝜇
M/M/1 system - the most commonly used type of queue - being characterized by only arrival
rate 𝜆 and service rate 𝜇, has special properties below.

Structural properties:

Interarrival times follow exponentially distributed with parameter 𝜆, and service times also are

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.8. CHAPTER CONCLUSION- HOMEWORK 77

exponentially distributed with another parameter 𝜇.

One server, No limitation on buffer and population, and FCFS service discipline.

1. Probabilistic properties
M/M/1 is a birth-death process with
𝜆𝑛 = 𝜆, 𝑛 = 0, 1, 2, . . . , ∞
𝜇𝑛 = 𝜇, 𝑛 = 1, 2, . . . , ∞
Probability of 𝑛 jobs in the system
(︂ )︂𝑛
𝜆
𝑝𝑛 = P[𝑋 = 𝑛] = 𝑝0, 𝑛 = 1, 2, . . . , ∞
𝜇
2. System workload is measured by utilization ratio meaning Traffic intensity (thoughtput)

𝜌 = 𝜆/𝜇

=⇒ 𝑝𝑛 = 𝜌𝑛 𝑝0

Knowledge box 4.3. (Main performance characteristics of the M/M/1)

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.8. CHAPTER CONCLUSION- HOMEWORK 78

We may use all following facts of key quantities.

The mean 𝐿 number of jobs in system and its variance V[𝑋]

⎧
⎪ 𝜌 𝜆
⎪
⎪ 𝐿 = E[𝑋] = =
1−𝜌 𝜇−𝜆
⎪
⎪
⎪
⎪
⎪
⎪ 1−𝑝 𝜌
V[𝑋] = =
⎪
⎪
𝑝2 (1 − 𝜌)2
⎪
⎪
⎪
⎪
⎨
(4.34)
⎪
⎪
⎪
⎪
⎪ 1
⎪
⎪ The mean service time E[𝑆] =
𝜇
⎪
⎪
⎪
⎪
E[𝑆] 𝜌 𝜌
⎪
⎪
⎩ The mean waiting time E[𝑊 ] = = ,
⎪
⎪
1−𝜌 𝜇 (1 − 𝜌)
Indeed, when a new job arrives, it finds the system with 𝑋 jobs in it, each job is processed in
E[𝑆] time unit. The new job so has mean waiting time
E[𝑆] 𝜌 𝜌
E[𝑊 ] = E[𝑆1 + · · · + 𝑆𝑋 ] = E[𝑆] E[𝑋] = =
1−𝜌 𝜇 (1 − 𝜌)
Furthermore

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.8. CHAPTER CONCLUSION- HOMEWORK 79

E[𝑆] 1
1. The mean response time E[𝑅] = E[𝑊 ] + E[𝑆] = =
1 − 𝜌 𝜇 (1 − 𝜌)
2. The mean queue length (waiting list)
𝜌2
E[𝑋𝑤 ] = 𝐿𝑤 = E[𝑋] − E[𝑋𝑠] = 𝐿 − E[𝑋𝑠] =
1−𝜌
𝑛
(︀ )︀
3. The probability of having 𝑛 customers in the system 𝑝𝑛 = P[𝑋 = 𝑛] = 𝜌 1−𝜌
give probabilities of idle state and busy state
𝜆
P[ server is idle ] = 1 − 𝜌 =⇒ P[ server is busy ] = 𝜌 =
𝜇

4.8.2 HOMEWORK

PROBLEM 4.1 (THEORETIC REVIEW).

Given the queue 𝑀/𝑀/3/20/1500/𝐹 𝐶𝐹 𝑆, check the following facts, and explain in your own
words if possible

Time between successive arrivals is exponentially distributed.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms
4.8. CHAPTER CONCLUSION- HOMEWORK 80

Service times are exponentially distributed, and three servers,

20 Buffers = 3 service + 17 waiting, After 20, all arriving jobs are lost,
Total of 1500 jobs that can be serviced. And Service discipline is FCFS.
Defaults:
Infinite buffer capacity, Infinite population size,
FCFS service discipline.

I) COMPUTATIONAL EXERCISES - PROBLEMS

PROBLEM 4.2.

Consider an insurance company that has two types of policy: Policy A and Policy B . Total
claims from the company arrive according to a Poison process at the rate of 9 per day. A randomly
1
selected claim has a chance that it is of policy A. Calculate, on a given day
3
a/ the probability that claims from policy A will be fewer than 2,

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.8. CHAPTER CONCLUSION- HOMEWORK 81

b/ the probability that claims policy B will be fewer than 2,

c/ the probability that total claims from the company will be fewer than 2.

GUIDANCE for solving. Apply the Thinning technique in Proposition 4.2. Brief inputs:

𝑁𝐴(𝑡)= number of claims of policy A∼Poisson process at rate 𝜆 𝑝 = 9.1/3 = 3 per day
𝑁𝐵 (𝑡)= number of claims of policy B∼Poisson process at rate 𝜆(1 − 𝑝) = 9.2/3 = 6 per day
N(t) = total number of claims ∼ Poisson process at rate 𝜆 = 9 per day.
0 1
−3 3 −3 3
a) P[𝑁𝐴(1) < 2] = 𝑝(𝑁𝐴(1) = 0) + 𝑝(𝑁𝐴(1) = 1) = 𝑒 .𝑒 = 4.𝑒−3 =?
0! 1!
0 1
−6 6 −6 6
b) P[𝑁𝐵 (1) < 2] = 𝑝(𝑁𝐵 (1) = 0) + 𝑝(𝑁𝐵 (1) = 1) = 𝑒 .𝑒 = 7.𝑒−6 =?
0! 1!
0 1
9 9
c) P[𝑁 (1) < 2] = 𝑝(𝑁 (1) = 0) + 𝑝(𝑁 (1) = 1) = 𝑒−9 .𝑒−9 = 10.𝑒−9 = 0.0123.■
0! 1!
PROBLEM 4.3. (Printer).

Jobs are sent to a printer at random times, according to a Poisson process of arrivals, with a
rate of 12 jobs per hour. The time it takes to print a job is an Exponential random variable,
SYSTEM PERFORMANCE EVALUATION Methods and Algorithms
 4.8. CHAPTER CONCLUSION- HOMEWORK 82

independent of the arrival time, with the average of 2 minutes per job.
1. What is the service (processing) rate?
2. What is the system’s utilization 𝜌? Is the printer stable? What is departure rate?
3. At an arbitrary time point, you count a total of 5 jobs in the printer, and assume that this is
the typical, average number of jobs in steady state. Compute the mean E[𝑊 ] of waiting time
𝑊 , the mean E[𝑅] of response time 𝑅.
4. What is the expected number of jobs in the queue only?

GUIDANCE for solving.

1 1
1. The service (processing) rate 𝜆𝑆 = 𝜇 = = =?
E[𝑆] mean service time
The service time 𝑆 is the time taken to print a job, here 𝑆 ∼ Exp(𝜆𝑆 ) is an exponential
random variable, with service rate 𝜆𝑆 , and the mean E[𝑆] = 2 = 1/𝜇 minutes per job. So
1 1
𝜇 = 𝜆𝑆 = = jobs per minute.
E[𝑆] 2

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.8. CHAPTER CONCLUSION- HOMEWORK 83

2. What is the system’s utilization 𝜌? By Equation (4.2),

𝜆𝐴 arrival rate 𝜆𝐴 1/5 2
𝜌= = = = = = 0.4.
𝜆𝑆 service rate 𝜇 1/2 5
since the arrival rate 𝜆𝐴 = 12 𝑗𝑜𝑏𝑠/ℎ𝑜𝑢𝑟 = 1/5 jobs per minute.
* What is departure rate? When the service is completed, the job instantly leaves the system.
Since the system’s utilization (load) 𝜌 = 0.4 < 1, hence the printer is stable/steady or in statistical
equilibrium.
For this stable queue, the departure rate is equal to the arrival rate, hence the throughput

𝜆 = 𝜆𝐴 = 𝜌 𝜇 = 1/5 jobs per minute.

3. The mean E[𝑊 ] of waiting time 𝑊 , the mean response time E[𝑅].
Call 𝑋 the number of jobs in the printer in steady state. We have 𝐿 = E[𝑋] = 5 jobs, and
𝜆𝐴 E[𝑅] = E[𝑋] =⇒ E[𝑅] = 𝐿/𝜆𝐴 = 5/(1/5) = 25 𝑚𝑖𝑛𝑢𝑡𝑒𝑠
(4.35)
𝜆𝐴 E[𝑊 ] = E[𝑋𝑤 ] =⇒ E[𝑊 ] = E[𝑋𝑤 ]/𝜆𝐴 =?

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

 4.8. CHAPTER CONCLUSION- HOMEWORK 84

1
𝑊 = 𝑅 − 𝑆 =⇒ E[𝑊 ] = E[𝑅] − E[𝑆] = E[𝑅] − = 25 − 2 = 23 𝑚𝑖𝑛𝑢𝑡𝑒𝑠.
𝜇
4. What is the expected number of jobs in the queue only? It is
𝑄 = E[𝑋𝑤 ] = 𝜆𝐴 E[𝑊 ] = (1/5) 23 = 23/5 = 4.6 jobs. ■

Studied concepts are also used to solve practical optimization problems in

Traffic and Transportation sciences as follows.

II) MODELING

MODEL 4.1 (Winner in Competition of providing services).

For 𝑛 = 2, an interesting problem regarding the competition between two exponential random
variables 𝑋, 𝑌 is computing the probability that one of the two variables, such as 𝑋, is less than
𝑌 . This kind of model might appear in service providing, where a provider completing his service
earlier than that of the other providers will be considered as the winner. See an application of
SYSTEM PERFORMANCE EVALUATION Methods and Algorithms
4.8. CHAPTER CONCLUSION- HOMEWORK 85

this model for Transportation Science with the following information.

DATA and ASSUMPTION: Consider two independently operating Poisson processes with rate
parameters 𝜆1 and 𝜆2 respectively.
QUESTION: What is the probability that an arrival from process 1 (a ”type 1” arrival as blue
buses) occurs before an arrival from process 2 (a ”type 2” arrival as red buses)?

GUIDANCE for solving.

Let the two independent inter-arrival times of interest be denoted by 𝐴 and 𝐵 for processes 1
and 2, respectively. We want to compute P[𝐴 < 𝐵]?

MODEL 4.2 (Train depot modeling).

Suppose that by any time 𝑡 the number of people that have arrived at a train depot is a Poisson
random variable with mean 𝜆 𝑡.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.8. CHAPTER CONCLUSION- HOMEWORK 86

ASSUMPTION: Suppose the initial train arrives at the depot at a time (independent of when
the passengers arrive) that is uniformly distributed over (0, 𝑇 ).

1/ What are the mean and variance of the number of passengers to the train?
2/ Is the load of trains (of that train depot) high?

How can we quantify this performance index (in fact a random variable)?

∙ ∙ ∙∙

We need the following fact for the above modeling problems.

Fact 4.1 (Min of exponential random variables).

Let 𝑇1, 𝑇2, · · · , 𝑇𝑛 be independent exponential variables, with 𝑇𝑖 ∼ Exp(𝛽𝑖), for 𝑖 = 1, . . . , 𝑛.

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

4.8. CHAPTER CONCLUSION- HOMEWORK 87

∑︀
Then the min variable 𝑇 = min{𝑇1, 𝑇2, · · · , 𝑇𝑛} ∼ Exp( 𝑖 𝛽𝑖 ) is exponentially distributed
𝑛
∑︁
with parameter 𝛽𝑖.
𝑖=1

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms

Syllabus relooked- WHAT NEXT?

Ch.1: Introduction to system performance evaluation (SPE)

Ch.2,3: Probabilistic models in Queueing Theory
Week 2, 3: Performance metrics (measures)?

Ch. 4 and 5: Stochatic process- Poisson Process- Birth & Death process
Week 4: Math model of 𝑀/𝑀/1 queue- Poisson arrival process- Inter-arrival times
Week 5: Birth & Death process- 𝑀/𝑀/1/𝐶 Finite Capacity System
Ch.6: Basic Queueing systems- Models and Analysis
Week 6: Exponential queues with many servers - 𝑀/𝑀/𝑘
Ch.7: Advanced Queueing systems
Week 7: Open Queueing systems
Week 8: Complex queue systems: A) Jackson networks or B) Non-Markovian Queues - WEEK 9: Midterm
Ch.8: SPE with Statistical Factorial Designs
Week 10: Background: Factorial Designs, full 2𝑘 design combined with regression model
Week 11: Multivariate ANOVA - Experimental error analysis ...
Ch.9: Statistical Simulation Modeling
Week 12: Discrete-event simulation and Monte Carlo simulation - Terminating simulation
Week 13: Comparison of Alternative System Configurations with means, proportions & variances
Ch.10: Analysis of Simulation Results And SPE Analytics
Week 14: Verification and Validation, Variance estimation
Week 15: Variance Reduction Technique (VRT) - Discussion on Project
Week 16: Team Presentation and FINAL REVIEW
List of Figures

4.1 Schematic queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

4.2 Schematic queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.3 Queueing systems with single queue . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.4 Visualization of a stochastic process 𝑋 . . . . . . . . . . . . . . . . . . . . . . . . 49

4.5 State-transition-rate diagram for birth and death process. . . . . . . . . . . . . . . . 66

4.6 State-transition-rate diagram for birth and death process. . . . . . . . . . . . . . . . 69

Index

capacity planning, 4 waiting time, 33

Little’s Law, 13, 33

processes of a queuing system, 4

response time, 34

stable system, 11
statistical equilibrium, 11
system design, 4

transient behavior, 11
transient removal, 11
SYSTEM PERFORMANCE EVALUATION Methods and Algorithms
INDEX 91

THE END

Copyright © Man VM. Nguyen 2023

SYSTEM PERFORMANCE EVALUATION Methods and Algorithms