Queueing Theory

The waiting or queuing phenomenon is the direct result of randomness in the

operation of service facilities.
Our objective in studying the operation of a service facility under random conditions
is to secure some characteristics that measure the performance of the system under
study. For example, a logical measure of performance is how long a customer is
expected to wait before being serviced. Another is the percentage of time the facility
is idle. The first measure looks at the system from the customer's standpoint, whereas
the second measure evaluates the degree of utilization of the facility. We can see that
the longer the customers waiting time, the smaller is the percentage of time the
facility would remain idle and vice versa. These measures of performance may
therefore be used to select the level of balance of service (or service rate) that will
strike a reasonable balance between the two conflicting situations.
We will discuss a number of queuing models that account for a variety of service
operations. The models we will develop will basically be applications of probability
theory and stochastic processes. The ultimate objective of solving these models is to
determine the characteristics that measure the performance of the system. We will
then go on to see how this information can be used in seeking an optimal design for
the service facility.
Basic elements of the queuing system
The basic elements of a queuing model depend on the following factors:
1

Arrivals and Service time distributions (Single or Bulk)

In queuing models, customer arrival and service times are summarised in terms of
probability distributions normally referred to as arrivals and service time distributions.
These distributions may represent situations where customers arrive and are served
singly (e.g. Banks or Supermarkets). In other situations, customers may arrive and /or
be served in groups (e.g. restaurants). The latter case is normally referred to as bulk
queues.

Service discipline (FCFS/FIFO, LCFS/LIFO, SIRO) and service priority.

This is the manner in choosing customers for service from the queue. The most
common and probably the fairest is the FCFS/FIFO (First Come First Served/First In
First Out), other disciplines used are LCFS/LIFO (La st Come First Served/Last In
First Out) and SIRO (Service In Random Order). Whereas service discipline
regulates the selection of customers from a queue, it is also possible that customers
arriving at a facility may be put in priority queues such that those with a higher
priority will receive preference to start service first. The selection of customers from
each priority queue may follow any service discipline.

Design of service facility (series, parallel or network stations)

The facility may comprise a number of stations in series through which the customer
passes before service is completed (e.g. processing of a product on a sequence of
machines). In this case, queuing may or may not be allowed between stations. The
resulting situations are norma lly known as queues in series or tandem queues. The
facility may include more than one server, thus allowing as many customers as the
number of servers to be served simultaneously (e.g. Bank Tellers). In this case, all
servers offer the same service and the facility is said to have parallel servers. The
most general design of a service facility includes both series and parallel processing
station. This results in what we call network queues.
4

Queue size (Finite of Infinite)

In some situations, only a limited number of customers may be allowed, possibly

because of space limitations (e.g. car spaces allowed in a drive- in bank). Once the
queue fills to capacity, customers are denied service and may not join the queue.
5

Calling source (Finite or Infinite)

This deals with the source from which customers are generated (arrivals of
customers). A calling source is finite when an arrival affects the rate of arrival of new
customers. In a machine shop with a total of M machines, the calling source before
any machine breaks down consists of M potential customers. Once a machine is
broken, it becomes a customer and hence incapable of generating new calls until it is
repaired. A distinction must be made between the machine shop situation and others
where the cause for generating calls is limited, yet capable of generating an infinity
of arrivals. For example, in a typing pool, the number of users is finite, yet each user
could generate a limitless number of arrivals, since a user generally need not wait for
the completion of previously submitted material before generating new ones.
6

Human behaviour (jockeying, balking and reneging)

A human server may speed up the rate of service when the queue builds up in size.
A human customer may go from one queue to another in hopes of reducing their
queuing time. Some human customers may also balk from joining a queue
altogether because they anticipate a long delay, or they may renege after being in the
queue for a while because their wait has been too long as they perceive it.
Role of the Poisson and Exponential Distributions
Consider the queuing situation in which the number of arrivals and departures during
an interval of time is controlled by the following conditions:
1

The probability of an event (arrival or departure) occurring between times t

and t+h depends only on the length of h, meaning that the probability does not
depend on either the number of events that occur up to time t or the specific
value of t. (Technically, we say that the probability function has stationary
independent increments).

The probability of an event occurring during a very small time interval h is

positive but less than 1.

At most one event can occur during a very small time interval h.

Let Pn(t)

Probability of n events occurring during time t

From condition 1 with n = 0 we have

P0 (t+h) = P0 (t)P0 (h) (means AND in probability theory)

By condition 2 we have 0 < P0 (h) < 1 for very small h

It can be shown that the solution to the equation is

P0 (t) = e-t where t 0 and is a positive constant which represents the rate of
arrivals or departures per unit time when the events represent arrivals (departures).
For h > 0 and sufficiently small, we have
P0 (h) = e-h which if expanded gives the following
P0 (h) 1 - h
Since condition 3 allows the occurrence of at most one event, it follows that
P1 (h) = 1 - P0 (h) h
This result means that the probability of an event occurring during a small interval h
is directly proportional to h.
To use these results for arrivals we replace the rate of event generation, by the
standard notation representing the arrival rate. thus, as we indicated previously we
have:
P0 (h) = e-h and P1 (h) = 1 - P0 (h) h since h is small.
If T is the time interval since the occurrence of the last arrival we can find that
f(T) = e-t
T > 0;
which is an exponential distribution.

We can now come to the following conclusions:

1

For the process described by the probabilities Pn(t), the time between the
occurrence of successive events follows an exponential distribution.

The expected value of the exponential distribution

E (T ) =

1
time units

represents the average time interval between successive occurrences of arrivals.

Thus
1
E(T)

occurrences/unit time

must represent the rate (per unit time) at which arrivals are generated.
3

The exponential distribution has the unique property that the time until the
next arrival occurs is independent of the time that elapsed since the occurrence
of the last arrival. This property is usually referred to as forgetfulness or lack
of memory of the exponential distribution.

Arrivals Process:
Define f(t)t = Probability that 1st arrival happens in (t, t+t)
i.e. no arrivals from 0 to t then 1 arrival from t to t+t
Assume the probability of arriving in a time t is t where is a constant arrival
rate i.e. depends on length of time t only and not on t.
This assumption may not always be true e.g. not usually true for a system that ages
e.g. probability of break down increases as age increases.
Define g(t) = probability of no arrivals in (0,t). We will find 2 equations for f and g
and solve them.
1 - g(t) = probability that 1st arrival is in (0,t)
f(t) t = probability of 1st arrival is in (t, t+t)
= [probability 0 arrivals (0,t)]*[probability of an arrival in (t,t+t)]
f(t) t = g(t)*( t) from basic assumption
So f(t) = g(t)

Probability that the 1st arrival is in (0, t+t) = 1 - g(t+t)

= Probability[1st arrival in (0,t) OR 1st arrival in (t, t+t)

= Probability[1st arrival in (0,t)] + probability[1st arrival in (t, t+t)]

=[1 - g(t)] + f(t) t
Therefore 1 - g(t+t) = 1 - g(t) + f(t) t
giving g(t+t) - g(t) = -f(t) t
Therefore

f(t)

- g(t+t) - g(t)
t

Let t -> 0 so f(t) = - dg

dt

Combining 1 and 2 we have g(t)

- dg
dt

Solving this equation using differential calculus we get

g(t)

e-t

f(t)

g(t)

e-t

g(t) = probability of no arrivals in (0,t)

g(t)

e-t

g(0)

e-0

e-0

1
e-0

1
1

Probability [0 arrivals in (0,0)] = 1

As t gets bigger g(t) gets smaller this is explained because the longer you wait the
smaller the possibility that no one has arrived.
f(t) t = probability of 1st arrival in (t, t+t)
f(t)

e-t

f(0)

e-0

Probability of 1st arrival in (0, t) = f(0) t = t

The bigger t is the greater the chance of a 2nd, 3rd etc., arrival as the 1st arrival has
been made.
1

System described by arrival/service distributions

Assume fixed arrival rate - doesnt depend on time

Can get likelihood of arrivals in particular lengths of time

We now wish to derive an expression for the probability Pn(t) of n arrivals during
time interval t.

Pn(t+h) = P(n arrivals during t and none during h) OR (n-1 arrivals during t and one
during h)
assuming h small and only one arrival possible
Thus
Pn(t+h) = Pn(t)P0 (h) + Pn-1(t)P1 (h) n=1,2,.....
P0 (t+h) = P0 (t)P0 (h)

n=0

We use the results developed earlier for P0 (h) = e-h and P1 (h) = 1 - P0 (h) h since h
is small to show that
Pn(t)

(t)n e-t
n!

n = 0,1,2,.........

This distribution is Poisson with mean and variance equal to t. Therefore the
interarrival time is exponential with mean 1/ and the number of arrivals during time
interval t is Poisson with mean t.
Arrivals rate number/unit time
1

Probability of 1st arrival in (t,t+t) = f(t) t = e-t t

Probability of no arrivals in (0,t) = g(t) = e- t

Service Processes:
Assume services happen at rate , which we also assume to be constant.
Service and departure rate are not necessarily the same.
is the rate at which the server can serve at, max. rate.
Departure rate could be smaller if fewer arrivals.
We can find, using the same arguments as before, that the probability of 1st service
completing in (t,t+t) = e-t t.
We can now find out how to get the numbers in the system for an M/M/1 queue. The
first M refers to the arrival distribution and is memoryless or Markov, the second M
refers to the service distribution and is the same as the arrivals and the 1 refers to the

number of servers. There is no limit on the capacity of the queue or the calling
source.
Number in server = 0 or 1
Number in queue = 0 to infinity
Number in system = number in service + number in queue.
Let Pn = Probability of n in system
Average number in system = 0P0 + 1P1 + 2P2 etc.,
We wish to find the values of the Ps. We will find two equations and solve.
Pn(t) = Probability of n in system at time t
Assuming only one thing can happen in time t which is a very small amount of time
i.e. one arrival or 1 departure.
Pn(t+t) = (i) Pn-1(t)*P(1 arrival in t) OR (ii) Pn+1(t)*P(1 departure in t) OR (iii)
Pn(t)*P(no arrivals and no departures) OR 0(t)2 (order of probabilities of 2,3,4 etc.
arrivals/departures)
Pn(t+t) = Pn-1(t)[ t] + Pn+1(t)[ t] + Pn(t)[{1- t}{1 - t}]
where
[1- t] = P(no arrivals) and (1 - t) = P(no departures)
Pn(t+t) = t[Pn-1(t) + Pn+1(t) - ( + )Pn(t)] + Pn(t) + (t)2 Pn(t)
Pn(t+t) - Pn(t) = Pn-1(t) + Pn+1(t) - ( + )Pn(t) + tPn(t)
t
Let t -> 0
dPn(t) =
dt

Pn-1(t) + Pn+1(t) - ( + )Pn(t)

This is the rate at which Pn is changing. After a long time the system is in a steady
state and the LHS = 0
0 = Pn-1(t) + Pn+1(t) - ( + )Pn(t)

This holds if n > 0 if n = 0 we can ignore the situation with n-1 as we cant have -1 in
the system.
We therefore have the situations with one in the system and one departing AND one
arrival and one departure. But if the time interval is small we can only have one event
happening in the time interval therefore if we have an arrival we cannot also have a
departure so therefore we can ignore the departure.
P(arrival) + P(no arrival) = 1 -> P(no arrival) = 1 - t
P(departure) + P(no departure) = 1 -> P(no departure) = 1 - 0

This is true because if there is no arrival we can not have a departure as we cannot
have -1 in the system. Also if there is one arrival we can assume that the probability
of a departure is also zero as we can only have one event happening per unit time.
This gives:
P0 (t + t) = P1 (t)(t) + P0 (t)[1 - t][1 - 0]
Solving and letting t -> 0 we get 0 = P1 (t) - P0 (t)
Combining equations 1 and 2 gives Pn(t) = ()n Pn(t)
()

As probabilities always sum to one we can show that P0 = 1 -

= 1 - Arrival rate
Service rate
= 1 where =

From this we get

Arrival
rates
Service

P =
n

(1 )

Also the average number in the system = 0P0 + 1P1 + 2P2 + ............
Which gives average number in system as =

Average number in server can be found to be equal to

Average number in queue can also be found to be equal to

Note that all of the above Operating Characteristics of the system depend on ? which
is the ratio between arrival and service rates, rather than on the arrival and service
rates themselves. When arrival rate is equal to the service rate we get infinite queues.
We cant get 100% utilisation a good level is about 70%. The more loaded the system
the more unstable it becomes.
Littles Law:
Average number in Queue = (Average time in Queue)
Average number in System = (Average time in System)
Average number in Server = (Average time in Server)

This is used to calculate the times for the various elements of the System and is
generally true except in rare cases.
A more general model of a queue is M/G/1 where G is a general service rate e.g.
Constant say 5 minutes exactly
Uniform service anything between 1 - 10 minutes is equally likely
Markov as before
The Operating Characteristics of the M/G/1 model are:
Average number in Server =

Average number in system =

(V ) +

2(1 )

where V is the variance of service time and therefore the larger V is the more erratic
the service is. In principle is the server can be kept working as fast as possible on
average and make him less erratic then a better throughput will be had. The minimum
value of V is 0 where all the service times are constant. This will lead to the smallest
queues. In this case the average number in the system will be:

Average number in system =

(0) +

2(1 )

(2 )
2(1 )

M/M/1 with Finite Queue Capacity

Maximum capacity of the system is N, i.e. N - 1 in queue and 1 in server.
We can derive the following equations:
Pn =

? n (1 - ?)
1 - ? N+1

Pn =

1
N+1

=1

1 ( N + 1) + N

Average number in system =

N +1
(1 ) 1

=
Average number in server

N
2
=

eff

N +1

=1

eff

Rate at which people join the system is given by:

eff = (1 - PN)
The average rate of balking (people turning away) = PN
Average Number = eff(Average Time)

Littles Law becomes:

To use Littles Law in this model we look at the rate people enter the system and not
the rate they arrive at the system.
M/M/S
This system has the same arrival and service distributions as before but has S (several)
servers. It can be shown that the probability of no one in the system can be found
from the following formula:

(1)

s 1

s! (1 ) +
s
n =0 n!

Average number in system = +

( s 1)! ( s ) 2

Average number in Server =

Average number in queue is found by subtraction. Times can be obtained from Littles
Law.
Finite source of arrivals (Machine Repair Man)
Example N VDUs and 1 CPU. When a job is submitted the terminal locks until the
job comes back to the terminal.
Maximum number of arrivals = Number of terminals = Machine Repair Man problem.
Formulas are as follows:
1
P0

=
i= 0

i
N!

( N i )!

10

 N!

( N n)!

Pn = N N ! i

i =0 ( N i )!

As N increases the model approaches and M/M/1 queue.

Average number in the system:

N

iN !

P ( N 1)!
0

i =1

For the Machine Repair Man Littles Law needs to be modified because is the
average rate that a terminal would like to submit jobs. But when a terminal is locked
it cannot do so. So the rate at which the jobs are actually submitted is defined as eff
which is usually less than . this gives us for Littles Law the following:
Average number in system = eff(Average time in system)
Where eff is the effective arrival rate into the system.
eff = (N - Average number in system)
(N - Average number in system) = number of VDUs that can still submit a job.
( - eff) = rate of turning arrivals away
N = maximum rate
Probability of not joining queue = Probability of N in system = PN
M/G/s
This is a system in which no waiting is allowed. There are s service facilities
available. The arrival distribution is a Poisson distribution as for the other Queueing
models. There is a general service distribution, which may follow any probability
distribution. The service rate is the same for all servers. Basically if a server is free
then a customer may enter the system but not otherwise. Telephone calls would be an
example, if the phone is engaged you don't wait you must dial again.
To find the best number of servers we estimate the probability that the servers are

Pj =

( / ) j / j!
s

( / )
11

/ i!

busy from the following formula.

Where = the mean arrival rate
= the mean service rate
s = the number of servers in the system
Pj = the probability that exactly s of the servers are busy
Another measure to keep monitored is the average number of servers busy which is
the same as the average number of units in the system.

L=

(1 Ps )

12