THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

REVIEW OF QUEUEING THEORY

Notes prepared for EE 6345 by Professor Cyrus D. Cantrell MayJuly 2002

c C. D. Cantrell (05/2002)

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

WHY LEARN QUEUEING ANALYSIS? Networks, and computers running multiuser, multitasking operating systems, can be viewed as interconnected queueing systems Uses of queueing analysis: Analyze and understand system behavior after the fact (i.e., using real data), Project from an existing system to a future system Develop an analytic model for use in designing a system Create a simulation that models a system Queueing theory can be used to analyze the performance of: Computer systems Networks Medical facilities, transportation systems, etc. ...

c C. D. Cantrell (05/2002)

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

THE DOCTORS FALLACY Its reasonable to assume that if the mean service time in a doctors oce is Ts minutes, then the oce can handle an arrival rate of 1 max = Ts patients per minute Unfortunately, accepting max patients per minute leads to a saturated system The utilization = Ts = =1 max The mean waiting time Tw = Patients get very unhappy... Near saturation, the waiting time changes exponentially in response to changes in utilization
c C. D. Cantrell (05/2002)

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

A SINGLE-SERVER QUEUE

Queue (implemented in buffer memory) Arrivals Server

Departures Dispatching discipline (FIFO, ...)

w = items waiting Tw = waiting time

Ts = service time = utilization

q = items resident in queuing system Tq = residence time

Stallings, High-Speed Networks, Fig. 7.2

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

QUEUEING IN CISCO OC-48 LINE CARD

Block Diagram of Cisco's OC-48c/STM-16c POS Line Card

MBus module Input interface Packet receive (Rx) Burst buffer Buffer memory

MBus interface

Silicon queuing engine Forwarding processor L3 switching accelerator Silicon queuing engine Output interface Packet transmit (Tx) Burst buffer Switch fabric interface Switch fabric

Buffer memory

H11725

c Cisco Systems

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

BASIC QUEUEING RELATIONS General: Littles formula: Time: Single server Utilization: Mean number of items: Multiple servers Utilization: Mean number of items: Trac intensity: Ts = N q = w + N u = Ts = N
c C. D. Cantrell (05/2002)

q = Tq w = Tw Tq = Tw + Ts = Ts q =w+

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

MULTIPLE SINGLE-SERVER QUEUES

N N Arrivals = arrival rate

Server 1

Server 2

Departures

Server N

Stallings, High-Speed Networks, Fig. 7.4

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

A MULTIPLE-SERVER QUEUE

N N

Server 1

Arrivals = arrival rate

Queue

Dispatching discipline

Server 2

Departures

N
Server N

Stallings, High-Speed Networks, Fig. 7.4

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

ASSUMPTIONS COMMONLY USED IN QUEUEING MODELS Item population There is an innite supply of items (mean arrival rate is constant) Innite queue size Real queues are nite because they are implemented in buer memory Dispatching discipline (how to decide which item to take care of) Most common is FIFO Arrival-time-interval and service-time statistics G general independent arrivals or service times M (negative) exponential distribution Equivalent to Poisson distribution of arrivals in a xed time T D deterministic arrivals or constant service time

c C. D. Cantrell (05/2002)

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

NOTATION FOR ARRIVAL AND SERVICE STATISTICS The notation for a queueing model is X/Y /N , where X denotes the arrival-time-interval distribution Y denotes the service-time distribution N denotes the number of servers Example: M /M /1 denotes a 1-server queueing model with Poisson arrival statistics and an exponential distribution of service times The distribution of voice trac arrival times is close to exponential The distribution of arrival times in a data network is non-exponential Trac is bursty Actual distributions are heavy-tailed and self-similar (more on this later) Voice trac models are not valid for data networks
c C. D. Cantrell (05/2002)

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

POISSON DISTRIBUTION (1) Probability of n events in time T , assuming a constant arrival rate and independent arrivals: pn = lim
N

N n

T 1 N

N n

T N

eT (T )n = n!

(probability of no events in N n time intervals of length T /N , times probability of one event in each of n intervals) Time interval distribution (probability of zero events in time T ) is exponential: p0 = eT

c C. D. Cantrell (05/2002)

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

POISSON DISTRIBUTION (2) Generating function:

G(s) =
n=0

pn(1 s) = e
n

n=0

(T )n (1 s)n = eT s n!

Factorial moments (easier to calculate than the usual moments): dmG E[n(n 1) (n m + 1)] = (1)m m ds Mean: E[n] = T Variance: 2 = E[n2] (E[n])2 = E[n(n 1)] + E[n] (E[n])2 = T = (T )m
s=0

c C. D. Cantrell (05/2002)

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

FORMULAS FOR M/M/1 QUEUES Mean number of items in system or waiting: 2 w= q= 1 1 Time in queue or waiting time: Ts Ts Tq = Tw = 1 1 Standard deviations: Ts Tq = Ts = Ts q = 1 1 Probability that the length of the queue, Q, is exactly L: Pr(Q = L) = (1 )L Probability that Q is greater than the buer length L ( items dropped):

Pr(Q > L) = (1 )
l=L+1

l = L+1
c C. D. Cantrell (05/2002)

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

MEAN QUEUE SIZE IN A SINGLE-SERVER QUEUE

Ts = Ts Ts = 0.5Ts

q
4 2

Ts = 0
0 0.0 0.2 0.4 0.6 0.8 1.0
Stallings, High-Speed Networks, Fig. 7.4

Utilization ()

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

MEAN QUEUEING TIME IN A SINGLE-SERVER QUEUE

10

T s = T s
6

Tr/TS

Ts = 0.5Ts
4

Ts = 0

0 0.0 0.2 0.4 0.6 0.8 1.0

Utilization ()

Stallings, High-Speed Networks, Fig. 7.4

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

BUFFER SPILL IN AN M/M/1 QUEUE

1.0 0.5 1.00 0.75 75 0.50 0.25 25 50 100

Pr(Q>L)

c C. D. Cantrell (05/2002)

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

FORMULAS FOR M/D/1 QUEUES Mean number of items in system or waiting: 2 q= + 2(1 ) Mean time in queue or waiting time: Ts(2 ) Tq = 2(1 ) Standard deviations: 1 q = 1 32 53 4 + 2 6 12 Ts Tq = 1 2 3 12 2 w= 2(1 ) Ts Tw = 2(1 )

c C. D. Cantrell (07/2002)

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

FORMULAS FOR M/M/N QUEUES Mean number of items in system or waiting: + N w = C(N, u) q = C(N, u) 1 1 C(N, u) is the Erlang C function (see next slide); u = N Time in queue or waiting time: C(N, u) Ts + Ts Tq = N 1 Standard deviations: Tq = Ts C(2 C) + N 2(1 )2 N (1 ) 1 w = C(1 + C) 1 C(N, u) Ts Tw = N 1

c C. D. Cantrell (05/2002)

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

ERLANG C FUNCTION C(N, u) is the probability that all N servers are busy in a queueing system with trac intensity u = Ts Formula: 1 K(u) where 1 K(u) and where K is the Poisson ratio function, C(N, u) =
N 1 l=0 N

u N

ul l! ul l!

K(u) =

l=0

Single-server queue: C(1, u) =

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

PLOT OF ERLANG C FUNCTION

1.00 0.75 25 0.50 0.25 10 5 20 15

1.0 0.8 0.6 0.4 0.2 30

c C. D. Cantrell (05/2002)

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

EXAMPLE OF A NETWORK OF QUEUES

Stallings, High-Speed Networks, Fig. 7.7

THE UNIVERSITY OF TEXAS AT DALLAS

Erik Jonsson School of Engineering and Computer Science

QUEUEING MODELS AND SIMULATIONS Professor Ann-Marie Martensson-Pendrill gives a Matlab program to compute the Erlang C function, along with many examples of values of C, time in queue, etc. Simulations are useful if one wants to escape from unrealistic assumptions (such as innite buer length) or use non-exponential arrival or service time distributions A standard tool for academic research is ns, which was developed at the Lawrence Berkeley Laboratory, based on S. Keshavs REAL Network Simulator Professor Ivor Pages Java applet simulates an M /M /N queue, with adjustable parameters

c C. D. Cantrell (05/2002)