Springer Series in Operations Research

Editors:
Peter Glynn Stephen M. Robinson

Springer Science+Business Media, LLC

Springer Series in Operations Research

Altiok: Performance Analysis of Manufacturing Systems

Birge and Louveaux: Introduction to Stochastic Programming
Bonnans and Shapiro: Perturbation Analysis of Optimization Problems
Bramel and Simchi-Levi: The Logic of Logistics: Theory, Algorithms,
and Applications for Logistics Management
Dantzig and Thapa: Linear Programming I: Introduction
Drezner (Editor): Facility Location: A Survey of Applications and
Methods
Fishman: Discrete-Event Simulation: Modeling, Programming, and Analysis
Fishman: Monte Carlo: Concepts, Algorithms, and Applications
Nocedal and Wright: Numerical Optimization
Olson: Decision Aids for Selection Problems
Yao (Editor): Stochastic Modeling and Analysis of Manufacturing
Systems
George S. Fishman

Discrete-Event Simulation
Modeling, Programming, and Analysis

With 83 Illustrations

, Springer
George S. Fishman
Department of Operations Research
University of North Carolina at Chapel Hill
CB#3180
210 Smith Building
Chapel Hill, NC 27599-3180
USA
[email protected]

Series Editors:
Peter Glynn Stephen M. Robinson
Department of Operations Research Department of Industrial Engineering
Stanford University University of Wisconsin-Madison
Stanford, CA 94305 1513 University Avenue
USA Madison, WI 53706-1572
USA

Library of Congress Cataloging-in-Publication Data

Fishman, George S.
Discrete-event simulation: modeling, programming, and analysis / George S. Fishman
p. cm. - (Springer series in operations research)
Includes bibliographical references and index.
ISBN 978-1-4419-2892-4 ISBN 978-1-4757-3552-9 (eBook)
DOI 10.1007/978-1-4757-3552-9
1. Simulation methods. 2. Discrete-time systems. 1. Title. II. Series.
T57.62.F58 2001
658.4'0352-dc21 00-052608
Printed on acid-free paper.
© 2001 Springer Science+Business Media New York
Originally published by Springer-Verlag New York, Inc. in 200 I
Softcover reprint of the hardcover I st edition 200 I
AII rights reserved. This work may not be translated or copied in whole or in part without the written permis sion
of the publisher Springer Science+Business Media, LLC, except for brief
excerpts in connection with reviews ar scholarly analysis. Use in connection with any form of information storage
and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or
hereafter developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are
not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and
Merchandise Marks Act, may accordingly be used freely by anyone.

Production managed by Lesley Poliner; manufacturing supervised by Erica Bresler.

Typeset by The Bartlett Press, Inc., Marietta, GA.

9 8 7 6 5 4 3 2 1

ISBN 978-1-4419-2892-4 SPIN 10785084

 To Paula Sue,

my wife and best friend

Preface

As tools for scientific inquiry, the methodologies of discrete-event simulation have been with
us for almost 50 years. They enable us to study discrete-event dynamical systems in which de-
lay is an intrinsic feature. Examples easily come to mind in manufacturing, communications
networks, transportation, health-care delivery, and public-sector services.
The methodologies embrace modeling, programming, input-data analysis, pseudo-
random variate generation, output-data analysis, and presentation of results. These 50 years
have seen these topics evolve in content, sophistication, and, most compellingly, accessibility.
In this last regard, the mushrooming PC environment of the past 20+ years has facilitated a
relatively rapid shift of many simulation techniques from concept to practical use. Witness
the many discrete-event simulation software systems available to the would-be simulationist.
Time was when a distinct demarcation existed between modeling and programming.
One laid out the dynamic logic of a system, converted it to executable computer code, and
executed the resulting simulation program. Today, these distinctions are less apparent. Tak-
ing advantage of a highly interactive PC environment available to many, vendors market
simulation software that offers macrostatements or modules that correspond to frequently
encountered modeling constructs, These macrostatements are collections of more elemen-
tary statements in an underlying computer language. Some apply directly to delay modeling;
viii PREFACE

others apply to sampling on a computer; still others streamline data collection and report
preparation. Using these macrostatements effectively merges modeling and programming
activities into a single effort.
In many respects, these developments are welcome. They eliminate much of the
drudgery and delay in getting a simulation program to execute. However, they have draw-
backs. The macro statements cover only a subset of all the dynamic logic that may be needed
to model a system. When no macro statement represents the logic of a particular problem, a
simulationist either has to resort to programming in the underlying language to create a cor-
rect model, or has to create a kluge from existing macro statements, sometimes awkwardly,
to create an equivalence. Most regrettably, a simulationist sometimes merely changes the
model to accommodate the available macrostatements.
Virtually all simulation software systems allow a simulationist to select sampling
distributions from libraries or menus. When the desired distribution is not offered, the
simulationist's only recourse may be to a general-purpose programming language like C.
Alternatively, he/she may choose to use whatever the simulation software offers and ignore
the distribution(s) that the problem setting dictates. While offering options for customizing
reports, automatic assessment of the statistical accuracy of a simulation's output is available
in few simulation software systems.
Since students often learn about discrete-event simulation through a particular soft-
ware's modeling, sampling, analysis, and reporting capabilities, their skills for overcoming
the impediments mentioned above may be severely limited unless they are well versed in
the basic concepts of discrete-event simulation. This book supplies a means for providing
this background. It gives a relatively detailed account of the principal methodologies of
discrete-event simulation. Modeling (Chapter 2), programming (Chapter 4), output analy-
sis (Chapter 6), making sense of the output (Chapter 7), pseudorandom variate generation
(Chapters 8 and 9), and input-data analysis (Chapter 10) are all treated in depth sufficient
to acquaint a would-be simulationist with the central issues.
The book also offers an excursion into the underlying models of data collection (Chap-
ter 3), explaining why using these models leads to sample averages that approximate the values
of unknown system parameters with errors that diminish as the simulation runs for longer
and longer amounts of computing time. It also characterizes these errors when a simulation is
run, more realistically, for a finite computing time and, therefore, a finite sample-path length.
By sensitizing the reader to the presence of error prior to running a simulation program, we
hope to give her/him a better understanding of experimental design considerations, which
are so important for carrying out simulation experiments in a knowledgeable way, and on
which Chapters 4 and 6 touch.
Chapter 5 offers another excursion, this time into the topic of computing time versus
size of problem. To obtain a specified level of statistical accuracy in output sample averages,
discrete-event simulation requires computing time that tends to grow with the size of the
system under study, sometimes superlinearly. If a simulationist is to avoid an excessive
amount of computing, he/she needs to be acquainted with the principal determinants of
this growth. Chapter 5 describes how this growth occurs for several delay models.
 PREFACE ix

The book is suitable for advanced undergraduates and graduate students in the man-
agement, mathematical, and physical sciences and in the engineering sciences. A background
in calculus, probability theory, and intermediate statistics is necessary. Familiarity with
stochastic processes considerably eases the study of Chapter 6. Knowledge of a program-
ming language significantly reduces the setup cost in mastering Chapter 4. The book also
serves as a ready reference for professionals who want to acquaint themselves with some
of the deeper issues of discrete-event simulation, such as estimating the accuracy of results
(Chapter 6) and the comparison of results for different scenarios (Chapter 7).
My students at the University of North Carolina at Chapel Hill have served as the test
bed for most of the material in this book. The course assigns Chapter 1 for background
reading on delay systems and as an introduction to discrete-event simulation. Detailed lec-
tures on Chapter 2, followed by selective lectures on sections in Chapter 3, and by rather
complete lectures on Chapter 4, work well to prepare students to do the exercises at the ends
of Chapters 2 and 4. In particular, Chapter 4 uses SIMSCRIPT 11.5 and Arena to illustrate
programming concepts. SIMSCRIPT 11.5 is a full-service programming language (like C)
with a comprehensive collection of macrostatements for discrete-event simulation. Arena
is a highly integrated simulation software system that in addition to providing macrostate-
ments in the SIMAN simulation language allows a user to engage menu-driven software that
performs many ancillary tasks that accompany simulation experimentation. These include
an input-data analyzer, an output-data analyzer, customized report generation, and output
graphical display. Instructors who choose simulation software other than SIMSCRIPT 11.5
or Arena can easily substitute lectures (for those on Chapter 4) on their preferred software.
However, students find intriguing a lecture on the variance-reducing technique known as
common pseudorandom numbers (Section 4.22).
Selective lecturing on Chapter 5 is an eye opener for most students. I customarily focus
on Sections 5.1 through 5.4.5. Lecturing on Chapter 6, especially on Sections 6.1 through 6.8,
with the aid of the LABATCH.2 software, available on the Internet, significantly heightens
the student's ability to assess the accuracy of results. Selective lecturing based on Sections
7.1 through 7.6.1 and, if time permits, on Sections 7.7 through 7.11, creates an awareness
among students that there is much more to output analysis and presentation that looking at
a general-purpose automatically generated report.
Lectures on Sections 8.1 through 8.6 provide the basics of random variate generation.
Examples taken from Sections 8.9 through 8.23 illustrate the concepts. For instance, Section
8.13 offers a good example ofa sophisticated algorithm that has bounded expected computing
time for Gamma sampling. I also lecture on Chapter 9, which addresses pseudorandom
number generation. I like to include Section 9.7, which describes a relatively new type of
pseudorandom number generator that ameliorates many of the limitations of the commonly
employed linear congruential generators. Although this and like generators are not now in
common use, they are a preview of things to come. Chapter 10 describes techniques for input-
data analysis. These can be covered selectively, depending on the types of data available for
classroom analysis.
X PREFACE

Four problems in the modeling exercises at the end of Chapter 2 are carried forward
through programming in Chapter 4, output analysis in Chapter 6, and comparison of results
in Chapter 7. Some problems demand more modeling and programming effort than others,
which require more effort in estimation and interpretation of results. To give students a
sense of satisfaction in solving a simulation problem from beginning to end, I customarily
choose one of the problems in Chapter 2 and assign its corresponding exercises there and
in Chapters 4, 6, and 7. Some exercises in Chapter 8 allow students to gain perspective on
the wide applicability of the methods of Sections 8.1 through 8.6. Others offer hands-on
experience in programming and executing algorithms. Those at the end of Chapter 9 offer
students an opportunity to hone their skills in analyzing pseudorandom number generators.
When I began writing the book in 1990, little did I know that such massive changes
would take place in simulation software during the following decade. I describe many of
these developments while retaining much of the well-established methodologies that have
so well served simulationists in the past. However, space considerations have inevitably led to
a selectivity that reflects my best judgment as to what a simulationist should know. The book
limits its discussion of variance-reducing techniques to those that I have found most useful in
the context of discrete-event simulation. While it addresses various aspects of experimental
design, more on this topic would be beneficial. Although important, the topics of verification
and validation have not made the cut, the rationale being that these are considerations that
apply to virtually every aspect of scientific modeling, not just discrete-event simulation.
More specialized topics, including regenerative simulation and infinitesimal perturbation
analysis, are also omitted. While of considerable conceptual interest, neither of these two
topics can claim to be an intrinsic part of discrete-event simulation as practiced today.
I am grateful to Christos Alexopoulos of Georgia Tech and Jim Henriksen of Wolverine
Software for their helpful comments on parts of the manuscript, and to Russell Cheng, Daryl
Daley, Debasis Mitra, and Les Servi for their direct or indirect help with several of the exer-
cises. Thanks also go to Pierre L'Ecuyer for his assistance in obtaining and operationalizing
software for the combined pseudorandom number generator described in Section 9.7. My
colleagues Vidyadhar Kulkarni, Scott Provan, and Sandy Stidham have all been generous
with their time and counsel, often steering me onto a much more constructive path than
I might have otherwise taken. I am most grateful to them. Thanks go to all my students
of the past decade who have subtly and, occasionally, overtly offered their comments as to
what works and what does not. Kathleen Dulaney, Betty Richmond, and Barbara Meadows
typed sections of this manuscript over the past decade. For their patience with my repeated
modifications and corrections I am again most grateful.
Errata for this book are available at http://www.or. unc.edu/ - gfish .

George S. Fishman
Chapel Hill, North Carolina
August 2000
Contents

Preface vii

1 Simulation in Perspective 1
1.1 Discrete-Event Systems . . . . . 5
1.2 Open- and Closed-Loop Systems 8
1.3 Time Averages . . . . . . . . . . 10
1.3.1 Exceedance Probabilities l3
1.4 Example: Reducing Waiting Time for Photocopying 14
1.5 Example: Increasing Throughput at an Incinerator 19
1.6 Why Discrete-Event Simulation 23
1.6.1 Detail........... 24
1.6.2 Modeling Dangers . . . . . 26
1.7 Technical Attractions of Simulation 26
1.8 Medium for Modeling . . . . . . . 27
1.9 Discrete-Event Simulation Programming Languages 29
1.9.1 Special Features . . . . . . . . 31
1.9.2 Object-Oriented Programming . . . . . . . 31
xii CONTENTS

1.9.3 Web-Based Discrete-Event Simulation 33

1.9.4 No Programming . . . . . . . . . . 34
1.9.5 Teaching Discrete-Event Simulation 34
1.10 References . . . . . . . . . . . . . . . . . . 35

2 Modeling Concepts 36
2.1 Next-Event Time Advance . . . . . . 39
2.2 Arithmetic and Logical Relationships 40
2.3 Event-Scheduling Approach . 41
2.3.1 Priority Order . . . . . . . . 52
2.4 Process-Interaction Approach . . . . 52
2.5 Active Processes and Passive Resources 57
2.6 Telecommunications Network Modeling 58
2.7 Lessons Learned 62
2.8 Exercises. 64
2.9 References . . . 69

3 Data Collection and Averages 70

3.1 Performance Statistics . 72
3.1.1 Queue Length . 74
3.1.2 Resource Utilization. 76
3.1.3 Waiting Time 77
3.2 Statistical Accuracy . . . . . 78
3.2.1 System State . . . . . 79
3.3 Conditional and Equilibrium Distributions . 81
3.3.1 Initial Conditions . . . . . . . . . . 82
3.3.2 Warm-Up Interval and Convergence 84
3.3.3 Queue Length, Resource Utilization, and Convergence . 85
3.3.4 Finite Sample-Path Length 88
3.4 Terminating Execution. 90
OptionA .. . 90
Option B . . . . . . 91
3.5 Waiting Time .. . 92
3.5.1 Little's Law. 93
3.6 Summary...... 94
3.7 Periodic Data Collection. 94
3.8 Lessons Learned 96
3.9 References . . . . . . . 97

4 Programming and Execution 98

4.1 Modalities for Programming 100
4.2 SIMSCRIPT II.5 . . . . . . . 103
 CONTENTS xiii

4.3 Entities....... III

4.4 Event Notices . . . 111
4.5 Summary Statistics. 112
4.6 Traffic Intensity . . 115
4.7 Pseudorandom Number Generation 116
4.8 Warm-Up and Path-Length Parameters 118
4.9 Event Management . . . . . . . 119
4.10 Starting and Stopping Execution 119
4.11 CALL Event . . . . . . 120
4.12 END~ESSAGE Event. 121
4.13 Filing Priorities . . . . 121
4.14 COMPLETION Event. 122
4.15 Getting an Executable Program 125
4.16 A First Run . . . . . . . . . . 127
4.17 Collecting Data . . . . . . . . 131
4.18 Process-Interaction Approach. 131
4.19 Processes and Resources in SIMSCRIPT II.5 132
4.20 Using Processes Exclusively . . . . 133
4.21 Using Processes and Resources . . . . . . 137
4.22 Common Pseudorandom Numbers . . . . 142
4.22.1 Other Variance-Reducing Methods 148
4.23 Point -Click-Drag-Drop-Connect Approach . 149
4.24 MIMll Arena Example .. 151
4.24.1 SIMAN Statements 154
Blocks . . . . . . . . . . . 154
Elements . . . . . . . . . . 155
4.25 Arena/SIMAN Airline Reservation Example. 156
4.25.1 Modeling with the BAAT Panels 162
4.26 Events in SIMAN and SLAM . . . 162
4.27 Macromodules........... 163
What is the sequencing discipline? . 163
What can the bridge crane do? . 164
Module Opaqueness 165
4.28 Lessons Learned 165
4.29 Exercises 166
4.30 References . . . 173

5 Search, Space, and Time 176

5.1 Memory Space Needs and Computing Time: A Simplified Perspective 178
5.1.1 FES Management Time per Job 183
5.1.2 Reducing the Size of the FES 183
Protocol A . . . . . . . . . . . . . . . 184
xiv CONTENTS

ProtocolB .. . 184
Protocol C .. . 185
5.2 List Processing . 186
5.2.1 Insertion and Deletion 189
5.3 Dynamic Space Management . 190
Algorithm SPACE_FINDER . . 191
Algorithm SPACE-RELEASER . 191
Algorithm SPACE1'INDER* . 194
Algorithm SPACE_RELEASER* 195
5.3.1 Aggregating Record Classes 195
5.3.2 Resizing . . . . . . . . 196
Algorithm SPACE_FINDER** . 197
5.4 Future-Event Set . . . . . . . . 197
5.4.1 Single Linear Event List 199
Efficiency. . . . . . . . . . . . . 201
5.4.2 Multiple Linear Event Lists 201
Scheduling . . . . . . . . 201
Selecting the Next Event . 201
5.4.3 Heaps...... 203
Algorithm SIFTUP(x) . . 204
Algorithm SIFTDOWN(x) 204
Space . . . . . . . . . . . . 205
5.4.4 Henriksen's Algorithm 207
Algorithm H -L . . . . . . . . . . 208
5.4.5 Best FES Manager . . . . 209
5.5 Broadening the Class of Delay Models. 211
Capacitated Travel . . . . . . . . 211
Uncapacitated Travel. . . . . . . . . . 212
5.5.1 Variable Transit Times. . . . . 212
5.5.2 Varying Number of Tasks per Job. 214
5.6 Lessons Learned 216
5.7 Exercises. 216
5.8 References 217

6 Output Analysis 218

6.1 Warm-Up Interval. . . . . . . . . . . . 220
6.1.1 Multiple Initial States . . . . . . 227
6.1.2 How Many Warm-Up Analyses? 228
6.1.3 Steps for Estimating the Length of the Warm-up Interval 228
6.2 Assessing the Accuracy of the Sample Average. 232
6.3 Formalisms........ 233
6.4 Independent Replications . . . . . . . . . . . 235
 CONTENTS XV

Recursive Computation . . 238

6.4.1 Allowing t to Grow 238
Brownian Motion . . . . . 238
Assumption of Weak Approximation 239
6.5 Strictly Stationary Stochastic Processes 241
6.6 Batch-Means Method . . . . . . . 246
6.6.1 LABATCH.2 Tableaus . . . 248
6.6.2 Systematic Variance Error . 248
6.6.3 Interim Review . . . . . . 253
6.6.4 FNB Rule . . . . . . . . . 253
6.6.5 Arena's Automatic Use of the Batch-Means Method 254
6.6.6 SQRT Rule. . . . . . . . . . . 256
6.6.7 LBATCH andABATCH Rules. 256
LBATCH Rule . . 257
ABATCH Rule . . . . . . . . . . . . . 259
6.6.8 Testing Ho . . . . . . . . . . . 260
6.6.9 Assessing the Adequacy of the Warm-Up Interval 264
6.7 LABATCH.2............. 265
6.7.1 Two Modalities . . . . . . . 267
6.7.2 Interacting with the Analysis 268
6.7.3 Implementation . . . . 270
6.7.4 User-Specified II and hI 270
6.7.5 Coding . . . . . . . . 271
6.8 Airline Reservation Problem . 271
6.9 Batch Means in Arena 273
6.10 Prebatching . . . 278
6.11 Ratio Estimation . 278
6.12 Lessons Learned 283
6.13 Exercises 284
6.14 References . . . 292

7 Making Sense of Output and Increasing Efficiency 294

7.1 Simultaneous Confidence Statements 297
7.2 Comparing Two Means . . . . . 300
7.3 Rationale for Degrees of Freedom .. 303
7.4 Ratio Analysis . . . . . . . . . . . . 305
7.5 Comparison of Means on Two Sample Paths 306
7.6 Multiple Comparisons . . . . . 306
7.6.1 Library Photocopying . 307
Scenario . . . . . . . . . . . . 307
Natural Ordering . . . . . . . 308
7.7 Common Pseudorandom Numbers Revisited. 312
xvi CONTENTS

7.7.1 How Scenarios Affect Correlation. 313

7.7.2 Multiple Comparisons 315
7.8 Optimal Sample-Path Lengths. 315
7.9 Unequal Sampling Costs . . . 316
7.10 Errors in Optimality . . . . . 319
7.11 opt and Multiple Replications 320
7.12 Lessons Learned 321
7.13 Exercises. 322
7.14 References . . . 324

8 Sampling from Probability Distributions 326

8.1 Inverse Transform Method . . . . 329
Algorithm ITM . . . . . . . . . . 329
8.1.1 Continuous Distributions . 330
Algorithm BISECTION . . . 330
Algorithm N-R . . . . . . . . . 330
8.1.2 Restricted Sampling . . 332
8.1.3 Discrete Distributions. 332
Algorithm DI . . . . . . . . . 333
Algorithm ITR . . . . . . . . . 333
Accumulated Round-off Error . 334
8.2 Cutpoint Method 334
Algorithm CM . . . . . . . . 336
Algorithm CMSET . . . . . . 337
8.2.1 Restricted Sampling. 337
8.3 Composition Method 338
Algorithm COMPM 338
Algorithm RWTE 340
8.4 Alias Method . . . 341
Algorithm ALIAS 342
Algorithm ALSET 343
8.4.1 Setting Up the Tables 343
8.5 Acceptance-Rejection Method. 345
Algorithm AR 347
Version 1 . . . . . . . . . 349
Version 2 . . . . . . . . . 349
8.5.1 Squeeze Method. 349
Algorithm ARS. . . . . . 350
8.5.2 Theory and Practice . 352
8.6 Ratio-of-Uniforms Method 352
Algorithm ROU . . . . . . . 354
8.7 Exact-Approximation Method. 358
 CONTENTS xvii

Algorithm EAM . . . . . . . . . . . . 359

Algorithm CA . . . . . . . . . . . . . 360
8.8 Algorithms for Selected Distributions . 360
8.9 Exponential Distribution .. 361
Algorithm EA . . . . . . . . . . . . 362
8.9.1 Weibull Distribution . . . . 362
8.9.2 Inverse Weibull Distribution 363
8.10 Normal Distribution .. 364
Algorithm NA . . . . . 365
8.11 Lognormal Distribution 366
8.12 Cauchy Distribution. 366
8.l3 Gamma Distribution 367
as l. . . . . 368
Algorithm GS* . . 369
a> 1 . . . . . . . 369
Algorithm GKM1 373
Algorithm GKM2 373
Algorithm GKM3 374
8.l3.1 Inverted Gamma Distribution 374
8.14 Beta Distribution 375
max(a, fJ) < 1 . 375
Algorithm AW 376
min(a, fJ) > 1 377
Algorithm BB* . 378
8.14.1 Beta-Prime Distribution 378
8.15 Revisiting the Ratio-of-Uniforms Method. 379
8.16 Poisson Distribution. 380
Algorithm PRUA* . . 383
8.17 Binomial Distribution 384
Algorithm BER. . . . 385
Algorithm BRUA* . . 386
8.18 Hypergeometric Distribution 387
Algorithm HYP . . . . 388
Algorithm HRUA* . . . 390
8.19 Geometric Distribution 390
Algorithm GEO . . . . 391
8.20 Negative Binomial Distribution 391
Algorithm NEGBIN . . . 392
8.21 Multinomial Distribution 392
Algorithm MULTN 394
8.22 Bernoulli Trials. . . . . . 394
xviii CONTENTS

Algorithm SKIP 394

8.23 Lessons Learned 395
8.24 Exercises... 395
Hands-On-Exercises. 407
8.25 References.. 410

9 Pseudorandom Number Generation 416

9.1 Linear Congruential Generators. 418
9.1.1 Period........ 419
9.1.2 Prime Modulus . . . . . 420
9.1.3 Multiplicative Inverse . . 420
9.1.4 Finding Primitive Roots. 421
9.2 Modulus 2fJ • • • • . • • • . • • 422
9.2.1 Mixed Congruential Generators. 425
9.3 Implementation and Portability . . . . . 426
9.4 How Random Are Pseudorandom Numbers? 427
9.5 Choosing Among Multipliers . . . . . . . . 433
9.5.1 Number of Parallel Hyperplanes .. 435
9.5.2 Should the Simulationist Be Concerned About a Theoretically
Poor Multiplier? . 436
Algorithm H-N . . . . . . . . 438
9.5.3 Dangerous Multipliers. 439
9.6 Multiple Replications . . . . . 441
9.7 Combined Generators. . . . . 442
9.7.1 Multiple Pseudorandom Number Streams 445
9.7.2 Cost ofImproved Quality. 446
9.8 Lessons Learned 446
9.9 Exercises. 448
9.10 References . . 450

10 Preparing the Input 452

10.1 Arrival Processess . . . . . . . . . 454
10.2 Empirical and Fitted Distributions 456
10.3 Empirical D.E Approach. . . . . . 457
10.3.1 Sampling Properties . . . . 459
10.3.2 Assessing the Approximation 460
10.3.3 Sampling from {Fn (x)} 461
10.3.4 The Quantile Method . 462
10.4 Characterizations from Theory 464
10.4.1 An Exponential Limit 464
10.4.2 Heavy Traffic 465
10.5 Expert Opinion . . . . . . . 466
 CONTENTS xix

10.6 Choosing a Distribution . . . . . . . . . 467

10.6.1 Exponential Tails . . . . . . . . 468
10.6.2 Sub- and Hyperexponential Tails 468
10.6.3 Lognormal Distribution. . . . . 468
10.6.4 Inverted Gamma and Inverse Weibull Distributions 469
10.7 Support . . . . . . . . . . . . . 470
10.8 Maximum Likelihood Estimation . . . 472
10.8.1 Beta Distribution . . . . . . . 479
10.8.2 Generalized Beta Distribution. 481
10.8.3 Beta-Prime Distribution . . . 481
10.8.4 Inverted Gamma and Inverse Weibull M.L.E.s 481
10.9 Adequacy of Fit . . . . . . . . . . . . . 482
10.10 Attributes and Fitting Distributions . . . . . . . 483
10.10.1 Attributes Affect Decision Flow . . . . . 483
10.10.2 Attributes Do Not Affect Decision Flow 485
10.11 Dependent Historical Data 485
10.11.1 Time Dependence . . . . . . . . . . 486
10.11.2 Local Dependence . . . . . . . . . . 486
10.12 Library Photocopying (Section 1.4 Example) 488
10.12.1 Interarrival Times .. 489
10.12.2 Likelihood Ratio Test 494
10.12.3 Off-Peak Hours .. 494
10.12.4 Number of Pages .. 495
10.12.5 Photocopying Time . 497
10.12.6 Relationship with Standard Linear Regression 501
10.13 Waste Generation (Section 1.5 Example) 502
Algorithm WASTE . . . . . . . . . . 505
10.14 Nonhomogeneous Poisson Processes 506
Algorithm NHPP . . . 508
10.14.1 Clearing Books 508
Algorithm CLEAR 510
10.15 Lessons Learned 513
10.16 References . . . . 514

Appendix More on the Batch-Means Method 516

A.l FNB Rule (Section 6.6.4) . . . . . . 517
A.2 SQRT Rule (Section 6.6.5) . . . . . . 520
A.3 Choosing t, I], and hi (Section 6.6.6) 522

Author Index 525

Subject Index 529

 CHAPTER 1

G. S. Fishman, Discrete-Event Simulation

© Springer Science+Business Media New York 2001
Simulation in
Perspective

Discrete-event simulation consists of a collection of techniques that, when applied to the

study of a discrete-event dynamical system, generates sequences called sample paths that
characterize its behavior. The collection includes:

• Modeling concepts for abstracting the essential features of a system into a coherent set
of precedence and mathematical relationships among its elements

• Specially designed software for converting these relationships into computer-

executable code capable of generating the requisite sample-path data

• Procedures for converting these data into estimates of system performance

• Methods for assessing how well these estimates approximate true, but unknown, system
behavior.

Modeling complex systems has become a way oflife in many fields, most especially in
the engineering, health, management, mathematical, military, social, telecommunications,
and transportation sciences. It provides a relatively low-cost way of gathering information
for decision making. Since the size and complexity of real systems in these areas rarely allow
4 CHAPTER 1. SIMULATION IN PERSPECTIVE

for analytical solutions to provide the information, discrete-event simulation executed on a

computer has become the method of choice. As with all tools of scientific inquiry, success
in applying discrete-event simulation to a particular problem depends on the depth and
breadth of its underlying model as an approximation of the system, and on the investigator's
or simulationist's skill in applying her/his collection of techniques to penetrate the mysteries
of the problem.
As a basis for acquiring these skills, this book describes concepts and methods and
provides guidance for:

• Modeling the behavior of a discrete-event system (Chapter 2)

• Translating the model into code executable on a computer (Chapters 3 and 4)

• Executing the code and generating output in response to one or more input scenarios,
some assigning different numerical values to system parameters and others based on
different strategies for dynamically operating the system (Chapter 4)

• Analyzing each output to infer the system behavior that each scenario induces. This
is done by assessing, by means of statistical methods, how well estimates of system
performance measures approximate their true, but unknown, values (Chapter 6)

• Comparing these inferred behaviorial patterns for the different scenarios (Chapter 7).

Knowledge of these topics is essential for an analyst who plans a discrete-event simu-
lation experiment, executes it, and wants to draw conclusions from its experimental results
that can withstand scientific scrutiny. In today's world of readily available software package
that virtually automates simulation modeling and execution, the newcomer to the method-
ology of discrete-event simulation may easily conclude that there is only one way to model
a system and one way to encode it for execution. Indeed, many courses on discrete-event
simulation focus on the features of a particular software package as the critical issues. While
this style of pedagogy may suffice for teaching students how to simulate relatively small
simple problems, it denies them the broader perspective about modeling alternatives and an
awareness of all options available to them. Chapters 2 through 4 offer the reader a broader
view of what is possible.
No methodology within discrete-event simulation has been as neglected as that for
assessing how well sample averages approximate true, but unknown, long-run time av-
erages that characterize system performance. Although some simulation software does
. provide a batch-means capability, the burden is on the user to figure out how to employ
it in a way consistent with good scientific practice. This is not easy. To reduce this bur-
den, Chapter 6 decribes implementable software, LABATCH.2, based on the batch-means
method that with minimal user effort can be applied to simulation output, either during
or after execution, to make this assessment. The software is available on the Internet at
http://www.or. unc.edu/ ---gfish!1abatch.2.html.
The book also addresses:
 1.1. DISCRETE-EvENT SYSTEMS 5

• Computational complexity as a function of problem size (Chapter 5)

• Sampling from diverse distributions on a computer (Chapter 8)

• Pseudorandom number generation (Chapter 9)

• Using empirical data to estimate the values of the input parameters of a simulation
model (Chapter 10).

In practice, dramatic increases in computing time and memory requirements frequently oc-
cur when a simulation user increases the size of the system being modeled. This consequence
often puzzles users, since the cause of these increases is not apparent to them. Our account
of computational complexity provides a reader with an understanding of how these increases
can arise, thus encouraging the choice of modeling techniques that do most to control the
inevitable increase in computing time and memory.
Virtually all simulation software provides for sampling from a finite collection of
continuous and discrete probability distributions. Chapter 8 expands the options open to
the simulationist by describing methods for sampling from selected theoretical and tabled
empirical distributions. It does so with an emphasis on sampling algorithms characterized
by bounded-mean computing times, regardless of the values assigned to the parameters of the
distribution. It also describes algorithms for a wider range of theoretical distributions than
most simulation software offers.
All simulation sampling on computers relies on a source of pseudorandom numbers.
Chapter 9 gives a concise description of the method of generating these numbers found in
most simulation software, the properties of these numbers, and procedures for generating
pseudorandom numbers with improved properties.
Before a simulation experiment can be executed, numerical values must be assigned
to its input parameters. These values come from either expert judgment or the analysis of
empirical data. Chapter 10 provides a concise account of the most essential details of this
data analysis for several environments that arise in simulation modeling.

1.1 DISCRETE-EVENT SYSTEMS

In a discrete-event system, one or more phenomena of interest change value or state at

discrete points in time, rather than continuously with time. For example, Figure 1.1 depicts
a bus route partially characterized by:

• Number of passengers on the bus

• Number of individuals waiting at each of the five stops

• Location of the bus.

6 CHAPTER 1. SIMULATION IN PERSPECTIVE

0 .... ·

•
Figure 1.1 Bus model

The first two descriptors are discrete, whereas bus location is continuous on the circuit
shown in Figure 1.1. However, the number of passengers on the bus can change only when
the bus location assumes one of the five values that uniquely identify the stops. Moreover,
the number waiting at a stop can change only when the bus arrives there or when a new
individual comes to the stop.
Clearly, the sequences of bus arrival times and individual arrival times, together with
bus capacity, determine how many passengers wait at each stop and bus occupancy at each
moment in time. They also determine how long each individual waits for the bus.
Manufacturing plants, inventory systems, distribution systems, communications
networks, transportation networks, health-care delivery systems, as well as many other en-
vironments are amenable to modeling as discrete-event systems. Virtually all measure their
performance in terms of delay, number waiting, throughput, and resource utilization.
Delay denotes time spent waiting for resources. Buffer occupancy denotes number of
items, jobs, or individuals waiting for resources. Number of finished units emerging from
'the system per unit time characterizes throughput, and proportion of time that resources are
busy relative to total time describes resource utilization.
As examples, throughput in a manufacturing setting denotes the number of finished
goods that a factory produces per unit of time, In a telecommunication network, the rate
at which messages pass from sender to receiver constitutes a measure of throughout. The
 1.1. DISCRETE-EvENT SYSTEMS 7

number of vehicles that cross a bridge per unit time characterizes throughput in a particular
type of transportation system.
Buffer occupancy denotes the number of items in the system awaiting processing or
service and is sometimes called queue length. Delay denotes the time that these items wait
and alternatively is called waiting time.
Occasionally, system loss rate is an additional performance descriptor. It measures
the rate at which objects leave a system, without successfully completing their purposes,
when they encounter delays exceeding specified tolerances. For example, work-in-process
within a manufacturing system may lose value unless delays between successive steps in the
production process are kept within specific limits. This deterioration, which often arises
because of the properties of materials and the production process, occurs in some forms of
semiconductor wafer fabrication.
In the bus example, passenger waiting times describe delay, number waiting at each
station describes queue length, number of completed passenger trips per unit time describes
throughput, and the ratio oftime-average seat occupancy and seat capacity describes resource
utilization. In this case, the buffer or waiting room at each stop has infinite capacity. However,
in many systems the buffer is finite, thus requiring the modeler to specify explicit rules
for what to do when an arrival occurs to an already-full buffer. More generically, every
discrete-event system embodies at least seven concepts:

• Work

• Resources

• Routing

• Buffers

• Scheduling

• Sequencing

• Performance.

Work denotes the items, jobs, customers, etc. that enter the system seeking service.
Would-be passengers are the work in the bus example. Resources include equipment, con-
veyances, and manpower that can provide the services, for example, the bus in Figure 1.1.
Associated with each unit or batch of work is a route delineating the collection of required
services, the resources that are to provide them, and the order in which the services are to
be performed. For the bus illustration, each passenger uses the bus to move from the stop at
which the passenger gets on to the stop at which he/she gets off.
Buffers are waiting rooms that hold work awaiting service. They may have infinite
capacity, as in the bus example, or may have finite capacity, as occurs at the retransmission
nodes (resources) in a telecommunication network that transmits packets of information.
8· CHAPTER 1. SIMULATION IN PERSPECTIVE

When buffers have finite capacity, explicit rules must be adopted that account for what
happens to arriving work that finds a buffer full.
Scheduling denotes the pattern of availability of resources. For example, suppose there
are two buses in Figure 1.1 and operating costs suggest that both buses be used only during
daily periods of high demand. A schedule would consist of the times at which each bus is to
provide service, taking into consideration time out for maintenance and cleaning.
Sequencing denotes the order in which resources provide services to their waiting work.
It may be in first -come-first -served order; it may be dictated by the amount of work awaiting
the resource or other resources from which it receives or transmits work. Sometimes the rule
for sequencing is called the queueing discipline.

1.2 OPEN· AND CLOSED·LOOP SYSTEMS

Collectively, the concepts of time delay, number waiting, resource utilization, and through-
put characterize performance measures in a queueing model. Originally formulated to study
problems in telephony, queueing models, in varying degrees of detail, are today the intrinsic
structures for studying many discrete-event systems in which the routing of items, the se-
quencing of jobs for service to be performed on the items at each station, and the scheduling
of resources to perform these services are principal determinants of delay, number of items
in system, resource utilization, and throughput.
Most queueing systems are either open-loop or closed-loop systems. Work arrives to an
open-loop system from outside the system at a rate independent of the state of the system
and beyond the control of system managers. Figure 1.2 depicts five examples of open-loop
systems. Analysts study systems like these for the purpose of finding work-routing, work-
selection (sequencing), and resource-scheduling policies that balance the competing needs
to keep queue length and waiting time low and resource utilization high.
The five formulations in Figure 1.2 reveal a small portion of the wide variety of behavior
that open-loop queueing systems can represent. Figure 1.2a illustrates the most elementary
model. A customer arrives at a single-server facility and waits in the buffer until the server
becomes available and selects her/him for processing, according to the queueing discipline.
Once processing is complete, the customer departs and, if any customers remain in the buffer,
the server selects one for processing. A barber shop with one chair, a medical practice with
a single physician, and a shoe-repair service with a single cobbler fit this model.
Figure 1.2b extends this concept to a multiple-server environment in which anyone of
r servers can process any customer. Figure 1.2c extends the concept of Figure 1.2a to one in
which a customer requires k distinct types of service in series. A generalization to multiple
servers at each station is easily made.
Figure l.2d introduces the concept of quality control in which a tester determines
whether or not the work performed on items is satisfactory. When it is not, the item must
be reprocessed by the server. In this example, p denotes the proportion of tests that are
 1.2. OPEN- AND (L.OSED-Loop SYSTEMS 9

arriVa1--Jl---.~81--------~.81-----------". departure
OJ

(a) single server

arriva1------I~ ~---------~departure

OJ,

(b) r servers in parallel

departure

(c) k servers in series

arriVal---.----1.~c:f)I-----1.~el-----I.~el-----1.~A/--P--~. departure
Jl fL _________co_ _ _ _ _ _ _ _ _ _ _ _y----'l-p

(d) single server with quality control

f---.----~departure

arrival

OJ,
(e) multiple service routes

Figure 1.2 Open-loop queueing systems

successful. Figure 1.2e exemplifies a system in which the a proportion q of the arrivals
requires service at stations 1 and 2 and the remaining proportion 1 - q requires service at
stations 1 and 3.
This is merely a microcosm of the many variations that can arise in a service environ-
ment in which each arrival comes equipped with a route through the system that indicates the
10 CHAPTER 1. SIMULATION IN PERSPECTIVE

services it requires, each server selects items for service according to its queueing discipline,
and there exists a schedule giving the time intervals during which each server is available to
do work.
Control over work arrival times distinguishes dosed-loop from open-loop systems.
Figure 1.3 illustrates the dosed-loop concept. A new item arrives for service at the moment
the server completes processing its old work. By enforcing this policy, the system manager
keeps throughput and resource utilization high and queue length and waiting time low. In
particular, queue length and waiting time are necessarily zero in this illustration.
Closed-loop systems characterize many manufacturing, transportation, and telecom-
munications systems with controlled access. In addition to choosing routing, sequencing,
and scheduling policies, the ability to control admission based on the state of the system offers
system managers a valuable tool for high throughput and resource utilization and low system
occupancy and delay. This motivates the study of admission policies presumably designed
to accomplish these objectives. From time to time, we return to this distinction between
open-loop and dosed-loop systems to emphasize the implications of each for discrete-event
simulation.

1.3 TIME AVERAGES

As mentioned earlier, queue length, waiting time, throughput, and resource utilization all
relate to system performance. To be useful concepts for decision making, we need a way
of summarizing what the output of a simulation run tells us about these phenomena.
Time averages provide a basis for doing this, as we illustrate for the single-server model
in Figure 1.2a.
Assume that the buffer in Figure 1.2a has infinite capacity and arriving items have
infinite patience. For t > s ::: 0 let

A(s, t) := number of arrivals in time interval (s, t 1,

arrival departure

Figure 1.3 Closed-loop single-server system

 1.3. TIME AVERAGES 11

N (s, t) := number of completions in time interval (s, t 1,

Q(t) := queuelength attime t.

B(t) := { ~ if the server is busy at time t,

otherwise,
(Ll)

Wi := waiting time of departure i,

and note the accounting identities

A(s, t) = A(O, t) - A(O, s),

N(s, t) = N(O, t) - N(O, s),

and

A(O, t) - N(O, t) = Q(t) - B(t).

Suppose that a simulation is run over a time interval (0, t 1and that these phenomena
are observed over the interval (s, t 1, s being a warm-up interval that reduces the influence
of the starting conditions on the behavior ofthe phenomena over (s, tl (Section 6.1) Then
the system has sample average arrival rate

- A(s,t)
A(s, t) := - - ,
t-s

average throughput

-( )._N(s,t)
N s,t .- - - - ,
t-s

average number waiting or queue length

Q(s, t) := -
t -
1
s
[t
s
Q(u)du, (1.2)

average server utilization

R(s, t):= -
1 [t B(u)du, (1.3)
t- s s

and average waiting time

_ 1
L
N(O,t)
W(s, t) = - - Wi. (1.4)
N(s, t) i=N(O,s)+l
12 CHAPTER 1. SIMULATION IN PERSPECTIVE

If the pattern of arrivals is deterministic at rate A and each item has processing time
I/w, then the system in Figure 1.2a is said to be deterministic and ..1(s, t) ~ A, where
notation "a(t) ~ bas t ~ 00" means that "a(t) converges to b as t grows without bound:'
Moreover, the limiting behaviors of Q(s, t), B(s, t), and W(s, t) depend on A and w. In
particular, as t ~ 00

I
N(s, t) ~ A
Q(s, t) ~ 0
if A < W
B(s, t) ~ Alw
W(s, t) ~ 0

and (1.5)

I
N(s, t) ~ W
Q(s, t) ~ 00
if A> W.
B(s,t) ~ 1
W(s, t) ~ 00

If interarrival times, service times, or both are random and have finite first and second
moments, then the limits (1.5) continue to hold as t ~ 00 with probability one, which
we denote by w.p.1. This characterization merely emphasizes that the limit is attained in
accordance with rules that acknowledge the random behavior inherent in the system.
The condition A < W is necessary for stability, and every queueing system must meet an
analogous condition to keep queue length and waiting time from growing without bound as
time elapses. For example, the multiserver system in Figure 1.2b requires A < WI + ... + Wr>
whereas the k-station system of Figure 1.2c requires A < min(wI' ... , Wk). Since the tester
in Figure I.2d induces total mean service time 1I W + (1 - p) I W + (1 - p)2 I W + ... , the
stability requirement is A < wp, whereas the dual route environment in Figure 1.2e requires
A < WI> qA < W2, and (1 - q)A < W3; or equivalently A <min [WI, wz/q, w3/(1 - q)].
A more general characterization of the systems in Figures I.2a and I.2c reveals the
central role that stability plays in performance. Consider a k-station series system with mi
servers each operating at rate Wi at station i for i = 1, ... , k, and arrival rate A. Let

and assume random interarrival and service times. Then under relatively mild conditions as
t ~ 00, throughput behaves as

N(s, t) ~ A min(1, lip) w.p.I,

 1.3. TIME AVERAGES 13

whereas sample average queue length (number in system waiting for service) and waiting
time (system delay time) behave as

_ 0(-) if P < 1,
Q(s,f)-+ { I
ooI-P w.p.I,
if P > 1,

and

_ 0(-) if P < 1,
W(s,f)-+ { I
ooI-P w.p.I,
if P > 1,

where the notation O(u(z)) denotes a function {v(z)} for which there exist constants c > 0
and Zo such that v(z) :::: cu(z) for all z 2: zoo
Recall that in an open-loop system items arrive at a rate A (> 0) independent of the
number of items in the system awaiting or receiving service and independent of the status of
all resources. The arrivals may be deterministic; for example, occurring in batches of fixed
size n at times 1/ A, 2/ A, 3/ A, .... Alternatively, they may occur randomly in time with mean
interarrival time 1/ A and random batch size. By contrast, the workload and its distribution
among buffers and servers in a closed-loop system feed into an action space that determines
the arrival pattern.

1.3.1 EXCEEDANCE PROBABILITIES

In addition to means, performance is sometimes viewed in terms of the exceedance

probabilities:

• pr[Q(f) > q] = 1- FQ(q), q 2: 0,

• pr[Wi ) > w] =1- Fw(q), w 2: 0,

where FQ and Fw are distribution functions (d.f.s). For example, pr[Q(t) > 0] denotes
the probability that at an arbitrarily selected time f at least one item is waiting for service,
and pr(Wi > 0) denotes the probability that an arbitrarily chosen customer i must wait.
Whereas means characterize central tendency, exceedance probabilities shed light on extreme
behavior, especially as q and w increase.
Suppose that a simulation is run for the time interval (0, fl. Then the sample time
averages

PQ(q, S, t) := - -
1
t - s
[t
s
I(q,oo)(Q(u))du, q > 0, (1.6)
14 CHAPTER 1. SIMULATION IN PERSPECTIVE

and

1
L
N(O,t)
Pw(w, s, t) := - - I(w,oo)(Wi ), w > 0, (1.7)
N(s, t) i=N(O,s)+l

denote the proportion of time (s, t 1that the queue has more than q items and the proportion
of customers who begin service in (s, t 1that wait more than w time units.
Under relatively widely applicable conditions, for fixed s

for each q ~ O. Also,

Pw(w, s, t) ~ 1 - Fw(q) w.p.l as t ~ 00

for each w ~ O. Sections 3.4 and 3.5 discuss these limits in more detail and consider them
when a simulation is run until a fixed number of items enter service. The photocopier
problem in the next section illustrates the use of means and exceedance probabilities.
We next describe several contexts in which simulation is an invaluable aid to discovery,
thus allowing the reader to study the methods in this book with a heightened awareness of
their potential value.

1.4 EXAMPLE: REDUCING WAITING TIME FOR

PHOTOCOPYING
Photocopying in a library offers a good example ofan open-loop queueing system. The Health
Sciences Library (HSL) at the University of North Carolina at Chapel Hill has photocopying
machines available for university-affiliated patrons, including students, faculty, and staff, and
for commercial customers. Figure 1.4 roughly illustrates work-flow in a two-floor version
of the problem. For a six-floor library, each buffer would have five flows from and five flows
to other floors, for a total of 30 dashed lines, in a corresponding flowchart.
The HSL has 11 machines; 1 in the basement and I, 2, 3, 3, I, located on floors 1
through 5, respectively. Customers usually make copies on the floor on which they find their
material. If all operational machines on a floor are busy, an arriving customer may choose
to go to another floor. Some patrons for go copying when the delay becomes too long.
Photocopying machines are subject to paper jams that make them inoperative. The
HSL employs a single dejammer to remedy these problems. When a machine becomes inop-
erative, the customer calls the front desk on the first floor to report the jam. The deskperson
receiving the call then signals the dejammer by beeper, indicating the floor on which the
jam has occurred. The dejammer services inoperative machines in the order in which he/she
receives notification of the jams.
 1.4. EXAMPLE: REDUCING WAITING TIME FOR PHOTOCOPYING 15

Floor 1

arrival---+\ buffer 1 r-------1~ copier f--------------3!I~departure

12

Floor 2

arrival - - + { l-----------1~ copier I--------------.:::::.departure

22

Figure 1.4 Library photocopying

Copiers are also subject to breakdowns requiring repair. When this occurs, the HSL
calls a service that dispatches a repairperson. The concomitant loss of operating time is
known to be a principal contributor to customer delay.
The purpose of studying this system was to evaluate the extent to which modifications
to the operation of this photocopying system could reduce waiting time. A discrete-event
simulation was developed to study the problem (Tzenova 1999). Simulation experiments
were run under several different scenarios, including:
• Replacing old copiers that require frequent repair with new copiers
• Adding new copiers to selected floors
• Moving copiers among floors to be more responsive to the demand pattern
• Each of the above with a 50 percent increase in customer arrivals.
Statistical tests applied to library data revealed significant differences at the 5% level
in arrival rates by floor, time of day, and day of week. Table 1.1 shows a partition into six
groups that captured these differences. Since data limitations prevented a detailed analysis
for the basement and floors 1,2, and 5, they were treated individually as groups 7 through
16 CHAPTER 1. SIMULATION IN PERSPECTIVE

Table 1.1 Partition of arrivals

Group Floor Peak hours Off-peak hours Day Mean interarrival
11 a.m.-4 p.m. 10-11 a.m., 4-5 p.m. time (minutes)
3 x All 4.04
2 3 x All 4.53
3 4 x Tuesday 3.38
4 4 x Tuesday 3.77
5 4 x All but 4.28
Tuesday
6 4 x All but 4.30
Tuesday
7 Basement 16.14
8 10.15
9 2 5.40
10 5 23.87

10. Since these arrival rates were notably smaller than those for groups 1 through 6, this
aggregation had no more than a negligible effect on simulation output.
For each scenario, differences in arrival rates by time ofday and day ofweek necessitated
four sets of runs to be executed; namely (off-peak hours, Tuesday), (peak hours, Tuesday),
(off-peak hours, all but Tuesday), and (peak hours, all but Tuesday). For each, sample path
data were collected for 1000 simulated five-day weeks. Omitting data collection during the
warm-up interval reduced the influence of the arbitrarily selected initial conditions in each
simulation run (Section 6.1).
The data were used to estimate:

• Mean waiting time

• Probability of waiting

• Probability of waiting more than five minutes

• Probability of an arriving customer not waiting for service when copiers are busy

plus additional performance measures. The three exceedance probabilities provide a picture
of extreme behavior, whereas the mean reveals central tendency. Figure 1.5a shows point
estimates and approximate 99% confidence intervals for the mean for each group for the
library's current operating policy. The study used the ratio-estimation methodology of Sec-
tion 6.11 with the LABATCH.2 statistical package in Section 6.7 to derive the confidence
intervals. This ratio approach was necessitated by the need to account for random variation
in the number of arrivals in a simulation run for a fixed number of days (Section 3.4). Figure
l.5b displays the corresponding point estimates of the exceedance probabilities.
 1.4. EXAMPLE: REDUCING WAITING TIME FOR PHOTOCOPYING 17

(a) Mean waiti ng time (min.)

6 .50

6.00
III
Q)
1il 5 .50
E
~
Q)
(ij
~
5.00
Q)

~
~
0
0> 4 .50
0>
'0
c:
<0
C 4.00
'0
a..
3.50

3.00
2 3 4 5 6 7 8 9 10
Group number

(b) Estimated exceedance probablities

0.80
II pr(wait>O) _ pr(wait>5) 0 pr(not served)

0.60

c:
0
1:
0
a. 0.40
e
a..

0.20

0.00
2 3 4 5 6 7 8 9 10
Group number

Figure 1.5 Photocopier waiting time analysis: Scenario 1

18 CHAPTER 1. SIMULATION IN PERSPECTIVE

(a) Mean waiting time (min.)

400

350

2'"
ClI
.§ 300
iii
Q)
tij
2:
.s
Q)
250
<I-
0>
0>
"C
<:

I
E'" 200
'0
CI...

150

100
2 3 4 5 6 7 8 9 10
Group number

(b) Estimated exceedance probabilities

050

pr(waiI>O) • pr(waiI>S) 0 pr(not served)

040

<:000
o
1:
o
Co
e
CI... 020

010

000
2 3 4 5 6 7 8 9 10
Group n umber

Figure 1.6 Photocopier waiting time analysis: Scenario 11

 1.5. EXAMPLE: INCREASING THROUGHPUT AT AN INCINERATOR 19

Among all scenarios considered, scenario 11, which replaced the four photocopiers
with the highest failure rate, led to the most substantial reductions in waiting time. These
were on the busiest floors, 3 and 4, which comprise Groups 1 through 6. Figure 1.6a shows
the means, and Figure 1.6b, the exceedance probabilities. In particular, note that for a given
group the corresponding confidence intervals in Figures l.5a and 1.6a do not overlap, offering
support to the contention that the difference in means is real. Also, note the reduction in
exceedance probabilities, especially for the two busiest floors 3 and 4. While the differences
are to be expected when new equipment replaces old, the real improvement on floors other
than 3 and 4 in Groups 7 through 11 shows the benefit to customers on these floors as well.
These and other alternatives need to be evaluated with regard to the benefit each offers
for the cost of the additional resources they require. Sections 6.11 and 7.6 return to this
example.

1.5 EXAMPLE: INCREASING THROUGHPUT AT AN

INCINERATOR
The process by which companies dispose of harmful liquid waste epitomizes a closed-loop
system. Feel Good Incorporated is a large pharmaceutical company with extensive research
and manufacturing facilities on four geographically close campuses or sites. Its research
and development activities generate liquid waste, which the company disposes of in its
own incinerator, located at its Environmental Safety Facility (ESF), and by arrangement
with independent contractors. Since local disposal is considerably less costly per unit than
contract disposal, the company would like to increase the throughput of its own incinerator.
Current operating policies and storage capacity limit its ability to do so.
Each site generates liquid waste that is stored in 55-gallon metal drums on site, if space
is available, until they can be transferred to the ESF, if it has available storage space. After
being brought to the ESF, the waste is eventually pumped out of the drums into one of
three vats. Two hold 500 gallons each and one holds 1000 gallons. After the emptied drums
undergo a time-consuming triple rinsing process, they are returned to the holding areas at
the manufacturing sites. The total number of 55-gallon drums is fixed.
Once a vat is filled, a sample of its liquid waste is sent to an independent laboratory
for assay. During this period, which can be as long as a week, the waste remains in the vat.
The assay determines the composition of the waste and specifies the temperature and time
duration for incineration of the vat's contents. The incinerator burns waste from one vat
at a time. After incineration, the ash residue is put in plastic drums and removed from the
incinerator complex.
Under this policy, more drums of waste are occasionally generated than can be stored
until transfer to the incinerator complex. Then Feel Good employs an outside contractor to
remove the excess drums and dispose of the waste. Therefore, we can think of Feel Good's
waste incinerating process as a closed-loop system to which admission is controlled by the
20 CHAPTER 1. SIMULATION IN PERSPECTIVE

availability of storage space at the four sites. Collectively, the four sites have space to store up
to 300 drums, and the ESF has space for 11 drums. As a consequence, waste can be stored on
the campuses only if space is available. Hereafter, we refer to a drum handled by an outside
contractor as lost.
Disposing of waste by outside contractor is considerably more expensive per drum
than disposal at the incinerator complex. Although the need for outside contracting rarely
occurs at the current waste generation rate, this rate is expected to increase in the future. In
anticipation of this growth, the company requested a study to:

• Estimate the volume ofoutside contracting to expect as the amount ofwaste generation
grows in the future

• To identify which of the several time-consuming steps of transporting dums, pumping

their contents into vats, assay, rinsing drums, and incineration is the principal retardant
of a higher incinerating rate. This was to be studied for a fixed number of drums and
storage space at the four campuses and at the ESE

A discrete-event simulation experiment (Ma 1999) was developed and executed for
selected annual waste-generation growth rates. Of particular interest for each growth rate
was It := mean annual number oflost drums. In the notation of Section 1.3, let

Ai (0, t) := number of generated drums at site i in the time interval (s, t l,

Qi(t) := number of stored drums at time t at site i,
Q'(t) := number of drums at time t at the incinerator being stored or processed,
N(s, t) := number of drums processed at the incinerator during the time interval (s, tl.

Then the number of lost drums in (s, t 1is

4 4
D(s, t):= LAi(s, t) - L Qi(t) - Q'(t) - N(s, t),
i=1 i=1

and the time-average annual number of lost drums over r years is

1 1 '
[I,(r) := -D(s, s + r) = - L D(s +j - 1, s + j),
r r j=1

where data collection begins at year s, thereby diluting the influence of initial conditions
in the simulation (Section 6.1) on the sample-path data. The summation partitions the
sample-path data into increments that serve as the basis for a statistical analysis to assess
how well [1,( r) approximates the unknown It (Section 6.6).
Table 1.2 shows [I,(r) for selected growth rates a. To assess the statistical error in
approximating It(r) by p,(r) due to randomness in drum generation in the simulation, a
 1 .5. EXAMPLE: INCREASING THROUGHPUT AT AN INCINERATOR 21

Table 1.2 Number of drums handled by outside contractor" (current

waste generation = 380 drums/year)

Growth rate Sample mean 99% Confidence interval b

U-L
ex M(r) L U g(r)
0< ::: .05 0 0 0
.06 .1065 0 .2562 2.41
.07 .9268 .6061 1.247 .69
.08 3.269 2.743 3.795 .32
.09 5.845 5.338 6.352 .17
.10 8.552 8.016 9.088 .13
.11 11.22 10.63 11.80 .10
.12 14.04 13.50 14.58 .08
.13 16.83 16.29 17.37 .06
.14 19.58 19.06 20.10 .05
.15 22.34 21.76 22.92 .05
.16 25.19 24.64 25.73 .04
.17 28.15 27.55 28.74 .04
.18 30.85 30.22 31.47 .04
.19 33.67 33.04 34.30 .04
.20 36.44 35.87 37.02 .03
.21 39.50 37.65 41.34 .09
.33 72.78 70.85 74.71 .05
.46 110.1 108.1 112.2 .04
.61 150.8 149.0 152.5 .02
" For 0 < ex ::: .20, the simulations were run for r = 10,000 years.
For ex ~ .21, they were run for r = 1,000 years.
b L := lower limit; U := upper limit.

statistical analysis using LABATCH.2 (Section 6.7) gave the approximating .99 confidence
intervals for j.L. That is, fl( T) is a point estimate for j.L, and [L, Uj covers j.L with approximate
probability .99. The difference U - L measures the absolute accuracy of fl(T), and (U -
L)/ fl(T) its relative accuracy. For example, a = .14has (U - L)/ fl(T) = .05, indicating that
the length of the interval is 5% of the sample mean. This suggests a relatively high level of
accuracy. Only a = .06 has a relatively inaccurate point estimate, but the absolute accuracy
measure U - L is tolerable for present purposes.
Figure 1.7 plots the sample mean fl := fl( T) versus a. The graph shows a straight line

fl = -19.46 + 279.9 a, a> .07, (1.8)

as computed by least-squares regression analysis. If 50 lost drums per year were a crucial
number, then the regression line reveals that an annual growth rate of

50 + 19.46
a = 279.9
= 0.248
22 CHAPTER 1. SIMULATION IN PERSPECTIVE

160

140

toCD
>- 120
~
0. __ mean= 19.46+279.9a
rn
E 100 a>.07
2
'C
'lii 80
..Q
'0
~ 60
J:J
E
:::l
I::

as 40
I::
CD
:::1!
20

0
0.1 0.2 0.3 0.4 0.5 0.6 0.7
Growth rate (a)

Figure 1.7 Mean number oflost drums per year vs. waste-generation growth rate

would generate that amount on average in one year. More generally, let

/L(a, t) := mean number oflost drums t

years in the future with annual
growth rate a for waste generation.

Then the regression line (1.8) gives

ln1.07
/L(a, t) = -19.46 + 279.9[(1 + a)t - 1], t > lnO + a) , a > 0, (1.9)

enabling us to make projections as in Table 1.3.

As illustration, the table reveals that the mean annual number of lost drums exceeds
50 in years 5, 4, and 3 for annual growth rates a = .05, .06, and .08, respectively. More
generally, the table provides a basis for estimating growth in the number oflost drums.
The study also made simulation runs to evaluate five changes in operating policy,
each intended to reduce the annual number of lost drums. One of these moved the taking
of samples for testing content and the concomitant time delay in waiting for the resulting
assay from the ESF to the four drum -originating campuses. Under this policy, the simulation
results revealed zero lost drums for annual growth rates 0 < a ::::: .10 over a five-year period.
None of the remaining four scenarios achieved this zero loss.
 1 .6. WHY DISCRETE-EvENT SIMULATION 23

Table 1.3 Mean annual number oflost drums'

Annual waste-generation growth rate a

Year 0.05 0.06 0.07 0.08 0.09 0.10 0.15

t
1 0 0 0.13 2.9 5.7 8.5 22.5
2 9.2 15.1 21.1 27.1 33.2 39.3 70.8
3 24.7 34.0 43.5 53.2 63.1 73.2 126.3
4 40.9 54.0 67.5 81.4 95.7 110.4 190.1
5 57.9 75.2 93.2 111.9 131.3 151.4 263.5
10 156.6 201.9 251.9 304.9 363.2 426.6 832.7
• Based on expression (1.9).

To effect this policy, the samples had to be taken from the individual drums at the sites
rather than from a vat at the incinerator after the contents of the drums had been pumped
in. This was deemed worth the effort to eliminate the time delay at the ESF due to waste
remaining in a vat until the results of its assay were available.

1.6 WHY DISCRETE-EVENT SIMULATION

To put discrete-event simulation into context as a tool of analysis, we next describe the
way in which it relates to modeling in general. The first step in studying a system is using
accumulated knowledge to build a model. A model can be a formal representation based on
theory or a detailed account based on empirical observation. Usually, it combines both. A
model serves many purposes. In particular, it:

1. Enables an investigator to organize her/his theoretical beliefs and empirical observa-

tions about a system and to deduce the logical implications of this organization

2. Leads to improved system understanding

3. Brings into perspective the need for detail and relevance

4. Expedites the speed with which an analysis can be accomplished

5. Provides a framework for testing the desirability of system modifications

6. Is easier to manipulate than the system

7. Permits control over more sources of variation than direct study of a system allows

8. Is generally less costly than direct study of the system.

Except in relatively simple cases, the values of system performance descriptors are un-
known, and determining then remains beyond the capacity of the analytical tools of queueing
24 CHAPTER 1. SIMULATION IN PERSPECTIVE

theory. Discrete-event simulation offers techniques that can approximate the values of these
descriptors to within a remarkably small error. The approximations come from analyses of
data observed on sample paths generated during execution of a simulation program corre-
sponding to the model of interest, usually on a queue-length process {Q(z), s < z < t}
and on a waiting-time process {WN(O,s)+l, .•. , WN(o,t)+d (Section 1.3). This book describes
many of these methods in detail.
Often one creates and executes a simulation of a complex system simply to study
alternatives. How does the system respond to a change in input parameter values? How does
it respond to a change in routing, sequencing, or scheduling rules? Simulation-generated
data often can provide sufficiently accurate answers to these questions, thereby enabling an
investigator to sharpen her/his understanding of the system. Occasionally, interest focuses
on the selection of one of several system configurations that leads to the best performance
with regard to explicitly stated criteria. Carefully constructed and executed simulations of
each system alternative can provide the sample data on which one can apply well-established
statistical tests to detect superior performance (Sections 7.2, 7.5, and 7.6).
Another context in which the benefits of simulation are considerable is in the devel-
opment of optimal or near-optimal policies for system management. This area involves a
partnership between simulation and analytical methods. In many situations one constructs
an analytical model of a system in order to determine an optimal policy, with regard to a
specified criterion, for managing the system. However, the need to simplify the representa-
tion to obtain an analytical solution often removes from the model critical characteristics of
the system that may affect performance. Here, the analyst who has found an optimal policy
for the simplified analytical model ponders the extent to which the policy remains optimal
in the presence of the features suppressed for analytical convenience. Discrete-event simu-
lation can playa vital role in resolving this issue. It enables one to build a (more realistic)
simulation model containing the features in question, first execute it using the analytically
based optimal policy, and then execute it using other management policies that institutional
considerations may suggest. Comparing the simulation results for the different policies en-
ables the simulationist to determine the degree to which the optimal policy prevails over the
institutionally based policies in the more realistic simulation. This joint use of analytical and
simulation methods considerably enhances the value of both methods.

1.6.1 DETAIL

As mentioned earlier, discrete-event simulation becomes a legitimate research tool

when known analytical methods cannot supply a solution to a problem. Once simulation
is adopted, certain admonitions made earlier in regard to modeling must be reexamined.
Principal among these considerations is the issue of detail.
When building a model, an investigator repeatedly faces the problem of balancing the
need for structural detail with the need to make the model amenable to problem-solving
techniques. Being a formalism, a model is necessarily an abstraction. However, the more
explicit detail a model includes, presumably the better it resembles reality. Also, including
 1.6. WHY DISCRETE-EvENT SIMULATION 25

detail offers greater opportunities for studying system response when structural relation-
ships within the model are altered. First, more combinations of structural change can be
considered, and second, more aspects of the response can be studied.
On the other hand, detail generally impedes problem solution. Added detail often
shifts the method for solving a problem from an analytical to a numerical one, thereby
giving up the generality that an analytical solution offers. Detail also increases the cost of
solution. However, the most limiting factor in the use of detail is that we rarely know enough
about the system under study to specify more than a modicum of its characteristics. Every
model must limit detail in some respect. To fill in a system description in place of detail,
one makes assumptions about system behavior. Since one does not want these assumptions
to conflict with the observable behavior of the system, testing them against observation is
desirable whenever possible.
Our earlier remarks stressed the fact that the amount of detail in a model is generally
inversely related to our ability to obtain an analytical solution. However, if the minimal model,
which contains the least detail needed for useful study, is not amenable to analytical solution
and we adopt an approach based on discrete-event simulation, we have the prerogative of
building as much detail into our model as we like without concerning ourselves about the
absence of an analytical solution. It is precisely this descriptive ability that holds the most
attraction for modelers, for the greater the amount of detail, the more realistic the model is
and consequently the closer they expect the results of a simulation run to conform to reality.
However, practice indicates that judicious restraint in regard to detail is often the better
policy. At least three reasons support this assertion:

• To include detail we must devote effort and time to preliminary observation of the
individual characteristics of the system under scrutiny. These actions induce a cost, the
justification for which a modeler has to weigh with respect to her/his objective. This
issue arises in most simulations, and no universal course of action fits all contingencies.

• Additional detail requires more programming, especially for the contingencies that
detail specifies. Moreover, the inclusion of great detail at the outset of a modeling
effort makes the job oflocating the sources of error in the resulting computer program
especially difficult because of the many potential trouble spots.

• Because of the increased number of calculations that a computer program based on

a detailed model must perform, we expect a concomitant increase in execution time
and need for computer memory. These increase cost. This third reason for restraint is
not always apparent until actual simulation execution begins and we see computing
cost quickly mounting (Chapter 5).

These sobering remarks provide a useful perspective for evaluating the degree of detail
worth having. Often, a simulationist gets a limited model running first and then introduces
detail where this model provides inadequate answers. This bootstrapping approach makes
26 CHAPTER 1. SIMULATION IN PERSPECTIVE

for a more judicious allocation of a modeler's time and effort and reduces debugging and
computing times and memory requirements.

1.6.2 MODELING DANGERS

Although modeling clarifies many relationships in a system, there are at least two
warnings that an investigator should always keep in mind. First, no guarantee exists that the
time and effort devoted to modeling will return a useful result. Occasionally, failure occurs
because the level of resources is too low. More often, however, the simulationist has relied
more on the simulation language and its tools and not enough on ingenuity when the proper
balance between both is the recipe that enhances the probability of success.
The second warning concerns the use of the model to predict beyond its range of
applicability without proper qualification. Being an abstraction, a model inevitably produces
numbers that serve as approximations to the true values of unknown performance measures.
Do these numbers have a tendency to overstate or understate the true values? How close are
the generated values to the true values? Good modeling analysis demands answers to both
these questions. For example, the .99 confidence intervals in Table 1.2 attest to the sample
means (point estimates) being good approximations to the unknown mean number oflost
drums. Most importantly, the answers demarcate the range of applicability of model output.
Neglecting to provide this qualification constitutes one major cause of model misuse in
practice. Chapter 6 addresses the issue of statistical accuracy.

1.7 TECHNICAL ATTRACTIONS OF SIMULATION

Discrete-event simulation on a digital computer, or computer simulation, offers many conve-
niences that make it an attractive tool for analysis. l It compresses time so that years of activity
can be simulated in minutes or, in some cases, seconds. This ability enables an investigator
to run through several competing operational designs in a very small fraction of the time
required to try each on the real system.
Computer simulation can also expand time. By arranging for statistics of interest to
be produced over small intervals of simulated time, an investigator can study by computer
simulation the detailed structure of change in a real system which cannot be observed in real
time. This figurative time dilation is especially helpful when few data are available on change
in the real system. For example, the "list" statement in the SIMSCRIPT n.5 programming
language offers an example of this capability.
One important consideration in any experiment is the ability to identify and control
sources ofvariation. This is especially important when a statistical analysis of the relationship

IThe term computer simulation should not be confused with a simulation study of a computer
system. Such studies have become major users of computer simulation.
 1 .8 . ME DIU M FOR MOD ELI N G 27

between independent (input) and dependent (output) factors in an experiment is to be per-

formed. In the real world, the ability is to a major extent a function of the system under study.
In a computer simulation, a simulationist, of necessity, must explicidy specify the sources
of variation and the degree of variation due to each source in order to make a simulation
run. This requirement enables her/him to eliminate unwanted sources of variation simply
by omitting them from the simulation. However, this ability also requires the simulationist
to devote sufficient attention to a system in order to derive an adequate understanding of
how to describe quantitatively the sources of input variation that he/she wishes to include.
Sections 4.7 and 4.22 illustate how this control of sampling variation can improve statistical
efficiency.
In a field experiment, errors of measurement inevitably arise in recording data. These
occur because no perfect measuring device exists for suppressing all external influences.
By contrast, only round-off errors occur in a computer simulation, since the programmed .
simulation produces numbers free of any superimposed variation due to external and un-
controllable sources. The round-off errors are due to finite word length in a computer. Using
extended-precision computation, as needed, can make these errors relatively negligible.
During execution, it is occasionally desirable to stop the experiment and review results
to date. This means that all phenomena associated with the experiment must retain their
current states until the experiment resumes execution. In field experiments a complete halt
of all processes is seldom possible. However, computer simulation offers this convenience,
provided that the termination part of the program contains instructions for recording the
states of all relevant variables. When the experiment resumes, the terminal states become
the initial states for those variables so that no loss of continuity occurs. Most simulation
languages offer an option for programming this feature.
The ability to restore the state of the simulation provides another benefit. At the end
of a computer simulation run, analysis of the results may reveal a situation explainable
by data that were not collected. Here, the simulationist can reprogram the data-collecting
subprogram to record the desired data and can run the simulation again with the same initial
conditions. The simulation runs identically as before with the additional data being made
available.
Simulation also enables us to replicate an experiment. To replicate means to rerun an
experiment with selected changes in parameter values or operating conditions made by the
investigator. For example, an independent replication is one in which the simulation model
is unchanged but the sequence of pseudorandom numbers used to drive the model in the
replication is independent of the sequence employed in all previous runs.

1.8 MEDIUM FOR MODELING

Modeling a system inevitably relies upon some medium to connect the realities of the system
into a collection of abstractions amenable to manipulation for purposes of solution. In
28 CHAPTER 1. SIMULATION IN PERSPECTIVE

physics, partial differential equations are the abstractions that result from applying the
infinitesimal calculus as the medium. Difference equations are often the abstractions that
result from employing the calculus of finite differences to model a dynamical economic
system.
Discrete-event delay systems are also amenable to abstraction using these two media,
and indeed, the analytical derivation of queue-length, waiting time, and resource-utilization
distributions relies crucially on them. As illustration, consider a single-server queueing
system, as in Figure 1.2a, with interarrival times AI, A z, ... and service times B I , B z, ....
Let Wi denote the waiting time in queue for arrival i and suppose the first arrival finds
the queue empty and the server idle. Then it is easily seen that for a first-come-first-served
queueing discipline,

WI = 0,
Wz = (BI - A z)+,
W3 = (Wz + Bz - A3)+,

where

if x :::: 0,
otherwise.

Known as Lindley's formula, this notation incorporates the calculus of finite differences
with one boundary constraint. In principle, this calculus can be used to model a system
with m servers in parallel at a single station but with multiple boundary constraints that
grow exponentially in number with m. Moreover, other queueing disciplines demand more
complicated notation.
Recognizing the limitations of this approach to modeling discrete-event dynamical
systems and aware of the need to adopt modeling conventions that could be easily converted
into executable computer code, early simulation methodologists devised two paradigms for
abstracting these systems. The event-scheduling approach focuses on the moments in time
when state changes occur (Section 2.3). The process-interaction approach focuses on the flow
of each entity through the system (Section 2.4). Both allow for variable updating and flow-
branching, characteristics that epitomize discrete-event delay systems. Both allow a modeler
to create a collection of flowcharts that incorporate all system dynamics needed to write a
simulation program conveniently.
In practice, experienced simulation users rarely turn to explicit flowcharting when
modeling a system. More often, they rely on the constructs available to them within the sim-
ulation programming packages they employ. These constructs implicitly embrace either the
 1 .9 . DIs eRE TE - EVE N T S I M U LA T ION PRO G RAM MIN G LAN G U AGE S 29

event-scheduling or process-interaction approach, more often the latter than the former.
Although skipping the flowcharting step saves considerable time for a modeler, the new-
comer to discrete-event simulation cannot possibly develop an appreciation of the options
available to her/him for modeling delay systems by totally omitting a study of the principles
of the event-scheduling and process-interaction approaches. Familiarity with these options
is critical in modeling large complex systems. Indeed, total reliance on language constructs
can occasionally keep the modeler from seeing the simplest way to account for behavior.
Chapter 2 describes these principles in detail.

1.9 DISCRETE-EVENT SIMULATION PROGRAMMING

LANGUAGES
Virtually all discrete-event simulation programming languages contain capabilities for:

• Maintaining a list of events scheduled to be executed as time evolves

• Maintaining other dynamical data structures

• Generating pseudorandom numbers

• Generating samples from a variety of distributions

• Computing summary statistics including time averages, variances, and histograms

• Generating reports

• Detecting errors peculiar to simulation programs.

Also, having the capability to:

• Perform conventional arithmetic operations and to evaluate functions such as

max(x, y), log x, sin x

adds to a language's appeal, although some languages that provide less than this capability
have managed to survive and prosper.
In today's PC environment, many simulation languages have become embedded in
larger software systems with enhanced features that include the capabilities to:

• Operate in Microsoft Windows 98, Windows NT, and Unix environments

• Display data graphically

30 CHAPTER 1. SIMULATION IN PERSPECTIVE

• Display simulation animation.

Some also have capacities to:

• Import CAD (computer-assisted design) drawings to provide icons for animation

• Draw icons including all standard CAD objects such as rectangles and ellipses

• Allow a user to incorporate other computer applications such as Excel spreadsheets,

Microsoft Word files, clipart, and Microsoft PowerPoint presentations

• Integrate database systems that comply with the Microsoft ODBC standard such as
Dbase, Access, Fox Pro, and Excel

• Allow user-coded inserts in other computer languages such as Visual Basic, C, and
C++.

The principal general-purpose discrete-event simulation languages include GPSSIH

(Wolverine Software), which is now a subset of the more general SLX simulation pro-
gramming language; SIMAN (Systems Modeling Corporation), contained within the Arena
general-purpose simulation system; SIMSCRIPT 11.5 (CACI); and SLAM, which is embedded
in the AweSim general-purpose simulation system (Symix Systems, Inc.lPritsker Division).
Because of frequent updating of these software products, we merely include as references
the name of the vendor identified with each. More current information is obtainable at their
sites on the Internet. For a listing of other simulation software, see Swain (1999).
In principle, two options exist for constructing a discrete-event sinulation program.
One allows a simulationist to enter statements via a command line. Sections 4.2 through
4.21 demonstrate this option using SIMSCRIPT 11.5. The other displays icons corresponding
to the language statements, or aggregates thereof, on the screen and allows a simulationist
to construct a program by pointing, clicking, dragging, and dropping these into a "model
window:' Sections 4.23 through 4.27 use SIMAN to illustrate this option. For several reasons,
the second option has come to prevail in practice. In a PC environment, working with a
collection of screen-based icons, one for each of the powerful statements in the language,
allows a user quickly to:

• Construct a block-diagram flowchart of a program

• Use menus associated with each icon to assign numerical values to parameters

• Specify operating statistics to be calculated during a simulation experiment

• Have the block diagram converted into executable code

• Execute the simulation experiment

• Display the final operating statistics.

 1.9. DISCRETE-EvENT SIMULATION PROGRAMMING LANGUAGES 31

The availability of PC-based simulation systems that provide all or most of these conveniences
has significantly reduced the time and effort for the newcomer to discrete-event simulation
to create an executing program.

1.9.1 SPECIAL FEATURES

Within specialized areas, additional reductions in programming effort are possible. As

illustration, many manufacturing and materials-handling systems use conveyors and auto-
matic guided vehicles (AGVs) to move items within factories and warehouses. Some also rely
on automated storage and retrieval systems (ASRS), robots, and bridge cranes. Modeling
these objects and the dynamics of their interactions with other parts of the system presents a
formidable challenge to the average simulationist. However, manufacturing and materials-
handling simulation has grown into an industry of its own, prompting software vendors to
develop and offer precoded modules that simulate these special behaviors.
Among these offerings are the advanced manufacturing template in Arena (Systems
Modeling Corporation), AutoMod (AutoSimulations, Inc), FACTOR/AIM (Symix Systems,
Inc.!Pritsker Division), ProModel (PROMODEL Corporation), SIMFACTORY 11.5 (CACI),
Taylor II (F & H Simulation, Inc.), and WITNESS (Lanner, Inc.). These concepts are usually
iconized on the monitor screen and have menus that allow a user to customize features to
the particular problem at hand. Sections 4.23 and 4.24 discuss limitations of this approach.

1.9.2 OBJECT-ORIENTED PROGRAMMING

In practice, code reuse plays a central role in establishing a workable balance between
programming effort and computational effciency. If a large-scale simulation program is to
be developed and then executed many times, merely with different numerical input, the
investment in programming effort to achieve highly efficient code can be negligible when
amortized over the useful lifetime of the program. By contrast, a study that requires a se-
quence of simulation programs that successively differ in logic in relatively small ways may
call for so much programming effort at each stage using a conventional simulation language
as to discourage one from carrying out the full investigation. In this environment, keeping
programming effort within an acceptable bound becomes the overriding objective. In the
current world of highly interactive computing, this second modality has grown in relative
importance and led to an emphasis on point-click-drag-drop programming in conjunc-
tion with object-oriented programming. Indeed, this concept underlies the icon/menu-driven
environment so commonplace on PCs and workstations.
An object essentially denotes a collection of data structures for storing critical data.
For example, in a network model a message is an object with length, priority, origin, and
destination. In manufacturing, a part is an object whose data include a route identifying
the sequence of processing stations it visits on the way to completion. The data also include
processing times and priorities for service at the stations. Each class of objects is associated
with functions or algorithms that contain recipes for operating on the data stored in instances
32 CHAPTER 1. SIMULATION IN PERSPECTIVE

of the objects. An airport simulation clarifies the meanings of class, object, instance, and
function.
Aircraft form a class, and particular types of aircraft are objects. An individual aircraft
denotes an instance of a particular type (object) of aircraft within the class of aircraft.
Associated with these are the functions oflanding an aircraft, taxiing on a runway, loading
an aircraft with cargo, unloading cargo from an aircraft, loading passengers onto an aircraft,
unloading passengers from an aircraft, refueling an aircraft, and aircraft takeoff. In principle,
programs created to represent the class called aircraft and to compute the functions of
landing, loading, unloading, refueling, and takeoff can be used in many different scenarios
of airport simulation. For example, one simulation may study congestion at an airport
dedicated exclusively to passenger traffic, while another may study capacity limitations at an
air-cargo terminal. Both simulations would use the aircraft class but with different mixes of
types and different utilization of functions.
Ease of programming and program reuse provide the principal motivations for adopt-
ing an object-oriented approach to discrete-event simulation. This is especially true in an
environment where an iconized collection of preprogrammed modules allows a simulation-
ist to construct a simulation model on the monitor's screen with relatively little knowledge
of the components of each module. If results obtained from executing the simulation en-
courage the analyst to consider an amended simulation program, the point, click, drag, and
drop capacities facilitate this modification by assembling a new program on-screen made up
of parts of the old program and new modules taken from the collection of preprogrammed
modules in the displayed collection of icons.
As examples, AutoMod, ProMod, and WITNESS employ objected-oriented pro-
gramming within their modules for manufacturing and materials handling. MedModel
(PROMODEL Corporation) uses object-oriented programming to create its modules within
a health-care delivery system. Other simulation software products use a semblance of the
object-oriented programming approach when constructing conveyors, etc.; but in reality
they do not entirely adhere to the formal rules for this approach to programming. For busi-
ness applications, Technology Economics, Incorporated offers FlowModel, which contains
a preprogrammed collection of business-oriented objects and functions.
This capacity for program reuse and reduced programming effort has limitations.
If the menu of preprogrammed objects and functions does not provide for a particular
application, then either simulating the application cannot be done using this collection, or
else programming is required to effect the simulation. Sometimes this last option results in
relatively awkward use of objects and functions for which they were not originally intended.
To provide a more global setting, CACI offers an object-oriented simulation language called
MODSIM III that allows a user to program her/his own objects and functions.
To regard the term "object" as merely data and a function as merely another word for
subroutine would be an oversimplication. Whereas a subroutine may use global as well as
local data structures for computation, an object must have all its required data stored within
its own local data structure, and a function in an object-oriented environment operates on
these local data. This distinction clearly makes an object and a function less dependent on
 1 .9 . DIs eRE TE - EVE N T S I M U L A T ION PRO G RAM MIN G LAN G U AGE 5 33

the overall problem than a subroutine would be, and this offers a considerable advantage in
making changes.
Customarily, a program consists of objects linked together by well-defined connections
(which are themselves objects). Each object is compiled separately, with the underlying sim-
ulation language lurking in the background and usually out of the simulator's consciousness.
When a program requires change, only those objects and connections affected by the change
need to be recompiled. If one thinks of compilation as a process of optimization with regard
to computational efficiency, then this procedure of selective recompilation (suboptimiza-
tion) inevitably results in reduced run-time effciencywhen compared to a globally compiled
(optimized) program. Also, a program consisting of objects requires more space for data
structures than a more traditionally written program would. Nevertheless, proponents of
the object-oriented approach to simulation regard the programming convenience that it
offers as outweighing the penalties incurred by increased space requirements and reduced
computational efficiency.
The concept of object-oriented programming is not new to simulation. SIMULA
(Birtwistle et al. 1973), one of the first simulation programming languages, used the concept.
Moreover, the property of inheritability of attributes, an intrinsic feature of object-oriented
programming, has been a feature of GPSS from its beginning as an IBM programming
product. In particular (e.g., Schriber 1991, p. 368), the SPLIT statement in GPSS creates
a specified number of copies of a process and endows each with the data structures and
functions in the original process. However, each copy executes independently of all remaining
copies. In this way a batch of identical parts entering a job shop can be SPLIT for processing
and later reassembled into a batch using the ASSEMBLE statement of GPSS. The SPLIT
statement allows the spawned instances of the object to be numbered sequentially, if desired,
for easy identification.
Virtually all object-oriented simulation programming languages translate their state-
ments, into C++ statements, which are then compiled into executable code. C++ is an
enhancement of the C programming language endowed with a syntax for object-oriented
programming.
At present, every simulation software product is of one of two forms. Either it embraces
the traditional sequential programming approach reminiscent of FORTRAN and C or it
embraces the object-oriented paradigm as in C++ and JAVA. As to whether this coexistence
will continue or one of these formats will come to dominate remains for the future to reveal.

1.9.3 WEB-BASED DISCRETE-EVENT SIMULATION

The Internet has created opportunities for new modes of modeling, programming, and
executing discrete-event simulation. Modules created in the JAVA programming language,
using its object-oriented programming features, reside at accessible websites. Modelers
browse these sites and retrieve off-the-shelf modules that appear to fit the problem at hand.
All this presumes that the browser is using a JAVA-based discrete-event simulation language
possessing all the features delineated in Section 1.9 and that the off-the-shelf modules meet
34 CHAPTER 1. SIMULATION IN PERSPECTIVE

a universally accepted standard that allows their incorporation, without incident, into the
browser's program.
More ambitious modes of web-based simulation are also possible. For example, under
certain conditions, a simulationist could run a web-based simulation program t times in
parallel to generate sample paths for independent replications by distributing the workload
over t distinct computing platforms. Conceptually, this approach reduces elapsed clock time
to obtain results by a factor of t. Alternatively, a web-based simulation program can be used
simultaneously by many users. See Whitman et al. (1998).

1.9.4 NO PROGRAMMING

To make their products less intimidating and thus more appealing to a wider range of
potential users, some vendors have introduced simulation software that attempts to mini-
mize or, if possible, to eliminate user-originated programming. The products encourage a
simulationist to point and click on icons, corresponding to precoded constructs available in
an on-screen template and to drag and drop these constructs into the simulation program
being created, thus reducing the user's time and effort spent programming. The modules
discussed in the last several sections exemplify these precoded blocks.
While noble in concept, the practice has limitations. Most notably, prebuilt constructs
often represent specific environments that are difficult to modify in order to accommodate
the problem at hand. Indeed, it may not be apparent to the simulationist that a misrepre-
sentation of problem logic is about to occur when inserting a prebuilt module into her/his
program. For example, see Section 4.28.

1.9.5 TEACHING DISCRETE-EVENT SIMULATION

Every introductory course in discrete-event simulation seeks to turn out students ca-
pable of performing discrete-event simulation experiments. Inevitably, this must include the
capability to create and execute programs. However, the programming skill level is prob-
lematical. Catering to the little-or-no programming philosophy tends to produce students
whose knowledge is restricted to the software to which they have been exposed and possibly
to a narrow subset of problems encountered within discrete-event simulation.
A considerably more desirable outcome for the course is to equip each student with
the knowledge of how to model a system and translate the model into executable code in at
least one language and, at the same time, to familiarize them with the breadth and depth
of concepts that they are likely to encounter when asked to work in a language other that
those taught in the course. This book embraces this strategy. While it does offer examples
in particular languages in Chapter 4, our account works best when the instructor uses it in
parallel with an adopted relatively comprehensive discrete-event simulation language such
as GPSS/H, SIMAN, SIMSCRIPT II.s, SLAM, or SLX. It works less well when used with
special-purpose simulation languages that encourage minimal programming.
 1 .1 O. REF ERE NeE s 35

What about object-oriented programming? Since the purpose of this book is to

acquaint the reader with the most basic concepts of discrete-event simulation, we have
chosen to forego object-oriented programming as a pedagogic medium in favor of an ac-
count that exposes the reader to the conceptual formalisms of the event-scheduling and
process-interaction approaches and with their conversion to executable code in two major
discrete-event simulation languages, SIMSCRIPT 11.5 and SIMAN. Their syntactical com-
prehensiveness motivated this choice. After having studied the present account, a reader will
be well prepared to work with any general-purpose discrete-event simulation language or any
object-oriented discrete-event simulation language, subject to familiarizing herself/himself
with its particular syntax.

1.1 0 REFERENCES

Birtwistle, G., OJ Dahl, B. Myhraug, and K. Nygaard (1973) SIMULA BEGIN, Auerbach,
Philadelphia.
Ma, Xudong (1999). Managing the hazardous liquid waste input to Glaxo Wellcome's
environmental safety facility incinerator, OR350 project, Operations Research
Department, University of North Carolina, Chapel Hill.
Schriber, T.J. (1991). An Introduction To Simulation Using GPSSIH, Wiley, New York.

Swain, J.J. (1999). 1999 Simulation software survey, ORMS Today, 26, 38-51
Tzenova, E. (1999). An analysis of operations for the health sciences library photocopier ser-
vice, OR350 project, Operations Research Department, University of North Carolina,
Chapel Hill.
Whitman, 1., B. Huff, and S. Palaniswamy (1998). Commercial simulation over the web,
1998 Winter Simulation Conference, D.J. Medeiros, E.F. Watson, J.S. Carson, and M.S.
Manivannan, editors, Association for Computing Machinery, New York, 335-341.
 CHAPTER 2

G. S. Fishman, Discrete-Event Simulation

© Springer Science+Business Media New York 2001
Modeling
Concepts

Every simulation study consists of at least 10 steps:

1. Translating the qualitative description of a system into a formal abstraction that explicitly
accounts for all logical and mathematical relationships

2. Identifying all parameters of the abstraction that require numerical values as input

3. Identifying all measures of performance whose values require estimation

4. Estimating the values of all unknown input parameters from available data, expert
opinion, etc.

5. Designing the set of sampling experiments, at least one for each distinct set of values
for the vector of input parameters

6. Converting the logical abstraction to executable code in a simulation programming

language

7. Incorporating into the code statements for computing time averages of performance
measures during program execution
38 CHAPTER 2. MODELING CONCEPTS

8. Performing the set of sampling experiments by executing the code repeatedly, at least
once for each vector of input-parameter values

9. For each measure of performance, evaluating how well its time average approximates
its unknown long-run average

10. Comparing corresponding sample time averages for each performance measure across
experiments

While this numbering allows for orderly progress from beginning to end, actual practice
may turn out to blur the distinction between steps, rearrange their order, or in some cases,
neglect or pay too little attention to them. For example, the availability of point-click-drag-
drop simulation software entices a simulationist to merge steps 1 and 6, possibly making
her or him oblivious to a wider range of modeling concepts than the selected language
implementation may accommodate. Frequently, the choice of long-run time averages to
estimate in step 3 depends solely on what the programming language automatically makes
available in converting the model to code in step 6. Too often, step 9 is ignored, making any
conclusions based on comparisons across experiments difficult to justify statistically.
This divergence between our idealized list ofsteps and practice stems from the premium
that simulationists often put on their time. There is no doubt that the point-click-drag-drop
option saves time in model development and coding, and produces models and, subse-
quently, code that well approximates many a system. However, relying exclusively on this
option limits the diversity of models that can be formulated, an issue of considerable rele-
vance when the system under study calls for more structure than direct application of this
software can accommodate. To remove a potential impediment, this chapter focuses on
step l. In particular, it presents concepts germane to the construction of a discrete-event
simulation model. Most importantly, it describes two alternative, but logically equivalent,
ways of representing system behavior. Section 2.3 describes the event-scheduling approach
and Section 2.4, the process-interaction approach. Each provides directions for writing a
computer program whose execution simulates system behavior.
Discrete-event simulation has a special vocabulary that we use throughout this book.
Its principal terms include:

System. A collection of objects that interact through time according to specified rules.

Model. An abstract logical and mathematical representation of a system that describes the
relationship among objects in a system.

Entity. An object in a system that requires explicit representation within a model of the system.
For example, servers, customers, and machines all denote objects in a service delay
system.

Attribute. A descriptor of a property of an entity. For example, a server may have a skill attribute
that identifies its skill level and a status attribute that identifies whether it is idle or busy.
 2 .1 . N EXT - EVE N TTl MEA D V A NeE 39

Linked List. A collection of records each identified with an entity and chained together
in an order according to values assumed by a particular attribute of each
entity. For example, arriving jobs in a system are entities. If they have to await
service, they may be filed in a list ordered by the value of their arrival-time
attribute.

Event. A change in state of a system.

Event notice. A record describing when an event is to be executed.

Process. A collection of events ordered in time. Time elapses in a process.

Reactivation point. An attribute of an event notice associated with a process. It specifies the
location within the process at which execution is to resume.

Future-event set. A linked list containing event notices ordered by desired execution time.

Timing routine. Procedure for maintaining the future-event list set and advancing simulated
time.

Activity. A pair of events, one initiating and the other completing an operation that
transforms the state of an entity. Time elapses in an activity.

The reader has already encountered several of these terms; for example, system and
model. Others, such as the concept of a linked list, are essential for an understanding of how
the computer code for a discrete-event simulation carries out its tasks. Briefly, each record
in a singly linked list has an attribute, called the successor address, that points to the record
that immediately follows it in the list. This facilitates search starting from the first record in
the list, called the header. In addition to a successor address, each record in a doubly linked
list has an attribute, called the predecessor address, that points to the record that immediately
precedes it in the list. Collectively, predecessor and successor addresses allow for search
beginning either at the header or the trailer, the last record in the list. Section 5.2 discusses
the concept oflist processing in more detail. Unless clarity demands otherwise, we hereafter
refer to a doubly linked list simply as a list.

2.1 NEXT-EVENT TIME ADVANCE

Discrete-event simulation concerns the modeling on a digital computer of a system in which
state changes can be represented by a collection of discrete events. Here the appropriate
technique for modeling a particular system depends on the nature of the interevent intervals.
These time intervals may be random or deterministic. If they are random, the modeling
technique must allow for intervals of varying lengths. If the intervals are deterministic, they
may vary according to a plan, or they may all be of equal length. When they vary, the modeling
techniques must again acknowledge the nonuniform nature of the interevent intervals. When
they are constant we can employ a considerably simpler modeling technique to structure
change in a model. For present purposes, we assume that they vary.
40 CHAPTER 2. MODELING CONCEPTS

In a discrete-event system, a state change implies that an event occurs. Since the states
of entities remain constant between events, there is no need to account for this inactive time
in our model. Accordingly, virtually all computer simulation programming languages use
the next-event approach to time advance. After all state changes have been made at the time
corresponding to a particular event, simulated time is advanced to the time of the next event,
and that event is executed. Then simulated time is again advanced to the scheduled time of
the next event, and the procedure is repeated. Most importantly, the procedure enables a
simulation to skip over inactive time whose passage in the real world we are forced to endure.
Hereafter, we sometimes refer to simulated time, more concisely, as time.
Although the next -event approach is the most commonly used, the reader should note
from the outset that other methods exist for advancing time and processing events. (e.g.,
Fishman 1973, pp. 38-40 and Pritsker et al. 1997, p. 25) Unfortunately, use of an alternative
necessitates programming it, hence the appeal of the next -event approach, which is a standard
feature of most discrete-event simulation programming languages.

2.2 ARITHMETIC AND LOGICAL RELATIONSHIPS

In modeling a system for eventual computer simulation, two different structures play sig-
nificant roles. One includes the mathematical relationships that exist between variables
(attributes) associated with the entities. For example, if Q is the number of patients waiting
in an outpatient clinic, then we set Q +- Q + 1 when a patient arrives and Q +- Q - 1
when a patient begins treatment. Sometimes, the specification of the mathematical relation-
ships for a system serves to describe completely the way in which state changes occur, as, for
example, in a model of a national economic system.
Logical relationships constitute the other set of structures that are used to describe a
system. In a logical relationship, we check to see whether a condition is true or false. If true,
we take a certain action. If false, we take an alternative action. For example, when a server
becomes available in a maintenance shop, he or she checks to determine whether a job is
waiting for service. If one is, service begins. If no job is waiting, then the server remains idle.
If several jobs are waiting, the server selects one according to an established logical-operating
or job-selection rule. A logical-operating rule describes a set of mutually exclusive conditions
with corresponding actions. For example, if several jobs are waiting, the rule may be to select
the job that has been waiting for the longest time. If there are n jobs waiting, this rule is
formally, For i = 1, ... , n, if job i has been waiting for the longest time, select job i for
serVIce.
Our purpose in discussing time passage and relational questions in this introduction
is to establish their importance in modeling a discrete-event system. The remainder of the
chapter describes modeling methods that account for these concepts in different ways.
The concepts of event, process, and activity are especially important in modeling a sys-
tem. To illustrate the relationships among them, we consider a job arriving at a maintenance
 2.3. EVENT-SCHEDULING ApPROACH 41

shop with two tasks to be performed. Figure 2.1 shows the arrival and the eventual service
that each task receives. It also shows the relationships that exist among events, processes, and
activities. Collectively, these concepts give rise to three alternative ways of building discrete-
event models. The event-scheduling approach emphasizes a detailed description of the steps
that occur when an individual event takes place. Each type of event naturally has a distinct
set of steps associated with it. The activity-scanning approach emphasizes a review of all
activities in a simulation to determine which can be begun or terminated at each advance
of the clock. The process-interaction approach emphasizes the progress of an entity through
a system from its arrival event to its departure event.
The development of the three concepts is related to the world views adopted by simula-
tion programming languages. Among the major ones, SIMAN, SIMSCRIPT 11.5, and SLAM
offer a user the choice of event scheduling or process interaction, whereas GPSS/H exclu-
sively offers process interaction. While activity scanning is a logically equivalent approach,
it is computationally less efficient for most discrete-event scenarios. Accordingly, we omit
further mention of it.

2.3 EVENT-SCHEDULING APPROACH

Every discrete-event system has a collection of state variables that change values as time
elapses. A change in a state variable is called an event, and this concept is the basic building
block of every discrete-event simulation model based on the event-scheduling approach. In

Process

,,r-------------------------------,
,
:4 Activity 2

..-- Activity 1 --..

I I time

Event 1 Event 2 Event 3 Event 4 EventS

arrival service service service service
begun on begunon ended on ended on
task 1 task 2 task 1 task 2

Figure 2.1 Event, process, and activity

42 C HAP TE R 2. MOD ELI N G CON C E P T 5

a queueing system, an arrival constitutes an event because it increases the number of items
in the system by one. A departure is also an event because it decreases the number of items in
the system by one. Beginning service and completing service are also events, for each signals a
change in state for one or more resources of the system and a change in status for a particular
item. To illustrate the significance of the event -scheduling approach, we describe an airline
reservation system. We encourage the reader to recognize that the concepts described here
are generic to all simulation modeling based on this approach.
The principal features of the airline reservation model are:

a. All customers call on the telephone.

h. Customers are willing to wait indefinitely for service.

c. No restriction exists on the number of customers that can wait at one time; i.e., an infinite
number of telephone lines exists.

d. Each customer wants a reservation for either a single-destination or a multiple-destination

trip.

e. If all reservationists are busy when a call occurs, a 9-second recorded message asks that
customer to wait.

f. The airline office has m reservationists. Calls occur at rate A, and intercall times are i.i.d.
random variables that follow the exponential distribution

A > 0, t 2: 0,

denoted by £ (1/ A). The proportion of customers who want a multiple-destination reser-
vation is p, whereas the proportion that desire a single-destination reservation is 1 - p.
Single-destination service times are i.i.d. random variables that follow the exponential
distribution

w > 0, t 2: 0, (2.1)

denoted by £ (1/ w). Multiple-destination service times are i.i.d. random variables that
follow the Erlang distribution

w > 0, t 2: 0, (2.2)

denoted by £2(2/w). The service time of an arbitrarily selected caller is then a random
variable with probability density function (p.d.f.)

f(t) = (1- p)!t(t) + plz(t) (2.3)

= w(1 - p + P wt)e- wt , o :s p :s 1, w > 0, t 2: O.
 2 .3 . EVE N T - 5 C H E D U LIN GAp PRO A C H 43

Note that a random variable from £2 (2, 1/w) is equivalent to the sum of two independent
random variables, each from £(I/w).

g. The purpose for simulating this system is to compare its performance as the number of
reservationists m grows while A, w, and p remain fixed. In support of this objective it
is of interest to estimate the long-run averages and, more generally, the distributions of
waiting time, number of customers awaiting service, and number of busy reservationists
as the number of reservationists m varies.

In terms of the 10 major steps in a simulation study described in the introduction,

the present description accomplishes steps 2 and 3. To effect step I, we first need to identify
the entities in the system and their attributes. For this problem we have two classes of entities:

Entity Attribute Possible attribute assignments

<
ONE-DESTINATION
CUSTOMER TYPE
MULTIPLE_DESTINATIONS

<
IDLE
RESERVATIONIST STATUS
BUSY

In the event-scheduling approach to modeling, a single flowchart usually incorporates

all state changes that occur at the same time. Recall that a state change is called an event. In
the airline reservation problem we can identify 10 types of state change. These elementary
events are:

Customer Reservationist
1. Cali 8. Remove from waiting line
2. Hang up 9. Become busy
3. Begin service 10. Become idle
4. End service
5. Begin recorded message
6. End recorded message
7. File in waiting line

Flowcharts provide a convenient way of describing behavior that is eventually to be

transformed into a computer program. Moreover, alternative flowchart representations are
possible for the same problem, the choice of which to use being a function of user familiarity
with different languages and the degree ofinsight that each representation offers. We mention
familiarity before insight because experience regrettably shows this to be an overriding
consideration in practice.
44 CHAPTER 2. MODELING CONCEPTS

1 6
5 7
CALL END MESSAGE

Figure 2.2
10 Elementary and compound
events
COMPLETION

Figure 2.2 shows the logic flow that enables us to incorporate these elementary events
into three compound events entitled CALL, END-.MESSAGE, and COMPLETION. From
the original description of the problem we see that whenever a call occurs 0), the caller
either begins service (3) if a reservationist is idle, or otherwise begins listening to a 9-second
recorded message (5). If service begins (3), then the reservationist who begins service also
becomes busy (9). While all four of these state changes are elementary events, they differ
in at least one important way: (1) is an unconditional event, whereas (3), (5), and (9) are
conditional events. An unconditional event occurs independently of the current state of the
system, whereas a conditional event occurs only when another event puts the system in a
prescribed state. In particular, (3) and (9) occur at the same time as (1) if and only if a
reservationist is idle; but (5) occurs simultaneously with (1) if and only if all reservationists
are busy. Collectively, the CALL compound event summarizes this behavior.
In a like manner, the END-.MESSAGE compound event summarizes state-change
possibilities whenever a recorded message ends (6), and the COMPLETION compound event
summarizes the elementary events that can occur when a caller completes service (4). Since
the beginning of service (3) coincides with a customer call (1), an end of the recorded message
(6), or a service completion (4), all three CALL, END-.MESSAGE, and COMPLETION events
incorporate the beginning of service representation. Because a customer hangs up (2) upon
service completion (4), this event (2) is modeled as part of the COMPLETION compound
event. Lastly, the idling of a reservationist (10) occurs when either no customers await
service or those that are waiting are listening to the recorded message. Consequently, it is
also modeled as part of the COMPLETION event.
Figure 2.3 defines the symbols used for flowcharting. We usually refer to the symbols as
blocks. The reader should focus attention on the "scheduling" and the "next-event selection"
blocks, since all simulation programming languages contain corresponding special subpro-
grams. If one chooses to program a simulation model in a nonsimulation language, he/she
 2 .3 . EVE N T - SC H E D U LIN GAp PRO A C H 45

must write code for both of these subprograms. As the description of the airline reservation
problem evolves, the convenience of having these subprograms available in a simulation
programming language becomes apparent.
At each moment in time, the simulated system is in a particular state. For example,
the system state for the airline reservation problem has entries for:

• Number of busy reservationists (i)

• Number of customers listening to recorded message (j)
• Number of customers waiting for service after having listened to the recorded message
(k)
• Remaining time until next call (S)
• For each caller 1 listening to the recorded message:
Remaining time until completion of message (Tz)

• For each busy reservationist 1:

Remaining time to complete service (Tz').

Note that the first three entries can assume only nonnegative integer values, whereas the last
three assume nonnegative values. That is, the first three are discrete-state variables, whereas
the last three are continuous-state variables. More generally, every simulation has a state
vector some of whose entries are discrete and others of which are continuous.
Selecting the next event for execution and its timing depend crucially upon the entries
in this state vector. As illustration, the remaining time until the next event in the airline
reservation problem is

v = min(S, h ... , Tt, T{, ... , Tj}.

Moreover, this event is a new call if S = V, the end of a recorded message if Tz = V for some
1 = 1, ... , i, or a completion of service if Tz' = V for some 1 = 1, ... , j. As we presently
show, one rarely has to concern oneself with maintaining this vector explicitly.
Although brief, this description of the airline reservation problem illustrates several
important properties generic to the event-scheduling approach:

• An event is executed if and only if a state change occurs.

• Simulated time remains constant while an event is being executed.
• A compound event is a recipe for executing a sequence of actions all at the same time.
• A discrete-event simulation consists of a sequence of instances of compound events of
possibly different types ordered according to their scheduled execution times, which
may be randomly generated.
46 CHAPTER 2. MODELING CONCEPTS

wy Flowchart entry

Determine
time of Computation
next CALL

~
T
Subprogram

Scheduling execution
of an event

Decision flow

Selection of next
event for execution

Figure 2.3
Transfer
Flowchart conventions

This last property immediately raises the following issue:

• How does one determine the order of execution by type of compound event and by
time, and how does one arrange for this schedule of executions in practice?

Every discrete-event simulation programming language provides a capability for scheduling

events and for executing them. This distinction between scheduling and execution is an
important one. In general-purpose programming languages, code within a program is exe-
cuted sequentially. In a simulation language, a time delay may occur. The act of scheduling
creates a record that identifies the event that one wants to execute and that contains the
simulated time at which this execution is desired. The time may be any time no less than
the current simulated time. This record is then added to a list, often called the Juture-
 2.3. EVENT-SCHEDULING ApPROACH 47

event set (FES), using a filing protocol that allows retrieval of the records in the order
of desired execution time. The form of this protocol differs among simulation languages
(Section 5.4).
A simulation programming language also provides a timing routine that operates on
the FES to make event execution occur in the desired order and at the desired simulated
times. Figure 2.4 illustrates the concept. The dashed line in the User's Program denotes
elapsed simulated time. To prime the future-event set, the User's Program first schedules
at least one event. Then it transfers control to the timing routine, which selects the event
with the smallest execution time for execution (block 1). By executing an event in block 3
we mean passing control to a block of programmed logic corresponding to the changes
in state that this particular event describes, executing the block, and returning control to
the timing routine in block 4. Note that the event corresponds to a particular instance of
this programmed logic. The act of destroying the event notice (block 4) makes the space it
occupies available for other uses. Rather than destroying the event notice, some simulations
selectively choose to save and reuse the space for a newly created event of the same class.
Doing so eliminates the need to find space for the new event. Section 5.3 addresses this issue.
Note that once the user's program passes control to the timing routine, it remains there,
or at the event it executes (block 3), until the timing routine finds the FES empty (block 5).
Then it terminates the simulation by transferring control back to the user's program (block 6).
This is one of several ways of ending a simulation.

2 Advance simulated
time to desired
execution time

Begin
simulation
5 4
No FES Destroy event
notice

Figure 2.4 Prototypical timing routine, event-scheduling approach

48 CHAPTER 2. MODELING CONCEPTS

While Figure 2.4 shows how to prime the FES with the initial events that start the
simulation, it provides no explicit account ofhow successive events are scheduled as simulated
time elapses. This scheduling actually occurs within executing events, as we now show. Figure
2.5 shows the steps executed each time a CALL event occurs. Here words in capital letters
denote terms that have meanings particular to our description of the airline reservation
problem. Block 1 denotes the creation of a record of an entity called CUSTOMER. The
record presumably contains space for predefined attributes of the CUSTOMER. In block 2,
TYPE denotes an attribute of CUSTOMER that contains information about whether the
CUSTOMER wants a single- or multiple-destination reservation. This record is a block of
contiguous words in memory that conceptually has the form

entity ('d predeccessor successor)

record: 1 entl' fcatton,
i' attn'b ute 1, ... ,attn'bute r,
address, address '
(2.4)

providing space for the values of r attributes and two pointers. Since CUSTOMER in the
"
present problem has one attribute TYPE (block 2), r = 1. The fields entitled predecessor
address and successor address are included only if the record is to be chained to a doubly
linked list (Section 5.2). For a singly linked list, only the successor (predecessor) address
is included if the search is to begin from the front (back) of the list. No predecessor and
successor address fields appear if the entity record is not to be linked to a list. For the present
problem, the END..MESSAGE (Figure 2.6) and COMPLETION (Figure 2.7) events make
use of a list called QUEUE that contains records of CUSTOMERs.

1Create a 2 Detennine 3 Detennine

record of TYPE of time of
CUSTOMER CUSTOMER next CALL

7Schedule an 5 4 Schedule
"'--L--l Attempt
END_MESSAGE next
SERVICE
in 9 seconds CALL
Yes

Figure 2.5 CALL event: event-scheduling approach

 2 .3 . EVE N T - S C H E D U LIN GAp PRO A C H 49

Event
END_
MESSAGE

1
Attempt File
SERVICE CUSTOMER in
QUEUE
Yes

Figure 2.6 END-.MESSAGE event: event-scheduling approach

2 3
Remove first
Arrange
CUSTOMER
SERVICE
from QUEUE

Yes

4
Put
RESERVATIONIST
in IDLE mode

Figure 2.7 COMPLETION event: event-scheduling approach

Blocks 3 and 4 in Figure 2.5 provide a mechanism for executing the next CALL event.
In particular, block 3 determines when the event occurs, and block 4 creates an event notice
corresponding to the event, assigns the type of event to its class attribute and its desired
execution time to its clock attribute of the event notice, and files this event notice in the
future-event set. In the present case, the class is CALL.
50 CHAPTER 2. MODELING CONCEPTS

2 4 Destroy
Determine 3 Schedule a
record of
service time COMPLETION CUSTOMER

7 PutRESER-
VATIONIST
in BUSY mode

Figure 2.8 SERVICE routine: event-scheduling approach

Conceptually, an event notice takes the form

event notice:
predeccessor successor)
(identification, class, clock, attribute 2, ... , attribute T,
address, address '
(2.5)

where the predecessor and successor address fields allow for chaining the record to the FES
and for backward as well as forward searching of the FES.
The steps in blocks 3 and 4 represent bookkeeping procedures essential to all simulation
modeling based on the next-event approach. A study of block 8 indicates why this is so. When
entry into this block occurs, program control transfers to the timing routine. As Figure 2.4
shows, this routine searches the future-event set, selects the first event notice, removes it from
the set, advances time to the desired execution time, and transfers control to the subprogram
corresponding to the event name in the event notice. Execution of the event then occurs.
This variable-time advance procedure skips all the intervening time between events and
concentrates on the next event. Doing so contrasts with fixed-time advance procedures that
increase time in small increments and, at each advance, execute all events that call for
execution at that time.
Returning to Figure 2.5, we observe that the subprogram in block 5 attempts to pro-
vide service to the CUSTOMER. Since attempts at service also occur in other flowcharts,
describing these steps separately, as in Figure 2.8, avoids redundancy. Note the use there of
 2.3. EVENT-SCHEDULING ApPROACH 51

another entity called RESERVATIONIST. The decision flow in block 6 of Figure 2.5 indicates
that if no RESERVATIONIST is free, a message indicating a wait is to occur. Notice that
the scheduling in block 7 identifies the time that the message ends as the event execution
time. Doing so enables the END~ESSAGE event (Figure 2.6) to check to see whether a
RESERVATIONIST has become available during the nine seconds. Here scheduling creates
an event notice for an END~ESSAGE and files it in the future event set. Also notice that
entry into block 8 occurs when the CALL event is complete.
Figure 2.6 illustrates the END~ESSAGE event. Block 3 describes a new feature
wherein the record of the CUSTOMER is filed in a list called QUEUE. Although filing
can occur based on any attribute of the CUSTOMER, it is customary to order this list so that
the next available RESERVATIONIST can find the record of the next CUSTOMER for service
first in the list. Often, time of filing is the ordering attribute; however, another attribute may
be the basis for ordering.
The COMPLETION event in Figure 2.7 describes actions that occur whenever a
RESERVATIONIST completes service to a CUSTOMER. The RESERVATIONIST checks
to see whether any CUSTOMERs await service (block 1). Each RESERVATIONIST is an
entity with an attribute STATUS whose value denotes BUSY or IDLE status. When a COM-
pLETIoN occurs this attribute of the corresponding RESERVATIONIST indicates the BUSY
mode. If no one is waiting for service, then block 4 indicates that STATUS should be changed
to IDLE, and simulation continues via selection in block 5.
If at least one CUSTOMER is waiting in the QUEUE, the RESERVATIONIST removes
the record of the first CUSTOMER from the QUEUE (block 2) and service is arranged
via the SERVICE routine (block 3). Whereas the transfers to this routine in the CALL and
END ~ESSAGE events are attempts to get service, here service is assured because of the
availability of the RESERVATIONIST. Also, note that the design of this event eliminates the
need to update STATUS when customers are waiting.
The decision flow in block 1 of the SERVICE routine (Figure 2.8) allows for the
contingency that a RESERVATIONIST is definitely available. Block 2 determines service time.
The scheduling of a COMPLETION in block 3 establishes the time at which a corresponding
COMPLETION event in Figure 2.7 for this RESERVATIONIST is to be executed. Here again
an event notice is created and filed in the future-event set. We now begin to see how crucial
a role this set plays in a simulation.
Block 4 destroys the record of the CUSTOMER entity about to begin service. The
action indicates that each CUSTOMER is a temporary entity, since it can be destroyed (Figure
2.8) as well as created (Figure 2.5). This contrasts with each RESERVATIONIST, which is a
permanent entity for the life of the simulation. The distinction is important, for as Section
4.2 shows, it leads to considerably different programming procedures.
A little thought makes the convenience of using temporary entities apparent. Unless
we destroy CUSTOMER records as they receive service, the number of such records grows
as simulated time elapses. Since each record occupies space in the computer, this growth can
conceivably consume all available space before desired termination of the simulation occurs.
Destroying records that are no longer needed reduces the probability of this occurrence.
52 C HAP TE R 2. MOD EL I N G CON C E PT S

Careful space management can also lead to a reduction in computing time. For sim-
ulation languages that rely on a dynamic space allocator to assign space to records as the
simulation program executes, destroying unneeded records makes the program run more
rapidly. That is, the more space that is available, the faster the space can be found and allo-
cated. Conversely, an accumulation of unpurged, unnecessary records reduces the amount
of space available to the dynamic space allocator and forces it to spend extra computing time
searching for the remaining unallocated space. Section 5.3 returns to this topic.
Note that Figure 2.8 ends with a transfer of control back to the calling event. Although
we may also regard events as subprograms, they have the distinctive feature of always re-
turning control to the timing routine. Throughout this book we maintain this distinction
between a routine and an event.

2.3.1 PRIORITY ORDER

The airline reservation problem assumes that intercall and service times are continuous
random variables. This implies that if TI and Tz are the random times at which two distinct
events occur, then pr( TI = Tz) = o. However, all computers have finite word size. If TIl and
T22 are the finite-digit representations of TI and Tz, respectively, in a computer executing a
simulation with these two events, then it is possible for Tll = Tzz . That is, pr(Tll = Tu) > o.
If an executing simulation encounters this situation, it needs guidance as to the order
in which it should execute the two or more events scheduled for execution at the same
time. To resolve this issue, we assign a priority to each event class. If event class CALL has a
higher priority than event class COMPLETION, then, when a CALL event has the identical
execution time as a COMPLETION event, the CALL event is executed first. Hereafter, we let
the notation CALL >- COMPLETION denote this priority order. For the airline reservation
problem, we assume CALL >- COMPLETION >- END-11ESSAGE. All simulation languages
provide an explicit means for making these declarations.
It is possible that two or more events within the same event class have identical sched-
uled execution times. When this happens, most simulation programming languages break
the ties by executing events in the order in which they are scheduled. In summary, ties are
broken first by class and then by time scheduled within class.

2.4 PROCESS·INTERACTION APPROACH

The process-interaction approach to modeling differs substantially from the event-scheduling

approach. It caters to the flow environment that characterizes queueing systems and pro-
duces results identical to those for the event-scheduling approach when the same logical
relationships, input, sampling methods, and output analyses are employed. The four prin-
cipal discrete-event simulation languages, GPSS/H, SIMAN, SIMSCRIPT 11.5, and SLAM,
all allow model representation based on this concept of flow logic.
 2 .4 . PRO C E 5 5 - I N TE RAe T ION A P PRO A C H 53

Before beginning our description, we offer one viewpoint that the reader will find
helpful when weighing the benefits of the two alternative modeling schemes. Whereas the
process-interaction method often appears conceptually simpler to a model builder, this
simplicity usually comes at the expense of less programming control and flexibility than
the event-scheduling approach offers. The truth of this assertion becomes apparent as this
section evolves. For each simulation problem, a model builder who plans to use a general-
purpose simulation language must decide whether or not to opt for conceptual simplicity
or programming control. Considerations that enter this decision include familiarity with
alternative languages, the availability of languages at a user's computing facility, and the
extent to which a particular simulation program product is to be made available to other
potential users with different backgrounds at other computing facilities. Alternatively, sup-
pose that a model builder employs a special-purpose simulation package, as often occurs
in telecommunication and manufacturing simulation. Since most of these special-purpose
packages use the process-interaction approach, the model builder gives up the ability to
choose, presumably in return for the benefit of reduced programming effort.
Process interaction relies on a conceptually different representation of behavior than
event scheduling does. In the event-scheduling approach, an event contains all decision flows
and updating that relate to the change in state that execution of the event requires. Since
time elapses between events, one can regard a simulation as the execution of a sequence
of events ordered chronologically on desired execution times. However, no time elapses
within an event. By contrast, the process-interaction approach, either implicitly or explicitly,
provides a process for each entity in a system. To illustrate this concept, we return to the airline
reservation problem of Section 2.3. Figure 2.9 organizes the elementary events of the problem
into two sequences, one corresponding to the temporary entity CUSTOMER and the other
to the permanent entity RESERVATIONIST. Each generic sequence is called a process, and
dashed lines denote simulated time flow. For example, elementary event 6 (finish nine-second
message) occurs nine seconds after elementary event 5 (begin nine-second message). Each
temporary entity moves through the system and consequently through time. Occasionally,
a temporary entity encounters an impediment to progress and must wait. For example, if
all RESERVATIONISTs (permanent entities) are answering calls when a new CUSTOMER
(temporary entity) materializes, this CUSTOMER must wait for service. The inclusion of
time flow in a process distinguishes this approach from event scheduling. Moreover, it calls for
a more indirect form of computer programming when moving from logic flow to executable
program.
Figures 2.10 and 2.11 depict prototypical representations of these processes. Each
CUSTOMER has an attribute TYPE that identifies whether a single-destination or multiple-
destination reservation is desired. TYPE plays the same role here as in the event-scheduling
approach in Figures 2.4 and 2.7. Note in block 4 of the CUSTOMER procedure that an entry
into the procedure at block 1 is scheduled at a future simulated time. This scheduling sets
the stage for the creation of the next CUSTOMER. This form of scheduling coincides with
that of the event-scheduling approach, which always specifies entry to the first block of a
flowchart. Consequently, block 4 creates an event notice for this entry and files it in the
54 CHAPTER 2. MODELING CONCEPTS

CUSTOMER RESERVATIONIST

Wait

Wait
Wait Wait

Figure 2.9 Processes

future event set for later execution. Block 6 also schedules an event, namely, a completion
of the busy message. Flow then proceeds to block 7, where the CUSTOMER waits until
the message is complete. After the nine seconds for the message elapse, flow continues to
block 8.
The flow through blocks 6 and 7 represents one distinguishing feature of the process-
interaction approach. When the simulation encounters block 6, it creates an event notice and
assigns current simulated time plus nine seconds to its clock attribute as the desired execution
time. This event notice also has a reactivation point attribute to which the scheduling action
assigns a pointer to block 8. Then it files the notice in the future-event set. Flow continues
to block 7, where control returns to the timing routine, which then selects the next event for
execution. Here, block 7 represents a deactivation point. In contrast to the event-scheduling
approach, here entry into a process for execution does not necessarily start at the first block.
Also, time elapses within the CUSTOMER process. The dashed line between blocks 7 and
8 indicates that execution of block 8 occurs at a later time than execution of block 7 does.
This time evolution within a process distinguishes it in a major way from an event where all
computation takes place at the same moment in time.
Since reactivation occurs in a specified number of seconds, the wait in block 7 is
unconditional. However, the wait in block 10 is a conditional one. Here, reactivation at block
 2 .4 . PRO C E 5 5 - I N TE RAe T ION A PPRO A C H 55

Create 2 Detennine 3 Determine 4 Schedule

a record of Type of call time activation
CUSTOMER CUSTOMER of next of next
CUSTOMER CUSTOMER

12 II Activate
13 Destroy Yes
Wait until RESER-
record of
completion VATIONIST
CUSTOMER
now

end of busy

Yes

No 7 Wait until
busy message
ends

Figure 2.10 CUSTOMER process: process-interaction approach

12 occurs once a RESERVATIONIST in Figure 2.11 selects the CUSTOMER for service.
At least two alternative modeling representations of a conditional wait occur in practice.
First, a notice that a CUSTOMER is waiting for a RESERVATIONIST may be filed in a list
associated with the RESERVATIONIST process. In Figure 2.10 this list is called QUEUE.
Then, whenever a RESERVATIONIST in Figure 2.11 completes service at block 7, a scan
of the relevant list results in the selection of the next CUSTOMER to receive service. The
notice corresponding to this CUSTOMER is then removed from the list in block 4. If no
CUSTOMER is waiting, the RESERVATIONIST becomes idle in block 3.
If several alternative types of servers exist in a simulation problem, each would have a
distinct representation as a process and each would have a corresponding list on which to store
notices of demands for service. By contrast, a second approach to modeling a conditional
wait has a single list in which to store notices of all service demands, regardless of the type of
server demanded. By aggregating all demands, this alternative reduces the number oflists to
one, thereby reducing space requirements within a program. However, it requires additional
search time on the part of each type of server, who now has to search through a longer list of
56 CHAPTER 2. MODELING CONCEPTS

Process
RESERVA-
TIONIST

Create record 5 Determine

of RESER- Wait until
VATIONIST activated

4 Remove ftrst 6 Schedule

CUSTOMER service
from QUEUE completion

8 Activate 7 Wait until

CUSTOMER
completion
now

Figure 2.11 RESERVATIONIST process: process-interaction approach

notices. Both options are available in most simulation languages, with multiple lists being
the more common approach. Regardless of which option is chosen, we hereafter refer to
these lists as the conditional-event set (CES).
Time also elapses in the RESERVATIONIST process in Figure 2.11. Unlike every in-
stance of the CUSTOMER process, however, each instance of the RESERVATIONIST process
remains in existence during the life of a simulation run. Each RESERVATIONIST has an un-
conditional delay at block 7 and a conditional delay at block 3. Collectively, these reveal the
interaction that occurs between processes. In particular, block 11 in Figure 2.10 signals the
idle RESERVATIONIST in block 3 of Figure 2.3 that service is required, whereas the removal
of a CUSTOMER from the QUEUE in block 4 of Figure 2.11 signals the waiting CUSTOMER
in block 10 of Figure 2.10 that service is about to begin.
In a simulation based on the process-interaction approach, the timing routine does
more work than in one based exclusively on event scheduling. As Figure 2.12 shows, this
extra work arises from the need to manage the conditional event set as well as the future
events list. For conceptual simplicity, this depiction relies on a single CES. Here, block 12 is
equivalent to a scheduling statement, whereas reactivating a process in block 5 is equivalent
to transferring control to a particular instance of a process. Also, after selecting a scheduled
process from the FES in block 3 and executing it in block 5, the timing routine scans the CES
in block 1 to identify any processes whose conditions for continued execution are now met.
If found, it removes these processes, one at a time, from the CES in block 11 and files them
 2.5. ACTIVE PROCESSES AND PASSIVE RESOURCES 57

Schedule 3 Select and 4 Advance time

creation of initial remove next to desired
processes process from reactivation
FES time

5
Timing Begin Execute process
routine simulation starting at
reactivation
point

12 File process
10 File
inFES with
current time as process
desired ineES
execution time

Produce
summary
statistics

Figure 2.12 Prototypical timing routine, process-interaction approach

in the FES with current simulated time as the desired execution time (block 12). As with
Figure 2.3, we emphasize that Figure 2.12 describes a prototypical timing routine. Other
designs may exist that manage processes in a different, but logically equivalent, manner.

2.5 ACTIVE PROCESSES AND PASSIVE RESOURCES

Perusal of Figures 2.10 and 2.11 reveals that the CUSTOMER process provides the stimulus
to action, whereas the RESERVATIONIST process merely responds to the demand of a newly
arriving CUSTOMER (block 11 of Figure 2.1O) or to the status of the QUEUE (block 2 of
Figure 2.11}. More generally, the demand for resources in the process-interaction approach
induces a sequence of steps very much akin to those that Figure 2.11 depicts.
Recognition of the commonality of these resource-related steps across simulations
based on the process-interaction approach has led to simulation software with macrostate-
ments that account for this behavior. For example, most simulation programming languages
have SEIZE or REQUEST commands that "activate" a resource and RELEASE commands
that indicate the completion of service and provide a signal to resources to seek out other
58 CHAPTER 2. MODELING CONCEPTS

work to do. In particular, these and like simulation statements do away with the need to
model the RESERVATIONIST process explicitly as in Figure 2.11.
The concept of active processes and passive resources originated with GPSS (Gordon
1969). Its appeal has been so great that virtually every simulation programming language
based on the process-interaction approach embraces it. Section 4.21 illustrates the concept
for SIMSCRIPT 11.5, and Sections 4.24 through 4.25 for SIMAN.
If its appeal is so great, why do we bother to describe the option in which
both CUSTOMER and RESERVATIONIST are active processes? Flexibility is the answer.
Macrostatements such as SEIZE and RELEASE correspond to recipes for performing a mul-
tiplicity of steps in sequence. They serve a constructive purpose, provided that the problem
at hand has a logic flow that corresponds to that in the statements. If it does not, the simula-
tionist has two options, to ignore the difference and take advantage of the macrostatements,
or to write her/his own sequence of statements in the selected language to perform the steps
required in the problem at hand. Our discussion provides this guidance for effecting the
latter option.

2.6 TELECOMMUNICATIONS NETWORK MODELING

Previous sections concentrate on the flow of discrete objects through a delay system. Each
object is identified and its path tracked from entry to exit. In principle, telecommunications
systems also fit this characterization. A packet (object) is transmitted (flows) from an origin
node (entry) to a destination node (exit). En route it visits a sequence of nodes (stations),
collectively called a route. Each channel or link connecting a pair of nodes has an upper limit
on bandwidth (service rate), and each node has a buffer with finite capacity where en-route
messages wait until selected for transmission.
Discrete-event simulation is often used to study how to set buffer sizes and node
retransmission rates in order to control message loss, due to encountering a buffer with no
waiting room, and to evaluate buffer utilization. However, major differences exist between
these simulation models and those that arise in the job-shop settings described earlier. In
particular, the volume of messages is so large that any attempt to identify and track each within
an executing simulation generally meets with failure. To resolve this problem, messages are
viewed as part of a fluid that flows from node to node. While the concept of discrete events
is preserved, the fluid representation considerably reduces the computational burden, thus
allowing a simulationist to study aggregate behavior without tracking its atomistic parts. This
section describes a prototypical telecommunications network simulation model. It shows
how the concepts of discrete-event modeling apply and notes those features generic to most
telecommunications networks encountered in practice. Section 10.1 returns to the topic of
fluid-flow arrivals.
Consider a system of k buffers in series as in Figure 2.13 with r exogenous sources
putting fluid into buffer 1 at rates aj (r), ... , a r (r) at simulated time r. Each source j
 2 .6 . TEL E COM M U N I CAT ION S NET W 0 RK MOD ELI N G 59

(x,

~
----
(Xl

P2 Pk
~ ------~
(X)
0.

(X,
~ 02

Figure 2.13 Fluid flow network

changes rate at discrete points in time. The times between changes for a particular source
may be Li.d. random variables, in which case they are sampled from a probability distribution.
Alternatively, they may be deterministic with possibly constant increments. The simplest rate
change structure has Ct j (r) alternating between a low value Ct jO and a high value Ct jl; but
other behavior is possible.
For each buffer i = 1, ... , k, let

Ci := capacity,
Qi (r) := volume occupied at time r,
ri,next := time of next input rate change,
Yi-I := maximal allowable outflow rate from buffer i - I into buffer i,

Buffer 0 is taken to be the exogenous source,

The aggregate exogenous input rate to buffer 1 is

fh (r) = min [Yo, Ctl (r) + ... + Ctr(r)].

Fluid flows out of buffer 1 at rate

ifQI(r) =0,
if QI(r) > 0,

and this is the inflow rate to buffer 2. More generally, the outflow rate from buffer i and the
inflow rate to buffer i + 1 for i = 1, .. . , k is

if Qi(r) = 0,
if Qi(r) > O.

Note that f3k+ I ( r) is the outflow rate of the network at time r.

Ifr < ri,next and f3i (r) > f3i+1 (r), then

[f3i(r) - f3i+I(r)] (ri,next - r) + Qi(r) > Ci,

60 CHAPTER 2. MODELING CONCEPTS

and buffer i overflows at time

and fluid is lost to the system. If r < ri,next and PHI (r) > Pi (r), then buffer i empties at
time

so that Qi (re) = O.
In this system the design parameters are the buffer capacities CI, •.. , Ck and possibly
the maximal outflow rates Yb ... , Yk. Over a simulated time interval [s, t), the sample
average unused capacity

1
B(s, t) = t _ sis
t t; [Ci -
k
Qi(r)] dr

and the sample loss rate

estimate performance. For each buffer i the sample average unused capacity

1 t
Bi(s, t) = t _ sis [Ci - Qi(r)] dr

and sample average unused bandwidth

may also be of interest.

More complex models exist, some with thresholds and volumes for each buffer that
indicate when to reduce inflow rates to limit the frequency ofoverflow or the buffer becoming
empty. These are more in the spirit of dynamic control problems in which interest focuses
on how the choice of threshold affects unused capacity and loss rate. Other models consider
the flow of several different fluids. For example, see Kumaran and Mitra (l998).
Four elementary events characterize our model:
1. Change in buffer inflow rate

2. Change in buffer outflow rate

 2.6 . TEL E COM M U N I CAT ION 5 NET W 0 RK MOD E LJ N G 61

FULL

EMPTY

Figure 2.14 Fluid-flow compound events

3. Buffer becomes full

4. Buffer becomes empty.

Note that at time r, fJI (r) denotes the exogenous rate to buffer 1; for i = 2, ... , k, fJi (r)
denotes the inflow rate to buffer i and the outflow rate for buffer i - I ; and fJH I(r) denotes
the output rate from the network. If, as in some models, controls were applied to outflow
rates, then the twofold definitions of fJ2 (r ), ... , fJk ( r) would not apply.
The elementary events form the compound events in Figure 2.14. These compound
events reveal how the event-scheduling approach can conveniently represent the iterative
updating of inflow-rate changes that potentially can occur at all buffers j ::: 1 when the
exogenous flow rate changes, or at all buffers j ::: i when buffer i becomes empty. Notice
that executing an overflow event allows for recording when overflow begins.
Figure 2.15 provides more details. Because fluid flows continuously, it is necessary
to schedule events to handle fluid exhaustion and overflow. The EMPTY event updates a
change in a buffer's outflow rate to reflect the fact that fluid can exit an empty buffer at a rate
no greater than the rate at which it enters it. In the current setting, the FULL event merely
acknowledges that a buffer has become full and fluid (messages) will overflow and be lost.
In other operating environments, a decision may be made to reduce the inflow rate prior to
or at overflow. This action calls for rate changes in previous as well as successive buffers. The
FULL event here allows changes only in successive buffers.
The RATE event executes every time an exogenous or endogenous rate change occurs.
It introduces the reader to a new concept, namely, a cancellation. When fJi (r) changes at time
62 CHAPTER 2. MODELING CONCEPTS

Compute Yes
Detennine
i=l? time to next
Qi exogenous
rate change

Cancel
FULL Yes FULL Schedule
for scheduled? next RATE
BUFFERi for
BUFFERi

Cancel
EMPTY
No Yes for
:>'-----r--=, scheduled?
BUFFERi

No Yes

Schedule
EMPfYat Schedule
't'e='f+~ FULL
Yi-Pi for BUFFER i

Yes
i > k?

No
(a) RATE event

Figure 2.15 Fluid flow events (t' := current simulated time, f3i := f3i (t') Vi)

t', the scheduled execution for an already scheduled FULL or EMPTY event may change.
Canceling an event causes the simulation to remove the corresponding event notice from
its FES and, possibly, to schedule a new FULL or EMPTY event. Indeed, this can happen as
many as k - i + 1 times in a RATE event. Routine UPDATE describes the three alternative
ways that a change in flow rate can occur.

2.7 LESSONS LEARNED

• An event denotes a change in state.

• Every discrete-event simulation model is composed of elementary events. These can

be combined to form compound events, leading to the event-scheduling approach. Al-
ternatively, they can be combined into processes, leading to the process-interaction
approach.
• For a particular problem, modeling with the event-scheduling and process-interaction
approaches both lead to identical output for identical input.
 2.7. LESSONS LEARNED 63

Event
EMPTY

Schedule
Record RATE for
overflow BUFFERi+l
now

Select
next
next event
event

(b) FULL event (c) EMPTY event

No Yes

Yes No

Generate
new

Done

Cd) UPDATE routine

Figure 2.15
64 CHAPTER 2. MODELING CONCEPTS

• During execution of program code for an event, simulated time remains fixed.

• During execution of program code for a process, simulated time may advance. Indeed,
execution may terminate within the code, and after an elapsed simulated time execution
may begin again at a reactivation point in the code.

• An event notice with an attribute for desired execution time is used to schedule
execution.

• An event or process notice with attributes for desired execution time and reactivation
point is used to schedule execution.

• Every discrete-event simulation language keeps its event notices in a future-event set
(FES).

• Every discrete-event simulation language has a timing routine that operates on the FES
to cause the simulation to execute events in the desired order.

• The process-interaction approach also has some form of a conditional-events set that
keeps track of when conditions are met that allow execution to continue for each
process.

2.8 EXERCISES
GENERAL INSTRUCTIONS

For each exercise modeled with the event-scheduling approach, define all entities,
attributes, lists, and elementary events, as well the compound events they form. For each
exercise modeled with the process-interaction approach, define all entities, attributes, lists,
elementary events, and the processes formed from them.

QUARRY-STONE CRUSHING

iff? 2.1 Hard Rock Corporation owns a quarry and three power shovels that load its truck
with stone to be delivered to its crusher. The distances between each shovel and the crusher
vary. Two 20-ton and one 50-ton trucks are assigned to each shovel. A shovel loads its trucks
in first-come-first-served order. However, unloading at the crusher occurs first based on
truck size, with largest going first and then on a first-come-first-served basis. Table 2.1 gives
loading, dumping, and travel times, where a + £(b) denotes the sum of a constant a and
an exponentially distributed random variable with mean b. Once unloaded, the stone is
crushed at the rate of 25 tons per minute.

a. Ignoring waiting times, determine the long-run arrival rate to the crusher of a
randomly selected truck in tons per minute.
 2.8. EXERCISES 65

Table 2.1 Loading, dumping, and travel times (minutes)

Truck Loading Dumping Travel to or from shovel

type time time 2 3
20-ton 3.5 + £(1.5) 1.5 + £(.5) .25 + £(.75) 2.5 + £(1) 1.75 + £(.5)
50-ton 8 + £(2) 2.8 + £(1.2) .9 + £(.30) 2.9 + £(.8) 1.9 + £(.55)

b. Determine the long-run unloading rate at the crusher of a randomly selected truck in
tons per minute.
c. If this were an open-loop system that generates stone to the crusher as fast as the
shovels can load an unlimited number of trucks two-thirds of which are 20-ton and
one-third of which are 50-ton, would it be stable? Explain your answer in detail.
d. Explain why travel to and from the crusher does or does not playa role in part c.

e. Model the problem using flowcharts based on the event-scheduling approach.

f. Model the problem using flowcharts based on the process-interaction approach.

t}1 2.2 A well-established demand exists for Hard Rock's crushed rock at A dollars per
ton. Management knows that the crusher can process more stone than is currently supplied to
it. To boost actual throughput, Hard Rock considers allowing an outside contractor to supply
additional stone to the crusher in 50-ton loads. The interarrival times between successive
loads would be i.i.d. from £(1/A), where A is to be determined by Hard Rock. Regardless of
origin, 50-ton trucks receive priority over 20-ton trucks, among 50-ton trucks selection is
based on first in first out, and among 20-ton trucks it is based on first in first out. External
trucks have unloading times 1.5 + £(1) in minutes.
Crushing costs B = O.SOA dollars per ton. Hard Rock provides stone to the crusher
at a cost of C = O.2SA dollars per ton. The outside contractor offers to provide rock at
D = O.26A dollars per ton. In order to maximize profit, Hard Rock management wants to
know what value to assign to A. Assume A = 32.

a. Determine the smallest rate A* such that the system would be unstable if A ::: A*.
b. Model the problem with external as well as internal input using flowcharts based on
the event-scheduling approach.

c. Model the problem with external and internal input using flowcharts based on the
process-interaction approach.

LOADING TANKERS

t}1 2.3 Tankers take on crude oil in an African port for overseas shipment. The port
has facilities for loading as many as three ships simultaneously. Interarrival times between
66 CHAPTER 2. MODELING CONCEPTS

Table 2.2 Tanker Types

Type Relative frequency Loading time (hours)
1 0.25 9(1.5, 12)
2 0.55 9(2.4, 10)
3 0.20 9(.75,48)

tankers follow cell), where £(0) denotes the exponential distribution with mean 0 hours.
Three types of tanker exist. Table 2.2 describes their characteristics, where g(a, fJ) denotes
the Gamma distribution with scale parameter a and shape parameter fJ (note that the mean
is afJ).
One tug serves the port. All tankers require the tug to move them to and from the
berths. When the tug is available, berthing (an empty tanker) time follows £(1), whereas
deberthing (a full tanker) time follows £(1.25). When not towing a tanker, tug transit to
pick up a tanker is called deadheading and takes time drawn from £(0.75).
Upon arrival at the berths the tug next services the first tanker in the deberthing queue.
If this queue is empty, it services the first tanker in the berth queue in the harbor, provided
that a berth is available. Upon arrival in the harbor, the tug services the first tanker in the
berth queue. If the queue is empty, it services the first tanker in the deberthing queue. If
both queues are empty, the tug remains idle.
To compete with other oil-loading ports, the port authority plans to institute a rebate
program based on total port time. This is the summation of waiting, towing, and loading
times. If a tanker's port time exceeds 48 but is no more than 72 hours, the authority will pay
its owner a rebate of A dollars. If it exceeds 72 but is no more than 96 hours, the rebate is
3A dollars. In general, if it exceeds 24i but is no more than 24(i + 1) hours, the rebate is
(2i - 1)A. Assume A = 10,000 dollars.
To estimate the cost of this program the authority commissions a simulation study to
characterize port time. The first step is modeling.

a. Show that this delay system is stable.

b. Model the problem using flowcharts based on the event-scheduling approach.

c. Model the problem using flowcharts based on the process-interaction approach.

~ 2.4 Additional Tankers. A shipper considers bidding on a contract to transport crude

oil from the port described in Exercise 2.3 to the United Kingdom. He estimates that shipment
would require five tankers of a particular type to meet contract specifications. Time to load
one of these tankers follows £ (21). After loading and deberthing, a tanker would travel to the
United Kingdom, unload its crude oil cargo, and return to the port. Round-trip time follows
£(240). Before the port authority agrees to accommodate the five new tankers, it wants to
determine the effect of the additional port traffic on tanker waiting time, tug utilization, and
berth utilization.
 2.8. EXERCISES 67

a. Describe how the additional tankers would affect the stability of the overall queueing
system in the port.

b. Augment the model in Exercise 2.3b using flowcharts based on the event-scheduling
approach.

c. Augment the model in Exercise 2.3c using flowcharts based on the process-interaction
approach.

TERMINAL PACKET ASSEMBLY/DISASSEMBLY CONTROLLER MODEL (CHENG

AND LAMB 1999)

iff?l 2.5 The control of packet assembly and disassembly is a topic of major interest to all
who provide communications network services to industry, government, and the public at
large. The problem described here is considerably simplified to allow students to work on it
without the impediments that very high transmission rates and network size impose. How-
ever, the issues that arise in the scaled-down version continue to be principal considerations
in higher-speed problems as well.
A terminal packet assembly/disassembly (PAD) controller model contains r input
buffers each of capacity m and one output buffer of infinite capacity. The PAD controller
receives characters from r = 10 terminals, one for each input buffer, and assembles them
into packets before transmitting them into a network. The characters from each terminal
arrive at its input buffer and wait. A packet is formed when either an input buffer becomes
full or a special control character arrives, whichever is first. If present, the control character is
included in the packet. When a packet is formed, it is tagged with a fixed number of overhead
characters and sent to the output buffer, where it waits in a FIFO queue. The output buffer
serves all input buffers and, if there are any to transmit, puts out characters at a constant rate
of one per time unit. The input characters into each buffer form a Poisson process with rate
A. Assume that r is constant, so that P = r A is the arrival rate of characters into the PAD
controller.
The probability that an arriving character is a special control character is q := .02.
Also, one overhead character is added to a packet when it moves from the input buffer to
the output buffer.
The purpose for studying this model is to identify the rate A and input buffer capacity
m that minimize

J.-t(m, A) := average character delay in the PAD controller model between

(2.6)
arriving at an input buffer and being transmitted to the network

subjectto the constraints .30 .::: P .::: 1 andm E {8, 9, ... , 32}. Note that J.-t increases without
bound as either P ~ 0 or P ~ Po < 1, where Po := system-saturation input rate.

a. Determine the mean packet size as a function of m and q.

68 CHAPTER 2. MODELING CONCEPTS

b. Determine Po.
c. Identify all elementary events and use them to form compound events.
d. Using the event-scheduling approach, prepare flowcharts that describe the details of
the events in this problem.

e. Use the elementary events to form processes as in Figure 2.9.

f. Using the process-interactive approach, prepare flowcharts that describe the details of
the processes in part e.

MOBILE·TELEPHONE SERVICE (DALEY AND SERV11999)

(iF 2.6 Friendly Communications Incorporated provides mobile and standard telephone
services by means of a collection of communication towers distributed throughout a region.
A customer traveling through the region while using the mobile service automatically has
her/his call handed off and managed (transmission and reception) by a sequence of towers
en route. At any moment in time, the tower providing this service to the customer is the one
geographically closest to him or her. Each tower can handle m calls at a time.
Each tower has the capacity for handling a total of m standard and mobile calls simul-
taneously. For a traveling mobile caller, there is always the possibility that when her/his call
is handed off from one tower to the next, that next tower is already serving m callers, and
hence the mobile caller loses the connection. This loss is highly undesirable.
Assume that the times between successive new standard calls for a tower are LLd.
random variables from £ (1/ As), whereas the times between successive handoffs of mobile
calls to the tower are i.i.d. from £ ( 1/Ah ). Also, assume that call-duration times are i.i.d. from
£ (1/w) and independent of type.
To manage loss the company decides to dedicate r (::::: m) lines for handoffs. In par-
ticular, let X s ( r) denote the number of standard call and Xh ( r) the number of mobile calls
at time r. Let r _ denote the instant of time prior to a new demand. The operating policy is:

• If a new request for standard service arrives at time r, it is provided if Xs (r) < m - r.
Otherwise, service is denied and the call is lost.
• If a new mobile call handoff request occurs at time r, it is assigned a dedicated line if
Xs(L) < r. If Xh(L) > rand Xs(L) + Xh(L) < m, it is assigned one of the m - r
nondedicated lines.
• If Xs (r _) + Xh (r _) = m, the mobile call is lost.
The purpose of this study is to evaluate how the choice of the number of dedicated
lines r affects

fih (m, r) := probability of denying service to a handed-off mobile call,

 2.9. REFERENCES 69

f3s (m, r) := probability of denying service to a new standard call,

and, in particular, the cost oflosing calls. If the cost oflosing a handed-off call is unity and
the cost of denying service to a new standard call is c( < 1), then it is of particular interest to
determine the r that minimizes long-run average total cost

(2.7)

a. Determine the traffic intensity for this problem.

b. Identify all elementary events and form compound events from them.

c. Using the event-scheduling approach, prepare flowcharts that describe the details of
the events.

d. Use the elementary events of part b to form processes as in Figure 2.9.

e. Using the process-interaction approach, prepare flowcharts that describe the details
of the processes in part d.

2.9 REFERENCES

Cheng, R.C., and J.D. Lamb (1999). Making efficient simulation experiments interactively
with a desktop simulation package, University of Kent at Canterbury, United Kingdom.

Daley, D. and L. Servi (1999). Personal communications.

Fishman, G.S. (1973). Concepts and Methods in Discrete Event Digital Simulation, Wiley, New
York.

Gordon, G. (1969). System Simulation, Prentice-Hall, Englewood Cliffs, N.J.

Pritsker, A.A.B., J.J. O'Reilly, and D.K. LaVal (1997). Simulation with Visual SLAM and
AweSim, Wiley, New York.
 CHAPTER

G. S. Fishman, Discrete-Event Simulation

© Springer Science+Business Media New York 2001
Data Collection
and Averages

In simulating a discrete-event delay system, a principal objective is to evaluate its performance

with regard to throughput, buffer occupancy, delay, resource utilization, and system loss.
Throughput denotes the rate at which a system provides finished product. As examples,
throughput in a manufacturing setting denotes the number of finished goods that a factory
produces per unit of time. In a telecommunications network, the rate at which messages pass
from origin to destination constitutes a measure of throughput. The number of vehicles that
cross a bridge per unit time characterizes throughput in a particular type of transportation
system. Buffer occupancy denotes the number of items in the system awaiting processing or
service and is sometimes called queue length. Delay denotes the length of time that an item
waits and alternatively is called waiting time. Occasionally, system time characterizes delay.
It is the sum of waiting time and service time. Resource utilization denotes the proportion of
system resources that are kept busy processing items.
System loss rate sometimes serves as an additional performance indicator. It measures
the rate at which objects are lost to or leave the system without successfully completing their
purposes when they encounter delays exceeding specified tolerances. For example, work-
in-process within a manufacturing system may lose value unless delays between successive
steps in the production process are kept within specified limits. This deterioration can arise
72 C HAP TE R 3. DA T A COL LEe T ION AND Av ERA G E 5

in some forms of semiconductor wafer fabrication as a consequence of the properties of

materials and the production process.
To evaluate performance, a discrete-event simulation uses data generated during ex-
ecution to compute finite-sample time averages and arithmetic averages as approximations
to the unknown true long-term averages that characterize system performance. Section 3.1
describes these averages and, in particular, the recursive forms that they customarily take.
The remainder of the chapter provides the reader with a preview of the statistical issues that
arise when assessing how well these finite-sample averages approximate their corresponding
long-term averages. Among these issues, the most notable are:

• Choosing the initial state (S (0)) in which to begin simulation

• Choosing a warm-up interval (k - 1) that is to elapse before data collection begins in

order to dissipate the influence of the initial state of the simulation on data used to
compute the finite sample averages.

• Choosing a run length (t ) that ensures that the finite-sample averages are representative
of the unknown true long-term averages.

Sections 3.2 through 3.5 explain the significance of S (0), k - 1, and t, leaving the issues of
choice until Chapter 6, which gives a considerably more detailed account of these issues.
The account of computational forms in Section 3.1 aims to give the reader an un-
derstanding of the data-accumulation techniques that simulation programming languages
often provide implicitly. For the cases where such techniques are not provided, the account
provides sufficient information for a user to program the steps explicitly. Subsequent sec-
tions aim at sensitizing the reader to the important role that statistical issues play in assessing
the quality of the numbers that a simulation generates as output. This topic has many facets,
and any simulation user who contemplates presenting simulation output to an audience
and convincing them that the output truly characterizes the sysem will benefit first from an
understanding of what the statistical issues are (Sections 3.2 through 3.5) and second from
an awareness of how to resolve them (Chapter 6).

3.1 PERFORMANCE STATISTICS

We again use the notion of Section 1.3. For t > s ::: 0, let

A (s, t) := number of arrivals in time interval (s, t 1, (3.1)

N (s, t) := number of completions in time interval (s, t 1, (3.2)
Q(t) := queue length at time t, (3.3)
B (t) := number of busy servers at time t, (3.4)
Wi := waiting time of departure i, (3.5)
 3 .1 . PER FOR MAN C EST A TIS TIC 5 73

from which follow the accounting identities

A(s, t) = A(O, t) - A(O, S),

N(s, t) = N(O, t) - N(O, S),

and

A(O, t) - N(O, t) = Q(t) + B(t). (3.6)

Occasionally, we use the more concise notation A(t) := A(O, t) and N(t) := N(O, t).
A discrete-event simulation program is an engine for producing sample paths {A ( r) -
A(s), s :::: r :::: t}, {N(r) - N(s), s :::: r :::: t}, {Q(r), s :::: r :::: t}, {B(r), s :::: r :::: t},
and {Wi; i = N(s) + 1, ... , N(t)}. These sample paths, sometimes called sample records or
realizations, provide histories of how the underlying processes in the simulation behave over
the finite time interval [s, t]. It is these histories that provide the raw data for computing
the summary sample statistics:

• average arrival rate

- A(s, t)
A(s,t):= ------, (3.7)
t-s

• average throughput

-( )._N(s,t)
N s,t .- - - - , (3.8)
t-s

• average queue length

Q(s, t) := - 1
t - s
It
s
Q(u)du, (3.9)

• average server utilization

R(s, t) := -1-
t - s
It
s
B(u)du, (3.10)

• and average waiting time

_ 1
L
N(t)
W(s, t) := - - - Wi. (3.11)
N(s, t) i=N(s)+l
74 CHAPTER 3. DATA COLLECTION AND AVERAGES

Note that expressions (3.7) through (3.1O) are time averages, whereas expression (3.11) is an
arithmetic average. Sometimes Q(r} and B(r} are called time-persistent variables. Sections
3.1.1 and 3.1.2 explain why this name is applied.
The sample paths contain considerably more information than these averages alone can
convey. A sequence of exceedance proportions often provides supplementary information
of value. Let

ifx E A,
otherwise.

Then the system also has the following sample performance indicators:

• proportion of time that queue length exceeds q

_
PQ(q, s, t} := -
1 [t I(q,oo)(Q(u}}du, q = 0,1, ... , (3.12)
t - s s

• proportion of time that more than b servers are busy

_
PB(b, S, t} := - -
1 [t I(b,oo) (B(u»du, b = 0,1, ... , (3.13)
t- s s

• and proportion of customers who wait more than w time units

_ 1 N(t)
Pw(w, s, t} := - (- )
N s, t
L
i=N(s)+l
I(w,oo)(Wi }, w > 0. (3.14)

While the averages (3.7) through (3.11) show central tendencies, the proportions (3.12)
through (3.14) can provide a penetrating view of extreme system behavior. A table or graph
of {PQ(q, s, t}; q = 0,1, ... } reveals how likely queue length is to be large. It is par-
ticularly useful in sizing the buffer or waiting room where jobs wait. A table or graph of
{PB(b, s, t}; b = 0,1"", m - I} reveals how well utilized all the servers are. A table or
graph of {Pw (w, s, t), W E {WI < Wz < ... }} characterizes the delay that an arbitrary
arrival to the system encounters. In practice, graphical display of these distributions enables
one to draw meaningful conclusions more quickly than tabular displays do.

3.1.1 QUEUE LENGTH

While concise with regard to mathematical notation, expressions (3.7) and (3.14) give
no clue as to how to compute these averages in practice. This section addresses this issue. Let

J Q (t) := number of changes in queue length in simulated time interval (0, t 1,

 3.1 . PER FOR MAN C EST A TIS TIC 5 75

tQi := time of the ith change in queue length for i = 1, ... , lQ(t),
tQo := O.

Suppose a simulation is run for the simulated time interval [0, S + T], where s ::: 0
and T > 0, and we wish to compute the average queue length over the interval [s, s + T].
Expression (3.9) implies

Q(s, s + T) = -
1 [s+r Q(u)du.
T s

Moreover, the property

Q(u) = Q(tQ,i-d,

reveals that {Q(u), s ::: u ::: s + T} is piecewise constant, so that expression (3.9) has the
equivalent form

which leads to the recursion

(3.16)

In addition to average queue length, it is often enlightening to know the proportion

of time that a queue exceeds a particular value during the time interval [s, s + T]. This is
especially so when an objective of the simulation is to identify how large a buffer should be in
76 CHAPTER 3. DATA COLLECTION AND AVERAGES

order to ensure that overflow occurs with a frequency no greater than a specified quantity. In
a manner analogous to that for average queue length, expression (3.12) admits the recursion

- tQ,JQ(s+r:) - s -
PQ(q, s, s + t') = t' PQ(q, s, tQ,JQ(s+r))

t' +s - tQ JQ(s+r)
+ t'
. x I(q,oo) (Q(tQ,JQ(s+r)))'

Virtually all discrete-event-simulation programming languages use the recursions

(3.16) and (3.17) to compute running time averages for queue length as simulated
time evolves. For the time average (3.9), recursive computation requires retention
of four values, tQ,i-l> tQi' s, and Q(s, tQ,i-d, to calculate Q(s, tQi). For exceedance
proportions (3.12) corresponding to queue lengths ql < ... < qk> it requires reten-
tion of 2 + k values, tQ,i-l> tQi' s, P Q (ql, s, tQ,i-d, ... , PQ(qk. s, tQ,i-d, to calculate
PQ(ql> s, tQi), ... , PQ(qk. s, tQi).

3.1.2 RESOURCE UTILIZATION

Computing averages for resource utilization is analogous to that for queue length. Let

JB (t) := number of changes in the number of busy RESERVATIONISTs

in simulated time interval (0, tl

and

tBi := time of the ith change in the number of busy RESERVATIONISTs for

i = 1, ... , JB(t).

Since

B(u) = B(tB.i-l), tB,i-l ::::: u < tBi, i = 1, ... , JB(t),

 3.1. PER FOR MAN C EST A TIS TIC 5 77

we see that {B(u), s :::; u :::; s + T} is also piecewise constant in time. Therefore, time average
(3.lO) has the alternative representation

_ 1 [ JB(s+r)
B(s, s + T) = - L B(tB,i-l)(tBi - tB,i-l) + B(tB.JB(s+r))
T i=JB(S)+l

x (s + T - tB,la(s+r)) - B(tB.JB(S))(S - tB.JB(S))} (3.18)

so that

For b = 0, 1, ... , m, exceedance proportions over [s, s + T] take the form

In closed-loop systems, resource utilization may also characterize throughput. For

example, at time u ::: 0 let B (u) = 1 indicate that the crusher is busy crushing and B (u) = 0
otherwise in Excersie 2.1. Since the crusher is capable of crushing 25 tons per minute,
25B(s, s + T) is the finite-sample average throughput over the interval (s, s + T].

3.1.3 WAITI .... G TIME

Computing the averages in expressions (3.11) and (3.14) for waiting time calls for a
different approach than for queue length and resource utilization. Whereas the sample paths
of queue length {Q (T), s :::; T :::; t} and resource utilization {B (T), s :::; T :::; t} are indexed
78 C HAP TE R 3. DA T A COL LEe T ION AND Av ERA G E S

onthecontinuousvariabler, the sample path of waiting times {W;; i = N(s)+I, ... , N(t)}
is a discrete collection of observations. Moreover, whereas a simulation needs to keep track
of the number in queue and number of busy servers to compute the averages (3.9), (3.10),
(3.12), and (3.13), it must now keep track of the arrival timefor each job in the system and
subtract it from the simulated time at which the job enters service in order to compute its
waiting time. Making arrival time an attribute of the temporary entity corresponding to job
or CUSTOMER in the airline problem allows for this last calculation.
To conserve on space for data storage, one can again resort to recursion. Let

i=N(s)+I,
(3.21)
i = N(s) + 2, ... , N(s + r),
and

i = N(s) + 1,
(3.22)
i =N(s)+2, ... ,N(s+r).

Then for each simulated time t E ([s, s + rl : N(s + r) > N(s)},

W(s, t) = VN(t)-N(s) (3.23)

N(t) - N(s)

and

P- ( )_ VN(t)-N(s)(W)
(3.24)
ww,s,t- () ().
Nt - N s

Section 3.5.1 describes a relationship known as Little's law that relates the arithmetic
average (3.11) for waiting time to the time average (3.9) for queue length, thus reducing the
amount of data required to compute both averages.

3.2 STATISTICAL ACCURACY

In principle, every simulationist wants:

• Estimates oflong-term averages uninfluenced by the initial conditions prevailing at

the beginning of simulation.

• Estimates whose statistical accuracy improves as the length of the sample record, on
which the estimates are based, increases.

• An assessment of how well the sample averages approximate their corresponding

unknown long-term averages.
 3 .2 . S TAT 1ST I CAL Ace U RAe y 79

Virtually every simulation programming language provides special capabilities for

computing time and arithmetic averages and producing reports that display them. Indeed,
some languages do this automatically for performance measures such as utilization, queue
length, and waiting time, often producing histograms as well. While these conveniences
reduce or eliminate the need for the simulation user to generate results, it in no way addresses
the three just-mentioned statistical issues; nor for a simulation executed for two different
sets of numerical values assigned to its input parameters does it provide a basis for evaluating
the extent to which the difference between the two resulting time averages, say for queue
length, is due to the differences in input values and the extent to which it is due to sampling
variation. Without this ability to identify the source of output differences, a simulationist
has no scientific basis for explaining how variation in input affects variation in output.
With rare exceptions, most simulation programming languages provide limited guid-
ance for performing statistical analyses. The traditional reason for this limitation has been the
difficulty in automating these analyses and making them accessible to simulation users who,
by justifiable inclination, focus on final results rather than on the intricacies of method. This
out-of-sight-out-of-mind approach can lull a simulationist into a false sense of confidence
about how meaningful a simulation's output is. In others, it can arouse suspicion when they
are asked to accept estimates oflong-term averages without some indication of how well the
estimates approximate the true values of these system parameters. However, techniques do
exist that considerably facilitate one's ability to incorporate subprograms into a simulation
to perform the desired assessment with minimal effort on the part of the simulation user.
Chapter 6 describes the statistical theory that underlies one of these approaches, the
batch-means method, and shows how to incorporate it into a simulation program. Although
we can safely defer detailed discussion of this topic until after we have an executable program,
some familiarity with the statistical properties of stochastic time averages is essential at this
point to understand the elements of the executable programs in Chapter 4. These properties
relate to the initial state, the warm-up interva~ and the sample-path length, and this chapter
provides this familiarity.

3.2.1 SYSTEM STATE

Recall that the sample path for a simulation run results from the repeated operation
of the timing routine on the simulation's future-event set (FES) as simulated time T evolves.
At time T, the FES dictates which event occurs next and when to execute it. When, as
frequently occurs, execution of an event includes the scheduling of a future event, it provides
a new entry in the FES on which the timing routine eventually operates. While interaction
between the timing routine and the FES and interaction between an executing event and
the FES completely characterize the way in which simulated behavior evolves through time,
it provides this characterization with an emphasis on computational issues. An alternative
approach accomplishes this same objective by focusing on the state of the system. Introducing
this approach here enables us to shift the focus of discussion from computational to statistical
considerations while preserving all essential features of a simulation's dynamical behavior.
80 CHAPTER 3. DATA COLLECTION AND AVERAGES

It is these statistical considerations that determine how well finite-sample averages, as in

expressions (3.7) through (3.14), characterize long-term behavior.
The state of a simulated system attime rl (> 0) conveys information that, together with
one or more random innovations, determines the state of the system at each future time
r2 > rl. More often than not, the probability laws governing these random innovations
depend explicitly on the state of the system at time rl. As illustration, the airline reservation
problem has the state vector

S (r) := (IB(r), Ie(r), S(r)), (3.25)

where

IB(r) := {i l , ... , iB(r)} = set ofbusyRESERVATIONISTs, (3.26)

Ie( r) := {h, ... , jCI'i} = set of CUSTOMERs listening to recorded (3.27)
message,
Q(r) queue length
B(r) no. of busy RESERVATIONISTs
C(r) no. of CUSTOMERs listening
to recorded message
SA(r) remaining time to next CALL
SRi] (r)
S(r) = remaining service times for (3.28)

SBiBI'i (r)
} the busy RESERVATIONISTs

SCj] (r) remaining message times

for CUSTOMERs listening

SCkl'i(r)
} to the recorded message,

and

C(r) := A(r) - N(r) - Q(r) - B(r). (3.29)

A little thought shows that a sample path {S (s), 0 :::: s :::: r} also provides sufficient
information to construct the corresponding sequence of waiting times {WI, ... , WN(r)}'
If the last event occurred (executed) at time r, then the next event or events execute at

r '(r) = r + t.(r), (3.30)

 3.3. CON D I T ION A LAN D EQUI LIB RI U M DIS T RIB UTI 0 N 5 81

where

If

then a CALL event occurs at time r'. If

then a COMPLETION event occurs at time r' for RESERVATIONIST i for an i E IE (r). If

then an END-11ESSAGE event occurs for CUSTOMER j at time r' for a j E Idr).
We immediately see that the clock attributes of the event notices in the FES at time r
are in one-to-one correspondence with the elements of the set

and that the ordering of the FES for execution is identical with this set, ordered from small
to large.
Hereafter, we frequently refer to S (r) as the state vector or the state at time r. Note
that the number of elemental entries in S (r) varies with r. In particular, these changes in
number can occur only at r '( r) for each r. Also, note that S ( r) contains both continuous
and discrete random variables as entries.

3.3 CONDITIONAL AND EQUILIBRIUM DISTRIBUTIONS

The centrality of {S (s), 0 :s s :s r} with regard to the future-event set and the determina-
tion of {B(s), 0 :s s :s r}, {Q(s), 0 :s s :s r}, and {Wi; i = 1, ... , N(r)} in Section 3.2.1
enable us to unify our approach to understanding the statistical properties of the sample
averages, as in expressions (3.7) through (3.14). Most critical in this assessment is the role
of the probability distribution of the random state vector S (r) and the behavior of this
distribution as a function of r . As before, we use the airline reservation problem to illustrate
concepts, but emphasize that the general case demands relatively minor modification.
82 CHAPTER 3. DATA COLLECTION AND AVERAGES

3.3.1 I.... ITIAL CO.... DITIO....S

To execute a simulation run, we must first assign values to S (0), using either a de-
terministic or a random rule. We do this indirectly by establishing initial conditions that
are to prevail at simulated time zero. For example, suppose we start the airline reservation
simulation with a CALL at!' = 0 to an empty and idle system. Then the logic of this problem
dictates the initial vector state

S (0) = {IB(O), IcCO) , S(O)},

where

IB(O) = lid,
IcCO) = 0,

and S(O) has the entries

Q(O) = 0,
B(O) = 1,
CeO) = 0,
SA (0) = time at which second CALL occurs,
SBi! (0) = service time for first completion.

Also,

Alternatively, if we were to start the simulation with a CALL scheduled in a time units, three
busy RESERVATIONISTs with remaining service times of b, c, and d time units, and no
scheduled END-MESSAGEs, then S (0) would have the entries

IB(O) = {il, i 2, i 3},

Ic(O) = 0,
A(O) = 3,
N(O) = 0,
Q(O) = 0,
B(O) = 3,
C(O) = 0,
 3.3. CONDITIONAL AND EQUILIBRIUM DISTRIBUTIONS 83

SA(O) = a,
SBi] (0) = b,
SBi,(O) = c,
SBi,(O) = d.

Moreover,

r '(0) = min(a, b, c, d).

Let T denote the set of all possible states that a simulation model can realize as simulated
time elapses. Then the choice of initial conditions and the method by which a simulation
user makes this choice induce an initializing distribution

no(A):= pr[S (0) E A], A ~ T, (3.31)

denoted by 7ro. In each of the two examples for the airline reservation problem, 7ro is degen-
erate, putting all its mass on a single state. These are special, but convenient, illustrations.
However, if the simulationist in the second example elects to sample values for the remaining
times to the CALL event and the three COMPLETION events, 7ro would be nondegener-
ate. Note that a simulationist does not literally sample from 7ro; however, her/his method
of assigning initial conditions is probabilistically equivalent to sampling from an implicit
7ro. Recognizing this equivalence enables us to characterize sample-path behavior as time
elapses.
For each r ~ 0 and DE T, let no('; riD) denote the conditional distribution of the
state set S (r), so that

no(A; r I D):= pr[S(r) E A I S(O) = D)], A~T. (3.32)

To facilitate a meaningful statistical analysis on sample-path data, the simulation must endow
this conditional distribution with at least two important properties:

• For every possible initial starting state D E T,

no(A; riD) --+ n(A) as r --+ 00 'VA ~ T, (3.33)

where n (A) is called the equilibrium or steady-state distribution for the subset of states
A in T. In particular, A ~ B ~ T implies 0 ~ n(A) ~ n(B) ~ n(T) = l.

• This limit (3.33) must hold for every initial distribution 7ro on T.
84 CHAPTER 3. DATA COLLECTION AND AVERAGES

Expression (3.33) asserts that as simulated time r elapses, the distribution of the
state vector S (r) converges to a distribution that is independent of the initial conditions
S (0) = V, regardless of which state VET is chosen to start the simulation. Moreover,
this occurs regardless of the distribution 7ro. These are extremely useful properties. Were
expression (3.33) not to hold, then the sample path generated during a simulation run
would have statistical properties that depend on the initial conditions, a feature antithetic to
our objective of approximating long-run averages for utilization, queue length, and waiting
times, which are presumably unique for a given simulation model with specified numerical
values for its input parameters. Unless otherwise explicitly stated, we hereafter assume that
the design of the simulation model ensures that expression (3.33) holds.
Let 7r := InCA), A ~ T}. Ali with 7ro, the interpretation of 7r must be in context.
Ali r increases, the dynamic logic of the simulation model implies that there exists a unique
distribution 7r, independent of S (0), that characterizes the behavior of S (r ). Moreover, this
dynamic logic, not actual sampling from 7r, dictates the value that S (r) assumes at time r.
A simulation user frequently picks the initial state arbitrarily with no randomizing
considerations at all so that 7ro is degenerate. Computational ease in assigning the initial
state and limited knowledge usually dictate its choice. However, some states turn out to be
better than others for the initial state inasmuch as they induce faster convergence to the
limit (3.33) and thereby require a shorter warm-up interval before data collection begins.
Section 6.1 returns to this topic.

3.3.2 WARM-UP INTERVAL AND CONVERGENCE

Since sUPA~T Ino(A; s + rlV) - n(A)1 tends to decrease ass (> 0) increases, initial
conditions tend to influence data collected over the interval [s, s + r Jless than data collected
over the interval [0, r J. That is, both sample records have the same sample-path length r,
but the one collected after deleting the warm-up interval [0, s) is affected less by the starting
conditions of the simulation.
Recall that the output of a simulation run includes time averages, as in expressions
(3.7) through (3.14), computed from sample-path data, that serve as approximations to
the long-run averages, queue length, utilization and waiting time. These long-run averages
characterize congestion. In particular, we say that congestion increases (decreases) as average
queue length and waiting time increase (decrease). For fixed simulated times r1 and r2,
the dependence between two state vectors S (r1) and S (r2) tends to increase (decrease) as
congestion increases (decreases). For example, the dependence between queue lengths Q(r 1 )
and Q(r2) increases (decreases) for fixed times r1 and r2 as the traffic intensity increases
(decreases).
Since S (0) denotes the conditions that prevail at the beginning of a simulation run, it is
clear that the choice of an appropriate warm-up interval depends on the extent of congestion
in the system. While we must defer a discussion of how to choose an appropriate warm-up
interval until Chapter 6, we can at this point identify issues that should sensitize the reader
to the role of congestion in analyzing data.
 3.3. CONDITIONAL AND EQUILIBRIUM DISTRIBUTIONS 85

Recall that every discrete-event simulation model has one or more service time distri-
butions to whose parameters a simulationist must assign numerical values before execution
can begin. Moreover, an open-loop system has an interarrival time distribution whose pa-
rameters also require numerical assignments. We call these values inputto the simulation. As
the values assigned to the mean service time, the variance of service time, or in an open -loop
system, to the arrival rate increases (decreases), long-run queue length and waiting time,
and therefore congestion, increases (decreases). Hence, increases in these input values alert
the simulationist to the need to increase the warm-up interval.

3.3.3 QUEUE LENGTH, RESOURCE UTILIZATION, AND CONVERGENCE

Since queue length Q ( r) and number of busy servers B ( r) at time r both are
entries in the state vector, the limit (3.33) has implications for these quantities. Let
'ITQ := {1l'Qq; q = 0, 1, ... } and 'ITB := {1l'Bb; b = 0, 1, ... , m} denote the marginal equilib-
rium probability mass functions (p.m.f.s) of 'IT with respect to queue length and number of
busy RESERVATIONISTs, respectively, where

1l' Qq := probability that q CUSTOMERs awaiting service in the queue

and
1l'Bb := probability that b RESERVATIONISTs are busy.

Then the limit (3.33) implies that for every fixed s,

pr[Q(s + r) = qlS (s)] -7 1l'Qq

and (3.34)
pr[B(s + r) = blS (s)] -?1l'Bb as r -7 00.

While these limits are reassuring, they merely tell us that if one runs a simulation for
a sufficiently long time r, he/she can expect Q(s + r) and B(s + r) to exhibit statistical
behavior that is independent of S (s). That is, the values that Q(s + r) and B(s + r) as-
sume tell us something about congestion, uninfluenced by the conditions that prevailed at
simulated time s. However, our principal interest is in the time averages {PQ(q, s, s + r);
q = 0,1, ... }, {PB(b, s, s + r); b = 0, 1, ... , m - I}, Q(s, s + r), and B(s, s + r), which
characterize the entire sample path. To ensure that these are meaningful approximations of
their corresponding long-term averages, we need more than the limits (3.34).

Let

PQq := 1 - 1l'Qo - 1l'Ql - ..• - 1l'Qq

and
PBb := 1 - 1l'BO - 1l'Bl - .•. - 1l'Bb.
86 CHAPTER 3. DATA COLLECTION AND AVERAGES

Under relatively mild restrictions and for q = 0, 1, ... and b = 0, 1, ... , m - 1,

PQ(q, s, s + r) --* PQq as r --* 00 w.p.I.

and (3.35)
PB(b, s, s + r) --* PBb as r --* 00 w.p.I.

That is, the sample proportions {1- PQ(q, s, s+r); q = 0, 1, ... } converge to {1- PQq; q =
0,1, ... } and the sample proportions {I - PB(b, s, s + r); b = 0,1, ... , m} converge to
{1- PBb ; b = 0, 1, .. . m}.
The notation w.p.1 means "with probability one." It is equivalent to the notation a.e.
for "almost everywhere;' which is also common in the statistical literature. Both imply the
strongest form of convergence that probability theory has to offer. When expression (3.35)
holds we say that PQ(q, s, s + r) is a strongly consistent estimator of PQq and PB (b, s, s + r)
is a strongly consistent estimator of PBb .
While the limits (3.34) are necessary for the sample paths {Q(r), r ::: s} and
{B ( r ), r ::: s} to exhibit equilibrium behavior independent of S (0) as r --* 00, the limits
(3.35) assure us that the time averages PQ(q, s, s + r) and PB(b, s, s + r) tend to become
increasingly better approximations for PQq and PBb, respectively, as the length of the sample
path r increases without bound. Were this not so, data collected on a single sample path
would have little value.
To describe convergence for Q(s, s + r) and B(s, s + r), we rely on their relationships
to {PQ(q, s, s + r), q = 0, 1, ... } and {PB(b, s, s + r); b = 0, 1, ... , m - I}, respectively.
Let

q = 1,2, ... ,
= sample proportion of time r that q CUSTOMERs are in the queue

and

I'l.PB(b, s, s + r) := PB(b - 1, s, s + r) - PB(b, s, s + r), b = 1, ... ,m,

= sample proportion of time r that b RESERVATIONISTs are busy.

Then a minor amount of algebra shows that

L qI'l.PQ(q, s, s + r)
00

Q(s, s + r) = (3.36)
q=l

and

L bl'l.h(b, s, s + r).
m
B(s, s + r) = (3.37)
b=l
 3 .3 . CON D I T ION A LAN D EQUI LIB RI U M DIS T RIB UTI 0 N S 87

To put limiting behavior for Q(s, s + 1') and B(s, s + 1') into perspective, we rely on
yet another pair of equivalent representations,

()()

Q(s, s + 1') = L PQ(q, s, s + 1') (3.38)

q=O

and

m-J

B(s, s + 1') = L PB(b, s, s + 1'), (3.39)

b=O

which follow from expanding and summing like terms in expressions (3.36) and (3.37),
respectively.
By definition,

m m
/-LB := L b:rrBb = L PBb (3.40)
b=O b=O

denotes the mean number of busyRESERVATIONISTs. If the limit (3.35) for PB(b, s, s + 1')
holds for each b = 0, 1, ... , m, then for fixed s and finite m

B(s, s + 1') -+ /-LB as l' -+ 00 w.p.I.

That is, the sample time average B(s, s + 1') converges to a constant that turns out to be the
probability average called the mean number of busy RESERVATIONISTs.
For queue length, an additional property must hold. By definition,

()() ()()

/-LQ:= Lq:rrQq = LPQq (3.41)

q=O q=O

denotes the mean queue length. If the limit (3.35) for PQ(q, s, s + 1') holds for each q =
0, 1, ... and PQq -+ 0 sufficiently fast as q -+ 00, then /-LQ is finite and

Q(s, s + 1') -+ /-LQ as l' -+ 00 w.p.I. (3.42)

The condition that /-LQ be finite is an important one. For a system modeled with finite
waiting room or buffers, it is clearly true that PQq = 0 for q ::: c, where c is the capacity
of the buffer. In this case the condition always holds. However, if a system is modeled with
infinite-capacity buffers, then the finiteness of /-L Q depends on the properties of the random
input. If the interarrival times and service times all have finite moments, then /-LQ is finite,
and as in expression (3.42), Q(s, s+1') approximates /-LQ with an error that tends to diminish
as l' increases.
88 CHAPTER 3. DATA COLLECTION AND AVERAGES

AN EXCEPTION

The preceding discussion assumes that an equilibrium distribution 1r exists and that
tLQ is finite, not unreasonable assumptions for many models of manufacturing, commu-
nications, and traffic systems. However, exceptions exist, and when they occur the concept
of long-term average becomes conjectural and statistical inference based on finite-sample
averages becomes suspect. An example illustrates the problem.
Consider a single-server queueing model with independent exponentially distributed
interarrival times (finite mean and variance) and i.i.d. service times with finite mean but
infinite variance. It is known that an equilibrium distribution (3.34) for queue length exists in
this case but it has an infinite mean. Therefore, the limit (3.42) does not hold. The reason for
this anomaly is not hard to find. While PQq decreases monotonically to zero with increasing
q, it does so too slowly for the infinite sum (3.41) to converge.

3.3.4 FINITE SAMPLE-PATH LENGTH

Since r is finite in practice, B (s, s + r) and Q (s, s + r) merely approximate tL B and

tL Q, respectively, regardless of the value of s. To justify their use in subsequent analyses,
one needs to assess how well they represent their respective long-run averages. For finite r,
two types of errors arise in these approximations, one systematic and the other random. As
Section 3.3.1 indicates, initial conditions influence the sequence of states through which the
simulation passes, especially near the beginning of a simulation. For example, for small r a
simulation that begins with an arrival to an empty and idle system generates sample paths
{B(r),O::::: r ::::: t} and {Q(r), 0::::: r ::::: t} that tend to lie below tLB and tLQ, respectively,
so that B(0, r) and Q(0, r) often understate these quantities. Conversely, a simulation that
starts with the arrival of a large number of jobs to a system whose servers are all busy
generates sample paths {B(r), 0 ::::: r ::::: t} and {Q(r), 0 ::::: r ::::: t} that tend to exceed tLB
and tLQ, respectively. Therefore, B(O, t) and Q(O, t) often overstate their respective long-run
averages. Both examples illustrate systematic error.
To address this issue for a fixed r and t := s + r, it is of interest to study the properties
of time averages, as the length of the discarded warm-up interval, s, varies. Under relatively
general conditions, including the limits (3.34) and the finiteness of tLQ and tLB,

Q(s, s + r) - tLQ ~ ~Q(r) w.p.l as s ~ 00 (3.43)

and
B(s,s+r)-tLB~~B(r) w.p.l ass~oo, (3.44)

where ~ B ( r) and ~ Q ( r ) denote zero-mean random variables that are independent of initial
conditions. Expressions (3.43) and (3.44) reveal that if one runs a simulation for s + r
simulated time units but uses the truncated sample paths {Q(x), s ::::: x ::::: s + r} and
{B(x), s ::::: x::::: s + r} to compute approximations Q(s, s + r) and B(s, s + r) of tLQ and
IJ.B, respectively, then Q(s, s + r) - tLQ and B(s, s + r) - tLB converge with probability one
 3 .3 . CON D I T ION A LAN D EQUI LIB RI U M DIS T RIB UTI 0 N S 89

to zero-mean random variables that are independent of the state in which the simulation run
began execution. Most importantly, this property of asymptotic unbiasedness holds regardless
of the initial state chosen from T. Section 6.1 returns to this topic.
The relationships between Q (s, s + r) and tl. Q (r) and between B (s, s + r) and tl.B (r)
yield additional properties of interest. In particular,

rE [Q(s, s + r) - JLQf ~ rvartl.Q(r) ~ a~(Q) as r ~ 00

and (3.45)
rE [B(s, s + r) - JLB]2 ~ rvartl.B(r) ~ a~(B) as r ~ 00,

where a~(Q) and a~(B) denote positive quantities. Therefore, for large r, Q(s, s + r) has
approximate mean-square error a~(Q)/r, and B(s, s + r) has approximate mean-square
error a~(B)/r, regardless of the length s of the warm-up interval. This observation raises a
question as to why we should concern ourselves with truncation at all. The answer is that for
two warm-up interval lengths S2 > Sl ::: 0, the mean-square errors converge more rapidly
to a~(Q)/r and a~(B)/r for S2 than for Sj, as r increases.
The properties

rl/2 [Q(s, s + r) - JLQ] -! N(O, a~(Q)) asr ~ 00

and (3.46)
rl/2 [B(s, s + r) - JLB] -! N(O, a~(B)) asr ~ 00,

where -! denotes convergence in distribution andN(al, a2) denotes the normal distribution
with mean al and variance az, provide the additionally essential ingredients for assessing
how well Q(s, s + r) and B(s, s + r) approximate JLQ and JLB, respectively, for large r.
Expression (3.46) asserts thatthe distributions of the statistics rl/2 [Q(s, s + r) - JLQ] and
r 1/2 [B(s, s + r) - JLB] both converge to normal distributions as the sample-path length r
increases without bound. As illustration of their importance for error assessment for queue
length, the first limit in expression (3.46) implies that the interval

(3.47)

includes the true unknown long-term average JLQ with approximate probability 1 - 8 for
large r, where

<1>-1(3):= [z E (-00, (0): (2n)-1/2 i~ e-x ' 12 dx = f3 J, 0< f3 < 1,

denotes the f3 -quantile of the standard normal distribution. A similar interval estimate exists
for JLB with B(s, s + r) and a~(B) replacing Q(s, s + r) and a~(Q), respectively.
90 CHAPTER 3. DATA COLLECTION AND AVERAGES

In practice, a~(Q) and a~(B) are unknown. However, under specified conditions it
is known that if a~(Q, r) and a~(B, r) are estimators of a~(Q) and a~(B), respectively,
that satisfy

as r ~ 00 w.p.I

and (3.48)
as r ~ 00 w.p.I,

thenforlarger, the interval [Q(s, s + r) ± <1>-1(1 - 8/2)Ja50(Q, r)/r ] includesJLQwith

approximate probability 1 - 8 and [R(S, s + r) ± <1>-1(1 - 8/2)Ja50(B, r)/r ] includes

JLB with approximate probability 1 - 8, where the errors in the probabilities diminish as
r increases. Here a~(Q, r) and a~(B, r) satisfying expression (3.48) are called strongly
consistent estimators of a~ (Q) and a~ (B), respectively. The challenge is to employ strongly
consistent estimators based on sample-path data and to keep this computation from being an
onerous burden for the simulationist. Sections 6.6 and 6.7 provide the wherewithal to do this.

NOTE

Collectively, the time-average properties of strong consistency, asymptotic unbiased-

ness with respect to warm-up interval, asymptotic normality, and the ability to
compute consistent estimates of a ~ (Q) and a~ (B) are crucial to any valid assessment
of how well time averages approximate long-term averages.

Limiting properties analogous to those in expressions (3.43) through (3.48) also hold
for PQ(q, s, s + r) as an approximation to PQq for each q = 0, 1, ... and for PB(b, s, s + r)
as an approximation to PBb for each b = 0, 1, ... , m.

3.4 TERMINATING EXECUTION

With regard to data collection, analysis, and run termination, simulationists frequently
employ one of two options:

OPTION A

• Begin the simulation at time 0

• Begin data collection at specified time s (2: 0)
• Complete data collection at a specified time s + r
• Terminate execution of the simulation at time s +r
 3 .4 . T E RMIN A TIN G Ex E CUT ION 91

• Calculate and display summary statistics based on sample-path data collected on the
time interval [S, s + T].

OPTION B

• Begin the simulation at time 0

• Begin data collection when the Mth job completes all service
• Complete data collection when the (M + N)th job completes all service
• Terminate execution of the simulation when the (M + N)th job completes all service
• Calculate and display summary statistics based on sample-path data collected over the
time interval (TM' T M+N ], where

Tj := time at which the jth job completes all service.

Option A implies that the simulated time [s, s + T ] for data collection is predetermined
but that the number of jobs, N(s, s + T) = N (T) - N (s), completed in the time interval
is random. Conversely, Option B implies that the length of the simulated time interval,
(TM' TM+N]' is random but the number of jobs N ( TM' TM+N) = N is predetermined. Of the
two, Option B has more appeal inasmuch as it allows the simulationist the freedom to specify
how much completed work is to occur during the period of observation.
When Option B is employed, the sample average queue length is

(3.49)

where in addition to the sample path {Q(u), TM ::; U ::; TM+ N } being random, the length of the
sample path is also random. For finite M and N, this new source of randomness contributes
to the error of approximating fJ.Q by Q(TM , TM+J. Although the limiting properties that
Sections 3.3.2 through 3.3.4 describe all assume Option A, it can be shown under mild
additional qualifications that as N ~ 00 under Option B, limiting behavior analogous
to expressions (3.42), (3.45), and (3.46) again holds, and as M ~ 00, limiting behavior
analogous to expression (3.43) also holds.
Like properties hold for the sample average number of busy RESERVATIONISTs

(3.50)

under Option B. Likewise, strong consistency, asymptotic unbiasedness, and asymptotic

normality continue to hold for PQ(q, T M , TM +N ) for each q = 0,1, ... and for each
PB (b, TM' TM+N) for each b = 0, 1, ... , m - 1. Section 6.11 describes an indirect method for
assessing the accuracy of Q(TM, TM+N) and B( TM, TM+N) under Option B.
92 CHAPTER 3. DATA COLLECTION AND AVERAGES

3.5 WAITING TIME

We now turn to the estimation of the long-term average waiting time. The analysis proceeds
analogously to that for utilization and queue length, the differences being the discrete index-
ing of the sample path WI,"" WN(r), the continuous range [0, (0) for Wi, and the effect
of alternative modes of terminating the simulation (Section 3.4). Recall that

1
L
N(s+r)
W(s,s+r):= Wi (3.51)
N(s + r) - N(s) i=N(s)+1

denotes the arithmetic average of waiting times and

_ 1 N(s+r)
Pw(w, S, s + r):= (
N s +r
) ()
- N s
L
i=N(s)+1
I(w.oo) (W;), 0::: W < 00,

the proportion of CUSTOMERs that wait longer than W time units for service. Most sim-
ulation languages either automatically compute or provide the means for computing the
waiting-time empirical distribution function {I - Pw(w, s, s + r), WE {WI> Wz > ... >
wr > O}}, for a user-specified integer r and WI, ... , W r .

Let {Fw (w), W :::: O} denote the equilibrium d.f. of waiting time. Under relatively
general conditions for Option B and for each fixed M,

(3.52)

Moreover, provided that

I-Lw := 100
W dF(w) = 1 00
[1 - F(w)]dw < 00,

it follows that

(3.53)

Thus for each W :::: 0, Pw(w, r M , r M +N ) is a stongly consistent estimator of 1 - F(w) and
W(rM' r M+N) is a strongly consistent estimator of I-L w.
As in the case of queue length, the finiteness of I-L w holds for many simulation models.
However, it does not hold when the interarrival-time or service-time distributions do not
have finite first and second moments.
For finite N, an error arises in approximating {I - F (w), w :::: O} by Pw (w, r M' r M+N)
and I-Lw by W( r M , r M +N ). To facilitate the assessment of this error, we rely on additional lim-
 3.5. WAITING TIME 93

iting properties. We describe them for W(.M' • M+N) and note that for each w ::: 0 analogous
limiting properties hold for Pw(w, .M' .M+N) as an approximation to 1 - F(w).
Under Option B, relatively general restrictions, and for finite N, we have

(3.54)

where I::!..w(N) is a zero-mean random variable that fluctuates independently of initial

conditions. Moreover,

(3.55)

where (1~(W) is a positive constant, and

as N -+ 00. (3.56)

These results facilitate the derivation and computation of confidence intervals for ILw
in Section 6.6 based on replacing (1~(W) with a strongly consistent estimator provided by
the batch means method.
If Option A is employed for data collection, then expression (3.11) gives the sample
average waiting time where in addition to the sample sequences W N(sl+!, .•. , W N(s+rl being
random, the number of waiting times N (s + .) - N (s) is random, contributing one more
source of error to the approximation of ILw for finite sand •. Then under mild additional
qualifications, limiting properties similar to expression (3.54) hold as s -+ 00 and properties
similar to expessions (3.53) and (3.55) hold as. -+ 00. Also, a parallel development ensures
strong consistency and asymptotic normality for Pw(w, s, s + .) for each w ::: 0, and the
methods of Section 6.6 again apply.

3.5.1 LlnLE'S LAW

Let A denote arrival rate. Then Little's law (e.g., Stidham 1974) states that for all
queueing systems for which ILQ, A, and ILw exist,

ILQ = AILw, (3.57)

long-run average queue length = arrival rate x long-run average waiting time.

This identity suggests a way of eliminating the errors of approximation in W(s, s + .)

due to a random number of elements, N(s + .) - N(s), in the average under Option A.
Since A is known, Q(s, s + .)/A provides an estimator of J-tw, free of this error. Conversely,
under Option B, IL N W(.M' • M+N) provides an estimator of IL Q free of the error induced in
Q(.M' • M+N) by a random sample path length, • M+N - • M' Regrettably, no comparable result
holds for the exceedance probabilities. Note that E[N(s + .) - N(s)] = ./A.
94 CHAPTER 3. DATA COLLECTION AND AVERAGES

3.6 SUMMARY

This chapter has introduced the reader to the mechanics of calculating time and arithmetic
averages that serve as estimates of system performance measures. It has also alerted the reader
to the presence of errors, to ways of limiting them, and to the need to assess the extent of
remaining errors. In particular,

• Recursive expressions are available that minimize the need for data storage for
computing time and arithmetic averages during simulation execution.

• Errors arise from several sources:

Sources of Errors of Approximation To Reduce Errors

Influence of initial conditions on evolution of Truncate sample path near its beginning
sample path
Sampling fluctuations along sample path Increase sample-path length, provided
long-run average exists
Random number of elements, N(T + s) - N(s), Use Little's law
when estimating /-tw under Option A
Random sample path length, 7:M +N - TN' when Use Little's law.
estimating /-tQ under Option B
• With regard to assessing the extent of error, strongly consistent estimators of the
asymptotic quantities a!(Q), a!(B), and a!(W) are needed. Chapter 6 describes
these and software for calculating them using the batch-means methods.

3.7 PERIODIC DATA COLLECTION

Although computing the finite sample averages Q(s, s + T), PQ(q, s, s + T) for each q and
B(s, s + r), and PB(b, s, s + r) for each b in Sections 3.1.1 and 3.1.2 provides us with
estimates of their corresponding true long-run performance measures, these computations
do not provide the necessary data to compute assessments of how well the sample averages
approximate the true averages. If data collection begins at time s and ends at s + r, then the
techniques of Chapter 6 rely on the availability of sample-path sequences for j = 1,2, ... ,

1 [s+jll.
Qj:= - Q(u)du,
!:::.s+(j-l)ll.
1 [s+jll.
PQj(q):= !:::. I(q,oo)(Q(u»du, (3.58)
s+(j-l)1l.
1 [s+jll.
Bj:= - B(u)du,
!:::. s+(j-l)1l.
1 [S+jll.
PBj(b) := - I(b,m] (B(u»du j = 1, ... ,t:= Lr/!:::.J,
!:::. s+(j-1)1l.
 3 .7 . PER I 0 Die D A T A COL L E CT ION 95

Collect
statistics

Figure 3.1 COLLECT event

where I::!.. is a positive time increment chosen, ifpossible, to make L / I::!.. an integer. For example,
{QI> ... , Qt} provides the data needed to compute an estimate a;,(Q), of a;,(Q), which
together with Q(s, s + I::!..t) is used to compute an approximating confidence interval for
/LQ.
To collect these data at simulated time u after a warm-up interval s, we rely on a
periodically recurring event as in Figure 3.1, where we assume that the first COLLECT event
is scheduled at time u ::: s + I::!... For example, one approach gathers the data

Qj := Q(s + jl::!..),
P'U(q) := I(q,oo)(Q(s + jl::!..)),
Bj := B(s + jl::!..),
P~j := I(b,m] (B(s + jl::!..)),

and at the end of the simulation run computes the averages

96 CHAPTER 3. DATA COLLECTION AND AVERAGES

as approximations to their respective sample time-integrated averages Q(s, s + r),

PQ(q, s, s + r), B(s, s + r), and PB(b, s, s + r). Since, for example,

f,s+(j-l)/1
S+j/1
Q
(
U
)d
U
-+ I as ~ -+ 0 w. p.I,
~ x Q (s + j~)
the error of approximation diminishes as the interval between observations, ~, decreases.
The appeal of this alternative approach is its simplicity, especially when the selected
simulation language does not provide an automated procedure for collecting the time-
integrated quantities (3.58).

3.8 LESSONS LEARNED

• A discrete-event simulation program is an engine for generating sequences of data

points, called sample paths, on waiting or delay time, queue length, and resource
utilization.
• The data are used to compute approximations, called sample averages, of system per-
formance measures. These include long-run average waiting time, queue length, and
resource utilization.
• The data can also be used to compute estimates of the probability distributions of wait-
ing time, queue length, and resource utilization. In particular, estimates of exceedance
probabilities allow assessment of extreme (rare) behavior.
• The initial conditions (state) that prevail at the beginning of a simulation run influence
the appearance of the sample paths and therefore, the values of sample averages based
on them. This influence tends to diminish as simulated time relapses.
• The rate of dissipation is inversely related to the level of congestion (delay) in the system.
• Most simulation languages provide a means for recursively updating the sample av-
erages as simulated time elapses, thus reducing memory space required to store large
data sets.
• For finite simulated time r, the sample averages are approximations of true unknown
long-run averages.
• Discarding data points on sample paths close to the beginning of a simulation run
(warm-up interval) reduces the influence of the part of the error of approximation
induced by the choice of initial conditions.
 3.9. REFERENCES 97

• As the length of a simulation run '[' increases, the errors of approximation in sample
averages, due both to initial conditions and random sampling fluctuations, decrease.
That is, the values of the sample averages tend to converge to the values of the unknown
corresponding long-run averages.

• As '[' increases, a statistical theory based on the normal distribution becomes

increasingly valid as a means of assessing how good the approximations are.

• Execution of a simulation terminates when either a prespecified simulation time has

elapsed (Option A) or when a prespecified number of actions, such as job completions,
has occurred (Option B).

• Under Option A, sample average waiting time and estimates ofits exceedance probabil-
ities are ratio estimates. Under Option B, sample averages of queue length and resource
utilization and estimates of their exceedance probabilities are ratio estimates. Error
analysis for ratio estimates calls for more detailed attention to sources of variation
than for simple sample averages. See Section 6.11.

• Little's law (Section 3.5.1) establishes a proportional equivalence between the long-
run averages of waiting time and queue length that can eliminate the issue of ratio
estimation.

3.9 REFERENCES

Stidham, S., Jr. (1974). A last word on L = AW, Oper. Res., 22, 417-421.
 CHAPTER4

G. S. Fishman, Discrete-Event Simulation

© Springer Science+Business Media New York 2001
Programming
and Execution

Converting modeling logic as in Chapter 2 into executable code is the next step in simulation
program development. To this logic must be added all the housekeeping functions of assign-
ing values to input parameters and arranging for the desired output to emerge at the end
of program execution. This chapter illustrates these steps, first using the SIMSCRIPT 11.5
simulation programming language and then the Arena general-purpose simulation system
based on the SIMAN simulation language. It also describes how the resulting programs are
executed and displays their output.
A survey in Swain (1999) reveals at least 54 products for performing discrete-event
simulation on a computer. These vary widely in the ways they allow program creation.
Moreover, variations exist in how a program, regardless of program-preparation modality,
is converted to executable code and how the mode of conversion affects portability and
execution time. Portability denotes the extent to which a program is executable on different
hardware platforms. Section 4.1 describes several modalities for program creation. When
viewed as special cases of this account, subsequent sections based on SIMSCRIPT 11.5 and
Arena/SIMAN take on considerable more meaning for the reader.
100 CHAPTER 4. PROGRAMMING AND EXECUTION

4.1 MODALITIES FOR PROGRAMMING

Simulation software can be characterized in many ways. Does it operate under Unix-based
operating systems (e.g., Sun and Dec workstations) and/or under Windows-based operating
systems (e.g., IBM-compatible PCs)? Can a simulationist create a program using a textual
editor for entry or using graphical entry? Does the software embrace traditional program-
ming paradigms or is it object oriented (Section 1.9.2)? Does execution rely on a compiled
program or is it performed in interpretive mode? Passing a program through a compiler
leads to optimized code with respect to execution efficiency. Interpretive code avoids the
compilation time but tends to execute more slowly.
Rather than answer these questions directly, we describe a general framework that
encompasses most simulation software and briefly indicate how four frequently employed
discrete-event simulation languages fit within this structure. The languages are GPSS/H
(Schriber 1991), SIMAN (Pegden et al. 1995), SIMSCRIPT 11.5 (CACI 1987, 1993a), and
SLAM (Pritsker 1995). To varying degrees, each provides the essential features for discrete-
event simulation programming as described in Section 1.9. In particular, all provide for a
process-interaction orientation. SIMAN, SIMSCRIPT 11.5, and SLAM also acconmodate the
event-scheduling approach.
Three of these four languages are also embedded in general-purpose simulation sys-
tems: GPSS/H in SLX supported by Wolverine Software Inc (Henriksen 1997); SIMAN in
Arena (Kelton et al. 1997), supported by Systems Modeling Inc.; and Visual SLAM in AweSim
(Pritsker and O'Reilly 1999), supported by Symix Inc. These relatively comprehensive sys-
tems allow for program creation by simulationists of varying skill levels. They also facilitate
animation and data transfer from and to external databases.
In today's PC-workstation environment, programming takes place either by textual
entry via an editor or by some variation of point, click, drag, drop, and connect. Hereafter, we
refer to this last option, concisely, as point and click.
Figure 4.1 displays five alternative modalities for creating an executable simulation
program. In Modality (a), the simulationist enters statements via a textual editor. The re-
sulting source program is compiled into binary machine code to form an object program to
which required library routines are added to form executable code called a load module. This
close relationship between simulation language and binary code tends to give this modality
an advantage in execution time. However, it restricts portability by requiring a vendor to sup-
ply a separate compiler for each unique type of platform. SLX and the IBM and VAX versions
of GPSS/H use this approach, the net result being highly efficient program execution.
Modality (b) also relies on textual entry but passes the resulting program through a
parser written in C/C++ that translates it into binary code or converts the source code to
C/C++ and then translates it into binary code. This approach increases portability across
computing platforms. SIMSCRIPT 11.5 embraces this option on Unix and VMS platforms.
For IBM-compatible PCs and Sun workstations, GPSS/H uses a variant of Modality (b). It
generates "P-code" for a virtual machine, and for selected hardware platforms provides an
interpreter to carry out execution. This strategy again aims at efficiency in execution.
 4 .1 . MOD A LIT I E 5 FOR PRO G RAM MIN G 101

Create program
(a) Translated
via textual
directly into
entry binary code
Create program
(b)
via textual
entry

Build program Simulation language

Translated
by selecting (c) statements .I Executable I
and sequencing corresponding via C/C ++ into 'I module
blocks to blocks binary code
T
Build program (d) Blocks
by selecting orresponding to
macromodules macro modules

Build program (e)

by selecting
macro modules

Figure 4.1 Alternative modalities for creating an executable simulation program

Modality (c) mimics Modalities (a) and (b) with one major difference. Templates of
icons are displayed on the screen, one for each language statement or block. The simulationist
merely points, clicks, drags, and drops the blocks into a "model window" and connects them
to create a source program. Currently, this is the most commonly employed modality, and
Arena/SIMAN, AweSimlVisual SLAM, and SIMLAB/SIMSCRIPT u.s all permit this option
on Windows-based PCs. Section 4.25 illustrates the concept with Arena/SIMAN.
In principle, Modality (d) performs as Modality (c) with one important difference.
The icons in the tableaus correspond to macromodules each of which represents a sequence of
simulation-language statements or blocks. For example, an "Arrive" macromodule includes
all statements associated with bringing new items into a system. A "Server" macromodule
includes all statements related to providing an item with service. Combining Modalities
(c) and (d) enables a simulationist to flesh out a program that macromodules alone cannot
create. Arena/SIMAN and AweSimlVisual SLAM offer this option, whose ostensible purpose
is to reduce the required language familiarity that a simulationist must have to construct a
program. Section 4.24 illustrates the concept for Arena/SIMAN.
Some macromodules in Modality (d) are relatively complex in content. For example,
the availability of macromodules that represent the behavior of conveyors and automatic
guided vehicles (AGV) to move items within factories and warehouses, automated storage
102 CHAPTER 4. PROGRAMMING AND EXECUTION

and retrieval systems (ASRS), robots, and bridge cranes considerably reduces programming
effort in building simulations of manufacturing and materials handling systems.
Modality (e) represents a strategy for packaging many of these special-purpose tasks
that, for example, manufacturing requires into a moderately large collection of macromod-
ules that correspond directly to off-the-shelf precompiled C++ subprograms or rely on a
C++ translator for object-program creation. This eliminates any reliance on a simulation
language per se. Many software offerings that cater to specialized modeling rely on this op-
tion. Its purpose is to attract potential users with limited or no knowledge of a simulation
programming language by reducing the barriers to entry. While the benefit is immediately
clear, a disadvantage emerges when a simulationist employs software that, while easy to use,
does not have the flexibility to represent the exact flow logic in her/his model. Virtually
all simulated software offers an option that allows a simulationist to merge user-created
programs with user-written code in C, C++, or other high-level language. Presumably this
would be the option one would have to take to overcome the limitation of Modality (e).
Since this approach requires a simulationist to be an able programmer in the selected lan-
guage, it undermines the philosophy of "no programming required." Because of this barrier,
some simulationists merely alter the original concepualization of their systems to make them
amenable to representation in terms of the iconized blocks and modules that their selected
simulation software offers. Although the extent to which this action undermines the quality
of a study varies from problem to problem, the practice is lamentable, at a minimum, and
jeopardizes scientific integrity, at worst. Section 4.27 describes potential pitfalls.
Rather than clarity of concept, today's highly interactive PC environment demands
rapidity in moving from a less than completely thought -through concept to execution. While
speed of creation is important, a large-scale discrete-event simulation with intricate flow-
logic inevitably requires some awareness of the nuts and bolts of simulation programming.
This is especially true if a program is to be used repeatedly in a computationally efficient way.
The extent to which pedagogy that focuses on this rapid movement from concept to execution
prepares the newcomer to simulation to address issues of model size and complexity is
debatable. There is a more basic issue.
The point-and-click approach also encourages a simulationist to skip the preliminary
steps in Sections 2.3 and 2.4 that explicitly identify events and flow logic in favor of creating the
model directly on the PC screen and, at the click of the mouse, converting the representation
to executable code. Although this protocol may work well for problems of small and moderate
size, large problems with complex flows present a challenge that is most easily met with
deliberate attention to the steps in Sections 2.3 and 2.4 prior to program creation. While
especially true for the student just learning simulation methodology, experience indicates
that more seasoned simulationists can also benefit from this focus on fundamental principles.
Accordingly, our discussion of SIMSCRIPT II.S relies heavily on the event-scheduling and
process-interaction approaches of Chapter 2. Our discussion of Arena/SlMAN focuses on
the process-interaction approach with passive resources.
Regardless of the extent of embeddedness, every discrete-event simulation program-
ming language provides a user with a syntax that supports either the process-interaction or
 4 .2 . S I M SC RIP T II. 5 103

event-scheduling modeling approach. Some simulation programming languages, including

the SIMAN, SIMSCRIPT II.S, and SLAM, offer both. Sections 4.2 through 4.21 use SIM-
SCRIPT U.5 to illustrate both approaches, and Sections 4.24 and 4.2S use SIMAN to illustrate
the process-interaction approach.
Of the two, the process-interaction approach has come to dominate, principally be-
cause it allows a syntax that relieves the simulationist of the considerable amount of code
writing inherent in the event-scheduling approach and because iconization of this syntax on
PC screens allows the convenience of the point-click-drag-drop mode of program creation.
To illustrate how syntax favors the process-interaction approach, consider a single-
server system as in Figure 1.2a. Every arrival to the system demands service from server 1.
If service is not available, the arrival waits. Otherwise, server 1 becomes busy and service
begins. This sequence of actions characterizes a substantial portion of what occurs when
service is demanded in a delay system. Because of the repeated occurrence of this sequence
of actions in many systems in a process-interaction environment, all languages based on
this paradigm contain a statement such as SEIZE 1 in GPSS/H that effects these actions.
Moreover, the statement QUEUE 1, preceding the SEIZE statement, allows the simulation
to collect statistics on queue length and waiting time. The structure of the event -scheduling
approach does not allow equally comprehensive statements.
In general, the process-interaction approach leads to a more powerful syntax in the
sense that far fewer statements per program are required than in one based on the event-
scheduling approach. The conciseness of this syntax has induced simulationists to use block
diagrams corresponding to its statements to model systems, thereby merging the modeling
exercise into program development. The concept of simulation block diagramming based
on syntax originated in 1962 with the IBM product GPSS developed by Geoffrey Gordon.
Several simulation languages, including GPSS/H, SIMAN, SIMSCRIPT U.5, and
SLAM, contain provisions for improving the statistical quality of results at relatively lit-
tIe cost by manipulating their pseudorandom number generators. Section 4.22 describes an
application of one of these options called common pseudorandom numbers. By including it
here, we wish to encourage the reader to recognize the opportunities that these languages
offer for statistical as well as programming refinements.
Section 4.27 returns to the issue of macromodules. As noted, these can save consid-
erable time programming complex concepts such as conveyors, AGVs, ASRSs, robots, and
bridge cranes. However, they should be used only when their implicit behavior is clearly
understood. Section 4.27 describes the limitations of proceeding otherwise.

4.2 SIMSCRIPT 11.5

Sections 4.3 through 4.21 describe how to convert the flowcharts for the airline reservation
problem in Chapter 2 into computer programs written in SIMSCRIPT U.5. It also describes
their input, execution, and output. Since our purpose is to acquaint the reader with the lan-
104 CHAPTER 4. PROGRAMMING AND EXECUTION

guage, the subprograms illustrate concepts, but not necessarily the most efficient coding. The
advantages of illustrative versus tightly written subprograms should not be underestimated.
SIMSCRIPT 11.5 represents the latest in a sequence of discrete-event simulation lan-
guages that use the SIMSCRIPT title. Markowitz, Karr, and Hausner (1963) developed the
original SIMSCRIPT at the Rand Corporation. SIMSCRIPT II, which represents a consid-
erable extension of the original features and capabilities, was developed at Rand by Kiviat,
Villanueva, and Markowitz (1969). SIMSCRIPT 11.5 is a substantial enhancement of SIM-
SCRIPT II supported by CACI (1987, 1993a, 1993b, 1997). All examples described here are
based on SIMSCRIPT 11.5.
The language has a compiler and is divided into five levels:

LEVEL 1. A simple teaching language designed to introduce programming concepts to

nonprogrammers.

LEVEL 2. A language roughly comparable in power to FORTRAN but departing greatly from
it in specific features.

LEVEL 3. A language roughly comparable in power to C or PASCAL but again with many
specific differences.

LEVEL 4. That part of SIMSCRIPT 11.5 that contains the entity-attribute-set features of
SIMSCRIPT. These features have been updated and augmented to provide a more
powerful list-processing capability. This level also contains a number of new data
types and programming features.

LEVEL 5. The simulation-oriented part of SIMSCRIPT 11.5, containing statements for time
advance, event processing, generation of statistical variates, and accumulation and
analysis of simulation-generated data.

By providing the features in level 5, SIMSCRIPT 11.5 relieves the user of the need
to provide these special-purpose routines. Indeed, every simulation programming lan-
guage provides these features, thereby giving them a considerable edge over general-purpose
programming languages.
The principal building blocks of SIMSCRIPT 11.5 are entities, attributes, and sets.
Entities denote objects of a system; attributes, characteristics of the entities; and sets, entity
groupings or lists. Figure 4.2 contains a SIMSCRIPT 11.5 program that incorporates the logic
specified in Figures 2.5 through 2.8 based on the event -scheduling approach. For convenience
ofexposition, we adopt the convention that all lowercase words denote SIMSCRIPT 11.5 code,
whereas all uppercase words denote user-defined concepts. In general, SIMSCRIPT 11.5
makes no distinction between lower- and uppercase letters. Occasionally, we use quotation
marks in the text to identify SIMSCRIPT 11.5 keywords. Line numbers are for the benefit of
the reader and should not appear in an executable version of this program. SIMSCRIPT 11.5
statements can begin in any column.
 4 .2. S I M SC RIP T II. 5 105

Every SIMSCRIPT II.5 program written to execute a simulation begins with a nonex-
ecutable subprogram called a preamble that defines the system's entities, attributes, sets, and
the relationships among them. It also defines all global variables and functions accessible
to all executable components (events and subroutines) within the program. In Figure 4.2,
line 2 of the preamble indicates that unless otherwise specified all variables are assumed to
have integer mode. Line 3 declares the existence of a set called QUEUE, which is the only
nonSIMSCRIPT II.5 word there. We may call this set QUEUE, LIST, or any other name we
like, provided that we avoid using the names of specified SIMSCRIPT II.5 system variables.
Upon encountering this set declaration for QUEUE in the preamble, SIMSCRIPT II.S
implicitly defines a global variable, n.QUEUE, that serves as the number of elements in the
set. During execution, SIMSCRIPT II.S updates n.QUEUE each time that either an addition
or deletion is made. Initially, n.QUEUE is set to zero. More generally, for each set declared
in a preamble, SIMSCRIPT II.5 defines a global variable with the same name prefaced by n.
Global variables are accessible from anywhere in a SIMSCRIPT II.5 program, in contrast to
local variables, which are accessible only from within the subprogram that defines them.

Figure 4.2 SIMSCRIPT II.S AIR_EY.sim program

AIR_EV.sim (February 1998)

,,------------------------------------------------------------------
PREAMBLE
,,------------------------------------------------------------------
1 preamble
2 normally mode is integer
3 the system owns a QUEUE

4 permanent entities
5 every RESERVATIONIST has a STATUS
6 define NO.BUSY as a variable
7 accumulate AVE.UTIL as the average, V.UTIL as the variance and MAX.UTIL
as the maximum of NO.BUSY
8 accumulate UTIL.HISTO (0 to n.RESERVATIONIST by 1) as the histogram
of NO.BUSY
9 accumulate AVE.Q as the average, V.Q as the variance and MAX.Q as the
maximum of n. QUEUE
10 accumulate Q.HISTO (0 to 32 by 1) as the histogram of n.QUEUE

11 temporary entities
12 every CUSTOMER has a TYPE, a START and may belong to the QUEUE
13 define START and WAITING. TIME as double variables
14 tally AVE. WAIT as the mean, V.WAIT as the variance and
MAX. WAIT as the maximum of WAITING. TIME
15 tally WAIT.HISTO (0 to 8 by .25) as the histogram of WAITING. TIME
16 define ZERO. WAIT as a variable

17 event notices include CALL

106 CHAPTER 4. PROGRAMMING AND EXECUTION

Figure 4.2 (cont.)

18 every COMPLETION has a CLIENT and a WORKER

19 every END_MESSAGE has a CALLER
20 priority order is CALL,COMPLETION and END_MESSAGE

21 define LAMBDA, OMEGA and P as double variables

22 define M to mean n.RESERVATIONIST

LAMBDA := call rate

" OMEGA := service rate for single-destination trips
P .= multiple-destination probability
M := no. of RESERVATIONISTs

23 define NUM, K and T as variables

"
NUM := no. of CUSTOMERs that have or are rece1v1ng service
K-l := no. of CUSTOMERS to skip before beginning data collection
T := no. of CUSTOMERs used for analysis

24 define RRHO to mean LAMBDA*(l+P)/(M*OMEGA)

25 define IDLE to mean 0

26 define BUSY to mean 1
27 define SUCCESS to mean 0
28 define NO. SUCCESS to mean 1
29 define ONE to mean 1
30 define MANY to mean 2

31 define SEEDS as a i-dim array

32 define KIND as a function
33 define S_TlME as a double function
34 end

,,----------------------------------------------------------------------
MAIN
,,----------------------------------------------------------------------
1 main
2 open unit 10 for input
3 open unit 15 for output
4 use unit 10 for input
5 use unit 15 for output
6 read LAMBDA,OMEGA,M and P INITIALIZE
" PARAMETERS
7 if RRHO>=1.0 print 1 line thus
ERROR-----ACTIVITY LEVEL>=l ERROR CHECK
8 stop PARAMETERS
9 otherwise

10 reserve SEEDS(*) as 4

intercall times use stream 1

single-destination service times use stream 2
 4.2. SI M SC RIP T II. 5 107

Figure 4.2 (cont.)

second half of multiple-destination service times use stream 3

selection of TYPE uses stream 4

11 for 1=1 to 4 do
12 read seed. v(I) ASSIGN SEEDS
13 let SEEDS(I)=seed.v(I) SAVE SEEDS
14 loop

15 read T and K
16 let NUM=1

17 schedule a CALL now ESTABLISH

18 create each RESERVATIONIST INITIAL
19 start simulation CONDITIONS
20 end

,,-----------------------------------------------------------------
event CALL
,,-----------------------------------------------------------------
1 event CALL saving the event notice
2 define ACTION as an integer variable
3 define T.CALL to mean exponential.f(1./LAMBDA,1)
4 create a CUSTOMER
5 let TYPE=KIND
6 let START(CUSTOMER)=time.v
7 reschedule this CALL in T.CALL minutes
8 call SERVICE yielding ACTION
9 if ACTION=NO.SUCCESS
10 schedule an END_MESSAGE given CUSTOMER in .15 minutes
11 else
12 add 1 to ZERO. WAIT
13 always
14 return
15 end

,,------------------------------------------------------------
routine KIND
,,------------------------------------------------------------
routine KIND
2 if random.f(4) < P
3 return with MANY
4 else
5 return with ONE
6 end

,,-----------------------------------------------------------
event END_MESSAGE
,,-----------------------------------------------------------
event END_MESSAGE given CALLER
2 define ACTION and CALLER as integer variables
3 let CUSTOMER=CALLER
4 call SERVICE yielding ACTION
108 CHAPTER 4. PROGRAMMING AND EXECUTION

Figure 4.2 (cont.)

5 if ACTION=NO.SUCCESS
6 file CUSTOMER in QUEUE
7 else
8 always
9 return
10 end

,,-------------------------------------------------------------
event COMPLETION
,,-------------------------------------------------------------
1 event COMPLETION given CLIENT and WORKER
2 define ACTION , CLIENT and WORKER as integer variables
3 destroy CUSTOMER called CLIENT
4 if QUEUE is empty
5 let STATUS(WORKER)=IDLE
6 let NO.BUSY=NO.BUSY-1
7 return
8 always
9 remove the first CUSTOMER from QUEUE
10 let RESERVATIONIST=WORKER
11 call SERVICE yielding ACTION
12 return
13 end

, ,---------------------'------------------------------------------
" routine SERVICE
,,---------------------------------------------------------------
1 routine SERVICE yielding ACTION
2 define ACTION as an integer variable

3 if event.v not equal to i.COMPLETION

4 for each RESERVATIONIST with STATUS=IDLE find the first case

5 i f none
6 let ACTION=NO.SUCCESS
7 return

8 else
9 let STATUS=BUSY
10 add 1 to NO.BUSY
11 always

12 schedule a COMPLETION given CUSTOMER and RESERVATIONIST in S_TlME minutes

13 call ACCOUNTING
14 let ACTION=SUCCESS
15 return end

,,---------------------------------------------------------------

,,---------------------------------------------------------------
1 routine S_TIME
2 if TYPE= MANY
 4.2. SIMSCRIPT 11.5 109

Figure 4.2 (cont.)

3 return with exponential.f(1./0MEGA,2)+exponential.f(1/0MEGA,3)

4 else
5 return with exponential.f(1./0MEGA,2)
6 end

,,---------------------------------------------------------------
routine ACCOUNTING
,,---------------------------------------------------------------
1 routine ACCOUNTING
2 define DEAD. TIME as a double saved variable
3 if NUM=K
4 reset totals of NO.BUSY,WAITING.TIME and n.QUEUE
5 let ZERO. WAIT = 0
6 let DEAD.TIME=time.v
7 always
8 i f NUM >= K
9 let WAITING.TIME=1440.*(time.v-START(CUSTOMER))
10 always
11 if NUM=K+T-1
12 call SUMMARY(DEAD.TIME)
13 stop
14 always
15 add 1 to NUM
16 return end

,,---------------------------------------------------------------
routine SUMMARY
,,---------------------------------------------------------------
routine SUMMARY(DEAD.TIME)
2 define DEAD. TIME as a double variable
3 define RUN. TIME to mean (time. v-DEAD. TIME)
4 define DUM1,DUM2 and DUM3 as double variables
5 start new page
6 skip 1 line
7 print 7 lines with M,LAMBDA,OMEGA,P,RRHO,K-1 thus

ANALYSIS OF SIMULATION RESULTS

NO. OF RESERVATIONISTS=** CALL RATE=***.*** SERVICE RATE=***.***

MULTIPLE-
DESTINATION PROB.=*.*** ACTIVITY LEVEL=*.*** NO. TRUNCATED=******
8 skip 1 lines
9 print 1 line with NUM-K+1 and RUN.TIME thus

STATISTICS based on ********* waiting times and ****.**** elapsed days

10 skip 1 line
11 print 4 lines with AVE.WAIT,V.WAIT,MAX.WAIT,AVE.Q,V.Q,MAX.Q,AVE.UTIL,
V.UTIL,MAX.UTIL thus
110 CHAPTER 4. PROGRAMMING AND EXECUTION

Figure 4.2 (cont.)

AVERAGE VARIANCE MAXIMUM

WAITING TIME
QUEUE LENGTH ***.**** *****
UTILIZATION ***.**** *****
12 skip 1 line
13 let DUMl = 1. - ZERO.WAIT/(NUM - K + 1)
14 let DUM2 = 1. - Q.HISTO(l)/RUN.TlME
15 print 4 lines with DUMl and DUM2 thus
PROPORTION
WAITING TIME WHO WAIT QUEUE PROPORTION
(minutes) THIS TIME LENGTH OF TIME
W> 0 *.**** Q> 0 *.****

16 let DUMl = 1.0

17 for 1=2 to 32 do
18 let DUMl = DUMl - WAIT.HISTO(I-l)/(NUM-K+l)
19 let DUM2 = max.f(0.,DUM2 - Q.HISTO(I)/RUN.TlME)
20 print 1 line with (I-l)/4,DUM1,I-l,DUM2 thus
>= *.** *.**** >*** *.****
21 loop
22 print 1 line with WAIT.HISTO(33)/(NUM-K+1),Q.HISTO(33)/RUN.TlME thus
>= 8.00 *. **** Q >=32 *. ****
23 skip 1 line
24 print 2 lines thus
NO. OF BUSY
RESERVATIONISTS PROPORTION

25 let DUM3 = 1.0

26 for 1=1 to M do
27 let DUM3 = max.f(0.,DUM3 - UTIL.HISTO(I)/RUN.TIME)
28 print 1 line with 1-1 and DUM3 thus
*.****
29 loop
30 skip 1 line
31 print 6 lines with SEEDS (1) ,SEEDS (2) ,SEEDS (3) ,SEEDS (4) ,seed.v(1),seed.v(2) ,
seed.v(3) and seed.v(4) thus

RANDOM NO. RANDOM NO. RANDOM NO. RANDOM NO.

STREAM 1 STREAM 2 STREAM 3 STREAM 4
---------- ---------- ---------- ----------
FIRST NO. ************ ************ ************ ************

LAST NO. ************ ************ ************ ************

32 skip 3 lines
33 return end
 4.4. EVENT NOTICES 111

4.3 ENTITIES

As in line 4 of the preamble, the words "permanent entities" must preface declarations
relating to such entities. Here the declared permanent entity RESERVATIONIST has an
attribute STATUS. The significance of this attribute becomes apparent later when we search
for an idle RESERVATIONIST. For each permanent entity, SIMSCRIPT 11.5 implicitly defines
a global variable n.entity. This quantity denotes the number of such entities in existence
at a specific time during execution. In the present case, n.RESERVATIONIST serves this
purpose. We assign it a positive integer value in the "main" program, prior to beginning
a simulation run. The words "temporary entities" (line 11) preface declarations relating to
such entities. Notice in line 12 that the temporary entity CUSTOMER has an attribute TYPE
and may be a member of the set QUEUE, defined in line 3. TYPE allows for identification
of a CUSTOMER as requesting a one-way or a round-trip reservation. Failure to assign any
membership to a declared set results in a diagnostic error message during compilation.

4.4 EVENT NOTICES

The words "event notices" in line 17 of the preamble prefaces such declarations. However, as
lines 17 through 19 indicate, two ways exist for declaring these notices. If the COMPLETION
and END...MESSAGE event notices were to have no attributes, then we could replace lines
17 through 19 by

event notices include CALL, COMPLETION and END...MESSAGE.

Similarly, if RESERVATIONIST and CUSTOMER have no attributes, the statements

permanent entities include RESERVATION

temporary entities include CUSTOMER

apply.
Note the use of the underscore in END ...MESSAGE. This acceptable use in SIMSCRIPT
11.5 provides a convenient way of making variable names more comprehensible to readers.
Also note the use of the period in line 6 of the preamble, which improves comprehension.
In this book we adopt the convention of using an underscore in the names of events and
routines and using a period in entity, attribute, and variable names. Although entity names
can have as many characters as one wishes, the first seven characters in the name of a routine,
global variable, or attribute must be unique, as must be the first five characters in the name
of an entity, event, or set.
Since two events may be scheduled for execution at the same time, the declaration in
line 20 removes any ambiguity regarding the order of execution. This statement indicates
that a CALL event has priority over COMPLETION and END...MESSAGE events and that
112 CHAPTER 4. PROGRAMMING AND EXECUTION

a COMPLETION event has priority over END~ESSAGE events. If two CALL events are
scheduled for the same time, SIMSCRIPT 11.5 executes them in the order in which their
corresponding events notices are created. Suppose that the priority declaration is

priority order is END~ESSAGE .

Then SIMSCRIPT 11.5 implicitly assigns priority in the order END~ESSAGE, CALL, and
COMPLETION. After ordering priorities as specified in the "priority order" statement,
SIMSCRIPT 11.5 goes back to the "event notices" statement and assigns successive priorities
to events declared there but omitted from the "priority order" statement. These priorities are
assigned in the order in which the events are declared. Note that in the present case omission
ofline 20 would not affect the priority order. Its value here is pedagogic.

4.5 SUMMARY STATISTICS

The "accumulate" statements in lines 7 through 10 and the "tally" statements in lines 14 and
15 of the preamble are declarations that cause summary statistics for utilization {B (s), s ::::
O}, queue length {Q(s), s :::: O}, and waiting time {Wi, i :::: O} to be updated each time these
processes change values during execution. See Section 1.3. To accomplish this in the present
problem, the preamble declares two global variables, NO. BUSY (line 6) and WAITING.TIME
(line 14). As a consequence of these declarations, SIMSCRIPT 11.5 responds to a request for
values ofAVE.UTIL,AVE.Q, and AVE.WAIT at simulated time r by computing (CACI, 1987,
p.269)

AVE.UTIL= B(O, r),

AVE.Q = Q(O, r), (4.1)
AVE.WAIT = W(O, r),

the time averages that Section 3.1 describes. Section 4.14 shows how inclusion of a "reset"
statement allows one to adjust these averages to account for the warm-up interval.
SIMSCRIPT 11.5 responds to a request for values of MAX.UTIL, MAX.Q, and
MAX.WAIT by computing

MAX.UTIL = max B(s),

O::::s~r

MAX.Q = max Q(s), (4.2)

0:9:::,
MAX.WAIT = max Wi,
l:::i:::N(,)
 4.5. SUMMARY STATISTICS 113

and to a request for the time variances V.UTIL and V.Q, and the variance V.WAIT, by
computing

(4.3)

By default, NO. BUSY has integer mode, whereas WAITING.TIME is a double precision
floating-point variable. An alternative would be to define WAITING. TIME as a "real vari-
able;' which would make it a single-precision floating-point variable. To reduce numerical
round-off error, we encourage double-precision declarations on 32-bit wordsize computers.
Line 8 of the preamble declares that SIMSCRIPT U.5 is to create a histogram of
NO.BUSY, and line 10 does likewise for n.QUEUE. SIMSCRIPT u.s responds to requests
for values of these histograms by computing

J(r)
UTIL.HISTO(I) = 8(B(r) - I + l)(r - tJ(r» + L8(B(ti-l) - I + l)(ti - ti-I),
i=1
1= 1, ... , n.RESERVATIONIST + 1, (4.4)

and

8(Q(tJ(r) - 1+ l)(r - tJ(r»

J(r)
+ L 8(Q(ti_l) - I + l)(ti - ti-I), 1= 1, ... ,32,
Q.HISTO(I) = i=1
(4.5)

L Q.HISTO(I),
32
r - 1=33,
1=1

where

if x = 0,

elsewhere.

Then UTIL.HISTO(I)/r gives the proportion of time that 1-1 RESERVATIONISTS are busy
for 1 :::: I :::: 33, and Q.HISTO(I) gives the proportion of time that I-I CUSTOMERs await
service for 1 :::: I :::: 32. The remaining ordinate, Q.HISTO(33), contains the amount of time
that 32 or more CUSTOMERs await service.
114 CHAPTER 4. PROGRAMMING AND EXECUTION

In terms of the exceedance proportions (3.12) and (3.13),

1 b
PB(b, 0, r) = 1 - - LUTIL.HISTO(I) (4.6)
r 1=0
= proportion of time that more than b
RESERVATIONISTs are busy

and

1 q
PQ(q, 0, r) = 1 - - L
Q.HISTO(I) (4.7)
r 1=0
= proportion of time that more than q
CUSTOMERs wait.

These are the quantities that we print in the output.

I
In a similar manner, the declaration in line 15 of the preamble leads to

~ f[.25(I-1),.251) (Wj ), 1= 1, ... ,32,

WAIT.HISTO(I) = NCr:) (4.8)
L I[32,oo)(Wj ), 1=33,
j=1

where

I[a,b) (x) := { ~ if a::::: x < b,

otherwise.

If waiting times are recorded in minutes, then at time r, WAIT.HISTO(I) contains the
number of CUSTOMERs whose waiting times were at least 0.25(1-1) but less than 0.251
minutes for 1=1, ... , 32, and WAIT.HISTO(33) contains the number of CUSTOMERs with
waiting times of at least eight minutes. With regard to the exceedance proportion (3.14),
recall that for all W E [0, (0),

Pw (W, 0, r) = proportion of waiting times that exceed w, (4.9)

whereas

1
L
lw/.25J
1 - -- WAIT.HISTO(I) = proportion of waiting times::: w. (4.10)
N(r) 1=1
 4.6. TRAFFIC INTENSITY 115

As line 12 of the CALL event shows, the integer variable ZERO.WAIT in line 16 of the
preamble accumulates the total number of CUSTOMERs who do not wait, and therefore

ZERO.WAIT .
1- N (r:) = proportion of CUSTOMERs who Walt. (4.11)

We use expressions (4.1 0) and (4.11) in the output. Since waiting time has a continuous
distribution function for all w > 0, the difference between the values that expressions (4.9)
and (4.10) generate should be negligible.

4.6 TRAFFIC INTENSITY

The "define" statement in line 21 defines global variables whose modes are double, whereas
line 23 defines global integer variables (by default). Table 4.1 relates SIMSCRIPT 11.5 and
problem equivalents. The "define to mean" statement in line 22 merely associates the global
variable M with the SIMSCRIPT 11.5 generated variable n.RESERVATIONIST. Although the
"define to mean" statement in line 24 serves a similar purpose to that of lines 25 through 30,
it illustrates good simulation procedure as well. For the moment, let us ignore the existence
of the recorded message, and note that 1/A is the mean intercall time and 1/w is the mean
service time for one-way trips. In the simulation we assume that the mean service time for a
round trip is 2/w, so that the mean service time in general is (1- p )/w+ 2p/w = (1 + p )/w.
Then the system traffic intensity is

mean service time ).,(1 + p)

p'- - (4.12)
.- no. of reservationists x mean intercall time - mw'

Since we customarily consider alternative combinations of A, w, and p, the possibility

arises that we may inadvertently select a setting such that p 2: 1. In a simulation with p > 1,
the number of waiting calls increases without bound. For example, this would occur if one
inadvertently specified the number of RESERVATIONISTs as m < A(1 + p )w, a potential
scenario when one is trying to reduce resources. Since this nonstationary situation is contrary
to intention, we regard a check on p < 1 as essential. This is done in the main routine, to

Table 4.1 Airline simulation

Problem SIMSCRIPT 11.5
variable equivalent
)., LruMBDA
(J) OMEGA
p p
m M, n.RESERVATIONIST
116 CHAPTER 4. PROGRAMMING AND EXECUTION

be described shortly. The use of line 24 provides us with a convenient way of expressing
RRHO, which corresponds to p in expression (4.12), concisely. When exact evaluation of p
is not possible, computable lower and upper bounds PL and Pu' respectively, can be useful.
If PL > 1, then P > 1; conversely, if Pu < 1 then P < 1.
We now consider the existence of the recorded message. Ifthe simulation runs for a long
period of time, then the number of calls that become available for service is independent
of the time delay due to the recorded message. Then A is the long-run arrival rate, and
consequently the activity level defined in expression (4.12) continues to apply.
If one is simulating a closed-loop system, then the traffic intensity is always less than
unity and no need exists for evaluating p. However, p < 1 in this case does not necessarily
imply a unique equilibrium distribution.
Line 31 defines SEEDS as a global one-dimensional array having an integer mode
because ofthe global declaration in line 2. Line 32 declares an integer function that determines
whether a one-way or round-trip reservation is desired. Line 33 also defines a double-
precision function S_TIME. We describe the contents of the KIND and S_TIME functions
shortly. Functions are similar to, but not identical to, routines. In the particular case of lines
32 and 33 we note that the subprogram associated with S_TIME has a double mode as a
result of the inclusion of "double" before "function." However, the function KIND has the
integer mode because of the default to line 2. The "end" statement tells the SIMSCRIPT 11.5
compiler that all global declarations have been made and what follows is the executable part
of the program.
The "main" routine is the first subprogram that SIMSCRIPT 11.5 executes. After defin-
ing input and output units in lines 2 through 5, it reads values (line 6) for LAMBDA, OMEGA,
M, and P from the specified input unit using a free-form format. Lines 7 through 9 execute
an error check using RRHO (line 24 of preamble) and demonstrate a simple "if" statement.

4.7 PSEUDORANDOM NUMBER GENERAliON

Many algorithms exist for randomly generating samples from diverse probability distribu-
tions, and each simulation language provides programs in its collection of library routines
for implementing many of these algorithms. All require a stream of pseudorandom numbers
as input. Theorem 4.1 describes one property that accounts for this link between a sample
from an arbitrary distribution and random numbers.

Theorem 4.1.
LetX bea random variable with distribution function (d.f) {F(x), a ::;: x ::;: b}, denoted
byF. Let

G(u) := inf [x: F(x) ~ uj, O::;:u::;:l,

 4.7. PSEUDORANDOM NUMBER GENERATION 117

which is called the inverse distribution function of X, and let U have the uniform distribution
on [0, 1], denoted brU(O, 1). Then G(U) has the d.f. F.

PROOF. Let Y := G(U). Then

pr(Y::: y) = pr[G(U)::: y] = pr[U::: F(y)]

= F(y) = pr(X ::: y).

This result has immediate implications for sampling from F. For continuous F, one can
sample U from U (0, 1) and solve F(X) = U for X. For example, if F(X) := 1 - e- AX for
° t
A > and x ~ 0, then X = - !n(I - U) has the exponential distribution with rate A. If
F is discrete with points of increase at, say, a, a + 1, ... , b - 1, b and F (a - 1) := 0, then
X = a + I is distributed as F, where

I := min [i: F(a +i - 1) < U ::: F(a + i)]. o

i=O.I ..... b-a

Chapter 8 describes other methods for generating random samples, all of which pre-
sume the availability of a sequence of independent random samples from U (0,1). To
approximate this idealization, every simulation language provides a procedure for gener-
ating a sequence {Ui := ZdM, i = 1,2, ... } where Zo, Zb Z2,'" are integers often
derived from the one-step recursion

Zi = AZi - 1 (modM), i = 1,2, .... (4.13)

Here A and M are integers selected to make the points {Ud sufficiently dense in (0, 1) and
to have distributional properties that "closely" resemble those of the idealized i.i.d. uniform
sequence. Expression (4.13) is called a pseudorandom number generator (png). GPSS/H,
SIMAN, SLAM, and SIMSCRIPT 11.5 all use pngs of this form.
To initialize the png at the beginning of a simulation experiment, one provides a seed

°
Zo for expression (4.13). This seed dictates the subsequence of numbers that expression
(4.13) generates during execution. SIMSCRIPT 11.5 actually has 1 pseudorandom number
streams, each of which requires a seed. The seed for stream I is stored in a system variable
seed.v(I), which assumes a default value if none is assigned by the user. Although Chapter
9 deals with issues related to pseudorandom number generation in detail, it is important
that the reader now recognize that well-designed computer simulation experiments call for
explicit identification of the first and last pseudorandom numbers used in each stream that
an experiment employs. One reason is reproducibility.
Recording the seeds, which are the first pseudorandom numbers used, enables one to
run a related experiment with the exact same stream of numbers but different values for
other input parameters, and then to compare experimental results with one major source of
variation, the differences in pseudorandom numbers, removed. Section 4.22 returns to the
118 CHAPTER 4. PROGRAMMING AND EXECUTION

topic of "common pseudorandom numbers" and shows how they can reduce the computing
time to compare the output of two runs of a simulation, with different input values for one
of its parameters, to within a given statistical accuracy.
Line 10 in the main program reserves space for the array SEEDS (line 31 of preamble).
Lines 11 through 14 read four user-specified seeds into the SEEDS array and then assign
SEEDS(I) to seed.v(I). Thus SEEDS(I) contains the first random number used in stream I.
When the simulation terminates, seed.v(I) contains the seed to be used in the next generation
from stream I. In particular, stream 1 generates pseudorandom numbers for interarrival
times, stream 2 for service times for single-destination trips, stream 3 for the additional
service time for multiple-destination trips, and stream 4 for determining values for TYPE.
The "loop"statement in line 14 demarcates the end of the DO loop that begins in line 11.
For a given integer I::::: 1, the statements
release seed.v(*)
reserve seed.v(*) as I
cause SIMSCRIPT 11.5 to resize seed.v. This would be required if I > 10 and should be
inserted in the main program prior to assigning seeds to seed.v.
SIMSCRIPT 11.5 allows coding of more than one statement on a single line. For
example, we could have written lines 11 through 14 as

for I = 1 to 4 do read SEEDS(I) let seed.v(l) = SEEDS(I) loop,

and SIMSCRIPT 11.5 would interpret this in identical fashion.
By recording the last pseudorandom numbers one can run the simulation program
again with these numbers as seeds but with the same values for other inputs, thereby creating
an independent replication of the experiment.

4.8 WARM-UP AND PATH-LENGTH PARAMETERS

Line 15 reads T, the number of calls to be completed before the simulation terminates,
and K, where K-l specifies the number of completed calls to be discarded before data
analysis begins for final reporting. This truncation of data reduces the influence of the initial
conditions that prevail at the beginning of the simulation on the values of the data collected
to estimate steady-state parameters. Initially we set K = 1 merely for illustration. We return
to the important issue of selecting an appropriate K in Section 6.1 and show the error that
potentially can arise by neglecting this issue.
The statement in line 18 creates n.RESERVATIONIST = M records of RESERVATION-
1ST, where M receives its value in line 6. Remember that each record of a RESERVATIONIST
includes a field for the attribute STATUS, which, by default, has the integer mode.
Line 17 creates an event notice CALL with a desired execution time equal to the current
simulated time, time.v, a SIMSCRIPT 11.5 system variable measured in double-precision
 4.10. STARTING AND STOPPING EXECUTION 119

decimal days. Since this action occurs at the beginning of execution and since SIMSCRIPT
II.5 initializes all system and declared variables to zero, time. v = 0.0 at this point in simulated
time.

4.9 EVENT MANAGEMENT

As we remarked earlier, event notices are filed in the future-event set in order of their desired
execution times. SIMSCRIPT II.5 actually has a more elaborate structure for filing and
retrieving event notices. For each class of events defined in the preamble, SIMSCRIPT II.S
establishes a distinct future-event set (e.g., Section S.4.2). When an event notice of a particular
class is created, it is filed in the set corresponding to that class in chronological order of desired
execution time. When control returns to the SIMSCRIPT II.S timing routine, it examines
the first entry in each of the future event sets and determines which event to execute next.
The priority ordering among events established in the preamble resolves ties. In the present
problem there exist three classes of events, CALL, COMPLETION, and END..MESSAGE.
This means that the timing routine examines the first entry in each of three sets each time
it selects an event.
This multiple-event set feature caters to simulation with few events but many extant
event notices per event. Conversely, the feature slows down the execution time of a simulation
with many events most of which rarely occur. To circumvent this inefficiency one can define
an omnibus event with an attribute whose value directs program flow within the event to one
of several user-defined subprograms that contain the appropriate statements for effecting
alternative types of state changes. This procedure simply reduces the number of event classes
and increases the number of events in the future-event set corresponding to the omnibus
event.

4.10 STARTING AND STOPPING EXECUTION

When SIMSCRIPT II.5 encounters line 19, it transfers control to SIMSCRIPT II.s's timing
routine, "time.r:' which selects the first event notice in its future event set, "ev.s." Once the
simulation and concomitant analysis are completed, a programmer may choose to write
code to terminate execution in one of several ways. One way returns control to the statement
following "start simulation:' which in this case leads to an end of execution in line 20. We
illustrate an alternative approach shortly in line 12 of the ACCOUNTING subroutine.
120 CHAPTER 4. PROGRAMMING AND EXECUTION

4.11 CALL EVENT

Line 4 of the CALL event directs SIMSCRIPT 11.5 to create a record corresponding to the
record image declared in the preamble for a temporary entity CUSTOMER. This record in-
cludes space for the attributes TYPE and START. At the same time SIMSCRIPT 11.5 treats the
word CUSTOMER as a global variable and assigns to it the address within the computer of the
newly created record. In effect, CUSTOMER becomes a pointer so that TYPE(CUSTOMER)
is the attribute TYPE of this newly created CUSTOMER for the remainder of the execution
of this CALL event.
Note that line 5 of the CALL event uses TYPE without a subscript. Doing so demon-
strates the implicit subscripting feature of SIMSCRIPT 11.5, which interprets TYPE to mean
TYPE (CUSTOMER). If one wishes to access the TYPE attribute of another customer whose
record address had been previously assigned during execution to a global variable CCUST,
then TYPE( CCUST) would be that attribute. Line 5 assigns to TYPE (CUSTOMER) the value
of KIND, an integer-valued function defined in line 31 of the preamble. The corresponding
subprogram appears below the CALL event in Figure 4.2. In line 2, random.f(4) denotes a
SIMSCRIPT 11.5 library subprogram that produces a number from the uniform distribution
on the interval (0, 1) using pseudorandom number stream 4. If this number is less than P, the
probability of multiple destinations, then SIMSCRIPT 11.5 assigns the value of MANY (see
preamble) to KIND. Otherwise, it assigns the value of ONE. The demarcating word "else"
is syntactically equivalent to "otherwise:' The justification for this assignment procedure is
easy to see. If U is uniformly distributed on [0, 1], then

pr( U ::::: p) = 1
P du = p. (4.14)

Line 6 of the CALL event assigns to START (CUSTOMER) the current simulated
time time.v. This will be used later to compute the CALL's waiting time. Line 7 directs
SIMSCRIPT 11.5 to use the event notice that generated the current execution of this CALL
event to schedule the execution of the next CALL event in T.CALL (line 3) minutes. Here
"reschedule" is syntactically equivalent to "schedule:' If"a" replaces "this;' then SIMSCRIPT
U.5 creates a new event notice. This difference is important with regard to program efficiency.
Use of "a" requires SIMSCRIPT 11.5 to find and allocate space to the new event notice. Use
of "this" eliminates this often time-consuming search and allocation procedure. The reader
should note that in order to use the CALL event notice again, the words "saving the event
notice" must appear in line 1. Otherwise, SIMSCRIPT 11.5 destroys the event notice when
execution of the corresponding event is about to begin.
Note also in line 7 that the event is scheduled to occur in T.CALL minutes. Since time.v
is in double-precision decimal days, the use of "minutes" causes SIMSCRIPT 11.5 to schedule
the next CALL in time.v+ T.CALL/1440.0 days, since there are 1440 minutes in a day. For
consistency CALL.RATE and SERVICE.RATE, read in the main subprogram, must be in calls
per minute.
 4.13. FI LIN G PRIO RI TIE S 121

Line 8 indicates an attempt to arrange service by calling the SERVICE subprogram.

It yields a value assigned to the variable ACTION, which is local to the CALL event. If no
success occurs (line 9), then a recorded message is played for the CUSTOMER that ends
in 9 seconds or, equivalently, 0.15 minutes (line 10). If success occurs, then the counter
ZERO.WAIT is incremented by 1 in line 12. The statement "always" in line 13 denotes the
end of the sequence of conditional statements beginning in line 9. Consequently, after line
13 is executed, a return to the timing routine occurs.

4.12 END_MESSAGE EVENT

When an END~ESSAGE event executes, its local variable CALLER contains the identifi-
cation of the CUSTOMER assigned in the schedule statement in line 10 of the CALL event
made at time time.v- .15/1440 days. Assignment of this identification to the global variable
or entity CUSTOMER in line 3 is necessary because during the elapsed time a new CALL
event may have occurred, and hence CUSTOMER may refer to a new entity. If an attempt
at service (line 4) fails again (line 5), then the CUSTOMER is filed in the set called QUEUE.
Ifline 12 of the preamble had failed to mention that a CUSTOMER could belong to the
set QUEUE, then the "file" statement here would induce an error message during execution
indicating an attempt to file an entity in a set for which no corresponding membership dec-
laration exists. In lieu of explicit declaration to the contrary in the preamble, CUSTOMERs
are filed in the order in which they encounter this file statement.

4.13 FILING PRIORITIES

Suppose that we add the statement

define QUEUE as a set ranked by low TYPE

after line 12 in the preamble. Then an encounter of the file statement in line 6 of the
END~ESSAGE event would cause an ordering first on TYPE, with one-way trips coming
first, and then, if ties occur, an ordering on time of filing. In the problem under study
the present preamble declares behavior closer to reality than the example given here does.
However, other contexts may call for the use of alternatives similar to this example.
Several additional issues deserve attention. Suppose that we deletelines 11, 12, and 13 of
the preamble and line 4 of the CALL event. Then the event notice CALL could serve as a record
of a temporary entity instead of CUSTOMER, "a" would replace "this" in line 7 of the CALL
event, and CALL would replace CUSTOMER in lines 3 and 6 of the END_MESSAGE event.
The reader should recognize that an event notice is a specialized temporary entity and can
function as such in many convenient ways. Although no great computational improvement
122 CHAPTER 4. PROGRAMMING AND EXECUTION

accrues from this alternative approach here, situations may arise in practice where it is
desirable to keep the number of temporary entities to a minimum. Note that set membership
for CALL in QUEUE must also be declared.
The local variable ACTION in the END~ESSAGE event is not the same as the local
variable ACTION in the CALL event. We chose the same labels to demonstrate that two or
more subprograms can have local variables with the same names. In fact, a local variable
can have the same name as a global variable, provided that the subprogram contains an
appropriate "define" statement. For example, if we include

define RRHO as a real variable

in the END~ESSAGE event, the local variable RRHO would be distinct from the global
variable RRHO defined in the preamble.

4.14 COMPLETION EVENT

Line 3 of the COMPLETION event instructs the simulation to release the space occupied by
the record of the temporary entity CUSTOMER called CLIENT. Failure to release space in
this manner may cause the simulation to run out of space when it needs to create records of
new temporary entities. Line 4 indicates that if the set QUEUE is empty, then the RESER-
VATIONIST called WORKER should have his attribute STATUS set to IDLE. Because of line
24 of the preamble, this action is equivalent to

let STATUS(WORKER) = 0 .

Lines 4 through 7 of the COMPLETION event imply that if no CUSTOMERs await service,
then NO.BUSY should be reduced by 1, the corresponding RESERVATIONIST should be
put in the idle mode, and control should return to the timing routine.
Recall that the prefix n. before a set name denotes the number of entries currently in
that set. Therefore, n.QUEUE denotes the number of CUSTOMERs in the QUEUE, so that
the statement

if QUEUE is empty

is equivalent to

if n.QUEUE =0.
The programmer has her or his choice of which to use.
If the set QUEUE is not empty in the COMPLETION event, then control passes to
line 9, where the "first" CUSTOMER is removed, since he or she is about to receive service
 4.14. COMPLETION EVENT 123

from the RESERVATIONIST with name WORKER who has just completed service. Failure
to include line 9 results in an increasing list length even though CUSTOMERs receive service.
The assignment of the name WORKER to the entity name RESERVATIONIST is neces-
sary if the SERVICE routine is to work correctly. The quantity event.v is a system variable that
contains the class number i.event of the event that began execution last. Class numbers are
assigned according to the priority order statement in the preamble. Class numbers for CALL,
END~ESSAGE, and COMPLETION are i.CALL, i.END_MESSAGE, and i.COMPLETION,
respectively, and assume values of 1,3, and 2, respectively. Now all three classes of events call
the SERVICE routine. In case of CALL and END_MESSAGE, a search for an idle RESER-
VATIONIST is necessary in lines 3 and 4 of the SERVICE routine. For a COMPLETION,
control passes to line 9, since the RESERVATIONIST is already known. S_TIME is a function
that we describe shortly.
If no RESERVATIONIST is idle (line 5), then the routine yields NO.SUCCESS, which
is 1, according to the preamble. When success occurs, RESERVATIONIST denotes the idle
server whose STATUS we want to change from IDLE to BUSY. Note that

let STATUS = BUSY

is equivalent to

let STATUS (RESERVATIONIST) = BUSY,

again demonstrating the implicit subscripting ability ofSIMSCRIPT 11.5. However, note that
substitution of the statement

let STATUS = IDLE

in line 5 of the COMPLETION event would probably cause an error, since there is little
likelihood that RESERVATIONIST = WORKER because of the passage of time. Line 10 of
the SERVICE routine merely increments the user-defined variable NO.BUSY by 1.
Line 12 indicates the scheduling of a COMPLETION for the current value of RESER-
VATIONIST. Line 13 calls a user-written subroutine ACCOUNTING that checks on the
progress of the simulation with respect to truncation, computes WAITING.TIME for the
CUSTOMER entering service ifNUM ::: K, and updates NUM. Line 14 of COMPLETION
assigns SUCCESS to ACTION for the benefit of the calling event.
The S_TIME subprogram determines service time from the Erlang distribution in
line 3 for a multiple-destination trip and from the exponential distribution in line 5 for a
one-stop trip. Note that S_TIME is double precision. Failure to make this declaration in the
preamble would result in S_TIME being treated as integer. SIMSCRIPT 1I.5's structured "if"
statement has the form

if a condition is true, perform the statements listed here

124 CHAPTER 4. PROGRAMMING AND EXECUTION

else if the condition is false, perform the statements listed here

always in either case continue execution after this key word.

IfNUM 2: K, the ACCOUNTING subroutine updates WAIT.TIME in line 9 with the

waiting time, time.v-START( CUSTOMER), of the customer who is about to begin service.
The factor 1440 converts time in double-precision decimal days to double-precision decimal
minutes. Lines 3 through 7 concern truncation. Recall that we want to collect summary
statistics onlyoncustomersK, K+ 1, ... , K+ T -1. The global variable NUM contains the
number of customers who have or are about to receive service. When NUM = K in line 3,
SIMSCRIPT 11.5 executes lines 4, 5, and 6. Line 4 resets the totals of all summary statistics,
thereby removing any dependence on the data for the previous K - 1 customers. That is, for
simulated time r 2: r KSIMSCRIPT 11.5 computes utilization summary statistics as

AVE.UTIL = B(r -l,r), (4.15)

[2
K

V.UTIL = 1 B (r)(r - tJ(T»)' (4.16)

r - r -l K

+. L
J(T) ]
B2(ti_d(ti - ti-l) - (AVE.UTIL)2,
,=J(TK_1)+1

MAX.UTIL = max B(s),

TK _ 1 :9::: T
(4.17)

J(T)
UTIL.HISTO(I) = 8(B(r) - I)(r - tJ(T») + L 8(B(ti-d - 1+1) (4.18)
i=J(TK_1)+1

x (ti - ti-d, I = 1, ... , n.RESERVATIONIST,

summary queue length statistics as

AVE.Q = Q(rK_1.r) (4.19)

Y.Q = 1
r - r
Q
K_ 1
[2 (r)(r - tJ(T») (4.20)

+. L
J(-r) ]
Q2(ti_l)(ti - ti-l) - (AVE.Q)2,
,=J(TK_1)+1

MAX.Q = max Q(s), (4.21)

t'K_l ~s:::t'

8(Q(r) - I + 1)(r - tJ(T»)

J(T)

Q.HISTO(I) = + L 8(Q(ti-d)(ti - ti-d, 1= 1, ... ,32,

(4.22)
i=J(TK_1)+1

L Q.HIST(l),
32
r - 1=33,
1=1
 4.15. GETTING AN EXECUTABLE PROGRAM 125

and summary waiting time statistics

1
L
N(r)
AVE.WAIT = Wi, (4.23)
+1

II: W?-
N(r) - N(rd i=N(rK)

V.WAIT = 1 [N(r)-N(rKH1](AVE.WAIT)2},(4.24)
N(r)-N(rK) i=N(rKl

MAX.WAIT = max Wj , (4.25)

N(rK)~j~N(r)

N(r)

L
i=N(rK)
h25(I-I),.25I] (Wi), 1= 1, ... , 32,
WAIT.HISTO(I) = (4.26)
N(r)
L /[32,00) (Wi), 1=33.
i=N(rKl

In the present case, note that N (rK ) = K and N (r) = K + T - 1. Also note that line 5
reinitializes ZERO.WAIT.
Line 6 records the current time that the SUMMARY subroutine needs to normalize
the time-integrated statistics properly. Note that line 2 defines DEAD. TIME as a double-
precision saved variable. The qualifier "saved" ensures that DEAD.TIME retains its value
between successive calls to ACCOUNTING. Omission of "saved" would cause DEAD.TIME
to be initialized to zero at the beginning of each execution of ACCOUNTING.
If NUM = K + T - 1, then the simulation has been executed for the number of
customers K + T - 1 specified in the main routine. It then calls the SUMMARY subroutine,
which prints out all input parameter values in its line 7, all summary statistics in lines 9
through 21, and then prints the user-specified input seeds and the final numbers for each
pseudorandom stream in line 22. The "stop" statement in line 13 of ACCOUNTING provides
one way to terminate the simulation's execution.

4.15 GETTING AN EXECUTABLE PROGRAM

SIMSCRIPT 11.5 is available for most mainframe computers, IBM -compatible personal com-
puters, ~nd Unix-based workstations. While each implementation has its own peculiarities,
they all share common features an understanding of which is essential in trying to obtain an
error-free executable program.
After preparing a source program in SIMSCRIPT 11.5, a user submits it for compilation,
the process ofconversion from the source program in the syntax ofSIMSCRIPT 11.5 to an object
program in the basic machine language of the computer. The SIMSCRIPT 11.5 compiler scans
the program. If the level of severity of the errors encountered is sufficiently high, the scan
terminates, the compiler produces a list of error messages for the portion of the program it
126 CHAPTER 4. PROGRAMMING AND EXECUTION

has scanned, and execution terminates. For example, an error in the preamble usually causes
a scan termination before the executable portion of the program is reached.
Alternatively, the compiler scans the program, and, if the level of error is below that
for scan termination, SIMSCRIPT 11.5 produces an object program that is storable in a
user-specified file. The compiler also produces a listing of the entire source program with
appropriate error messages. A natural question to ask is: What happens to statements with
errors? SIMSCRIPT 11.5 ignores them during compilation, even though object programs
are created for every subprogram. Consequently, the object programs corresponding to
subprograms with errors are worthless and should be discarded. When the user examines
her/his program listing, she/he identifies the errors, corrects them in the source program,
and attempts to recompile the program successfully.
SIMSCRIPT 11.5 also allows a user to compile and execute a program all in one sub-
mission. The appeal is apparent. If compiler-detected errors exist in the program, execution
usually terminates with an execution error message. Such error messages also may occur when
no compilation errors arise, since the user may have failed to construct an internally consis-
tent executable program. During execution, stategically user-placed LIST statements in the
source program can provide information about entities, attributes, sets, and other variables.
In addition to the diagnostic at compilation, SIMSCRIPT 11.5 also provides SIMDEBUG
for tracing the sequence of routine calls during a simulation that lead to an execution error
(CACI 1993b). Invoking the -d option at compilation enables this capability for subsequent
program execution.
Unless a user has considerable confidence that the source program contains no errors
detectable during compilation, he/she is well advised to use this combined compilation and
execution procedure sparingly. This advice follows as a consequence of the mandatory link-
age editing step that follows the compilation step and precedes the execution step. The linkage
editing step links the object program together with a series of routines in the SIMSCRIPT
11.5 library that are needed for execution but have not been previously incorporated. These
routines differ from those automatically compiled into the object deck. For example, in our
. airline program there are more than 50 such routines, including those for Erlang, exponen-
tial, and uniform variate generation. The combined object program plus SIMSCRIPT 11.5
routines form a load module. Linkage editing takes time, and a user who repeatedly uses
the compilation and execution option with a program with probable compilation errors is
wasting time.
We wish to point out that once a program has been successfully compiled and executed
the linkage editing step can be avoided on future runs. To accomplish this a user submits his
object program for linkage editing and then stores the resulting load module in a predesig-
nated file. To execute the program one submits instructions to take the load module from
the data set and execute it. This procedure takes negligible time.
This relatively brief description of SIMSCRIPT 11.5 creates a familiarity, not an in-
timacy, with the language. The reader who intends to do the SIMSCRIPT 11.5 exercises in
this book will find regular access to the CACI references indispensable for success. More-
over, thorough understanding of the process of conversion of a source program written in
 4.16. A FIR S T RUN 127

SIMSCRIPT II.5 into an object program coded in the basic machine language of the computer
can lead to considerable cost saving and to reduced frustration in trying to get an executable
object program. We describe several issues deference to which should reduce the needed
effort.

4.16 A FIRST RUN

Before executing AIR_EV.sim, we need to specify numerical values for the input variables
to be read in lines 6,12, and 15 of the main program. Since that routine designates unit 10
as input in line 2, we create an input file called SIMUI0 and load it with the input data.
Suppose that past experience indicates that)... = 1, (J) = .50, and p = .75. Then expression
(4.7) reveals that m > 3.5 is necessary for a stationary solution to exist for this problem.
Since m must be integral, we assign m = 4. For the initial run SIMUlO contains

main
program

line 6: .50 4 .75

line 12: 604901985 1110948479 1416275180 40579844
line 15: 1000

Line 6 provides the input for the "read" statement in line 6 of the main program. Line 12
provides the seeds for the four pseudorandom number streams. These seeds come from
entries in Table 4.2, which lists numbers that successively occur 1,000,000 numbers apart in
the stream of numbers that the SIMSCRIPT II.s pseudorandom number generator
produces. The entries are from column 0 and rows 0, 1,2, and 3, implying that successive
numbers are 10,000,000 apart in the pseudorandom number sequence. For AIR_EV.sim, this
choice of seeds implies that for runs made with T + K :::: 10 7 the four streams would not
overlap. Nonoverlapping streams are essential for independent sampling with regard to each
source of variation in this problem. "Line IS" ofSIMUlO provides values for T and K.
Of necessity, virtually every discrete-event simulation begins with guesses for K and T
and later revises them based on information that the preliminary run reveals. The present
account follows this approach. By setting K = 1, AIR_EV.sim uses all the data generated
during a run to compute statistics. Since this is a preliminary run, setting T = 1000 is merely
a guess on how long to run the simulation to obtain meaningful data.
Figure 4.3 displays the output for these choices. It summarizes descriptive statistical
behavior over the 0.6818 x 24 = 19.632 hours of simulated time that elapsed up to the
moment at which the 1000th CUSTOMER entered service. Tableaus similar in format to
this one are commonplace output for many discrete-event simulations. Unfortunately, they
contain no information on how well these sample averages approximate the long-term
128 CHAPTER 4. PROGRAMMING AND EXECUTION

Table 4.2 Seeds for SIMSCRIPT 11.5 Pseudorandom Number Generator (spaced 1,000,000 apart) Z 1,000,000 (10i+ j)

i \i 3
0 604901985 1911216000 726466604 622401386 1645973084 1901633463 67784357 2026948561 1545929719 547070247
1110948479 1400311458 1471803249 1232207518 195239450 281826375 416426318 380841429 1055454678 711617330
1416275180 788018608 1357689651 2130853749 152149214 550317865 32645035 871378447 108663339 199162258
40579844 1357395432 829427729 1967547216 1955400831 2003797295 2094442397 2037011207 234502811 989756374
1912478538 740735795 1417788731 457273482 1315849725 270156449 440381980 1658090585 766876137 695181594
1140617927 1763098393 862536069 944596746 1832412366 1370121812 349487370 1490679974 1137328972 717010525
1768908897 214786952 1001221991 810390021 466350553 285688338 511678575 417093409 1857363469 833556803
1168765321 1578167688 904774509 1230498508 349030873 1280945183 411345101 1477279210 523541596 44009158
8 1034152236 171671287 1486370683 758021537 675302256 766501909 1570038230 127735274 531051296 481363190
9 36897498 377244078 632839225 1113362767 996065998 1127104879 1070367500 1992322523 548733793 1489852891
10 1719061561 3280042 1657019267 1741267743 691072269 1090974938 763220480 1468416723 111113372 129823184
II 705619618 224040976 749810472 2013253460 792658328 86620226 1386345831 1735570696 1896821842 278150891
12 331083792 1598034603 236635316 252926587 471825789 37034748 376315326 2146164618 1999725174 1243665337
13 1336631754 438764943 957241771 1713987536 1948054280 1998497269 575575728 677819252 501253489 1945486566
14 1245688777 2000276877 1503544457 464753035 200460239 1434443717 1353395916 133529216 1070327290 1928287458
15 1384322130 893932629 1181351847 606871877 523684234 1002300248 791546478 1328653688 246224997 48717991
16 2127710456 497489986 869470589 61678709 1145621181 1695821109 369707566 2110142430 763197278 11382857
17 1246856358 1714587858 873019592 1279964778 1396199241 1399930491 2078774631 985879465 486028376 1478854650
18 568269458 1657131937 1622687092 91313597 2080365494 1650386438 1871205625 563020663 1728323861 590715859
19 513501749 793321696 1443190012 1950895003 597186867 50824927 911425540 398409796 1672302183 1870294907
20 1827350617 727130390 774781557 1955524233 87547656 1061350608 1491284628 2029219172 1260286972 272221686
21 300327217 1787384331 2014193199 666978949 301608013 948450895 1453252697 419227541 1678821481 1765439952
22 177708831 1077885023 540589041 2007603355 1543903327 2017128524 1405189468 1003195553 337684924 1565288286
23 1745588809 485902835 1763594466 1082423631 194255479 1912029349 1219836142 1648556563 1508071873 549212365
24 1096265301 851153926 456184535 1408460673 1685622150 190036881 142510264 1465148875 459174572 597953519
25 606282883 1721436086 975289131 1376880454 1041829784 1596248327 2108842626 413468719 1204553786 9011429
26 626121779 203036030 1837216393 2086017536 697945492 1610367790 1512734966 925844033 1356793564 1549516128
27 649444368 510292152 1742847415 1285151791 1006048803 1538805236 1767548554 1446954807 1842805571 1969150158
28 2135785600 906045904 784888097 1048325198 416733559 993764136 1303965501 121604326 1780949928 1259010827
29 2102580471 741705624 1349641363 1376265619 653106395 271655202 1907239477 1015954941 1938162956 371946026
30 1842460388 543303725 1390660531 1138405358 31074996 1712456909 1247559532 1617264816 755962474 804894664
31 1186319334 494117518 1634124003 1027439985 319326476 1043817176 1323412056 668757796 406168599 1317790466
32 625592935 1520076665 1900759891 1684472723 209831417 2007811944 587647848 1424142232 6938939 427509950
33 1174409129 210461470 539786056 1629541058 1819341982 43434266 1391309379 244792174 750490187 423505883
34 904658495 245177844 841290082 1402203157 429993932 1836064376 1038894007 1923849909 1600794244 1950123859
35 960647611 974174009 1934369904 808615644 1879784225 795995555 1922561547 69238569 961625762 163574669
36 1095421583 554291019 432345723 855636598 971945436 881294334 1783677724 2088374054 1270902641 1182268061
37 2004539543 216813966 1304722830 629904199 834124981 898116231 864082217 389917225 1278587457 1869657489
38 1111329860 1594031833 571644852 1053085806 1019423302 482661750 1134475119 1104046462 1929367108 1097726063
39 295080740 1818474208 1129948309 75253501 1876330799 506183851 1692164827 873044608 1908879123 1897274087
40 1373006294 445158089 292594684 1305725195 847492301 1067706417 1718084174 133626496 971565385 1931931986
41 1677252989 312760713 891757911 776146802 462527160 1917080512 1349286825 909926550 1756656652 1570226967
42 105992480 1444798270 523658664 1024065462 299634195 884544980 1281114105 2106315124 869738253 1165668988
43 1036972300 462764735 314834790 222634789 1852359919 1085683972 115959636 695883743 1545554827 1372269350
44 287358587 2022531413 535827657 1657359673 904942635 317419570 576435644 1843235552 1761208654 912499785
45 1230888777 1724910189 1288149046 897051070 1758822711 1977931462 451692590 1554328303 1935881922 49510711
46 14291789 46830509 499758343 758082642 1018016755 1635908329 1577325787 423614699 1901481113 763520381
47 251686092 860899664 1253266716 487207054 1133665090 858964589 2104231773 554226602 126548109 568187190
48 71320167 1595587069 1886787081 883887920 1470032706 817226546 1572454264 766098912 1700253028 230254384
49 1488181869 1104477402 2087453595 1544780480 826720767 1926533288 23680406 97080449 2139633377 994728742

averages of interest. In particular, they contain not a clue as to how large K should be to
make the influence of initial conditions negligible on the displayed summary statistics.
To make this issue of statistical accuracy more salient, AIR-EV.sim was rerun with only
T changed from 1,000 to 1,000,000. This was motivated by the observation that since a run
with T = 1000 took less than a second, a run with T = 1,000,000 would take no more than
17 minutes. Figure 4.4 shows the corresponding output, which differs substantially from
that in Figure 4.3. For example, T = 1000 gives a sample average waiting time of 5.88179,
whereas T = 1,000,000 gives 4.29191. Because the summary in Figure 4.4 is based on a
larger sample path, one is inclined to believe that it is more representative of the long-run
averages. However, it provides no other indication of how good the approximations are and,
in particular, provides no guidance as to the choice ofK to mitigate the effects of starting the
simulation in the empty and idle state. Chapter 6 addresses both these issues in detail and
describes techniques for estimating a suitable initial truncation and for assessing how well
the time averages based on simulated data approximate the corresponding true long-run
averages of interest.
 4.16. A FIRST RUN 129

ANALYSIS OF SIMULATION RESULTS

NO. OF RESERVATIONISTS= 4 CALL RA TE= 1. 000 SERVICE RATE= .500

MULTIPLE-
DESTINATION PROB. = .750 ACTIVITY LEVEL= .875 NO. TRUNCATED=

STATISTICS based on 1000 waiting times and 6818 elapsed days

AVERAGE VARIANCE MAXIMUM

WAITING TIME +5.88179E+000 +3.30388E+001 +2.86073E+001
QUEUE LENGTH 5.8620 +3. 55975E+001 30
UTILIZATION 3.7135 +5.90100E-001 4

PROPORTION
WAITING TIME WHO WAIT QUEUE PROPORTION
(minutes) THIS TIME LENGTH OF TIME
W> 0 .8550 Q > 0 .7825
>= .25 .8420 > 1 .7183
>= .50 .8270 > 2 .6475
>= .75 .8120 > 3 .5660
>= 1.00 .7970 > 4 .4923
>= 1.25 .7760 > 5 .4352
>= 1.50 .7560 > 6 .3860
>= 1.75 .7370 > 7 .3212
>= 2.00 .7180 > 8 .2637
>= 2.25 .6980 > 9 .2036
>= 2.50 .6780 > 10 .1577
>= 2.75 .6590 > 11 .1261
>= 3.00 .6340 > 12 .1030
>= 3.25 .6100 > 13 .0877
>= 3.50 .5890 > 14 .0802
>= 3.75 .5710 > 15 .0785
>= 4.00 .5540 16 .0752
>= 4.25 .5280 17 .0664
>= 4.50 .5170 > 18 .0574
>= 4.75 .4970 > 19 .0442
>= 5.00 .4800 > 20 .0369
>= 5.25 .4650 > 21 .0278
>= 5.50 .4480 > 22 .0247
>= 5.75 .4330 > 23 .0221
>= 6.00 .4170 > 24 .0190
>= 6.25 .3990 > 25 .0149
>= 6.50 .3820 > 26 .0110
>= 6.75 .3660 > 27 .0080
>= 7.00 .3490 > 28 .0040
>= 7.25 .3240 > 29 .0008
>= 7.50 .3110 > 30 O.
>= 7.75 .2950 > 31 O.
>= 8.00 .2810 Q >=32 O.

NO. OF BUSY
RESERVATIONISTS PROPORTION
> 0 .9925
> 1 .9616
> 2 .9064
> 3 .8530

RANDOM NO. RANDOM NO. RANDOM NO. RANDOM NO.

STREAM 1 STREAM 2 STREAM 3 STREAM 4
---------- ---------- ----------
FIRST NO. 604901985 1110948479 1416275180 40579844

LAST NO. 279098565 1319150004 1298427771 1408862871

Figure 4.3 AIR_EV.sim sample output (1000 callers)

130 CHAPTER 4. PROGRAMMING AND EXECUTION

ANALYSIS OF SIMULATION RESULTS

------------------------------

NO. OF RESERVATIONISTS= 4 CALL RATE= 1.000 SERVICE RATE= .500

MULTIPLE-
DESTINATION PROB.= .760 ACTIVITY LEVEL~ .875 NO. TRUNCATED=

STATISTICS based on 1000000 waiting times and 694.0694 elapsed days

AVERAGE VARIANCE IlAXIMUM

WAITING TIME +4. 29191E+000 +3.10938E+00l +5. 57573E+00l
QUEUE LENGTH 4.1841 +3. 51154E+00l 64
UTILIZATION 3.4963 +9. 13621E-00l 4

PROPORTION
WAITING TIME WHO WAIT QUEUE PROPORTION
(minutes) THIS TIME LENGTH OF TIME
W> 0 .7336 Q> 0 .6161
>= .25 .7070 > 1 .5262
>= .50 .6794 2 .4484
>= .76 .6525 3 .3816
>= 1.00 .6261 > 4 .3242
>= 1.25 .6004 5 .2757
>~ 1.50 .5757 6 .2350
>= 1.75 .5516 > 7 .2006
>= 2.00 .5281 > 8 .1713
>- 2.26 .5054 > .1461
>= 2.50 .4838 > 10 .1249
>= 2.75 .4630 > 11 .1070
>- 3.00 .4429 > 12 .0917
>= 3.25 .4237 13 .0785
>= 3.50 .4055 14 .0673
>= 3.75 .3880 > 15 .0578
>- 4.00 .3711 > 16 .0496
>= 4.25 .3548 >17 .0425
>- 4.60 .3394 > 18 .0362
>- 4.75 .3249 > 19 .0309
>= 6.00 .3108 > 20 .0263
>= 6.26 .2974 > 21 .0223
>= 5.50 .2846 > 22 .0189
>= 5.75 .2724 > 23 .0160
>= 6.00 .2608 > 24 .0137
>= 6.26 .2496 > 25 .0117
>= 6.50 .2389 > 26 .0100
>= 6.75 .2284 > 27 .0085
>= 7.00 .2189 > 28 .0073
>= 7.25 .2097 > 29 .0062
>= 7.50 .2008 > 30 .0063
>- 7.75 .1922 > 31 .0046
>- 8.00 .1843 Q >=32 .0046

NO. OF BUSY
RESERVATIONISTS PROPORTION
> 0 .9854
> 1 .9341
> 2 .8436
> 3 .7331

RANDOM NO. RANDOM NO. RANDOM NO. RANDOM NO.

STREAM 1 STREAM 2 STREAM 3 STREAM 4
---------- ---------- ---------- ----------
FIRST NO. 604901985 1110948479 1416275180 40579844

LAST NO. 425804541 1400311458 1881615938 1163233127

Figure 4.4 AIK.EY.sim sample output (1,000,000 callers)

 4.18. PROCESS-INTERACTION ApPROACH 131

4.17 COLLECTING DATA

Although AIR-EY.sim produces a tableau of summary statistics by executing its SUMMARY
subroutine at the end of a simulation run, it does not preserve a record of the actual sample
sequences on which the summaries are based. For example, it averages the waiting times in
AVE.WAIT in the ACCOUNTING subroutine but does not save the actual waiting times. As
Chapter 6 shows, these data are essential for assessing the influence of initial conditions on
AVE.WAIT and for estimating its variance. Adding the statements

define W as a double variable

to the preamble betweens lines 20 and 21,

reserve W(*) as T

to the main routine between lines 9 and 10, and

letW(NUM-K+l) = 1440.*(time.v-START(CUSTOMER))

to the ACCOUNTING routine between lines 8 and 9 directs SIMSCRIPT 11.5 to save the
waiting times in decimal days in the double-precision array W, which is 8T bytes in length.
Therefore, the size of the array grows linearly with T, which conceivably can create a space
problem as T becomes excessively large. Section 3.7 describes procedures for collecting
sample-path data on queue length and resource utilization.

4.18 PROCESS·INTERACTION APPROACH

As Chapter 2 indicates, process interaction relies on a conceptually different representation
of behavior than does event scheduling. In the event-scheduling approach, an event contains
all decision flows and updating that relate to the change in state the execution of the event
requires. Since time elapses between events, one can regard a simulation as the execution of
a sequence of events ordered chronologically on desired execution times. However, no time
elapses within an event.
By contrast, the process-interaction approach provides a process for each entity in a
system. Each temporary entity moves through the system and consequently through time.
Occasionally, a temporary entity encounters an impediment to progress and must wait.
For example, if all reservationists (permanent entities) are answering calls when a new
call (temporary entity) occurs, this call must wait for service. The inclusion of time flow
in a procedure and in its corresponding flowchart distinguishes this approach from event
scheduling. Moreover, it calls for a less direct form of computer programming in moving
132 CHAPTER 4. PRO G RAM MIN G AND EXECUTION

from flowchart to executable program. A process can also characterize a permanent entity.
For example, a process can describe the behavior of each reservationist in the airline problem.
From the event-scheduling perspective of Section 2.3, the airline reservation prob-
lem concerns interactions between two types of entities, CUSTOMERs (temporary) and
RESERVATIONISTs (permanent). One formulation of the process-interaction approach, as
in Figures 2.10 and 2.11, provides a process for each CUSTOMER and a process for each
RESERVATIONIST. Our SIMSCRIPT II.S rendering of this problem follows the logic in these
figures. Here, each CUSTOMER has an attribute TYPE that identifies whether a one-way
or two-way reservation is desired. TYPE plays the same role here as in the event-scheduling
approach in Section 2.3.

4.19 PROCESSES AND RESOURCES IN SIMSCRIPT 11.5

SIMSCRIPT II.5 offers a capability for performing a simulation using the process-interaction
approach using the executable process statements

activate a process name in time expression

create a process name
work time expression
wait time expression
interrupt process name called name
resume process time called name
suspend
reactivate process name called name
destroy process name called name

and using the executable resource statements

request quantity units of resource name (index) with priority expression

relinquish quantity units of resource name (index).

Italics in this list denote numbers and variables that the simulation programmer assigns. In
practice, one can use the process-interaction option in SIMSCRIPT II.5 to simulate a system
in two different but equivalent ways. One relies on the process statements exclusively. The
other exploits the resource statements "request" and "relinquish" as well. Section 4.20 de-
scribes the approach based exclusively on processes, and Section 4.21 describes the approach
using processes and resources.
 4.20. U SIN G PROC E SSE sEx C L U S IV ELY 133

4.20 USING PROCESSES EXCLUSIVELY

Figure 4.5 lists a SIMSCRIPT 11.5 program, based on Figure 4.2, which produces an output
analysis identical to the SIMSCRIPT 11.5 program in Figure 4.3. Line 3 of the preamble
defines two sets, QUEUE and IDLE. LIST. The key word "processes" (line 4) indicates that
subsequent statements define the processes in the simulation. In particular, a CUSTOMER
has the attributes TYPE and START and can be linked to QUEUE. A RESERVATIONIST has
an attribute DEMAND and can be linked to IDLE. LIST. If CUSTOMER had no attributes
and set membership, then one could write

processes include CUSTOMER.

Since a process has no system variable n.process analogous to n.entity for a permanent entity,
we use M (line 19) to serve this purpose globally for RESERVATIONISTs.
The main routine agrees with that in Figure 4.2, with two notable exceptions. Firstly,
the statement in line 18

for 1= 1 to M activate a RESERVATIONIST now

causes SIMSCRIPT 11.5 to create a process notice for each RESERVATIONIST. This process
notice contains fields for explicitly declared attributes in the preamble as well as fields to
store scheduling information. In the event-scheduling approach, SIMSCRIPT 11.5 assigns a
future events set ev.s(l) to each event notice where the priority order declared in line 16 of
the preamble determines I. SIMSCRIPT 11.5 makes a similar assignment for process notices.
In particular, I = 1 for RESERVATIONIST process notice and I = 2 for a CUSTOMER
process notice.
In addition to allocating space for each RESERVATIONIST process notice, the afore-
mentioned "activate" statement indicates in each time.a field that the desired execution time
is time.v = 0 and files the notices in ev.s(l) in order of creation. In short, "activate" per-
forms the same task for a process that "schedule" does for an event. The term "reactivate" is
synonymous with "activate:' Secondly, the statement

activate a CUSTOMER now

in line 19 of "main" causes creation and filing in ev.s(2) of a process notice for CUSTOMER
with time.a = O.
One issue regarding "activate" deserves explicit mention. Whereas the article "a" in
the aforementioned "activate" statement causes creation and scheduling, the statement

the
activate CUSTOMER now
this
134 CHAPTER 4. PROGRAMMING AND EXECUTION

Figure 4.5 SIMSCRIPT 11.5 AIK.PRl.sim program: processes

" AIR_PR1.sim (February 1998)

,,-----------------------------------------------------------------
PREAMBLE
,,-----------------------------------------------------------------
1 preamble
2 normally mode is integer
3 the system owns a QUEUE and an IDLE. LIST

4 processes
5 every CUSTOMER has a TYPE, a START and may belong to the QUEUE
6 define START and WAITING. TIME as double variables
7 define SERVER as a variable
8 tally AVE. WAIT as the mean, V.WAIT as the variance and
MAX.WAIT as the maximum of WAITING. TIME
9 tally WAIT.HISTO (0 to 8 by .25) as the histogram of WAITING. TIME
10 define ZERO. WAIT as a variable
11 every RESERVATIONIST has a DEMAND and can belong to the IDLE.LIST
12 define NO.BUSY and NO.RESERVATIONIST as variables
13 accumulate AVE.UTIL as the average, V.UTIL as the variance and MAX.UTIL
as the maximum of NO.BUSY
14 accumulate UTIL.HISTO (0 to 10 by 1) as the histogram
of NO.BUSY
15 accumulate AVE.Q as the average, V.Q as the variance and MAX.Q as the
maximum of n. QUEUE
16 accumulate Q.HISTO (0 to 32 by 1) as the histogram of n.QUEUE
17 priority order is RESERVATIONIST and CUSTOMER

18 define LAMBDA, OMEGA and P as double variables

19 define K,M,NUM and T as variables
20 define RRHO to mean LAMBDA*(l.+P)/(M*OMEGA)
21 define ONE to mean 1
22 define MANY to mean 2
23 define SEEDS as a i-dim array
24 define KIND as a function
25 define S_TIME as a double function
26 end
,,---------------------------------------------------------------
MAIN
,,---------------------------------------------------------------
1 main
2 open unit 10 for input
3 use unit 10 for input
4 open unit 15 for output
5 use unit 15 for output

6 read LAMBDA,OMEGA,M and P INITIALIZE

PARAMETERS
7 if RRHO>=1.0 print 1 line thus
ERROR-----ACTIVITY LEVEL>=l ERROR CHECK
8 stop " PARAMETERS
10 otherwise

11 reserve SEEDS(*) as 4
12 for 1=1 to 4 do
 4.20. U SIN G PROC E SSE sEx C L U S I VEL Y 135

13 read seed.v(I) ASSIGN SEEDS

14 let SEEDS(I)=seed.v(I) SAVE SEEDS
15 loop
16 read T and K
17 let NUM=l
18 for 1=1 to M activate a RESERVATIONIST now
19 activate a CUSTOMER now
20 start simulation
21 end
,,---------------------------------------------------------------
process CUSTOMER
,,---------------------------------------------------------------
1 process CUSTOMER
2 define PATRON as a variable
3 define T.CALL to mean exponential.f(l./LAMBDA,l)
4 let TYPE = KIND
5 let START = time.v
6 let PATRON = CUSTOMER
7 activate a CUSTOMER in T.CALL minutes
8 let CUSTOMER = PATRON
9 if the IDLE. LIST is not empty
10 remove the first RESERVATIONIST from the IDLE.LIST
11 let DEMAND = CUSTOMER
12 activate the RESERVATIONIST now
13 add 1 to ZERO. WAIT
14 else
15 wait .15 minutes
16 if n.IDLE.LIST = 0
17 file CUSTOMER in the QUEUE
18 else
19 remove the first RESERVATIONIST from the IDLE.LIST
20 let DEMAND = CUSTOMER
21 activate the RESERVATIONIST now
22 always
23 always
24 suspend
25 end
,,---------------------------------------------------------------
process RESERVATIONIST
,,---------------------------------------------------------------
1 process RESERVATIONIST
2 'AAA' if QUEUE is empty
3 file RESERVATIONIST in the IDLE. LIST
4 suspend
5 else
6 remove the first CUSTOMER from the QUEUE
7 let DEMAND = CUSTOMER
8 always
9 add 1 to NO.BUSY
10 call ACCOUNTING
11 work S_TIME minutes
12 subtract 1 from NO.BUSY
13 activate the CUSTOMER called DEMAND now
14 go to AAA
15 return end

See Fig. 4.1 for KIND, S_TIME, ACCOUNTING, and SUMMARY routines.
136 C HAP TE R 4. PRO G RAM MIN G AN D EXEC UTI 0 N

causes the scheduling of the reactivation of an already existing process named CUSTOMER.
Both the CUSTOMER and RESERVATIONIST statements offer illustrations of the "activate"
statement in this context.
Once control passes to the timing routine via the "start simulation" statement in line
20 of "main;' the routine scans the first entry in each list to select the next process, that
is to continue execution of scheduled events. Since line 16 of the preamble indicates that
RESERVATIONIST has priority over CUSTOMER, SIMSCRIPT II.s first activates a RESER-
VATIONIST. Control then passes to the RESERVATIONIST process, where a check of the
QUEUE occurs. Since this set is empty at the beginning of the simulation, the RESER-
VATIONIST is filed in the IDLE.LIST and then is "suspend"ed. The "suspend" statement
deactivates t.his RESERVATIONIST. In particular, it terminates execution, sets the implic-
itly assigned attribute sta.a (RESERVATIONIST) from "active," denoted by I, to "passive;'
denoted by 0, and records the address of the next statement as the reactivation point. Note
that failure to assign the RESERVATIONIST to a set, before invoking "suspend," can result
in a loss of that process to the simulation. After the first RESERVATIONIST is suspended,
control passes to the timing routine, where execution of the remaining M -1 RESERVATION-
ISTs occurs in like manner. After all are suspended, the timing routine finds one remaining
process, namely, the CUSTOMER, whose activation was scheduled in the main program.
The first executable statement in the CUSTOMER process (line 4) determines TYPE.
Line 5 assigns the call time to START, and line 6 stores the name of the current CUSTOMER
in a local variable PATRON. The need for this step is a consequence of the next activate
statement, which schedules the occurrence of the next call. Once the activate statement has
been executed, the global variable CUSTOMER refers to the next CUSTOMER, not the
currently active one. However, the statement

let CUSTOMER = PATRON

restores the designation to the currently active CUSTOMER and facilitates implicit
subscripting thereafter.
If the condition in line 9 is true, at least one RESERVATIONIST is idle. Then line 10
identifies the first of these, removes it from the IDLE. LIST, and line 12 signals the RESERVA-
TIONIST to provide service to the CUSTOMER identified in line 11. Recall that DEMAND
is an attribute of RESERVATIONIST. SIMSCRIPT II.s then executes the "suspend" state-
ment in line 24, which deactivates this CUSTOMER process. It is important to recognize
that the actual execution of the RESERVATIONIST reactivation occurs after this "suspend"
statement is executed.
If the IDLE.LIST is empty in line 9, then the program transfers to and executes line
14. Here, SIMSCRIPT II.s sets the time.a attribute of the CUSTOMER process notice to
time.v + 0.15/1440 days and files the notice in ev.s(2). It also indicates in the notice that the
subsequent line 16 is the reactivation point. After the wait, the CUSTOMER again checks
for the availability of a RESERVATIONIST in line 16. If at least one is idle, SIMSCRIPT II.5
executes lines 19,20,21, and 24. If none are idle, it executes lines 17 and 24.
 4.21. U5 I N G PRO C E SSE 5 AND RES 0 U RC E 5 137

In the CUSTOMER process, line 12 or 21 causes SIMSCRIPTIL5 to setsta.(f.IDLE.LIST)

to "active" and time.a (f.IDLE.LIST) to time.v, where f.IDLE.LIST is the address of the first
entry in IDLE.LIST. Moreover, line 11 or 20 assigns the address of the executing CUSTOMER
to the DEMAND attribute of identified RESERVATIONIST.
When the RESERVATIONIST process begins to execute in line 9, after being sus-
pended in line 4, its DEMAND attribute identifies the CUSTOMER to which service is being
provided. Like "wait;' the "work" statement in line 11 denotes an unconditional wait of
S_TIME minutes. After this time elapses, line 12 adjusts the value of NO.BUSY and line 14
signals the appropriate CUSTOMER that service has been completed. The RESERVATION-
1ST then transfers to line 2 to determine its next action. It next deactivates at either line 4
or 11. Then the CUSTOMER process, signaled in line 14 of the RESERVAIONIST process,
resumes execution at its line 24.
The KIND and S_TIME functions and the ACCOUNTING and SUMMARY subrou-
tines are identical to those in Figure 4.2. Although not used in the airline problem, two
additional statements deserve attention at this point. The statement

interrupt CUSTOMER called HARRY

enables a process to deactivate a currently active CUSTOMER process whose address is

stored in HARRY. In particular, sta.a(HARRY) is set to "passive;' the time remaining until
execution of HARRY is to continue is stored in his time.a attribute, and the process notice
is removed from ev.s(2), the list of scheduled events corresponding to the CUSTOMER
process. A subsequent

resume CUSTOMER called HARRY

enables a process to activate a currently inactive process called HARRY. In particular, it

sets time.a(HARRY) +- time.a(HARRY) + time.v, sta.a(HARRY) +- "active" and files the
process notice in ev.s(2) for eventual execution. Note the difference between "suspend" and
"interrupt:' The first allows self-inactivation; the second allows inactivation only of another
process.

4.21 USING PROCESSES AND RESOURCES

By exploiting the "resource" feature ofSIMSCRIPT 11.5, one can create an entirely equivalent
simulation of the airline reservation problem. Whereas a "process" behaves as a temporary
entity, a "resource" behaves as a permanent entity. Moreover, implicitly executable statements
associated with a "resource" can lead to reduced coding in some modeling simulations. Fig-
ure 4.6 describes a SIMSCRIPT 11.5 program that incorporates a "process" and a "resource;'
CUSTOMER and RESERVATIONIST, respectively. This program again employs the same
KIND and S_TIME functions as in Figure 4.2 but, because of a difference in implicitly created
system variables, requires modified ACCOUNTING and SUMMARY subroutines.
138 CHAPTER 4. PROGRAMMING AND EXECUTION

Figure 4.6 SIMSCRIPT 11.5 AIRJ>R2.sim program: processes and resources

AIR_PR2.sim (February 1998)

,,------------------------------------------------------------------
PREAMBLE
,,------------------------------------------------------------------
1 preamble
2 normally mode is integer
3 processes
4 every CUSTOMER has a TYPE and a START
5 define START and WAITING. TIME as double variables
6 tally AVE. WAIT as the mean, V.WAIT as the variance and
7 MAX.WAIT as the maximum of WAITING. TIME
8 tally WAIT.HISTO (0 to 8 by .25) as the histogram of WAITING. TIME
9 define ZERO. WAIT as a variable

10 resources include RESERVATIONIST

11 accumulate AVE. UTIL as the average, V. UTIL as the variance and MAX. UTIL
as the maximum of n.x.RESERVATIONIST
12 accumulate UTIL.HISTO (0 to 10 by 1) as the histogram
of n.x.RESERVATIONIST
13 define NO.RESERVATIONIST as a variable
14 accumulate AVE.Q as the average, V.Q as the variance and MAX.Q as the
maximum of n.q.RESERVATIONIST
15 accumulate Q.HISTO (0 to 32 by 1) as the histogram of n.q.RESERVATIONIST

16 define LAMBDA,OMEGA and P as double variables

17 define K,M,NUM and T as variables

18 define RRHO to mean LAMBDA*(1.+P)/(M*OMEGA)

19 define ONE to mean 1
20 define MANY to mean 2

21 define SEEDS as a 1-dim array

22 define KIND as a function
23 define S_TlME as a double function
24 end

,,---------------------------------------------------------------
" MAIN
,,---------------------------------------------------------------
1 main
2 open unit 10 for input
3 use unit 10 for input
4 open unit 15 for output
5 use unit 15 for output

6 let n.RESERVATIONIST=1
7 create each RESERVATIONIST
8 read LAMBDA,OMEGA,M and P " INITIALIZE
9 let u.RESERVATIONIST(1)=M
 4.21. USING PROCESSES AND RESOURCES 139

10 if RRHO>=1.0 print 1 line thus

ERROR-----ACTIVITY LEVEL>=l ERROR CHECK
11 stop PARAMETERS
12 otherwise
13 reserve SEEDS(*) as 4
14 for 1=1 to 4 do
15 read seed.v(I) " ASSIGN SEEDS
16 let SEEDS(I)=seed.v(I) " SAVE SEEDS
17 loop
18 read T and K
19 let NUM=l
20 activate a CUSTOMER now
21 start simulation
22 end

,,------------------------------------------------------------------
" process CUSTOMER
,J _________________________________________________________________ _

1 process CUSTOMER
2 define PATRON as a variable
3 define T.CALL to mean exponential.f(l./LAMBDA,l)
4 let TYPE=KIND
5 let START=time.v
6 let PATRON=CUSTOMER
7 activate a CUSTOMER in T.CALL minutes
8 let CUSTOMER=PATRON
9 if u.RESERVATIONIST(l) > 0
10 request 1 unit of RESERVATIONIST(l)
11 add 1 to ZERO.WAIT
12 otherwise
13 wait .15 minutes
14 request 1 unit of RESERVATIONIST(l)
15 always
16 call ACCOUNTING
17 work S_TIME minutes
18 relinquish 1 unit of RESERVATIONIST(l)
19 return end

,,------------------------------------------------------------------
routine ACCOUNTING
,,------------------------------------------------------------------

Replace NO.BUSY by n.x.RESERVATIONIST(l) and n.QUEUE by

n.q.RESERVATIONIST(l) in ACCOUNTING routine in Fig. 4.1.

,,------------------------------------------------------------------
routine SUMMARY
",J _________________________________________________________________ _

Replace Q.HISTO(l) by Q.HISTO(l,l), Q(HISTO(I) by Q.HISTO(l,I),

Q.HISTO(33) by Q.HISTO(33,I), and UTIL.HISTO(I) by
UTIL.HISTO(l,I) in SUMMARY routine in Fig. 4.1.

See Fig. 4.1 for KIND and S_TIME routines.

140 CHAPTER 4. PROGRAMMING AND EXECUTION

The preamble now has a new keyword "resources" in line 10. It prefaces the declaration
of the list of resources or servers to be used in the program when all have attributes. Since
in the present case we elect to define RESERVATIONIST without explicit attributes, the
statement

resources include RESERVATIONIST

suffices. The preamble also contains modifications of the "accumulate" and "tally" state-
ments. The need for these modifications becomes apparent to the reader as we describe the
effects that using "resources" induces in the main routine and CUSTOMER process.
Although similar to those in Figures 4.2 and 4.5, the main routine differs in several
important ways. In particular, n.RESERVATIONIST in line 6 of "main" refers to the number
of different classes of RESERVATIONISTs here, not to the number of reservationists (see
Section 4.3). This is a consequence of defining RESERVATIONIST as a resource instead of
as a process. The implicitly generated variable u.RESERVATIONIST(I) denotes the number
of available RESERVATIONISTs in class 1 and requires assignment of a value. Note that data
structures are created for each resource class (line 7), not for each resource. A comparison of
the CUSTOMER processes in Figures 4.5 and 4.6 indicates notable differences. In particular,
in Figure 4.6 the system variable u.RESERVATIONIST( 1) in line 9 of the CUSTOMER pro-
cess enables a CUSTOMER to determine whether a RESERVATIONIST is available. If one is,
the CUSTOMER executes the "request" statement in line 10. If one is not, the CUSTOMER
executes the "wait" statement in line 13 and upon reactivation executes the "request" state-
ment in line 14. The "request" and "relinquish" statements are powerful preprogrammed
bodies of code designed to mimic the steps commonly encountered in demanding resources
in a queueing simulation.
If a RESERVATIONIST is idle, the CUSTOMER executes the "request" statement in
line 10. If not, the CUSTOMER executes the "wait" in line 13 and upon reactivation executes
the "request" in line 14. The "request" statement invokes a SIMSCRIPT 11.5 system routine
designed to mimic the two alternative types of action that arise in a queueing simulation
when resources are demanded. To understand the options, a knowledge ofSIMSCRIPT 1I.5's
implicit behavior is necessary.
Whenever a preamble declares a "resource;' SIMSCRIPT 11.5 creates two associated
lists, q.resource and x.resource. At execution of a "request" statement SIMSCRIPT 11.5 creates
a record of a special temporary entity, qc.e, whose attributes include who.a, assigns the
address of the requesting process to who.a( qc.e), and files it in an implicitly created list or
set of resources owned by the requesting process. If u.resource is greater than or equal to the
number of requested units ofthe resource, the qc.e record is also filed in x. resource. Otherwise,
it is filed in q.resource to await service. If the desired number of units of the resource are
available, the process continues execution. For example, "work S_TIME minutes" would be
executed in line 17 of the CUSTOMER process. If the desired number of units of resource are
unavailable, execution ceases and control passes to the timing routine with the next executable
 4.21. USING PROCESSES AND RESOURCES 141

statement as reactivation point. For the airline reservation problem, the "request" statement
incorporates blocks 8, 9, 10, and 11 in Figure 2.10.
The statement

relinquish 1 unit ofRESERVATIONIST(l)

is unusually powerful in content, essentially corresponding to the logic in Figure 2.11. When
executed in line 18 of the CUSTOMER process, SIMSCRIPT 11.5 removes the temporary
entity record qc.e from x.RESERVATIONIST(l) and from the list of resources owned by
CUSTOMER who.a(qc.e) and files the CUSTOMER process notice for who.a(qc.e) in the
corresponding list of scheduled processes with its time.a(who.a(qc.e)) = time.v, which is
"now:' Next, it destroys the qc.e record and checks to see whether q.RESERVATIONIST(l)
is empty. If it is, then u.RESERVATIONIST(l) is augmented by 1. Otherwise, it removes
the first qc.e from q.RESERVATIONIST(l), files it in x.RESERVATIONIST(l), and files the
process notice corresponding to who.a( qc.e) in the single future event set with time.a( who.a)
= time.v. In the present example, ev.s(l), the list of scheduled CUSTOMER processes, is the
future-event set.
The programming convenience that the "request" and "relinquish" statements offer
is self evident. They reduce programming effort by replacing a sequence of more elemental
statements, as in Figure 4.5, that collectively would effect the same modeling logic. This
saving in effort occurs k times in a simulation with k stations in series; it can become
substantial as k increases. Every simulation that employs the process-interaction approach
contains macrostatements analogous to "request" and "relinquish:'
In exchange for this reduction in effort, a simulation user gives up explicit control over
some parts ofprogram flow. Although not necessarily detrimental, this loss of flexibility needs
to be understood explicitly. For example, the steps SIMSCRIPT 11.5 takes are not apparent
from the program in Figure 4.6. In particular, note that the RESERVATIONIST automatically
becomes committed when it is released if another process is waiting. This action can cause
a problem in other simulations. For example, consider the tanker problem in Exercise 4.7,
where a TANKER (process) may arrive in the harbor while the TUG (resource) is towing
out another TANKER and yet another TANKER has completed loading and has requested
the TUG. Here, the TANKER in the harbor would have to

request 1 unit ofTUG(l) with priority A,

where A is an integer larger than the priority with which the TANKER at the berth requested
the TUG. Although this would work here, one needs to avoid this statement if the TANKER
arrives while the TUG is heading in to the berths, since it would lead to an order of servicing
contrary to that intended.
At least one other feature deserves attention. It is entirely possible to employ events,
processes, and resources in the same SIMSCRIPT 11.5 simulation. SIMAN and SLAM II also
provide this option, which can introduce considerable flexibility in modeling a system with-
142 CHAPTE R 4. PROG RAMM I N G AN D EXEC UTIO N

activate
work
wait

Figure 4.7 SIMSCRIPT 11.5 process-interaction approach

out the restriction of adopting either the event-scheduling or process-interaction approach

exclusively.
Figure 4.7 illustrates the effect of each SIMSCRIPT 11.5 statement in using processes
and resources. In particular, a process comes into existence in one of two ways. The "activate"
statement creates a process and schedules the time of its first execution. Since the "create"
statement only creates the process, at some later time an "activate" statement is needed to
schedule its first execution. The "suspend" statement inactivates the process in which the
statement occurs, but it remains extant. The statement "work" or "wait" simply reschedules
continued execution for a later time. If a process is "interrupt"ed, its scheduled execution is
canceled, and it becomes inactive. The "resume" statement restores the scheduled execution
of the process. A "request" statement may lead to a delay. If so, the occurrence of a "relinquish"
statement for the desired resources in another process causes SIMSCRIPT 11.5 to schedule
continued execution of the requesting process at the current time. The "return" statement
obliterates an executing process from the simulation. The "destroy" statement does likewise
for an inactive process.

4.22 COMMON PSEUDORANDOM NUMBERS

Recall the discussion of Section 4.7 regarding the seeding of pseudorandom number streams.
Suppose we assign individual streams to each of r sources of variation and for each i =
 4.22. COMMON Ps EU DO RAN DOM NUMB E RS 143

1, ... , r assign the same seed to stream i on each of two runs of a simulation program each
with different values for an input parameter that does not affect sample generation. Then
sample averages on each run are consequences of the same random input sequences. This
implies that the difference between corresponding sample averages on the two runs arises
principally, if not entirely, from the difference in the parameter's values. An example based
on waiting time for the airline reservation problem illustrates the benefit that can arise from
this property.
It is of interest to evaluate the reduction in waiting time that materializes when the
number of reservationists increases from m = 4 to m = 5. A simulation for m = 4 generated
a sample record of waiting times Xko Xk+l, ... , Xk+t-h and a simulation with m = 5 using
the exact same seeds generated a sample record of waiting times Yko ... , Yk+t-h where the
choice of k ensured that the influence of initial conditions was negligible (Section 6.1) and
where t = 100,000.
The sample waiting times are

_ 1 k+t-l
Xkt =-
t
L j=k
Xj = 4.072 minutes

and

_ 1 k+t-l
Ykt =-
t
L
j=k
Yj • = 0.7347 minutes.

Their difference is

_. 1 k+t-l _ _
Zkt =-
t
L
j=k
Zj = X kt - Ykt = 3.337 minutes,

where Z j := X j - Yj • Under commonly encountered conditions (Section 3.5),

- 2
tvar Xkt --+ 1100 (X),
- 2
tvar Ykt --+ 1100 (Y)'
- 2
tvar Zkt --+ 1100 (Z), as t --+ 00,

so that ikt, -rkf> and Zkt have approximate large-sample variances 11~(X)/t, 11~(Y)/t,
and I1Jo(Z)/t, respectively. With LABATCH.2, as described in Sections 6.6 through 6.7.5,
144 CHAPTER 4. PROGRAMMING AND EXECUTION

generated

&!(X) = 3429,
&!(Y) = 60.92,
&!(Z) = 2847,

as estimates of a~(X), a~(Y), and a~(Z), respectively.

If {Xi} and {Yj } were independent, due to seeds that ensured nonoverlapping
pseudorandom number streams on each run, then Zkt would have asymptotic variance
[a~(X) + a~(Y)l!t estimated by [&~(X) + &~(Y)l!t = 3490 x 10- 5 • However, the es-
timated asymptotic variance, obtained by analyzing the sample record Zt, ... , Zk+t-l> is
&~(Z)/t = 2847 x 10- 5 • This difference implies that common random numbers require
only

as many observations to achieve the same variance for the sample average difference Zkt
as independent sample records. Moreover, this result occurs with no additional computing
time per run.
Estimating the difference in the proportion of customers who wait reveals an even
greater benefit of common random numbers. Let

if customer j on run 1 waits,

X J...-- { 1
o otherwise,

and

if customer j on run 2 waits,

yJ..._- { 1
o otherwise.

Then Xkt and :rkt estimate the long-run proportion of customers that wait for m = 4 and
m = 5 reservationists, respectively. The same sample records yielded
Xkt = 0.7345, :rkt = 0.3748, Zkt = 0.3597
and

&!(X) = 5.392, &!(Y) = 2.624, &!(Z) = 1.316.

 4.22. COMMO N PSE U DO RAN DOM NUMB E RS 145

Therefore, common random numbers require only

more observations as independent sample records to achieve the same variance for the sample
average difference Zkt.
Why do the savings in sample sizes vary so substantially for the difference of means
and the difference of proportions? Recall that

varZkt = var(ikt - .rkt)

= varikt - 2 cov(ikt , .rkt) + varYkt
~ G!(Z)/t
= [G!(X) - 2Goo (X, Y) + G!(Y)l!t,

where

Therefore,

G!(Z) 2p(X, Y)
1-
----::---=---::--- -
G&,(X) + G&,(Y) - <7oo (X) +
--:-=---:-::::-:-
<7oo(Y) ,
<7oo(Y) <700 (X)

where

- - Goo(X, Y)
p(X, Y) := corr (Xkt, Ykt) ~ () ()'
Goo X Goo Y

Thus variance reduction is a function both of the correlation between sample averages and
of the ratio Goo (X) / Goo (Y). Low correlation diminishes the potential variance reduction, as
does a large or small ratio Goo(X)/Goo(Y).
For sample average waiting times,

[J(X, Y) = 0.7033

and

aoo(X)
-;;------(
Goo Y
) = 7.502.
146 CHAPTER 4. PROGRAMMING AND EXECUTION

For sample proportions,

p(X, Y) = 0.8906

and

The higher correlation and smaller relative difference between uoo(X) and uoo(Y) account
for the greater benefit of common pseudorandom numbers for sample proportions than for
sample average waiting times.
To realize these variance reductions calls for more than common pseudorandom num-
bers on each run. It requires that a separate pseudorandom number stream be dedicated to
each source of random variation and that the seed for stream i be identical across runs. Most
major simulation languages provide access to multiple pseudorandom number streams and
allow user assignment of their seeds.
At least two additional issues deserve attention. The seeds for the streams must be
chosen so that the sequences they generate do not overlap; otherwise, the sequence would be
dependent, contrary to intention. The second issue concerns the number of pseudorandom
numbers that stream i generates on each run and their use. The airline reservation simulation
executed until exactly k + t customers entered service. The structure of the problem ensured
that each stream generated exactly the same number of pseudorandom numbers on the run
for m = 4 and on the run for m = 5, and that each number was used for exactly the same
purpose on both runs.
Common pseudorandom number streams may also induce a variance reduction when
stochastic variation differs on the two runs. Suppose that the purpose for simulating the
airline reservation model is to study the effect on waiting time of changing p from .75 to
.90. If common pseudorandom number seeds are used, then both runs would have the same
sequence of intercall times but different sequences of service times. Let

Ai := ith exponential sample generated by stream 2,

Bi := ith exponential sample generated by stream 3,
Vi := ith uniform sample generated by stream 4.

For specified p, the simulation then generates a sequence of service times

 4.22. COMMON PSEUDORANDOM NUMBERS 147

where
i
Ji(p) := L I[O,pj(UI), i 2: 1,
1=1

and Bo := O. Therefore, the difference between the ith service times on runs with p = .75
andp = .90 is

Ci (.75) - C i (.90) = [B]j(.75) - B]j(.90) 1I[O,.75j (Ui ) - B]j(.90)1(.75,.90j(Ui ),

which rarely is zero.

Although corresponding service times differ, the commonality of the intercall time
sequence can induce a reduced variance for the difference of corresponding sample averages.
As illustration for waiting time, two runs with m = 4 and t = 100, 000 yielded Xkt = .7345
as an estimate ofpr(W > 0 I p = .75) and -rkt = .8851 as an estimate ofpr(W > 0 I p =
90), thus giving Zkt = Xkt - -rkt = -.1506 as an estimate ofpr(W > 0 I p = .75)-
pr(W > 0 I p = .90). Moreover, LABATCH.2 (Section 6.7) gave

a~(X) = 5.392, a~(Y) = 4.836, anda~(Z) = 2.152.

Thus for each run, a sample size

times as large would be required to estimate this difference with the same estimated accuracy.
In the present case, relatively small changes in the S_TIME code in Figure 4.2 can make
the two sequences of service time more similar. In particular:

Between lines 1 and 2, insert:

define DUMI and DUM2 as double variables

Between lines 4 and 5, insert:

DUMI = exponential.f(I/OMEGA,2)

DUM2 = exponential.f(1I0MEGA,3)

return with DUMI

Delete line 5.

These changes induce service times

C;(p) = Ai + BiI[o,pj(Ui ),
148 CHAPTER 4. PROGRAMMING AND EXECUTION

so that

C(.75) if 0 ::: Ui ::: .75 or.90 < Ui ::: 1,

C;(.90) ={ :
. Ci (.75) + Bi if .75 < Ui ::: .90.

That is, only 100 x (.90 - .75) = 15 percent of the service times differ. Simulations run
with these changes produced X = 0.7292 for p = 0.75, Ykt = 0.8806, Xkt - Ykt =
-0.1514, a~(X) = 6.028, a~(Y) = 5.789, and a~(Z) = 1.590, so that

revealing a substantial variance reduction.

This improvement comes at a cost, although a relatively small one. A run using the
original code generate pt times on average from stream 3. In the present case, the two runs
generate (.75 + .90)t exponential times on average. With the modified code, they generate
2t exponential times, 2/1.65 = 1.21 times as many as previously required. Although this
result implies higher simulation computing times with the modified code, this increase is
relatively negligible when compared to the saving in sample size it induces to achieve a
specified accuracy.
However, there is a more important consideration. Whereas LABATCH.2 (Section
6.7) computes a~(X) and a~(Y) without storing sample records generated on each run,
computing a~ (Z) requires storage of at least one of these records, as discussed in Section
6.11, to compute subsample averages based on these data. Therefore, assessing the statistical
accuracy of Xkt - Ykt as an estimate of the true differences requires additional computer
space and additional time to compute a~(Z). Section 6.11 returns to this topic.

4.22.1 OTHER VARIANCE-REDUCING METHODS

These examples illustrate a relatively easily implementable procedure to reduce the

computing time required to obtain a difference in sample averages within a specified statis-
tical accuracy. For its effectiveness, the method of common pseudorandom number seeds
requires that a separate pseudorandom number stream feed each source of stochastic varia-
tion. To remain synchronized during the two runs, the method may also require that variates
be generated but not used, as in the case for m = 4 and p = .75 versus m = 4 and p = .90.
For a more extended discussion of the use of common pseudorandom number streams and
its relationship to other variance-reducing techniques, see Sections 7.7 through 7.11.
Other methods exist for reducing variance, including antithetic variates, control vari-
ates, importance sampling, and conditional sampling. See Fishman (1996) and Law and Kelton
(1991) for example. Although scenarios arise in which each of these can improve compu-
tational efficiency, all, with the exception of antithetic variates, demand substantially more
 4.23. POI N T - eLI C K - D RAG - D R0 P - CON NEe TAp PRO A C H 149

effort than common pseudorandom number seeds to implement and to evaluate. This de-
mand lowers their appeal in practice for simulators who prefer more easily implementable
online methods.

4.23 POINT-CLICK-ORAG-OROP-CONNECT APPROACH

Some languages provide a screen-based block modeling approach to constructing a discrete-
event simulation program. It relieves the simulationist of the onus of entering code via a
textual editor, thereby reducing the frequency of typing errors and shortening the time it
takes to move from concept to execution.
As a consequence ofits emphasis on flow through a system, virtually every screen -based
block-modeling approach embraces the process-interaction paradigm (Section 2.4). The
concept originated with the GPSS simulation language (Gordon 1969). It allows a simulation-
ist to string together macrostatements called blocks that correspond to the transformations
of a temporary entity from its entry into a system through its completed processing and
departure.
Although discrete-event simulation in the 1960s era was performed in a mainframe
computing environment that allowed, at most, a primitive user/computer interaction during
program development, the ease of program construction using blocks gave this approach
considerable appeal. The advent of the PC with its emphasis on interactive programming and
computing substantially enhanced block-oriented programming. Virtually all commercially
available simulation software offers this mode of program construction.
Enhanced interactive capability has led to further software development. By packaging
frequently occurring sequences of blocks into macromodules, vendors can offer a simula-
tionist a menu of these modules that, in principle, he/she can string together in a logical
sequence to produce a simulation program with little or no knowledge on the user's part
of the underlying programming language's constructs. In practice, a blend of this approach
and block-oriented programming becomes necessary as the logical complexity of the systems
under study grows.
As illustration, this and the next several sections describe the point-click-drag-drop-
connect approach to simulation programming using the Arena modeling system (Kelton et al.
1998; Systems Modeling Corporation 1995a, 1995b, 1995c) based on the SIMAN simulation
programming language (Pegden et al. 1995). Arena allows simulationists with varying levels
of familiarity with SIMAN to construct models and execute them. It does so by providing a
collection of panels and allowing simulationists to construct models using entries in them.
At the language level, the Block and Elements panels contain the SIMAN statements. The
Basic Process, Advanced Process, and Advanced Transfer panels contain aggregates of these
statements that correspond to frequently encountered discrete-event constructs. Hereafter,
we refer to them collectively as me BAAT panels. The BAAT three panels allow a modeler
to create flowcharts using relatively standardized block representations on the screen. Like
150 C HAP TE R 4. PRO G RAM MIN G AN D EXEC UTI 0 N

--
SIMAN Parser (in C/C++)
Blocks converts SIMAN
I Modules I f---------. f----- Executable
and statements to module
Elements machine code
Figure 4.8 Arena discrete-event simulation program development

the previous three, the Common, Support, and Transfer Process panels, contain aggregates
of SIMAN statements, again for frequently encountered constructs. Their representations
on the screen attempt to identify them with their principal underlying activities. Hereafter,
we refer to these three as the CST panels. Arena 4.0 contains these eight panels. Arena 3.51
contains the first two and the last three.
In principle, pointing, clicking, dragging, dropping, and connecting modules from,
for example, the Common panel and dropping them into a model window allows a user with
little to no familiarity with SIMAN to create a simulation program. Analogous operations
applied to the Blocks and Elements panels allow a more knowledgeable user of SIMAN to
construct a simulation program directly from its basic blocks or macromodules. Arena also
offers the user the option to program directly in SIMAN via a textual editor. Figure 4.8
depicts program creation in the Arena environment.
As with all other simulation languages, SIMAN embraces the elementary concepts of
entity, attribute, event, list, process, and future-event set described in the introduction to
Section 2. It includes a grammar that caters to the process-interaction approach. Although
it is not possible to create a SIMAN program that embraces the event-scheduling approach
of Section 2.3, Section 4.28 describes SIMAN's EVENT block, which allows a simulationist
to introduce elements of that approach within the framework of process interaction. The
SIMAN literature often refers to event scheduling as event orientation and to process inter-
action as process orientation. Its event -scheduling capabilities can be traced to early work of
Kiviat at U.S. Steel, who developed the GASP discrete-event simulation language (Pritsker
and Kiviat 1969). Its process-interaction features can be traced back to GPSS (Gordon 1969).
Arena embraces SIMAN's process-interaction concepts as the basis for its modeling
environment. SIMAN's time-advance protocol follows the next-event approach (Section
2.1). It maintains its future-event set (FES) in a single list as a heap (Section 5.4.3). Its
pseudorandom number generator has the form (4.13) with A = 16807 and M = 231 - 1,
and it allows for multiple streams each with its own seed (Section 4.7). The seeds may be
specified by the user. Otherwise, SIMAN employs its default seeds. Section 9.4 describes the
properties of this generator. For reference, see Pegden et al. (1995).
As mentioned, the BAAT panels are available with Arena 4.0. Using them produces
spreadsheets below the model display for entities, queues, resources, etc. The spreadsheets
facilitate program construction. However, there are qualifications to using these and the CST
panels. In a particular problem specification may exceed the capacity of these panels to model
 4.24. M/M/1 ARENA EXAMPLE 151

the problem completely. When this is so, the simulationist may augment the aggregates taken
from these with statements from the Block and Elements panels. The "WHILE" statement
illustrates one area in which this merging can be beneficial.
In addition to the modules provided by these panels, Arena's Professional Edition
allows a user to create her/his own modules, out of SIMAN blocks, and collect them into
a template (panel). Arena provides a capability for dynamic animation as simulated time
elapses during execution and a means for postsimulation graphical display of sample paths,
histograms and other selected concepts. Arena also allows for the incorporation of user-
written code in Visual Basic, and C/C++. However, the burden for this incorporation falls
on the simulationist.
The description here is based on Arena Version 4.0. We illustrate Arena first using the
high-level Common panel and then, at the other extreme, the Block and Elements panels.
We then describe several features of the BAAT panels. In conformity with SIMAN reference
material, we capitalize all SIMAN block, element, and module names. To distinguish between
SIMAN code and user-specfied variables, etc., we capitalize only the first letters of the latter.

4.24 MlMI1 ARENA EXAMPLE

We first illustrate Arena program construction exclusively by means of the Common panel,
some of whose modules are depicted in Figure 4.9. Consider a single-server queueing system
with i.i.d. exponentially distributed interarrival times with mean 1.1111 and i.i.d. exponen-
tially distributed service times with unit mean. Jobs receive service in first-come-first-served
order. There are five types of state changes or elementary events (Section 2.3):

1. Arrival 4. Server becomes idle

2. Job begins service 5. Server becomes busy.
3. Job departs

These can easily be incorporated into a single process as in Section 2.4. In the Common panel,
the ARRIVE module serves as arrival generator, the SERVER module as work processor, and
the DEPART module as disposer of jobs after they complete service. Figure 4.10 shows a
model constructed by pointing and clicking on these objects, one at a time, in the Common
panel and dragging and dropping them in an opened model window labeled mml.doe.
Double clicking with the left mouse key on the icon for the dropped ARRIVE module
opens a dialog box that provides space for time of first arrival number of jobs per arrival
and time between arrivals. The defaults for the first two are 0 and 1, respectively, and we
allow those to prevail. Clicking on the time between arrivals brings up a list of distributional
options for sampling interarrival times. The menu there lists 13 distributions, including two
empirically based options, one for continuous random variables and the other for discrete
random variables. Clicking on Expo(mean) and replacing "mean" by 1.1111,1 specifies the
exponential distribution with mean 1.1111 using pseudorandom number stream 1.
 ..
152 CHAPTER 4, PROGRAMMING AND EXECUTION

--./
r-- .
~'- -I
L. '
\ .5 ..1 .CZ '/
- •~L.
,
. -1
C-' ' .L. 9
• --- L... ~Q
Arrive • Depart I • Server " Inspect " AdvServer ~ Process
-,
i
!
Arrive 1 Depart 1
Server 1 Inspect 1 AcfvServet 1

, Enter ~ ~ Leave ., Actions' " Menu ~ Expressions J Queue Recipes ,

•
Elller2 Aclions 1 Queue 1

I Sequences I ~ Simulate Statics Statistics Storage , , Variables I

Storage 1

__tl. Container 1
IResource : , Container !
R8S0Urce 1

Number Available

Figure 4.9 Modules in Arena Common Panel

---';

Arrive 1
Server 1 Depart 1

•
Simulate
MM 1 Model
10000

Figure 4.10 Arena/SIMAN mml.doe model

Clicking on the "connect" option in the dialog box sets the stage for linking the
ARRIVAL and SERVER modules. Bringing down the "object" menu, clicking on "connect;'
and clicking once on the exit node of the ARRIVAL module and then once on the entry node
to the SERVER module connects them. Similar steps for the SERVER module with mean
replaced by 1,2 specifies exponential service times with unit mean using pseudorandom
number stream 2. Connecting the SERVER and DEPART modules is accomplished by first
 4.24. M/ M/ 1 ARE N A EXAM P L E 153

Model statements for module: Arrive 1

PROJECT. MMI Model;

42$ CREATE. \:expo(LlIII.I); ATI'RIBUTES: QueueTime;

3$ STATION. Arrive I; QUEUES: Server LR_Q,FIFO;

51$ TRACE, -I,"-Arrived to system at station Anive lin":;
6$ ASSIGN: Picture=Default; PlcruRES: Default;
27$ DELAY: 0.;
55$ TRACE. -I ,"-Transferred to next moduIeln"::NEXT(O$); RESOURCES: Server I_R.Capacity(l.).-.Stationary;

STATIONS: Arrive I:
Depart I:
Server I;
Model statements for module: Server I
TALLIES: Server I_R_Q Queue Time;

0$ STATION. Server 1; DSTATS: NQ(Server I_R_Q).# in Server I_R_Q:

158$ TRACE. -I."-Arrived to station Server lin":; MR(Server I_R).Server I_R Available:
121$ DELAY: 0.;
165$ TRACE. -I."-Waiting for resource Server I_RIn":;
NR(Server I_Rl.Server I_R Busy;
82$ QUEUE. Server LR_Q:MARK(QueueTime);
83$ SEIZE. I:
Server I_R.I; REPLICATE. 1,0.0.IOOOO.Yes.Yes.lOOO;
192$ BRANCH, I:
If.RTYP(Server I_R).eq.2,193$,Yes:
If.RTYP(Server I_R).eq.I.95$.Yes;
193$ MOVE: Server I_R.Server I;
95$ TALLY: Server LR_Q Queue Time.INT(QueueTime), I;
202$ DELAY: 0.0;
TRACE, -I."-Delay for processing time EXPO(I.2)ln":;
84$ DELAY: EXPO(I.2);
166$ TRACE. -I,"-Releasingresourceln":;
85$ RELEASE: Server I_R.I;
149$ DELAY: 0;
172$ TRACE. -I."-Transferred to next moduleln"::NEXT(2$);

Model statements foc module: Depart 1

2$ STATION. Depart I;
233$ TRACE. -I,"-Anived to station Depart lin":;
203$ DELAY: 0.;
240$ TRACE, -I."-Disposing entityln":;
232$ DISPOSE;

(a) MODfile (b) EXPfile

Figure 4.11 Arena/SIMAN mml.mod and mml.exp files

clicking on the "connect" option in the dialog box and following steps analogous to those
used previously.
A SIMULATE module dragged from the Common panel and dropped in the open
mml.doe window creates the basis for executing the model. Double clicking on it brings
up its dialog box with entries for "title;' "number of replications;' "beginning time;' and
"length of replication." In the present setting, the last three of these are set to 1, 1000, and
10,000.
154 CHAPTER 4. PROGRAMMING AND EXECUTION

Clicking on "SIMAN" and then on "View" in the Run menu displays the corresponding
SIMAN code. Figure 4.11 shows it partitioned into a MOD file, or Model file (left side),
and an EXP file, or Experiment Source file (right side). The MOD file contains the executable
SIMAN code whereas the EXP file contains all nonexecutable global declarations plus a
list of final results to be tabulated. In the present case, Arena has automatically included
these output quantities in TALLIES and DSTATS statements. TALLIES relates to output
quantities computed from discrete-indexed sample-path data, for example, waiting time.
SIMAN refers to these as discrete-valued variables. DSTATS relates to output computed
from continuous-time indexed sample-path data; for example, queue length. SIMAN refers
to these as time-persistent variables.
Of particular interest is the partition of the MOD file into three sets of SIMAN
blocks that correspond to the ARRIVE, SERVER, and DEPART modules. The reader quickly
appreciates the benefit that point and click from the Common panel offers.
Clicking on "Go" in the "Run" menu executes the program with animated activity in
the mml.doe window. Alternatively, clicking on "Fast Forward" suppresses the animation
and induces considerably faster execution. When execution terminates, Arena asks the user
whether he/she wishes to view its automatically generated output summary. We return to
this topic in Section 4.25, where we examine the output for the airline reservation program.
Note that Arena has relieved the simulationist of the need to compile, link, and execute
the program and, by default, has provided for the computation of summary statistics. It also
uses default seeds for the two pseudorandom number streams.
What does the SIMAN code in the MOD and EXP files tell us? As a basis for answering
this question, Section 4.24.1 describes a collection of SIMAN Block and Element statements
and then uses the latter to explain the contents of the EXP file in Figure 4.11 b. Then Section
4.25, which studies the Arena/SIMAN version of the airline reservation problem, use the
Block elements to explain the contents of the MOD file for that model.

4.24.1 SIMAN STATEMENTS

SIMAN contains executable statements, or blocks, and nonexecutable global state-

ments, or elements. Those in Figures 4.11 and 4.13 have the following generic forms:

BLOCKS

ASSIGN: variable or attribute name = expression or numerical value:

repeat;

COUNT: counter name, increment;

CREATE: batch size, offset time: interval, maximal number of batches to be created;

DELAY: time duration, storage name;

DISPOSE;
 4.24. M/M/1 ARENA EXAMPLE 155

ENDIF;
IF:
QUEUE, queue name, capacity, transfer address when capacity exceeded;
RELEASE: ,resource name, quantity to release:
repeat;

SEIZE, priority: ,resource name, number of units:

,
repeat;

TALLY: tally name, name of quantity to be recorded, number of entries to be recorded;

ELEMENTS

ATTRIBUTES: ,number, attribute name (index), initial value:

repeat;

COUNTERS: number,
,counter name, limiting value, initial value, option, output file:
repeat;

DSTAT: ,number, expression, name, output file:

v
repeat;

EXPRESSIONS: number, name (... ,... ), expressions:

, J

repeat;

PROJECT: title, analyst name, date, summary report;

QUEUES: ,number, name, ranking criterion:
,
repeat;

REPLICATE: number, beginning time, replication length, initialize system, initialize

statistics, warm-up period;
RESOURCES: ,resource ID number, name, ~apacity or number of units:
repeat;

SEEDS: ,stream ID name or number, seed value, initialize option (yes or no) :,

TALLIES: ,number, na"',e, output file:

repeat;

VARIABLES: ,number, name (index), initial values, . .. :

v

repeat;
156 CHAPTER 4. PROGRAMMING AND EXECUTION

The notation "repeat;" indicates that several records of multifield information may be entered
by merely listing them sequentially, each followed by a colon, except for the last, which is
followed by a semicolon. The "number" entry in each element with "repeat" designates
a unique numerical identifier. Whenever the "name" entry in the same record is invoked
in SIMAN, it refers to this unique identifier. For a complete listing of SIMAN Blocks and
Elements, see Pegden et al. (1995), Systems Modeling Corporation (1995b), or the Help
menu in the Arena screen display.
The content of the EXP file in Figure 4.11 b provides a basis for familiarizing ourselves
with several of the Elements statements. Recall that this file displays the nonexecutable global
declarations that the user-specified model in Figure 4.10 automatically creates in the SIMAN
code.
ATTRIBUTES identifies QueueTime as an attribute of the process. RESOURCES de-
clares a single server, Server LR, Capacity (1,), and QUEUES associates a set Server LR_Q
with this resource having the FIFO discipline. In the present context, STATIONS is self-
explanatory but contains additional information in other models (e.g., Pegden et al. 1995,
p. 214). We defer our discussion of the TALLIES and DSTATS statement until Section 4.25.
To follow an entity's progress through a system for diagnostic purposes, SIMAN pro-
vides the TRACE element. It allows a user to specify those entities and variables that are to
be monitored during the simulation, presumably for the purpose of checking system flow
for errors. Arena also provides for an automatic trace if one clicks on Run Control on the
Run menu, then on Trace, and then on Enable Block Trace. Before using this option, the
simulationist is encouraged to read the accompanying Help file.

4.25 ARENAlSIMAN AIRLINE RESERVATION EXAMPLE

For a simulationist with limited knowledge of SIMAN and for whom speed of modeling is a
high priority, the appeal of working exclusively with the Common panel is apparent. If we
try to use this panel to model the airline reservation problem, an obstacle arises when we
try to represent the branching at block 5 in Figure 2.10 related to the busy message. Since the
Common panel in Arena version 3.51 does not display a module for effecting branching, we
could resort to making selections from it and from Arena's Support panel, which contains
logic functions for effecting branching. However, we move away from the higher level of
functionality that these panels offer and resort to modeling at the level of Arena's Blocks
and Elements panels. Recall that the Blocks panel contains all SIMAN executable statements,
whereas the Elements panel contains SIMAN nonexecutable global declaration and control
statements.
Figure 4.12 shows the Arena/SIMAN airline reservation model derived from the Blocks
and Elements Panel, and Figure 4.13 shows the corresponding Arena-generated MOD and
EXP files.
 4.25. ARENA/SIMAN AIRLINE RESERVATION EXAMPLE 157

Oeate 0---1 Assign ~ If ~ Delay ~ Endlf

"""
InlercalLTime lYJll NR(ReservationisQ=N_Reservationisl Message_Time

~ OJeue ~ Sei2B ~ Tally ~ Count ~ Delay ~

Custlmer_O Reservationisl Waiting_TIme CO"1lleled_Calls Service_Time(Type)

-t Release ~ Dispose

Res ervalm B t

I POOJEGr I IATTRIBUTES I ICUEUESI I OOJNTERS I

Timft..h CustlmecO CO"1lleled_calls
AirHne Reservation
Type

ISEEDS I IVARIABLES I I RESOJFCES I ITALLESI

Waiting_time
1 Message_lme Reservationisl
2 N_reservationisl
3
4

I RERJCAlE I I EXPRESSIQ\IS I I CSTATSI

InlercalLdme NO(Custlmer_O)
Service_time NR(ReservationisQ
Type_dislribution

Figure 4.12 Arena generated SIMAN Blocks and Elements for airline reservationist
problem

CREATE in Figure 4.13 iteratively brings new processes into existence, each of which
we hereafter call Customer, for convenience of exposition. Furthermore, "batch size" = 1
indicates that these Customers are created one at a time. Creation starts at "offset time" = O.
Successive Customers are created at "interval" = EXPO (1 , 1) as defined in EXPRESSIONS in
Figure 4.13. These are exponentially distributed samples with unit mean using pseudoran-
dom number stream 1. MARK is a SIMAN-defined function that assigns the current time
in the user-specified attribute Time-In of the Customer.
ASSIGN determines Type (ofCustomer), declared as its first attribute in ATTRIBUTES,
by sampling from the user-specified Type.J)istribution in EXPRESSIONS using stream 4.
IF allows the subsequent DELAY to be executed for Message_TimeJ:n VARIABLES if no
Reservationist is available (NR(Reservationist) = = N-Reservationist;). ENDIF delimits the
IF range.
158 CHAPTER 4. PROGRAMMING AND EXECUTION

PROJECT. "Airline
2$ CREATE. Reservation".··G.S.F.".712211999,Yes.No.Yes.No,No.No.No.No;
1:lntercaiLTime:MARK(Time_IN):NEXT(3$);

ATTRIBUTES: I.Type:
3$ ASSIGN: TYP""Type_Distribution;
2,Time_ln;
11$ IF: NR(Reservalionist)=N_Reservationist;
4$ DELAY: Message_Time"Other:NEXT(5$);
VARIABLES: I.Message_Time,CLEAR(System),0.15:
2,N_Reservationist,CLEAR(System).4;
5$ ENDIF;
6$ QUEUE. Customer_Q;
SEEDS: 1,1973272912,No:
7$ SEIZE. I.Other:
2,1964472944,No:
Reservalionist.I:NEXT(8$);
3,1l20463904,No:
4,1351423507,No;
8$ TALLY: Waitin&...Time.INT(Time_in),I;
0$ COUNT: Completed_Calls, I;
QUEUES: Customer_Q,FirstinFirstOut;
1$ DELAY: Service_Time(Type)"Other:NEXT(9$);

RESOURCES:
9$ RELEASE: Reservationist.l ; Reservationist.Capacity(4),,stationary,COST(0.0,0.0,0.0);
10$ DISPOSE: No;
COUNTERS: Completed_Calls,IOOO,Replicate;

TALLIES: I,Waitin&...Time,"Waitin&...Time.dat";

DSTATS: I,NQ(Customer_Q),QueueJ-ength:
2,NR(Reservationist),#_Busy_Reservationists;

REPLICATE, I,0.0"Yes,Yes,0.0.,,24.0,Ho=;

EXPRESSIONS: intercaiLTime,EXPO(I. I):

Service_Time(2),EXPO(2, 2).EXPO(2, 2)+EXPO(2, 3)
TypeJ)istribution.DISC(0.25, I, 1.0, 2, 4);

(a) MOD file (b) EXP file

Figure 4.13 Arena-generated SIMAN code for airline reservation problem

QUEUE files a Customer process in a user-specified list called CustomeLQ, which has
infinite "capacity." If the specified "capacity" were limited, and a Customer were denied entry
because of space, the "balk label" would redirect the entity to an alternative block. SEIZE
functions similarly to REQUEST in SIMSCRIPT 11.5. Here "priority" = 1 denotes first-
come-first-served for the Reservationist resource whose total number is N..Reservationist
= 4 in VARIABLES. Once an entity flows from the- SEIZE block, TALLY collects "value
of tally" = INT(Time_In) and stores these data in "name of quantity" = Waiting_time for
subsequent analysis and display. Time-..ln, the second attribute of Customer, declared in
ATTRIBUTES, stores the simulated time at which the Customer was created in the CREATE
block. INT(Timdn) is a SIMAN defined function that computes the elapsed time between
current simulated time (at TALLY) and Time-..ln.
COUNT adds 1 to the user-declared Completed_Calls in COUNTERS. When the
limit of 1000 completions, specified in the COUNTERS element, is reached at COUNT,
SIMAN terminates the simulation run. Thus we have two ways of controlling run length,
one based on elapsed time (Option A in Section 3.4) in the MIMI 1 example and one based on
number of Customers receiving service (Option B in Section 3.4) in the airline reservation
problem.
 4.25. ARE N A / S I MAN A I RLIN ERE S E RV A T ION EXAM P L E 159

The next DELAY provides service to the Customer using the service time distribution
defined under EXPRESSIONS. IfType = 1 (single destination), it samples from EXPO(2,2),
the exponential distribution with mean 2 using pseudorandom number stream 2. If Type
= 2 (multiple destinations), it augments that sample by EXPO(2,3) using pseudorandom
number stream 3. After time elapses at this second DELAY, SIMAN RELEASEs one unit of
the resource Reservationist. Then DISPOSE eliminates the record of this Customer process
from the simulation.
Recall that SIMAN uses a pseudorandom number generator of the form (4.27). The
SEEDS element allows us to specify Zo, ZIOOOOOO, Zzoooooo, and Z3000000 for the four seeds
using entries in Fishman (1978, Table A.2). Note that exclusive use of the Common panel
does not allow this option. Also, note that the third field of each stream entry indicates that
the streams are not to be reinitialized to the original seeds on successive replications. Since
the REPLICATE element indicates 1 replication in its first field, this is not an issue here.
Recall that the DSTATS element allows us to specify the continuous-time sample paths
on which data for time-persistent statistics are to be collected and analyzed. NQ and NR are
SIMAN-defined system variables. In the present setting, NQ(CustomeLQ) denotes queue
length for the user-specified CustomeLQ set and NR(Reservationist), the number of busy
resources of the user-specified resource Reservationist. Note that the user-specified names,
Queue Length and #j3usy_Reservationists, in field 3 refer to the respective Arena-assigned
indices, 1 and 2, in field 1.
In the EXP file of Figure 4.l3, DSTATS declares that data on two time-persistent
phenomena are to be collected and summarized. One is NQ(CustomeLQ), the number
in the set designated by CustomeLQ. It is to be named Queue-Length (field 3), which
implicitly refers to index 1 in field 1. The other is NR(Reservationist) (field 2), the number
of busy RESOURCES called by the user-designated name, Reservationist.1t is to be named
# _Busy_Reservationists, which implicitly refers to index 2 in field 1.
The executable TALLY statement in the MOD file in Figure 4.l3 computes waiting
times as INT(TimeJ:n) in field 2 (see Section 4.22) and refers to them as Waiting_Time
in field 1. The TALLIES statement must have this designation in field 2. In the present
case, it also declares that these data are to be stored in the user-specified file, Waiting_time
.DAT, in binary form. Double quotation marks must precede and follow the file name. The
DSTATS element also allows for data storage. Section 6.9 describes how to use this file for a
user-specified batch-mean analysis.
Figure 4.14 displays the user-specified statistical summary of output. The sample
average waiting time 5.6114, queue length 5.8416, and number of busy reservationists 3.5064
are comparable to the values of their corresponding quantities, 5.8818, 5.862, and 3.7135,
in Figure 4.3 for SIMSCRIPT 11.5. Arena has a capability for computing an approximating
confidence interval for the time long-run averages and displaying its half-length. The default
significance level for the confidence level is 0.95. In the present case, the entry (Corr) indicates
that the data are correlated so that a half-length cannot be computed by the Arena's built-
in method. See Section 6.6 for a discussion of Arena's method of computing a confidence
interval.
160 CHAPTER 4. PROGRAMMING AND EXECUTION

ARENA Simulation Results

Operations Research - License: 20010064

Summary for Replication 1 of 1

Project:Airline Reservation Run execution date :10127/2000

Analyst:G.S.F. Model revision date: 7122/1999

Replication ended at time : 979.939

TALLY VARIABLES

Identifier Average Half Width Minimum Maximum Observations

5.5749 (Carr) .00000 32.076 1000

DISCRETE-CHANGE VARIABLES

Identifier Average Half Width Minimum Maximum Final Value

Queue_Length 5.7490 (Carr) .00000 37.000 21.000

# _Busy _Reservationists 3.5036 (Carr) .00000 4.000 4.000

COUNTERS

Identifier Count Limit

Completed_Calls 1000 1000

OUTPUTS

Identifier Value

System.NumberOut .00000

Simulation run time: 0.03 minutes.

Simulation run complete.

Figure 4.14 Arena air.out output

After run termination, a simulationist can click on "Output" in the Tools menu and
then on "Graph:' This gives options for graphical and tabular displays. Figure 4.15 shows
the table of the resulting waiting-time histogram. Clicking on "Data File" in the File menu
brings up a dialog box, one of whose options is to export the saved data to a user-specified
textual file for subsequent analysis.
Histories may be saved at the modular level using the STATS module in the Common
panel. It provides options for DSTATS, TALLIES COUNTERS, and FREQUENCY variables.
 4.25. ARENA/SIMAN AIRLINE RESERVATION EXAMPLE 161

Histogram Summary
Waiting time

Cell Limits Abs. Freq. ReI. Freq.

Cell From To Cell Cumul. Cell Cumul.
I -Infinity 0 0 0 0 0
2 0 0.25 305 305 0.305 0.305
3 0.25 0.5 25 330 0.025 0.33
4 0.5 0.75 27 357 0.027 0.357
5 0.75 I 22 379 0.022 0.379
6 I 1.25 16 395 0.016 0.395
7 1.25 1.5 23 418 0.023 0.418
8 1.5 1.75 29 447 0.029 0.447
9 1.75 2 30 477 0.03 0.477
10 2 2.25 14 491 0.014 0.491
II 2.25 2.5 20 511 0.02 0.511
12 2.5 2.75 16 527 0.016 0.527
13 2.75 3 28 555 0.028 0.555
14 3 3.25 14 569 0.014 0.569
15 3.25 3.5 20 589 0.02 0.589
16 3.5 3.75 16 605 0.016 0.605
17 3.75 4 13 618 0.013 0.618
18 4 4.25 13 631 0.013 0.631
19 4.25 4.5 8 639 0.008 0.639
20 4.5 4.75 7 646 0.007 0.646
21 4.75 5 11 657 0.011 0.657
22 5 5.25 16 673 0.016 0.673
23 5.25 5.5 7 680 0.007 0.68
24 5.5 5.75 14 694 0.014 0.694
25 5.75 6 3 697 0.003 0.697
26 6 6.25 9 706 0.009 0.706
27 6.25 6.5 4 710 0.004 0.71
28 6.5 6.75 11 721 O.OII 0.721
29 6.75 7 6 727 0.006 0.727
30 7 7.25 9 736 0.009 0.736 Figure 4.15
31 7.25 7.5 4 740 0.004 0.74
32 7.5 7.75 8 748 0.008 0.748 User-specified Arena
33 7.75 8 5 753 0.005 0.753 waiting-time histogram
34 8 +Infinity 247 1000 0.247

Recall that SIMAN maintains its future-event set (FES) or event calendar as a single
list. CREATE schedules arrivals by inserting the current Customer record into this FES with
reactivation time being the next InteLcall time. The first DELAY does similar linkage with
current time + 9 seconds as the reactivation time. The second DELAY performs similarly.
QUEUE effectively makes use of the lists defined in QUEUES to store records.
SEIZE selects and removes records from the Customer _Q as resources become available
at the RELEASE statement in another Customer process. Although SIMAN effects the
same logic for its QUEUE-SEIZE-RELEASE sequence as SIMSCRIPT 11.5 does for its
q.resource-x.resource-request-relinquish sequence, its list-processing programming code
differs substantially.
162 C HAP TE R 4. PRO G RAM MIN G AN D EXECUTION

SIMAN's TRACE element allows a simulationist to generate a time history of events

during a simulation run. This history can be helpful in debugging a model, especially for
identifying unintended impediments to entity flow from node to node.

4.25.1 MODELING WITH THE BMT PANELS

As already mentioned, the BAAT panels permit modeling at a higher level of code
aggregation than exclusive reliance on the Blocks and Elements panels allow while retaining
more modeling flexibility than the more aggregated models in the CAT panels allow. As with
the other panels, each module in the BAAT panels has a dialogue box that enables a user to
specify which of several actions are to be taken. However, the BAAT menus in these boxes
offer more options and actions than those in the Block and Elements panels.
As illustration, the BAAT panels contain a multiple-purpose "process" module. We use
quotation marks here merely to distinguish Arena's use of the word "Process" from our more
generic use of the term to describe the principal concept in the process-interaction approach.
See Chapter 2. The dialogue box of the "Process" module contains a menu with items

Delay
Size, Delay
Size, Delay, Release
Delay, Release

where Seize, Delay and Release are SIMAN statements with corresponding entries in the
Blocks panel. A user clicks on the choice that fits the problem at hand. For example, a
modeler of the airline reservation problem may use Seize, Delay, Release or Delay, Release,
depending on how the time spent waiting is to be collected in the module. More generally,
we emphasize the versatility of this and other BAAT modules that allow a modeler to make
multiple Block selections in a single module.
Although space considerations limit what we can show here, we encourage the reader
to model the airline reservationist problem using the BAAT panels with the same input
specification depicted in Figure 4.13 based on Blocks and Elements. In principle, the outputs
should be identical.

4.26 EVENTS IN SIMAN AND SLAM

As mentioned, both SIMAN and SLAM allow simulation modeling and programming using
the event-scheduling approach. Unlike SIMSCRIPT 11.5, however, for which the event-
scheduling syntax is an integral component of the language, Arena/SIMAN requires skill
in C (Pegden et al. 1995, Appendix F), and AweSimNisual Slam requires skill in C or Visual
Basic (Pritsker et al. 1995, Section 13.2). Regrettably, a consequence of these requirements
is to limit accessibility to the event-scheduling approach in these languages.
 4.27. MACROMODULES 163

From the point of view of pedagogy, this should in no way deter the reader from
studying the concept of event scheduling in Chapter 2 and the programming constructs to
which it leads in Sections 4.2 through 4.17. Familiarity with these ideas provides a consid-
erably deeper understanding of the inner workings of discrete-event simulation than the
higher-level functionality of the process-interaction approach allows. The familiarity can be
of considerable help in trying to understand precedence relationships in complex models.

4.27 MACROMODULES

Macromodules can incorporate considerably more functionality than the elementary blocks
of a simulation language. The MIMll example well illustrates this, where three macromod-
ules in Figure 4.lO lead to the relatively detailed MOD and EXP files in Figure 4.11. Relying
on macromodules dearly saves time and has great appeal for the experienced program-
mer. Moreover, the minimal programming to which it leads allows those with little if any
familiarity with a language to model complex systems and execute simulations of them.
For both experienced and inexperienced programmers, there is an issue of correct-
ness. In particular, the user of this approach needs to be assured that the generated code
does, indeed, do exactly what helshe wishes. Studying the self-documenting code in Figure
4.11 provides one way to gain this assurance. In practice, many a simulationist pays scant
attention to this code. Regardless of the simulation language, this can lead to a serious mis-
representation of modeling logic, as the next several examples show. This is especially so at
the higher level, for example, in modeling conveyors, where many alternative logic protocols
exist and only one ofwhich may be embraced in the macromodules of a particular simulation
language.
Three examples taken from Banks and Gibson (1997) illustrate the dangers inherent
in this minimal approach to simulation programming based on macromodules:

WHAT IS THE SEQUENCING DISCIPLINE?

Suppose we use a popular programming simulation software package, which is

oriented toward manufacturing; call it Software X. The goal is to simulate the loading
of a trailer with lO different rolls of carpet, to travel to the destination, then to unload
the carpet in the reverse order ofloading. Using the "LOAD" command accomplishes
the placement of the carpet on the trailer. So far, so good. Using the "UNLOAD"
command removes the carpet, but in the order that it was loaded. Obviously, the trailer
should be unloaded in reverse order. To accomplish the desired result (provided that
the modeler knew that the software was not reproducing reality) requires the use of
"dummy" locations: fake entities used to allow work-arounds for such problems.
164 CHAPTER 4. PROGRAMMING AND EXECUTION

Dummy locations are used frequently in no-programming software to accom-

plish a desired result, and this can lead to much confusion. Not only is the frequent
need to resort to this artifice confusing to the modeler, the use of dummy locations
causes other aggravations. For example, in a complex process at a machine, dummy
locations may have to be added to portray the various steps in the process. Then, when
a report on the utilization of the machine is desired, even more dummy locations have
to be added.
Using a different, "programming" simulation software, an order list construct
can be used. This allows the modeler to specify to the model that the carpet rolls be
unloaded in accordance with the value of an attribute that contains the time of their
arrival. Unloading the rolls from lowest to highest time of arrival would accomplish
the desired purpose. Although we would consider this a "programming" approach, it
seems to us more straightforward and easier to build than the dummy location work-
around. If the customer later decides that the truck should be unloaded in order of
color instead, this too can be easily programmed using the same order list construct.

WHAT CA.... THE BRIDGE CRA.... E DO?

The words sound right. You want to model a bridge crane. The no-programming
software has a bridge crane! The model must include an automated guided vehicle
(AGV). Okay, the no-programming software has an AGV. This looks easy, and the
model is constructed rapidly. It takes only four hours to build with animation, and
that's four-and-a-half days faster than with the package that doesn't advertise no-
programming.
But you need to dig deeper into what the bridge crane module really does. You
need to dig deeper into those AGV movement rules. What scheduling logic is being
used? What parking rules are used? Is this the same as your bridge crane? Does your
AGV system work this way?
There is much danger in treating these capabilities as a black box. You may
find that you haven't accounted for the minimum spacing of the AGVs, and that is
important. You may find that the control algorithm for the bridge crane is not the
same as the one your system is using, and it can't be changed.
We might go as far as saying that by not requiring you to explicitly specify or
"program" model logic, no-programming packages hide critical details that you may
need to see in order to understand and verify the model's behavior and results. If you
were told to design a machine, you wouldn't simply describe the machine's outside
view, fill its surface with some pretty colors from your paintbrush type software, and
then turn on the automatic inside machine design program. You would need to design
the inside of that machine carefully, including all of the spindles, bearings, gears, races,
and their interactions.
 4.28. LESSONS LEARNED 165

MODULE OPAQUENESS

Here's an example of the hidden dangers: A model was developed of a simple,

high-speed packaging conveyer line using the approach promoted by Software Y. The
modeler selected from among the high-level blocks provided, clicked and filled in the
Windows "check boxes:' and accepted some defaults. The modeler made one critical
error, however, by not specifying a previous conveyer section resource to be released
after gaining access to the next section; the modeler inadvertently left the default
"release before" option selected. This model construct used an "internal" queue in
the queue-size logic, which is hidden from the user. This queue actually allowed an
unlimited number of package entities to build up when the downstream conveyer
section filled, providing a totally erroneous result. The model's performance reports
provided no clue about the problem, because the software doesn't report on these
hidden queues. Had the modeler used a lower-level programming construct instead,
this error would have been much easier to detect.

4.28 LESSONS LEARNED

• Major discrete-event simulation languages include GPSS/H, SLAM, SIMAN, and
SIMSCRIPT 11.5.

• All allow simulation programming using the process-interaction approach. Although

several also allow programming via the event-scheduling approach, SIMSCRIPT 11.5
is by far the easiest to use in this regard.

• Maintaining a separate pseudorandom number stream for each source of stochastic

variation allows for control of stochastic variation within a run and across runs.

• Recording first and last seeds for each pseudorandom number stream allows for easy
reproducibility of results. This is especially helpful in checking out a program. It also
facilitates the use of common pseudorandom number streams.

• The method of common pseudorandom number streams is a technique that may lead
to a variance reduction in comparing sample averages computed from data on two
simulation runs, each based on a different assignment for parameter values or on
different system operating rules.

• SIMSCRIPT 11.5 is a full-service programming language that allows all the arithmetic
and file management operations in languages like FORTRAN and C and, in addition,
has a syntax catering to discrete-event simulation. Unix and Windows implementa-
tions are available, the latter allowing a point-click-drag-drop approach to program
construction. Its principal concepts are events, processes, entities, attributes, sets, and
event notices. It admits both the event-scheduling and process-interaction approaches
166 CHAPTER 4. PROGRAMMING AND EXECUTION

to modeling. Its principal strength comes from the high level of generality it offers for
modeling the competing demands for resources that arise in a wide variety of delay
systems. However, this high generality puts more of a burden on the simulationist
with regard to programming than do other languages, such as GPSS/H, SLAM, and
SIMAN, that provide constructs considerably more focused on commonly encoun-
tered flow logics. Nevertheless, SIMSCRIPT 11.5 offers many pedagogic benefits for
teaching students how to model much of this logic, a particularly valuable asset when
one encounters a nonstandard flow logic that some of these other languages either
handle awkwardly or cannot handle.

• SIMAN is a discrete-event simulation language whose syntax principally consists of

constructs that facilitate the programming of the competing demands for resources,
so characteristic of demand-driven delay systems. By incorporating a considerable
amount of commonly encountered system logic into these constructs, SIMAN reduces
the effort required to create an executable simulation program for many delay systems
encountered in practice. However, the tight coupling between its arithmetic and list-
processing operations on the one hand and its demand-resource constructs on the
other limits its ability to model some demand-driven delay systems. For these, the
SIMAN coding may appear awkward or may not be possible.

• For the same problem, a program coded in SIMAN tends to execute faster than one
coded in SIMSCRIPT 11.5. SIMAN and SIMSCRIPT 11.5 are C-based.

• Arena is a comprehensive simulation modeling system that relies on SIMAN as the

means for programming a discrete-event simulation. Windows-based, it encourages a
point-click-drag-drop approach to program construction. It contains macromodules,
frequently required aggregates of SIMAN constructs, which further reduce program-
ming effort. It also provides an input-data analyzer and automatic report generator.
A user can easily tailor the content and form of these reports.

4.29 EXERCISES

GENERAL INSTRUCTIONS

These e.xercises provide opportunities for hands-on experience in writing simulation

programs. As the specifications in each exercise reveal, considerable detail is necessary before
a program can be executed. There is also the question of correctness. One way to address
this last issue and to enhance the learning experience is to assign two sets of pseudorandom
number seeds to each student. One set common to all students consists of the class seeds. If
all students write logically equivalent simulation programs that preserve the same degree of
numerical accuracy, then using the class seeds should result in the same numerical results
for all. When this does not occur, students should compare their approaches to resolve
 4.29. EXERCISES 167

differences. The instructor can assign seeds in a way that ensures nonoverlapping sequences
for the streams assigned to each source of variation for each student as well as nonoverlapping
sequences among students.
While this ability to compare answers is a classroom luxury not available in prac-
tice, its ability to sensitize students to how seemingly small coding differences can generate
substantially different sample paths makes the approach worthwhile.
The second set of assigned seeds are unique to the student. Once differences in runs
using the class seeds are resolved, each student uses the individual seeds to execute her/his pro-
gram. For N students, this provides N independent sets of statistical results to the instructor
for analysis.
In addition to a complete listing of the simulation program, each student's submission
should contain a listing of the values of all input parameters. Each should contain two sets
of output, one using class seeds and the other, the student's seeds. Each of these outputs
should list initial and final seeds for each pseudorandom number stream. All results should
be listed to four significant digits.

QUARRY-STONE CRUSHING (EXERCISES i.1 AND i.i)

tjfJl4.1 Prepare a flowchart for an event that repeatedly executes at one-minute intervals
and collects data on:

1. status of each shovel

11. queue length at each shovel

iii. queue length at the crusher

iv. tons of stone unloaded at the crusher during the past minute (throughput).

For an illustration, see Figure 4.4.

tjfJl 4.2 This exercise establishes a basis for estimating profit without external deliveries.
Prepare and execute a simulation program based on the event-scheduling approach using
the flowcharts in Excercise 2.1e.

a. Begin the simulation with all trucks at their respective shovels and with the 50-ton
truck ready to begin loading. Priority order is arrival at the crusher, arrival at a shovel,
and data collection.

b. Table 4.3 shows how trucks are identified with shovels. Assign pseudorandom number
stream i to truck i for i = 1, ... , 9 to generate times for loading, transit to crusher,
dumping, and transit to shovels. At site 1 the ordering of trucks is 1, 2, and 3. At site
2 it is 4, 5, and 6. At site 3 it is 7, 8, and 9.

c. Run the simulation for six hours.

168 CHAPTER 4. PROGRAMMING AND EXECUTION

Table 4.3 Truck identification

Shovel
Site Truck Tonnage
1 1 50
2 20
3 20
2 4 50
5 20
6 20
3 7 50
8 20
9 20

d. For debugging purposes, all students should use the same seeds for corresponding
streams. For personal runs, each student should use distinct seeds taken from Table 4.2.
Doing so allows us to regard personal runs as statistically independent replications.

Note: Assigning distinct seeds within a row in Table 4.2 to the nine streams called
for in part b ensures that the sequences generated by these streams do not overlap,
provided that each truck makes no more than 1,000,000/4 = 250,000 round
trips from its shovel. Assigning a distinct row to each student and a distinct row
for class seeds ensures that the sample paths generated by each student and by the
class seeds are statistically independent.

e. Compute estimates of

i. crusher throughput

ii. shovel 1 utilization

lll. shovel 2 utilization

iv. shovel 3 utilization

v. overall shovel utilization

vi. mean queue at crusher

vii. mean queue at shovel 1

viii. mean queue at shovel 2

ix. mean queue at shovel 3.

f. Prepare graphs of

i. crusher throughput over five-minute intervals

 4.29. Ex ERe I S E S 169

ii. crusher queue length at five-minute intervals.

g. Describe the influence of initial conditions in the graphs of part f.

t%? 4.3 This exercise establishes a basis for comparing profits with and without external
deliveries. Prepare and execute a simulation program based on the event-scheduling ap-
proach using the augmented flowcharts in Exercise 2.2b that incorporate stone delivered by
the outside contractor.

a. Use the initial conditions in Exercise 4.2a augmented by the arrival at the crusher of a
truck from the external source.

b. Assign stream 10 to inter arrival times for external deliveries and stream 11 for
corresponding unloading times.

N.B. These assignments preserve the same stochastic variation associated with the
original nine trucks, so that changes in performance are due to the addition of
external trucks and the stochastic variation that their interarrival and service times
induce.

c. For your first run take 1/).. = 16 minutes.

d. Run the simulation using the same seeds for streams 1 through 9 as in Exercise 4.2b.

e. Follow the same debugging practice as in Exercise 4.2d.

f. Compute estimates of all performance measures listed in Exercise 4.2e.

g. Prepare graphs of:

1. crusher throughput per five minutes

11. crusher queue length at five-minute intervals.

h. Describe the influence of initial conditions in the graphs of part g and compare them
to what you observed in the corresponding graphs in Exercise 4.2f.

i. Discuss the effects of external deliveries on profits and utilization of Hard Rock's
equipment. See Exercise 2.2.

t%? 4.4 Independent Runs. Rerun Exercise 3 using the last pseudorandom numbers
produced there as the seeds for corresponding seeds in the new run. For j = 1, ... , 72, let

ZZj := crusher throughput during time [5(j - 1), 5 j) for Exercise 4.2
Z3j := crusher throughput during time [5(j - 1), 5j) for Exercise 4.3
Z4j := crusher throughput during time [5(j - 1), 5j) for the current exercise.
170 CHAPTER 4. PROGRAMMING AND EXECUTION

Plot {Z3j - ZZj; j = 1, ... , 72} and {Z4j - ZZj; j = 1, ... , 72} on the same graph and
interpret the differences between them, ostensibly due to the use of common pseudorandom
numbers.

~ 4.5 Prepare and execute a simulation program based on the process-interaction ap-
proach using the flowcharts in Exercise 2.1 f and Exercise 4.1. Follow the directions in Exercises
4.2a through 4.2e and show that this program produces the identical results as in Exercise
4.2e.

~ 4.6 Prepare and execute a simulation program based on the process-interaction ap-
proach using the flowchart in Exercise 2.2c and Exercise 4.1. Follow the directions in Exercises
4.2a through 4.2e and show that this program produces the identical results as in Exercise
4.3f.

TANKER LOADING (EXERCISES 2.3 AND 2.4)

~ 4.7 Using the flowcharts for Exercise 2.3b based on the event-scheduling approach,
prepare and execute a simulation program for the tanker problem.

a. For events that occur at the same time, assume the priority order: tanker arrival, tanker
berthing, loading completion, picking up a tanker at a berth, picking up a tanker in
the harbor, and tanker departure.

b. Start the simulation with all berths idle, the tug in the harbor, and the arrival of a
tanker in the harbor.

c. Run the simulation until the nth tanker has sailed from the harbor, where n = 1000
tankers.

d. Assign the following pseudorandom number streams:

Pseudorandom
Source of variation stream no.
Interarrival times
Towing in 2
Towing out 3
Deadheading in 4
Deadheading out 5
Type 6
Loading times
type 1 7
2 8
3 9

Deadheading denotes the movement of the tug with no tanker in tow.

 4.29. EXERCISES 171

e. Data Collection and Analysis. Let Tn denote the simulated time at which the nth
tanker has sailed from the harbor. Use data from the simulated time interval [O,Tnl to
estimate system parameters. In addition to identifying input parameter values clearly,
your output should contain averages, standard deviations, and maxima for:

1. Tanker waiting time

11. Number of waiting tankers in harbor and for deberthing

iii. Number ofloading berths

IV. Number of committed berths (reserved for tankers being towed in and holding
loaded tankers waiting to be towed out). A "committed" berth is one that has a
nonloading tanker occupying it or is awaiting a tanker that the tug is either towing
or is deadheading to get.

Based on the port time definition in Exercise 2.3, it should also contain estimates of:

v. pr(port time> 24i hours) for i = 1, ... ,10

VI. Average rebate per tanker

V11. Average rebate per day.

t$? 4.8 Using the flowcharts for Exercise 2.3c based on the process-interation approach,
prepare and execute a SIMSCRIPT U.5 simulation program for the tanker problem, treating
tankers and tugs as processes. Include steps a through e of Exercise 4.7.

t$? 4.9 Based on the augmented flowcharts in Exercise 2.4b for the event -scheduling
approach, prepare and execute a simulation program using the same input as in Exercise 4.7.
For the additional tankers, use

Pseudorandom
Sources of variation for new tankers stream no.
Round-trip times 10
Deadheading in 11
Deadheading out 12
Towing in 13
Towing out 14
Loading times 15

Include steps a through e of Exercise 4.7.

172 CHAPTER 4. PROGRAMMING AND EXECUTION

~ 4.10 Do the same as in Exercise 4.9 for the process-interaction approach as modeled
in Exercise 2Ac.

~ 4.11 Do the same as in Exercise 4.9 for the mixed approach modeled in Exercise 2Ad.

TERMINAL PAD CONTROLLER MODEL (EXERCISE 2.5)

The simulation programs developed in Exercises 4.12 and 4.13 establish a mechanism
for generating sample-path data to be employed in Chapters 6 and 7 to estimate how mean
delay in transmission /L(m,).,) in expression (2.6) varies as a function of character arrival
rate)" and packet size m.

~ 4.12 Using the flowcharts for Exercise 2.5b based on the event-scheduling approach,
prepare and execute a simulation program for the terminal PAD controller problem.

a. Give precedence to arrivals at the output buffer over arrivals at input buffers and to
transmission completions over arrivals at the output buffer.

b. Begin with a randomly scheduled arrival to a system of r = 10 empty input buffers,

an empty output buffer, and an idle transmitter. Perusal of Section 5.1.2 suggests how
to determine which input buffer receives an arriving character.

c. Assign pseudorandom number stream 1 to generate interarrival times and stream 2

to identify control characters across all input buffers. If the inverse transform method
(Table 8.1) is used to generate interarrival times, this turns out to be a reasonably good
way to control stochastic variation in character generation while varying arrival rate
)., and maximal packet size m.

d. For a first run, take)., = 1/20 characters per time unit and m = 32 and run the
simulation until the output buffer transmits the t = 2500th nonoverhead character.

e. Let

Ii := time delay for nonoverhead character i between arrival at an input buffer

and transmission at the output buffer for i = I, ... , t.

Collect and use T1 , ••• , 4, to estimate /L(m,).,) and the exceedance probabilities
pr(Ii > w) for each w E {350, 400, 450, 500, 550, 600}.

f. Graph T1 , ••• , Tt • What does this reveal about the warm-up interval?
 4.30. REF ERE NeE 5 173

rf}? 4.13 Using the flowcharts for Exercise 2.6 based on the process-interaction ap-
proach, prepare and execute a simulation program for the terminal PAD controller problem
according to parts a through f in Exercise 4.12.

MOBILE·TELEPHONE SERVICE (EXERCISE 2.6)

The simulation programs developed in Exercises 4.14 and 4.15 provide a means to be
employed in Chapters 6 and 7 for generating sample-path data for estimating the loss prob-
abilities fJh (m, r) and fJs (m, r) in expression (2.7), as functions of number of transmission
lines m and number oflines r dedicated to handoffs.
rf}? 4.14 Using the flowcharts for Exercise 2.6 based on the event-scheduling approach,
prepare and execute a simulation program for the mobile-telephone service problem.
a. Give precedence to hand-off arrivals over standard arrivals and to call completions
over all arrivals.
b. Begin with one randomly scheduled hand-off and one randomly scheduled arrival to
an otherwise empty and idle system.
c. Assign pseudorandom number stream 1 to interarrival times of han doffs, stream 2 to
standard interarrival times, stream 3 to duration times for handoffs, and stream 4 to
duration times for standard calls.
d. Run the simulation with m = 11, r = 0, Ah = 2, As = 4, (j) = 1 for a total of t = 1000
calls regardless of type. Hint: The quantity Ah / (Ah + As) denotes the probability that
a new call is a mobile call.
e. Devise and implement a data collection plan to estimate fJh (m, r) and fJs (m, r) in
expression (2.7).
f. Devise and implement a graphical analysis to assess the influence of initial conditions
on the sample-path data of part e.
g. Note any distinction between the statistical properties of the estimates of fJh (m, r) and
fJs(m, r).

rf}? 4.15 Using the flowcharts for Exercise 2.6e based on the process-interaction ap-
proach, prepare and execute a simulation program for the mobile telephone service problem
according to parts a through g in Exercise 4.14.

4.30 REFERENCES

Banks, J., and R. Gibson (1997). Simulation modeling: some programming required, JIB
Solutions, 29, 26-31.
174 CHAPTER 4. PROGRAMMING AND EXECUTION

CACI (1987). SIMSCRIPT II.S Programming Language, CACI Products Company, La Jolla,
California.

CACI (1993a). SIMSCRIPT II.S Reference Handbook, CACI Products Company, La Jolla,
California.

CACI (1993b). SIMDEBUG User's Manual: SIMSCRIPT II.S Symbolic Debugger, CACI
Products Company, La Jolla, California.

CACI (1997). SIMSCRIPT II.S Reference Handbook, CACI Products Company, La Jolla,
California.

Fishman, G.S. (1996). Monte Carlo: Concepts, Algorithms, and Applications, Springer-Verlag,
New York.

Gordon, G. (1969). System Simulation, Prentice-Hall, Englewood Cliffs, N.J.

Henriksen, J.O. (1997). An introduction to SLX, 1997 Winter Simulation Conference Pro-
ceedings, S. Andrid6ttir, K.J. Healy, D.H. Withers, and B.L. Nelson, editors, IEEE,
Piscataway, N.J.

Kelton, W.D., RP. Sadowski, and D.A. Sadowski (1998). Simulation with Arena, McGraw-Hill,
New York.

Kiviat, P.J., R Villanueva, and H.M. Markowitz (1969). The SIMSCRIPT II.S Programming
Language, Prentice-Hall, Englewood Cliffs, N.J.

Law, A.M., and W.D. Kelton (1991). Simulation Modeling and Analysis, McGraw-Hill, New
York.

Markowitz, H.M., H.W. Karr, and B. Hausner (1962). SIMSCRIPT: A Simulation

Programming Language, Prentice-Hall, Englewood Cliffs, N.J.

Pegden, CD., R.E. Shannon, and RP. Sadowski (1995). Introduction to Simulation Using
SIMAN, second edition, McGraw-Hill, New York.

Pritsker, A.A.B. (1995). Introduction to Simulation and SLAM II, fourth edition, Wiley, New
York.

Pritsker, A.A.B., and P.J. Kiviat (1969). Simulation with GASP II, Prentice-Hall, Englewood
Cliffs, N.J.

Pritsker, A.A.B., and J.J. O'Reilly (1999). Simulation with Visual SLAM and AweSim, second
edition, Wiley, New York.

Schriber, T.J. (1991). An Introduction to Simulation Using GPSSIH, Wiley, New York.

Swain, J.J. (1999). 1999 Simulation software survey, ORMS Today, 26, 38-5l.
 4.30. REF ERE NeE s 175

Systems Modeling Corporation (199Sa). ARENA Template Reference Guide, Sewickley,

Pennsylvania.
Systems Modeling Corporation (199Sb). ARENA User's Guide, Sewickley, Pennsylvania.

Systems Modeling Corporation (199Sc). SIMAN Reference Guide, Sewickley, Pennsylvania.

 CHAPTER 5

G. S. Fishman, Discrete-Event Simulation

© Springer Science+Business Media New York 2001
Search, Space,
and Time

Every execution of a computer program uses memory space and consumes computing time.
In particular, a discrete-event simulation expends a considerable proportion of its running
time executing searches for new space and creating and maintaining order among the myriad
of entity records and event notices it generates as simulated time evolves. In spite of their
relative importance, current PC workstation environments, with their substantial memo-
ries and reduced, if nonexistent, emphasis on execution within a specified computing time
constraint, make these topics appear less important to the simulationist than they were in
the past. Moreover, every simulation programming language implicitly provides a means for
managing space and performing searches during execution of virtually any program written
in the language, further removing these issues from a simulationist's consciousness.
This computing environment is reassuring to a potential simulationist, for it relieves
her/him of the need to be concerned about memory space and computing time for simu-
lations of small and moderate sizes. Indeed, most advertising for simulation languages and
packages focus on examples that illustrate this speed of execution. However, as the number of
entity and event classes in a simulation increases, it is not unusual to see the corresponding
computing time increase rapidly. What may have taken seconds for a small-size simula-
tion now takes minutes for its moderate-size extension. What may have taken hours for a
178 CHAPTER 5. SEA RCH, SPA eE, AND TIME

moderate-sized simulation now takes days for its large-size extension. When such an increase
in computing time occurs, it often puzzles and frustrates the simulationist.
Simulation languages manage space and conduct searches in different ways. As a
consequence, programs based on the same simulation logic but written in different languages
vary in their execution times and space needs. It is not the purpose of this chapter to
choose the one among these languages that performs best with respect to time and space
conservation. Indeed, it is problematic to determine whether a single language dominates
when one considers the widely varying characteristics of many simulations ..
This chapter has a twofold purpose: to characterize the way in which memory space
needs arise during a simulation and to describe prototypical modes of memory manage-
ment and search that characterize the procedures that many simulation languages employ.
An awareness of these concepts enables a simulationist to anticipate potential computing
problems during the design phase of a study, rather than encountering them as a consequence
of costly execution.
To establish a formalism for discussing space and time, Section 5.1 describes a series-
parallel queueing model whose parameters we employ throughout the chapter to characterize
how space and time needs vary. Section 5.1.1 addresses the space issue generically for the
future-event set, and Section 5.1.2 illustrates how thoughtful analysis prior to modeling can
significantly reduce space needs during execution.
This illustration has more general implications. Rather than regarding a problem's
description as a recipe to be translated into programming code and then executed, the illus-
tration encourages simulationists to apply their skills of mathematical analysis to transform
the description into a probabilistic equivalent that increases computational efficiency.
To set the stage for a discussion of space management, Section 5.2 describes the essential
features oflist processing, with emphasis on predecessor-successor relationships. Section 5.3
introduces a succession of progressively more sophisticated procedures for this management.
Section 5.4 focuses on the management of the future-event set. In particular Section 5.4.1
discusses single linear event lists, Section 5.4.2 multiple linear event lists, Section 5.4.3 heaps,
and Section 5.4.4 Henriksen's Algorithm. An example, carried through the four subsections,
provides a basis for understanding and comparing the time needs of each method of FES
management.
Section 5.5 describes how the restricted formalism of Section 5.1, which Sections 5.1.1
through 5.4.5 employ to illustrate concepts, can be extended to a broader class of delay
models.

5.1 MEMORY SPACE NEEDS AND COMPUTING TIME: A

SIMPLIFIED PERSPECTIVE
While every discrete-event simulation model induces its own peculiar demands with regard
to memory space and computing time, all share common properties. This section describes
5.1. MEMORY SPACE NEEDS AND COMPUTING TIME 179

arrivals

Figure 5.1 Series-parallel queueing model

several of these properties as they arise in the context of a commonly encountered class of
delay problems. Once acquainted with these properties, the reader will have a perspective for
understanding what motivates the discussion of alternative data structures for space search
in Section 5.2 and for managing the FES in Sections 5.4.1 through 5.4.4. To broaden the range
of applicability of these concepts, Section 5.5 relaxes several of the modeling assumptions
made in the present section.
Consider a simulation model based on the k-station queueing system in Figure 5.1,
where r statistically independent arrival processes feed jobs to station 1. Every job receives
service at each station j from one ofits m j servers for j = 1, ... , k. Upon completion of
serviceatstationk,ajobdepartsthesystem.Fori = 1, ... , r,arrivals of type i occur at times
Til ~ Ii2 ~ ... with identically distributed interarrival times {Ii,/+l - Iiz; I = 1,2, ... }
with mean 1/Wai.
Each server at station j has identically distributed service times with mean l/wsj for
j = 1, ... , k. Then one may think of Wa := Wa\ + ... + War as the rate at which jobs arrive
to the system at station 1 and of m jWsj as the rate at which station j processes waiting jobs
for j = 1, ... , k.
At each simulated time r, interest focuses on

B(r) := B\(r) + ... + Bdr)

(number of busy servers)

and

(number of jobs awaiting service),

180 CHAPTER 5. SEARCH, SPACE, AND TIME

where

B j (r) := number of busy servers at station j

and

Q j ( r) := number of jobs awaiting service at station j.

For Q (r) + B (r) to have an equilibrium probability distribution, we require that

p := max p. < 1,
max l::;j~k J

where

j = 1, ... ,k.

At simulated time r, the executing simulation's future-event set contains r + B(r)

event notices, one scheduling the next job arrival for each type i for i = 1, ... , r and B (r)
scheduling the completions of service for each of the busy servers.
At that same time r, the executing simulation has also assigned memory space to the
records of the B (r ) + Q ( r ) temporary entities called jobs. In the present setting each record
has an attribute identifying the type of job. More generally, every model specification dictates
the number of attributes that each temporary entity requires and, hence, its concomitant
space needs. Associated with each station j is a set containing the records of the Qj ( r) jobs
awaiting service there. For present purposes, we refer to these k sets collectively as QUEUE. It
occupies space proportional to Q ( r ). We defer discussion of the computational complexity
that alternative job selection rules from QUEUE imply until Section 5.2.1
In some cases, a record of a temporary entity can also serve as an event notice, thus
reducing space requirements. We assume that these circumstances do not prevail here.
We first examine how space needs vary with numbers of servers and buffer capacities.
As simulated time revolves, B(r) and Q(r) vary, and therefore space needs vary. In the
present setting,

0.::: B(r) .::: kin w.p.l, (5.1)

where

_ 1
m:= -(ml
k
+ ... +mk). (5.2)
5.1. MEMORY SPACE NEEDS AND COMPUTING TIME 181

Were station j to have a finite buffer capacity, C j' for each j = 1, ... , k and were jobs lost
to the system when they arrive at station j and find its buffer full (i.e., Qj ( r) = C j ), then

0::: Q(r) ::: kG w.p.I, (5.3)

where

Let

Pmin:= min Pj,

l:'Oj:'Ok

and observe that for each fixed r ::: 0,

B(r) -+ km as Pmin -+ 1 w.p.I (5.4)

and

Q(r) -+ kG as Pmin -+ 1 w.p.I.

In a highly congested system, these properties imply that at an arbitrarily selected time r:

• The number of event notices in the FES, r + B (r), is most likely to be r + km.
• The number of extant temporary entities, B( r) + Q( r), is most likely to be k(G + m).

Hereafter, we assume a highly congested system.

Suppose that traffic intensity Pj and service rate Wsj at station j remain fixed as its
number of servers m j increase. These conditions imply that the arrival rate Wa must also
increase. Then for each j = 1, ... , k,

and

implying that:

• The number of event notices in the FES tends to grow without bound as mincreases.
182 CHAPTER 5. SEARCH, SPACE, AND TIME

• The number of records in QUEUE becomes relatively small as Iii increases.

We next examine how space needs vary as a function of traffic intensity and the
amount of stochastic variation in interarrival and service times. In a highly congested but
uncapacitated system (Cj = 00; j = 1, ... , k), memory space needed for event notices
remains proportional to r + klii. However, space needs to accommodate the records of
temporary entities behave differently. As an illustration, suppose that Tl :s Tz :s ... denotes
the ordered interarrival times Uj=l {Tjz; 1= 1,2 ... } for the r types of arrivals and assume
that {Iz+! - 1/; I = 1,2, ... } is an Li.d. sequence with mean l/wa and variance a~. Also,
assume that the service times at station j have variance, a~. Let

a
Ya := _a_ = coefficient of variation of interarrival times
l/wa

and

a·
Ysj := _S_l_ = coefficient of variation of service times at station j.
l/wsj

Small Ya implies that interarrival times vary little from their mean, whereas large Ya implies
large deviations from this mean. An analogous characterization applies to Ysj for service
times at station j.
It is known (Kollerstrom 1974) that if interarrival times to station j are i.i.d. and traffic
intensity Pj is less than but close to unity, then

(5.5)

This expression reveals several important properties:

• For fixed Ysj and Ya, v grows superlinearlywith Pj.

• For fixed Pj and Ysj' v grows quadratically with Ya'

• For fixed Pj and Ya' v grows quadratically with Ysj·

As an example, a model with Pj = .99 and Ya = Ysj = l/./i, as would be the case if
interarrival and service times came from Erlang distributions with two phases (Section 8.13),
gives v ~ 49.50. However, if interarrival and service times were exponentially distributed,
then Ya = Ysj = I, so that v ~ 99.00. The sensitivity to stochastic variation is apparent.
These results, based on mean-value behavior, convey only part of the story regarding
space needs in an uncapacitated queueing system. Suppose one runs the simulation for
5.1. MEMORY SPACE NEEDS AND COMPUTING TIME 183

exactly r = t simulated time units. Then

G(t):= sup [B(z) + Q(z)],

O~z::9

where

G(t) ::: r + km + sup Q(z) w.p.l

O~z::9

gives the maximal number of temporary entities in the system during the simulation. For
every fixed positive integer q, pr[suPO~Z::9 Q(z) > q] is an increasing function of t, implying
that for given input Wa!, ... , War, Ws!, ... , Wsb m!, ... , mb a long simulation run is likely
to demand more space to store its records than a short run.

5.1.1 FES MANAGEMENT TIME PER JOB

Each job passing through the system induces 2( 1 + k) interactions with the FES, one
to schedule the next arrival of this type, k to schedule its completions at each of the k stations
it must visit, and 1 + k to select the next event to execute when the job arrives at station 1
and completes service at stations j = 1, ... , k.
Let ga (q) denote the average computing time required to schedule an additional event
when the FES contains q event notices and let gb(q) denote the average computing time
required to select the next event from an FES containing q event notices. Then at simulated
time r at which an interaction with the FES occurs, each job passing through the system
consumes

(1 + k)[Ega(r + B(r)) + Egb(r + B(r)] ::: (1 + k)[ga(r + km) + gb(r + km)] (5.6)

computing time on average interacting with the future-event set. Each simulation language's
protocol for managing the FES determines the particular form that ga(.) and gb(.) take. For
example, Section 5.4.3 shows that simulation languages that structure their future-event sets
as binary heaps have ga(B(r)) = O(logz B(r)) and gb(B(r)) = O(logz B(r)). Therefore,
for a simulation of a highly congested system that uses these languages, each job's interaction
with the FES takes O( (1 + k) logz (r + km)) time on average. Regardless of the protocol,
{ga (q)} and {gb (q)} both are increasing functions of q.

5.1.2 REDUCING THE SIZE OF THE FES

The last -mentioned property suggests that any reduction in the size of the FES reduces
its average management time. Here we describe one opportunity for reduction. Suppose that
for each arrival of type i = 1, ... , r, the interarrival times {'Ii.I+! - 'IiI; I = 1,2, ... }
184 CHAPTER 5. SEARCH, SPACE, AND TIME

form a sequence of i.i.d. exponentially distributed random variables with exponential p.d.f.

denoted by cal E (1/Wai)' Then

Ai(r) := number of arrivals of type i in the simulated time interval (0, r 1

has the Poisson distribution

denoted bycaIP(wair). As we have described the model, one would proceed as follows:

PROTOCOL A

Each time r at which an arrival of type i occurs, the simulation program:

• Creates a record for the arriving job and assigns i to its type attribute.

• Samples the next type i interarrival time fl T from e (1/Wai ).

• Creates and files an event notice in the FES for this next arrival with execution time,
r + flT.
Therefore, at every simulated time r, the FES contains at least r event notices, one for each
arrival type.
We can reduce this number by exploiting a property of the Poisson distribution. Since
A I ( r ), ... , Ar ( r) are statistically independent:

• A(r) := AI(r) + ... + Ar(r) has the Poisson distribution P(war).

• The corresponding sequence of interarrival times for the aggregated arrival process
are exponentially distributed from (1/Wa ).

• pr(next arrival is of type i) = ::' i = 1, ... , r.

As an immediate consequence, there exists an alternative procedure for generating arrivals:

PROTOCOL B

When an arrival occurs at time r, the simulation program:

• Samples type I from the p.m.f. {wadwa; i = 1, ... , r}.

5.1. MEMORY SPACE NEEDS AND COMPUTING TIME 185

• Creates a record for the arriving job and assigns I to its type attribute.

• Samples the next interarrival time t::..T from E (1/wa ).

• Creates and files an event notice in the FES for this next arrival with execution time
r+t::..T.

Protocol B generates an arrival process with statistical properties identical to those induced
by Protocol A. However, with regard to benefits and costs, Protocol A:

• Requires r memory spaces to store Wal, •.. , War.

• Requires O(r + km) memory space to store the event notices needed to maintain the
FES.

• Consumes 0((1 + k)[ga(r + kin) + gb(r + km)]) average time per job interacting
with the FES.

On the other hand, Protocol B:

• Requires one additional sample I of arrival type drawn from {Wal

Wa
, ••• , W a ,}. This can
Wa
be done in O( 1) time, independent of r, using memory space proportional to 2r, either
by the cutpoint method of Section 8.2 or the alias method of Section 8.4.

• Requires one memory space to store Wa.

• Requires O( 1 + km) memory space to store the event notices needed to maintain the
FES.

• Consumes O( (1 + k )[ga (1 + km) + gb (1 + km)]) average time per job to interact

with the FES.

The statistical equivalence of samples generated by Protocols A and B exists if and only
if {1I,1+1 - 111; I = 1, 2, ... } are i.i.d. exponentially distributed random variables for each
type i = 1, ... , k. If these times have any other distributions, applying Protocol B would be
inappropriate. However, for some simulation formulations, the assumptions of the underly-
ing problem are that interarrival times for the aggregate arrival process are i.i.d. with known
distribution and with known selection p.m.f. {qj := pr(that an arrival is of type i); i =
1, ... , k}. Then we can effect the same procedure as in Protocol B as follows:

PROTOCOLC

When an arrival occurs at time r, the simulation program:

• Samples type I from {qj; i = 1, ... , k}.

• Creates a record for the arriving job and assigns I to its type attribute.
186 CHAPTER 5. SEARCH, SPACE, AND TIME

• Samples the next interarrival time t::.T from its interarrival time distribution.

• Creates and files an event notice in the FES for this next arrival with execution time
r + t::.T.
When it applies, Protocol C allows one to reap the same space and time benefits as in Protocol
B.
If a modeler has empirical arrival-time data {Iiz; I = 1, ... , Nd for i = 1, ... , r,
these revelations regarding the advantages of Protocol C should encourage her/him to follow
these steps when modeling the arrival process (see Chapter 10):

• Form the sample sequence

{Tz; 1= 1, ... , N := Nl + ... + NT} = ordered Ur=l {IiI; 1= 1, ... , N i }.

• Form the interarrival time sequence

{Zz := 1/+1 - 1/; I = 1, ... , N - 1}.

• Test hypothesis H: ZJ. ... , ZN-l are i.i.d.

• If H is accepted:

• Fit a p.d.f. to Zl, ... , ZN-l.

• Estimate qi by qi = Ni / N for i = 1, ... , r.

5.2 LIST PROCESSING

Simulation languages usually employ one of two memory management techniques. One,
as in SIMSCRIPT 11.5, searches and allocates space each time the simulation calls for the
creation of a record for a new temporary entity or for a new event notice. The other, as in
GPSS/H, SIMAN, and SLAM, uses a block-array concept to accomplish the same purpose. At
the start of execution it allocates a block of contiguous memory space, which the executing
simulation uses to store new temporary entity records and event notices. Once a record
or event notice is no longer needed, both approaches make previously committed space
available for future use.
Regardless of the space allocation approach, all simulation languages rely heavily on
the concept of list processing to keep order among records and event notices as simulated
time elapses. List processing denotes a myriad of operations executable on a data structure
in a digital computer. The data structure is called a list. The list contains records each of
which has fields to store information. The operations include (Knuth 1973):
 5 .2 . LIs T PROC E S SIN G 187

• Gaining access to the jth record of a list to examine or change its content.

• Inserting a new record after the jth but before the (j + 1)st record.
• Deleting the jth record from the list.

• Combining lists.

• Dividing one list into several.

• Determining the number of records of a particular kind in a list.

• Ordering records in a list in ascending order based on the values in specified fields.

• Searching the list for records with given values in certain fields.

In the nomenclature of discrete-event simulation, list is synonymous with set.

In discrete-event simulation, list processing arises so frequently that a highly experi-
enced programmer, although familiar with these concepts, would find explicit programming
of each occurrence an onerous task. Fortunately, computer programming languages exist
that, in addition to having the list processing capability, contain libraries of subprograms
that perform the specialized list processing that discrete-event simulation demands.
A concept fundamental to an understanding of list processing is the predecessor-
successor relationship. Consider a simulation of ships arriving at a port to unload cargo.
Since the port has limited facilities, a ship may have to wait before it can unload its cargo.
A record characterizing each arriving ship contains fields in which relevant data are stored.
Fields 1, 2, and 3 of the records in Figure 5.2a contain the ship's identification, its arrival
time at the port, and its cargo tonnage, respectively. The address in the leftmost field of each
record denotes the location of the first word of the record in the computer's memory bank.
These addresses reveal that the four records are haphazardly located in memory. Usually
this is due to the fact that the records were created at different simulated times and that the
executing program searched for and allocated space at each of these times (Section 5.4.1).
Section 5.4.2 describes an alternative memory management that relies on reserving a fixed
block of contiguous memory at the beginning of simulation execution and then maintaining
the list within this block.
Suppose only the four ships corresponding to the four records in Figure 5.2a are waiting
to be unloaded. To keep track of them we put their records in a list. To form a list we need a
criterion for ordering the records. Suppose this criterion is taken to be the ascending order
of arrival times. A look at the records indicates that they should have order 14, 15, 16, and 17
in terms of identification. To make this ordering operational for the computer we introduce
the concept of a predecessor-successor relationship.
Note that each record in Figure 5.2a has a predecessor address in field 4 and a successor
address in field 5. A record's predecessor address identifies the record that precedes it in the
list. Since we are ordering on arrival times, record 14 has no predecessor and is called the
header. However, record 15 has a predecessor, whose record begins at address 01081. But this
188 CHAPTER 5. SEARCH, SPACE, AND TIME

FIELD
2 3 4 5
ship arrival cargo predecessor successor
Address identification time tonnage address address

header ~ 01081 14 107.3 1111891 1 null 00011

00011 15 123.2 11557871 01081 03447

03447 16 151.7 11008881 00011 23415

trailer ---+ 23415 17 155.8 11369251 03447 null

(a) Ordered by arrival time

FIELD
2 3 4 5
ship arrival cargo predecessor successor
Address identification time tonnage address address

01081 14 107.3 111891 03447 23415

trailer ---+ I 00011 15 123.2 11557871 23415 null

header~1 03447 16 151.7 1100888 1 null 01081

23415 17 155.8 11369251 01081 00011

(b) Ordered by tonnage

Figure 5.2 Predecessor-successor relationship

 5 .2 . LIS T PRO C E 5 5 I N G 189

new
record -LI_0_8_0_91----''--_1_8_--'--_1_67_._3--'-_1_4_7_24_4--'-_ _ _ . 1 - -_ _- '

,-----'---'---'-----'-----------'---'-----'---------'-----------------------------'1
i !
trailer
!
! i
before
I000~1 I I08091 I i insertion
!i !
'_._._-_._._._._._._._._._._._._-_._._---_._._._--._.--------------------------- ...:i

r------------------------------------------·
, '
:,
,
~ after
~:
insertion
,,
,, ,,
L__________________________________________ I

(c) Adding a new record when ordered by tonnage

Figure 5.2 (continued)

is simply record 14. Similar predecessor relationships apply for records 16 and 17. A reverse
relationship exists for the successor relationship. Record 17, the last record in the list, has no
successor and is called the trailer.
Ordering the records in this way facilitates the selection of the next ship for unloading
when a berth becomes idle. In particular, one selects the header record (1) and makes record
2 the new header. This selection presumes that ships receive service in the order of their
arrival. Suppose that service is to occur in increasing order of cargo tonnage (field 3). Then
Figure s.2b shows the predecessor-successor relationships that prevail for the four waiting
ships. Also suppose that a new ship, 18, arrives at time 167.3 (field 1) with a cargo of 147244
tons (field 3) and "enters" the queue. Figure S.2c shows the new record and the old and new
predecessor-successor relationships that prevail when cargo tonnage determines order. Note
that in order to chain record 18 to the list one need change only the predecessor address of
record 15, the successor address of record 17, and assign predecessor and successor addresses
to record 18. These operations correspond to the concept of inserting a record into the list.
Deleting a record from the list entails converse operations.

5.2.1 INSERTION AND DELETION

A discrete-event simulation maintains lists of records of entities waiting for resources

to become available. The QUEUE list in Chapter 4 is an example. If first-in-first-out is the
190 CHAPTER 5. SEARCH, SPACE, AND TIME

prevailing discipline for ordering the records upon insertion in the list, as in Figure S.2a,
then a single linear list has constant insertion and deletion times, regardless of its length.
If the ordering discipline depends on an attribute of the record other than simulated
time at insertion as in Figure S.2b, then relying on a single linear list becomes problematic.
For example, suppose that ordering within the QUEUE in the airline reservation example is
based first on TYPE (either ONE or MANY) and then on simulated time at insertion. If the
list contains N records, insertion can take O(N) computing time, as would occur when all
N records have TYPE = MANY, the record to be inserted has TYPE = ONE, and the list is
entered from its front end.
Maintaining two lists, one for each TYPE, would again lead to constant computing
times for insertion and deletion, independent of list lengths. However, two lists create a need
to check a record's TYPE to choose the appropriate list for insertion and a need to check
both lists for deletion when the higher-priority list is empty.
More generally, the computing times to insert and delete records according to the values
of an attribute other than simulated time of insertion can, in principle, be made independent
of the lengths of the lists by using multiple lists as above. However, the computing time
required to determine the list in which to insert a record or from which to delete a record
tends to increase with the number oflists. If k lists hold N records in total and N » k, then
multiple lists tend to offer a computational advantage. If k > N, as occurs when there are
relatively few waiting customers virtually all of whom differ with regard to their values for
the critical attribute, then a single list is likely to offer an advantage.
Every simulation language has built-in procedures for maintaining lists. Some use
multiple linear lists as above, others use special data structures, such as heaps that do not
maintain completely ordered lists but have O(log2 N) insertion and deletion computing
times. Section S.4.3 describes binary heaps in the context of managing the future-event set,
which is a list ordered on desired execution, rather than scheduling, time for an event.
While its supporting documentation rarely spells out the exact modality that a language
has adopted for list processing, presumably for proprietary reasons, list processing times all
have common tendencies. They grow with the number of sublists employed to accommodate
the primary criterion for insertion and with the number of entries per sublist if the secondary
criterion for insertion is other than simulated time of insertion. The present discussion is
intended merely to sensitize the first-time user of a simulation language to a potentially
major consumer of computing time.

5.3 DYNAMIC SPACE MANAGEMENT

During execution, a simulation must maintain records of temporary entities and of event
notices in its FES to ensure continued evolution. As time elapses the number of these entity
records and the number ofthese event notices vary, often substantially. Accommodating these
 5.3. DYNAMIC SPACE MANAGEMENT 191

records and event notices and managing their creation and eventual elimination account for
a major portion of computing time.
Every computer program begins execution with an upper bound on the memory
allocated to it. Sometimes this is literally at the individual program level. More often, today's
computing environments impose this upper bound at the level of the user. That is, he/she is
given some partition of memory within which to do computing in a stand-alone computing
environment. This partition can be virtually the entire uncommitted memory capacity of the
computer. In a network environment, the size of this partition usually reflects the competing
needs of all network users.
At the beginning of a simulation run, some of this space becomes reserved for the
program and data records that it needs to begin execution. Several languages also set aside
large block arrays within which the simulation stores records of temporary entities and event
notices as they arise during the simulation's execution. Others provide this space on demand.
The present account describes both approaches.
Algorithms SPACE_FINDER and SPACE-RELEASER describe elementary prototypes
of dynamic space management:

ALGORITHM SPACE_FINDER

1. Search of space

Yes No
3y. Mark space as occupied. 3n. Terminate simulation.
4y. Assign space.
Sy. Continue simulation.

ALGORITHM SPACE_RELEASER

1. Mark space as unoccupied.

2. Merge with contiguous space.

3. Continue simulation.

As execution progresses, a discrete-event simulation makes successive requests for

varying amounts of space. For example, as a simulation of the airline reservation problem
evolves, it creates records of temporary entities in block 1 and event notices, to schedule
future-event executions, in blocks 4 and 7 of Figure 2.5. Since the allocation of space for every
creation reduces the space available to satisfy future space requests, a simulation program that
merely consumes space could conceivably exhaust its availability and terminate prematurely.
192 CHAPTER 5. SEARCH, SPACE, AND TIME

The likelihood of this occurrence increases with the length of time for which the simulation
program is to be run. To reduce the frequency of such an occurrence, an effective space
management policy releases space as soon as it is no longer needed, so that it can be merged
with contiguous uncommitted space and subsequently assigned to future demands as they
arise. For example, the airline reservation simulation destroys temporary entity records for
CUSTOMERs in block 4 of Figure 2.8 and destroys event notices for executed events in
block 4 of Figure 2.5. Therefore, good programming practice encourages one to:

• Release space as soon as it is no longer needed.

Let dynamic space manager (DSM) denote the procedure that a simulation language
employs to search for and assign space. For simplicity, one can regard the DSM as a sub-
program to which a simulation program transfers control each time it requests or indicates
that it no longer needs a space segment. Let an uncommitted space segment of length n de-
note a sequence of n successive words in computer memory that are not already assigned.
Assume that the DSM maintains a space roster, a linked list of records each containing the
address of the first word of a noncontiguous uncommitted space segment and its length. A
noncontiguous uncommitted space segment oflength n denotes a sequence of n successive
uncommitted words in memory preceded and succeeded by previously allocated words.
Each attempt to assign space in Algorithm SPACE_FINDER requires a search of the
space roster to identify the locations of uncommitted space segments and their lengths.
Conversely, every attempt to merge space in Algorithm SPACE_RELEASER requires a search.
Collectively, allocating and merging space consume a substantial portion of computing time
in a simulation.
For a given simulation, let <¥l denote the mean computing time expended per space
allocation and let <¥2 denote the mean computing time expended per space merge. Suppose
the simulation is executed until N jobs, or CALLs in the airline reservation problem, receive
service. If f denotes the frequency with which new space is allocated per job or CALL
during this simulation, and if we assume that a space merge eventually follows each space
allocation, then the DSM consumes mean computing time f . (<¥l + «2) per job or CALL.
Clearly, choosing efficient space allocation and merge procedures makes <¥l and <¥2 smaller
than they would otherwise be, and that is clearly desirable. Moreover, adopting a simulation
programming strategy that keeps f as small as possible is also desirable. To control f, we
repeat our earlier admonition:

• Aggregate elementary events that occur at the same simulated time into compound
events.

Since allocating space for an n2 -word record takes more time in general than allocating
space for an n 1 -word record, where n2 > n 1, the additional admonition:

• Keep the number of attributes of events and entities to the required minimum.
 5 .3 . DYNAMIC SPACE MANAGEMENT 193

predecessor successor
address address
A(1,I) A(1,2) ... A(1,r - 1) A(1,r) D(1)
A(2,1) A(2,2) ... A(2,r - 1) A(2,r) D(2)

AU,1) AU,2) ... AU, r - 1) A(i, r) D(i)

A(n,1) A(n,2) ... A(n, r - 1) A(n, r) D(n)

(a)

ACi,1) A(i,2) AU, r - 1) AU, r) DU)

iI
2 ) n i2
3

trailer--+ )-1 n
header--+ ) 2

n- 1
n 2 ) - 1
iI, i 2, ... , i n - 4 is a permutation of {l, ... , n} \ {2, ) - 1, ), n}
(b)

Figure 5.3 Block arrays and ALIST

To effect space allocation and merge, a dynamic space manager can take alternative
forms. For example, some simulation programming languages rely exclusively on linear
linked lists to keep track of uncommitted space (e.g., Figure 5.2). With each successive space
allocation, the computing time required to find an uncommitted segment of space of any
given length tends to increase, usually because more search is required. Other languages rely
on a block array approach, combined with a linked list, to reduce this search time. This
alternative can significantly reduce search time.
For the moment, assume that all space requests are for segments of r contiguous words.
Figure 5.3 depicts an n x r block array A and an n x 1 block array D. Each row of A denotes
a space segment of length r, including fields for predecessor and successor addresses that
allow chaining linked committed rows of A into a doubly linked linear list called ALIST. We
emphasize that this is merely one way to maintain a list. Section 5.4.3 describes the concept
of a heap, which provides the basis for alternative list-maintenance procedures that become
increasingly more efficient as the number of records in the list grows. The D array contains
the addresses of uncommitted rows in A.
194 CHAPTER 5. SEARCH, SPACE, AND TIME

Since all rows in A are initially uncommitted, we begin with

D(i) +- i, 1 .::: i .::: n,

and

#VACANT +- n.

Then Algorithm SPACE...FINDER* describes one way to satisfy a demand for space oflength T
in less computing time on average than Algorithm SPACE...FINDER. IfA has no uncommitted
rows (#VACANT = 0), the algorithm executes its right branch in computing time that varies
with space availability in the remaining memory. However, if A has at least one uncommitted
row (#VACANT > 0), the algorithm executes its left branch in constant computer time,
independent of the availability of space in the remaining memory. This occurs because no
need exists to search for space. As n increases relative to the number of space segments
occupied at a moment in simulated time, the constant-time left branch comes to dominate.
Figure 5.3b illustrates chaining in the ALIST where rows j, 2, n, and j - 1 comprise the
ALIST, indicating that these are committed, whereas the remaining n - 4 rows listed in the
D array are uncommitted. Note that the ALIST also may contain committed space segments
not in the A array.

ALGORITHM SPACE_FINDER*

Yes No
2y. i +- D(1). 2n. Execute Algorithm SPACE-FINDER.
3y. If #VACANT > 1,
D(1) +- D(#VACANT).
4y. #VACANT +- #VACANT - 1.
Sy. Assign record to A(i, 1), ... , A(i, r).
6y. Insert new record last in ALIST.
7y. Continue simulation.

Algorithm SPACE..RELEASER* shows one way of managing released space. In partic-

ular, it maintains the D array of uncommitted rows in A with computing time independent
ofn and T.
 5 .3 . DYNAMIC SPACE MANAGEMENT 195

ALGORITHM SPACE_RELEASER*

1. Remove record from ALIST.

Record is in
A array?
Yes No

3y. #VACANT +- #VACANT + 1. 3n. Execute Algorithm SPACE_RELEASER.

4y. D(#VACANT) +- row no. of
released record.
Sy. Continue simulation.

For the left branch of Algorithm SPACE_FlNDER*, let f31 denote the mean computing
time expended per space allocation and let f3z denote the mean computing time expended
per space release. Note that f31 :s al and f3z :s az. Let P denote the probability that space is
available in the A array to satisfy a space demand (left branch). Then mean computing time
for Algorithm SPACE_FlNDER* is

f· P(f31 + f3z) + f· (1- p)(al + az) :s f· (al + az),

encouraging a large initial space allocation n to make p close to unity.

5.3.1 AGGREGATING RECORD CLASSES

Collectively, Algorithms SPACE_FlNDER* and SPACE_RELEASER* demonstrate the

benefits of employing block arrays to manage space demands of fixed length r. Moreover,
modifications ofthis logic allow one to accommodate k different types ofspace requests where
type i requires ri fields for 1 :s i :s k. One approach applies Algorithms SPACE_FlNDER*
andSPACE-RELEASER* to k pairs of arrays (AI, Dd, ... , (Ak, Dd, where Ai hassizeni x ri
and Di has size ni x 1 for 1 :s i :s k. For rl ::::: rz ::::: ... ::::: rko a second alternative maintains
a single A array of size n x rl and a single D array of size n x 1. In this case, a separate ALIST
of committed rows would not have to be maintained for each demand type. To compare
these alternatives, let

Pi := probability that the next request is for an ri -field segment

and

Li := number of committed rows in Ai prior to next request.

196 CHAPTER 5. SEARCH, SPACE, AND TIME

Since

PI x pr(Ll :::: nd + P2 X pr(L2 :::: n2) + ... + Pk X pr(Lk :::: nk)

:::: pr(Ll + ... + Lk :::: nl + ... + nk), (5.7)

the second alternative allows one to choose an n :::: n 1+... +nk without necessarily increasing
the frequency of search for space in remaining memory. Moreover, if rl = ... = rb then
nrl :::: (nl + ... + nk)rh implying a reduction in space as well.

5.3.2 RESIZING

Since the frequency of search for new space in Algorithm SPACE-FINDER* dimin-
ishes as n increases, computing time is reduced, but at the expense of reserving more,
potentially unused, space for the A and D block arrays. To balance computing time and
space considerations, some DSMs start with a small or moderate n but allow for an upward
resizing of the arrays if the test #VACANT = 0 is true in Algorithm SPACE-FINDER*. Al-
gorithm SPACE-FINDER** displays a prototypical procedure for performing these steps,
where (j(n) (> n) denotes an integer-valued function of n. For example, (j(n) := n doubles
the numbers of rows in A and D each time it exhausts the already available space.
Using Algorithm SPACE-FINDER** instead of Algorithm SPACE-FINDER* replaces
the need to search for space for individual records by a considerably less frequent need
to search for block arrays to augment A and D. While each resizing of the block ar-
rays may consume considerably more computing time, the lower rate at which resizing
is necessary generally favors the approach. Again, we emphasize that applying Algorithm
SPACE_FINDER** to each of k pairs of arrays (AI, Dd, ... , (At, Dk ) allows considerable
flexibility in how memory is allocated to different uses during the course of a simulation's
execution.
Our exposition here is intended merely to make the reader aware of the fact that dy-
namic space management that incorporates block-array data structures generally executes
more efficiently with respect to computing time and less efficiently with respect to space
usage. Since computing time considerations usually dominate, one may well prefer a simu-
lation programming language or package that employs these concepts. Since vendors rarely
mention this topic in their promotional literature, it falls to the potential user of a language
or package to elicit this information.
The greatest perceived benefit of Algorithm SPACE-FINDER** arises from the obser-
vation that it reserves blocks of space and requires little time when releasing space, and that
the frequency with which it encounters the relatively high cost of resizing is inversely related
to the block size n.
There remains the issue of unused space blocks. Suppose that k blocks of size n x r have
been allocated. If the demand for space falls below k blocks during a simulation, virtually no
language attempts to unallocate the unused space, since this would incur the relatively costly
effort of relocating records spread throughout the k blocks into the retained k - 1 blocks.
 5 .4 . FU T U RE - EVENT SET 197

Accordingly, there is a tendency for allocated space to increase as simulated time evolves.
However, the elapsed simulated time between successive space increases tends to decrease,
thus slowing the rate of increase.

ALGORITHM SPACE_FINDER**

Yes No
2y. i *- D(1). 2n. Augment A by B(n) rows.
3y. If#VACANT > 1, 3n. Augment D by B(n) rows.
D(1) *- D(#VACANT). 4n. #VACANT *- B(n).
4y. #VACANT *- #VACANT - 1. Sn. DU) *- n + i, i = 1, ... , B(n).
Sy. Assign record to AU, 1), ... , A(i, r). 6n. n *- n + B(n).
6y Insert new record in ALIST. 7n. Assign record to AU, 1), ... , AU, r).
7y. Continue simulation. Sn. Execute Algorithm SPACE_FINDER.

5.4 FUTURE-EVENT SET

Every discrete-event simulation that incorporates random behavior consumes computing
time to:

• Search for space

• Manage the future-event set
• Perform other list processing
• Generate pseudorandom numbers
• Sample from probability distributions
• Perform arithmetic operations
• Perform statistical analysis

• Generate reports.

Of these, managing the future-event set accounts for the largest proportion of comput-
ing time. Although every simulation programming language provides special-purpose data
structures to accommodate the FES and operations on it, individual differences do exist
198 CHAPTER 5. SEARCH, SPACE, AND TIME

among languages, and as a consequence, the same simulation problem programmed in

different languages can induce substantially different computing times. Nevertheless, all
methods for managing the future-event sets share at least four capabilities in common:

a. Insert a new event notice in the FES.

b. Delete the next-event notice in the FES.

c. After a deletion, find the next-event notice among those remaining on the FES.

d. Find an event notice whose attributes assume particular values and delete it from the
FES.

The next-event notice denotes the one whose clock attribute value is smallest among all
event notices in the FES. Action a corresponds to scheduling an event, b to choosing the next
event to execute, c to identifying which event notice is to be executed next, and d to canceling
the scheduled execution of an event. Since the cancellation rate customarily is much smaller
than the insertion, deletion, and search rates of actions a, b, and c, respectively, cancellations
contribute little to the overall computing time of a simulation run. Accordingly, we say no
more about them and, unless otherwise stated, hereafter subsume action c into action b.
Ideally, we prefer to employ the simulation programming language that manages the
future-event set most efficiently with regard to space and time. Regrettably, empirical studies
have repeatedly shown that no method demonstrates uniformly superior performance with
regard to average computing time when tested on different types of simulation models
(McCormack and Sargent 1981, Chung et al. 1993). Two factors principally determine the
computing time required to perform actions a and b: the number of event notices in the
FES and the distribution of simulated time between scheduling and executing an event. In
particular, they influence the sample sequence of insertions and deletions observed during
the simulation, the computing time devoted to action a, and, possibly, the computing time
devoted to action b.
Let a and b denote executions ofactions a and b, respectively. For a particular simulation
with n events in the FES, recall from Section 5.1.1 that g. (n) and gb (n) denote the respective
average times that these actions consume. At a particular moment in simulated time, suppose
that the FES contains k event notices and experiences a sample sequence of 20 insertions
and 20 deletions in the order

abababababababababababababababababababab. (5.8)

Then the average computing time consumed is 20 [g. (k) + gb(k + 1) J. Alternatively, suppose
the 20 insertions and 20 deletions occur in the order

aaaaabbbbbaaaaabbbbbaaaaabbbbbaaaaabbbbb. (5.9)
 5.4. FUTURE-EvENT SET 199

Then average computing time is

4[ga(k)+ ga(k + 1) + ga(k + 2) + ga(k + 3) + ga(k + 4) (5.10)

+ gb(k + 5) + gb(k + 4) + gb(k + 3) + gb(k + 2) + gb(k + 0],

revealing a potential difference in average computing time, depending on the form that
the insertion/deletion sequence takes. In addition, two different types of simulation faced
with the same insertion/deletion sequence have different average times ga (k) and gb(k) as a
consequence of different distributions of elapsed time between scheduling and executing an
event.
The absence of a universally dominant procedure with regard to average performance
forces us to focus principally on a worst-case computing time measure of peformance that
depends on the number of event notices in the FES but is usually independent of distribu-
tional properties. This measure reveals how computing time grows in the worst case as the
size of the simulation grows. Its limitation is that it rarely informs us as to how a procedure
performs relative to others for small and moderate size simulations. Nevertheless, the worst-
case measure can be invaluable in explaining why a particular simulation software package
peforms well for a relatively small demonstration problem but consumes dramatically more
computing time when employed to simulate a moderate or large system.

5.4.1 SINGLE LINEAR EVENT LIST

As shown in expression (2.5), the record of an event notice has multiple fields for
storing identification, time of execution (clock) other attributes, predecessor address, and
successor address. Since the discussion of the next several sections focuses on the ordering
of execution times, all depicted future-event sets show only the values of the clock attributes
for event notices, and arrows point to their predecessors and successors.
Suppose that the FES is maintained as a doubly linked list as in Figure 5.4 with deletion
from the low end, and insertion from the high end, where the number in each circle denotes
the scheduled execution time for the corresponding event and where the simulation succes-
sively schedules four events with desired execution times 51, 42, 34, and 28 before it deletes
the event notices with the most iminent execution time. Note that insertions made in this
order take 4 + 7 + 12 + 15 = 38 comparisons. More generally, inserting a new event notice
in a doubly linked linear FES that already contains N event notices takes O(N) worst-case
computing time, but deletion times are constant, independent of N.
Recall from Section 5.3 that ga (N) denotes mean computing time expended per inser-
tion, whereas gb (N) denotes mean computing time per deletion. Clearly, their values depend
on the data structure adopted for the FES and how the scheduling procedure operates on it.
In the present setting ga(N) is a constant, independent of the number of entries N in the
FES. However, empirical studies have repeatedly shown that gb(N) varies with the form of
the interevent-time distribution and cannot be easily characterized as a function of FES size,
(e.g., McCormack and Sargent 1981 and Chung et al' 1993). Because of these limitations on
200 CHAPTER 5. SEARCH, SPACE, AND TIME

First insert
4 comparisons

4 Second insert
7 comparisons

34 Third insert
12 comparisons

Fourth insert
28 15 comparisons

Delete @ : no comparisons

Figure 5.4 Inserting and deleting event notices in a doubly linked linear futute-event
set (entries are clock attribute values)

average behavior our assessment of alternative methods of managing the FES concentrates
on worst -case analyses. In particular, we focus on how computing time is affected as the size
of a simulation grows.
Consider the open-loop queueing system in Figure 5.1 with r ~ 1 types of arrivals,
where interarrival times are random and where each arriving job has k tasks of which
task i must be performed by one of the mj servers of type i for i = 1, ... , k. Assume
that the relationship between arrival and service rates is such that the simulation finds all
servers busy almost all of the time. This high traffic intensity environment characterizes
many manufacturing and telecommunications systems. Moreover, it implies that service
completion at one station (deletion from the FMS) immediately induces the scheduling of
service for this server and a task waiting in its buffer (insertion in the FMS).
During its tenure in the system, each job induces the scheduling of 1 + k events, one
for the next arrival and k for completions at each of the k banks of servers. Suppose that the
FES is a single linear doubly linked list of event notices ordered by desired execution times.
Then, as previously stated, gb(N) is relatively constant, regardless of the number of event
notices N in the FES. Since virtually all servers are busy at any moment in simulated time,
the FES contains approximately 1 + ml + ... + mk event notices. It also contains r event
notices, one for each distinct arrival type. Therefore, the scheduling of an additional event
notice takes at most m 1 + ... + mk + r comparisons. Since this occurs k times per job during
the simulation, worst-case computing time per completion is proportional to k 2 iii + kr,
where expression (5.2) defines iii.
For fixed k and r, this reveals that worst-case computing time grows linearly in iii,
the mean number of servers per station; but, for fixed iii and r, worst-case computing time
 5.4. FUTURE-EvENT SET 201

grows as k 2 , where k denotes the number of stations in series. For this type of list, series
queueing problems clearly demand more computing time than parallel queueing problems
do, as k grows relative to iii 1/2.

EFFICIENCY

As the remainder of this chapter shows, other methods of managing the FES can be
expected to consume considerably less computing time than a single linear linked list as the
number of simultaneously scheduled events in the FES grows. However, we cannot totally
dismiss the value of this data structure. When there are few simultaneously scheduled events,
it is known that this form of event management is computationally more efficient on average
than all others that have been considered. For example, results in Jones (1986) indicate that
a linear linked list is best for fewer than 10 simultaneously scheduled events.

5.4.2 MULTIPLE LINEAR EVENT LISTS

One approach to reducing list-processing time relies on multiple linear lists in the FES,
one for each type or class of event. When scheduling an event of type i, one peforms:

SCHEDULING

1. Create an event notice.

2. Determine and assign desired execution time to its clock attribute.

3. Employ linear search to insert the event notice into list i.
4. If first in list, update the header address.

5. Iflast in list, update the trailer address.

When an event terminates execution, control passes to the timing routine, which
performs:

SELECTING THE NEXT EVENT

1. Search the list of header addresses to identify which event type has the smallest value
for its clock attribute.

2. Remove this event notice from its list.

3. Update the header address.

4. Advance simulated time to the value of this event's clock attribute.
5. Execute the event.
202 CHAPTER 5. SEARCH, SPACE, AND TIME

Event class 1 Event class 2 Event class 3

r
@ First inse~t:
~ 1 companson

t34' Third insert: 42 Second insert

4 comparison
V!Y 3 comparisons

28 Fourth insert:
8 comparisons

Delete @: 2 comparisons

Figure 5.5 Inserting and deleting event notices in a doubly linked multiple linear
FES (entries are clock attribute values)

These prototypes characterize SIMSCRIPT 1I.5's approach to event management.

Figure 5.5. displays the same entries as in the single list FES of Figure 5.4, but ordered
in sublists by event class. The simulation again successively schedules events with desired
execution times 51, 42, 34, and 28 before it executes the next event with execution time 2.
Insertion takes 1 + 4 + 3 + 8 = 16 comparisons in contrast to the 38 comparisons for the
single list in Figure 5.4. However, deletion takes two rather than one comparisons because of
the need to check which among the first entries in the three lists has the smallest execution
time.
For the open-loop system in Figure 5.1 with r ~ 1, we can conceive of 1 + k event
classes, list 1 for the r arrival event notices that must always exists and list i + 1 for each of the
at most, mi completions at station i for i = 1, ... ,k. Therefore, scheduling for list 1 takes
O(r) comparisons, whereas scheduling for list i + 1 takes O(mi) comparisons. However,
a deletion always takes 1 + k comparisons. Since each job passes through the k stations,
the number of scheduling comparisons per job is OCr + kiii) but the number of deletion
comparisons per job is O(k + k 2 ).
For this multiple list formulation, worst -case number ofcomparisons per job is O(kiii +
r) + O(k + k 2 ). For fixed k and r, this bound grows linearly in iii, revealing an increasing
benefit, due to a smaller coefficient, when compared to the single list approach. For fixed
iii and r, it grows as k 2 but its coefficient is now reduced from iii, in the single list FES
of Section 5.4.1, to 1. Although substantial differences in computing times can occur as
the number of stations in series k grows, the proportionality to k 2 in both the linear- and
multiple-list approaches can require substantial computing times as k increases.
 5.4. FUTURE-EvENT SET 203

Levell

Level 2

Level 3

Level 4

Level 5

Figure 5.6 Binary heap displaying key( v) (doubly linked)

5.4.3 HEAPS

The concept of a binary heap provides another data structure for managing the FES
(Williams 1964, Evans 1988). Consider a directed tree all of whose edges point toward its root
node 1. Figure 5.6 illustrates the concept, where for present purposes we again regard the
numbers at each node as the execution times for events. The children of node v at level i are
the nodes at level i + 1 connected to v by an edge, and v is the parent of each of its children.
A node with no children is called a leaf The depth of node v is the number of edges in the
path connecting v to the root 1. A heap is a rooted tree each of whose nodes v has a value
key( v) that is the smallest among the keys of all nodes in its subtree. A heap in which each
node has at most two children is called a binary heap.
The data structure associated with a binary heap is an array in which nodes are stored
in order from top to bottom; its root occupies position 1, its children occupy positions 2
and 3, and, more generally, the nodes at depth i occupy positions 2i , i + 1, ... , i+ 1 - 1.
If the binary heap is maintained so that its N elements always occupy the first fleft-justified
positions as in Figure 5.6, then every element of the tree h~s depth no greater than Llog2 N J.
The bidirectional edges in Figure 5.6 emphasize that the list maintains predecessor
and successor addresses at each node and is thus doubly linked. In conformity with our
definition of a binary heap and for visual clarity, Figure 5.7 displays unidirected edges.
In terms of the FES, a node represents an event notice and key( v) denotes the value
of the clock attribute of event notice v. To insert or delete an event notice, one relies on
procedures akin to Algorithms SIFTUP(x) and SIFTDOWN(x). The first takes an element
x of the tree whose key(x) has been decreased and moves it up the tree to its appropriate
place. The second takes an element x whose key has been increased and moves it down the
tree to its appropriate place.
204 CHAPTER 5. SEARCH, SPACE, AND TIME

Algorithm SIFTUP(x)

Until SUCCESS:
Repeat
If x is the root, SUCCESS
Otherwise:
Let y be the parent of x
Ifkey(y) :::: key(x), SUCCESS
Otherwise:
Swap entries in x and y.

Algorithm SIFTDOWN(x)

Until SUCCESS:
If x is a leaf, SUCCESS
Otherwise:
Let y be the child of x with minimal key
Ifkey(y) :::: key(x), SUCCESS
Otherwise:
Swap entries in x and y.

To insert event notice x into the FES, one:

• Inserts x as leaf with key(x)

• Executes Algorithm SIFTUP(x).

Before assigning elements x to position I, one can think ofkey(l) = 00.

An n x 1 block array, as in Figure 5.3, can serve as a basis for maintaining an FES
that employs the binary heap concept. If the FES holds 1 - 1 < n event notices prior to an
insertion, then D(l) points to the root or first event notice, and forl :::: i :::: (I - 2) /2, D(2i)
and D(2i + 1) point to children whose parent's address is stored in D(i). Then insertion via
Algorithm SIFTUP(x) makes x the lastleafin the heap by assigning the address ofx to D(l).
To delete the next event notice from the FES, one:

• Removes the root event notice from the heap.

• Moves the last leaf y to the root position.

• Performs SIFTDOWN(y).
 5.4. FUTURE-EvENT SET 205

Figure 5.7 shows a binary heap FES for the same event notices displayed in the single
linear list of Figure 5.4 and in the multiple linear list in Figure 5.5. The successive insertion
of event notices with times 51, 42, 34, and 28 takes 1 + 2 + 3 + 3 = 9 comparisons and
o+ 1 + 2 + 2 = 5 switches, in contrast to 38 comparisons and one link in the single linear
list case and 16 and one link in the multiple linear list case. The single link comparisons
operation in each of the linear list cases corresponds to the final step in the search, where
the new event notice is chained to the list. Link and switch operations are comparable in
computing time.
The subsequent deletion of the first event notice from the FES in Figure 5.7e takes
one leaf-to-root transfer, three switches, and three comparisons. A leaf-to-root transfer is
comparable in computing time to a switch. Although more work than for deletions in either
Figure 5.4 or 5.5, the combined work for insertion and deletions for the binary heap favors
this approach to list management over the single and multiple linear list approaches.
Since the particular clock times in Figures 5.7a through 5.7e dictate the numbers of
comparisons per insertion and deletion, it is of interest to consider more global results. For
four successive entries into a heap with 19,20,21, and 22 nodes, the worst-case numbers of
comparisons are Llog2 19J = 4, Llog 2 20J = 4, Llog 2 21J = 4, and Llog 2 22J = 4, respec-
tively. For deletion following these four insertions, the worst-case number of comparisons
is Llog 2 23 J = 4.
More generally, with D maintained as a binary heap with N entries, insertion and
deletion each take O(log2 N) computing time, an unequivocal advantage over single and
multiple linear lists as I ---,)- 00. Also, canceling a particular event notice in a heap follows
the same steps as a deletion and therefore has O(log2 N) time. However, there is at least
one limitation of this formulation that needs to be overcome. As just described, the binary
heap makes no provision for priority order when two or more events have the same desired
execution time. This can be overcome as follows. Let OLD denote the old event notice in the
heap and let NEW denote the new event notice. If OLD and NEW have the same execution
times but NEW has a priority order that favors it over OLD, swap the locations of OLD and
NEW in the heap. Otherwise, do not swap them.

SPACE

Suppose that a simulation attempts to schedule an event when its FES already holds
N event notices. Then the simulation programming language must provide for upwardly
resizing the arrays and reconstituting the heap in the augmented array. While choosing
N large reduces the frequency of resizing, it may also lead to considerably unused, but
committed, space over long periods of simulated time. An alternative approach provides
memory space incrementally, as needed.
A binary heap with N entries has Llog 2 NJ levels, where level i accommodates i-I
event notices for i = 1, ... , N - 1. Figure 5.6 illustrates the concept for N = 19, where
Llog 2 19J = 5 and levels 1 through 4 have I, 2, 4, 8 entries, respectively, and level 5 has
four entries and space to accommodate 12 additional entries. When the FES has exactly i-I
206 CHAPTER 5. SEARCH, SPACE, AND TIME

a. First insert @: I comparison and no switches b. Second insert @: 2 comparisons and I switch

~~
11/ 34 ,-\

'~9. I~~
c. Third insert
6

@: 3 comparisons and 2 switches d. Fourth insert @: 3 comparisons and 2 switches

e. First deletion (1): I leaf-to-root transfer. 3 comparisons. and 3 switches

Figure 5.7 Inserting and deleting event notices in a binary heap future-event set
(doubly linked; entries are clock attribute values)

entries in its i levels, and an event is to be scheduled, the dynamic space allocator finds a
block array oflength i+ 1 for level i + 1 and connects it pointerwise to level i, much as in
the example in Figure 5.6. The address of the leftmost entry at level i + 1 now becomes
the trailer for the list. Since the space at this level is a block array, the location of the next
unoccupied record at level i + 1 is implicitly determined.
 5.4. FUTURE-EvENT SET 207

In summary, each time it runs out of space this FES doubles its size. Moreover, it takes
O(i) computing time to link the new space at level i + 1 to the old space.
For the prototypical simulation described in Figure 5.2 with r 2: 1 arrival types, each
+
job passing through the system takes O(2k log2 (km r» list processing time to maintain an
FES structured as a binary heap, where the constant 2 merely emphasizes that deletion as well
as insertion has a worst -case bound proportional to k log2 (km + r). As the number of visited
stations increases, this computing time grows as k log2 k. Conversely, as the average number
of servers increases, the bound grows as log2 m. As k, m, and r increase, these properties
make clear the benefit that a binary heap offers for FES management when compared to the
O(k 2m + kr) worst-case timefor the single linearlist of Section 5.4.1 and the O(km + r) +
O(k(k + 1) worst-case time of the multiple linear list approach in Section 5.4.2.
It is known that as N, the number of scheduled events in an FES maintained as a
heap, increases, the average computing time required to maintain the heap is O(lOg2N)
(e.g., Jones 1986). However, it is also known that for small N a heap is considerably less
efficient than a linearly linked list for FES management. Presumably, the large setup time
encountered each time a heap is modified accounts for its relatively poor performance for
smallN.

5.4.4 HENRIKSEN'S ALGORITHM

An alternative approach to event management in Henriksen (1977, 1983) also reduces

the time complexity ofFES managment as compared with either single or multiple linear lists.
It formulates the FES as a single linear doubly linked list ordered from largest to smallest, but
augments the list with two pseudo-event notices, one with clock attribute set to any negative
value and the other with clock attribute set to 00. In essence, these become the permanent
trailer and header records, respectively, for the list.
The algorithm also maintains a binary search tree with a five-field record corresponding
to each node. The fields contain a pointer to an event in the FES, the time at which this event
is scheduled to be executed (clock), pointers to the event's predecessor and successor in the
FES, and a pointer to the record in the tree with the next lowest scheduled execution time. The
tree has M := 2K - 1 nodes for some integer K 2: 1. For conciseness, let D(l), ... ,D(M)
denote the pointers in the tree to M events in the FES with the property that clock(D(i»
::: clock(D(i + 1) for 1 ::: i ::: M - 1 with D(M) = 00. Here clock(D(i» denotes clock
attribute in the record corresponding to a node in the binary search tree and serves as the
key (see Section 5.4.3) for maintaining the tree.
Today, most computers use byte-addressable memory. For example, a computer whose
memory consists of 32-bit words allows a user to address each of the four bytes in a word in
bits 1-8,9-16,17-24, and 25-32. However, the piece of memory needed to store the address
of the byte must itself be a 32-bit word. When we use the term word hereafter, we usually
mean a piece of memory that can contain a memory address, e.g., a 32-bit word.
208 CHAPTER 5. SEARCH, SPACE, AND TIME

Finding and deleting the event notice corresponding to the next event for execution
takes constant computing time regardless of the number of entries in the FES. To insert a
new event notice with desired execution time T in the FES (scheduling), one:

• Performs a binary search on D to find the pointer:

j = min[i : clock(D(i)) ~ T, 1:::: i :::: MJ .

• Scans the FES backwards starting from DU) and inserts the new event notice in its
appropriate location.

During a scan, the procedure keeps a COUNT of the number of unsuccessful com-
parisons it has performed. If the COUNT reaches a predetermined number [ and j > 1,
the procedure alters DU - 1) to point to the last scanned event notice, resets the COUNT
to zero, and continues scanning the FES until success or until the COUNT again reaches [.
Alternatively, if j = 1 and COUNT reaches [, the procedure increases the size of the D array
to M' = 2K + 1 - 1 = 2M + 1, reinitializes its entries, and continues scanning the FES until
success.
The expanded tree is initialized by setting its leftmost leaf (Le., D(M')) to point to the
pseudo-event notice at the rightmost end of the FES and D(l), ... ,D(M' - 1), the pointers
at all other nodes of the tree, to point to the pseudo-event notice at the leftmost end of the
FES. By assigning the "unused" pointers D(l), ... , D(M' -1) to the header entry in the FES
at the moment of resizing, the procedure can later reassign them, one at a time, whenever
COUNT = [ and j > 1. Algorithm H-L describes the major steps for insertion.

ALGORITHM H-L

Purpose: To insert an event notice into the FES using Henriken's Algorithm.
Given: j, K, M, and{D(l), ... , D(M)}.
Method:
Start scan at DU).
COUNT+- O.
Repeat:
COUNT +-COUNT+1
scan the next event notice
ifCOUNT=I:
if j > 1, reset DU - 1) to point to the last scanned event notice
otherwise:
increase size ofD from M = 2K - 1 to M' = 2K+I - 1
set D(M') to point to the trailer event in the FES and D(l), ... ,
D(M' - 1) to point to the header event in the FES
j -+ M'
 5.4. FUTURE-EvENT SET 209

COUNT~ 0
Until success.

Whereas a binary heap has the property that the key for each node is no greater than
the key for any node below it in the tree, a binary search tree has the property that the key
for each node is no greater than that for any element in the original node's left subtree but
is no smaller than that for any element in the original nodes right subtree.
As I decreases the number of scans until success diminishes, as does the frequency of
resizing D. However, decreasing I increases the likelihood of increasing M, the size of D, and
thereby increasing the cost of reinitializing D. Henriksen (1977) suggests I = 4. Kingston
(1986a,b) describes procedures that reduce the cost of initalization and, in finding the next
event in the FES, for reducing the size of D.
Figure 5.8 displays the steps for insertion and deletion from an FES based on Hen-
riksen's Algorithm using the four successive insertions with clock attribute times 51, 42, 34,
and 28 followed by a deletion, as in Figures 5.4, 5.5, and 5.7. Each insertion induces at most
Llog 2 7J = 2 comparisons to find the pointer to the location at which to start the list search.
In addition to the comparisons, insertion requires two pointer changes in Figures 5.8a and
5.8c. Note that no comparison is made with the trailer clock entry of 00.
In an FES with N elements, Henriksen's Algorithm takes 0(1) computing time to find
the next event to execute and O(log(N / I)) + 0(NI/2) +0(1) worst-case time to insert anew
event notice (Kingston 1986a, 1986b). However, substantial empirical evidence indicates
that insertion takes mean computing time proportional to log2N for given I and that this
result is insensitive to distributional properties for a wide variety of simulations (Henriksen
1994 and McCormack and Sargent 1981).
In the context of simulating a queueing system as in Table 5.1 with r arrival types, these
results suggest O( (k + 1)(klii + r )1/2) worst-case computing time and an average computing
time proportional to average (k + 1)log2 (klii + r) for fixed t. As the average number of servers
per station, Iii, increases for fixed k, this computing time grows as Iii 1/2 at worst and as log21ii
on average. As the number of stations k increases for fixed Iii, the computing time grows
as k 3 / 2 at worst and as klog 2 k on average. When compared to the single or multiple linear
list approaches with standard scanning, the benefits of Henriksen's Algorithm are apparent.
Algorithm H -L is prototypical of the approaches to event management in SLX, GPSS/H,
AweSim, and SLAM.

5.4.5 BEST FES MANAGER

Table 5.1 summarizes the bounds on computing time for managing the FES by the four
data structures just described. However, at least one additional issue remains to be discussed,
namely, an FES with few event notices. As Section 5.4.1 intimates, a linearly linked list is most
efficient on average for these. Hence, a language that employs a binary heap or Henriksen's
Algorithm may consume more time than necessary to manage an FES with few entries on
average. However, we anticipate that any excess computing time will be small in absolute
210 C HAP TE R 5. SEA RCH, SPA eE, AND TIM E

(a) First insertion

At most 2 comparisons
to find pointer,
1 pointer shift,
4 comparisons to insert

At most 2 comparisons to
find pointer, 1 comparison
to insert

At most 2 comparisons
to find pointer, 1
comparison to insert

At most 2 comparisons
to find pointer, 4
comparisons to insert,
1 pointer shift
8
(e) First deletion

o comparisons to
delete 0

Figure 5.8 Inserting and deleting event notices in an indexed future-event set: Hen-
riksen's Algorithm (entries in list are clock attribute values; entries in index array are
ordered pointers; dashed arrows denote pointers before shift during search)

terms. Moreover, we can regard this excess as a relatively small penalty when compared to the
efficiencies that heap-like structures and Henriksen's Algorithm offer for large future-event
sets. The circumstances that are yet to be investigated fully are those in which an FES has
few events notices most of the time and occasionally, but rarely, grows substantially in size.
Do these rare but substantial excursions favor simple or sophisticated structures?
Since studies of more sophisticated heap structures for FES management fail to reveal
unequivocal superiority for anyone ofthem (e.g., Jones 1986 and Chung et al. 1993), it is
not possible to choose a simulation programming language based on its universally accepted
computing time superiority. Nevertheless, it is clear that for a given problem, GPSS/H and
SLAM, based on Henriksen's Algorithm, and SIMAN, based on the binary heap concept,
execute faster than SIMSCRIPT 11.5, which relies on multiple doubly linked linear lists.
 5 .5 . BROADENING THE CLASS OF DELAY MODELS 211

Table 5.1 Managing the future-event set for a saturated system (k := number of stations in
series, rn := average number of servers per station, r := number of distinct arrival types)

Data structure Insertion time/job Deletion time/job

Linear linked list OWrn + kern + 1) + r) 0(1)

Multiple linear O(krn + r) 0(k 2 + k)

linked list

Binary heap O((k + 1)log2(krn + r)) O((k + 1)log2(krn + r))

Henriksen's Algorithm O((k + 1)(krn + r)I/2) 0(1)
Indexed and linear lists
ex (k + 1)log2(krn + r)
on average

5.5 BROADENING THE CLASS OF DELAY MODELS

The prototypical delay model in Section 5.1 identifies the number of station, k, the average
number of servers per station, fit, and in the capacitated case, the average buffer size, c, as
the major influences on interaction time per job with the FES. The model assumes that
upon completing service at station j, a job moves instantaneously to station j + 1 for
j = 1, ... ,k - 1. We next investigate the FES interaction time for a more inclusive class of
simulation models.

CAPACITATED TRAVEL

Suppose there are 1j pallets or conveyances at station j to move work to station j + I

and that each job requires a conveyance to move from station j to station j + 1. After a
pallet, cart, conveyance, etc., arrives at station j + I, it must return to station j to pick up
the next job. Let

_ I k-l
1:=- Llj.
. k - 1 j=l

Since there are at most (k - I)J event notices associated with the arrival of a job at a
station or the arrival of the conveyance back at its home station, the FES contains at most
r + kfit + (k - I)J event notices.
If each pallet has unit capacity, then each job is responsible for k - I interactions with
the FES for pallet arrivals and for an additional k - I for their return to their originating
stations. Therefore, the total number ofFES interactions is k + I + 2(k - I) = 3k - I, so
that, for example, heap insertion and deletion times per job each are O«3k - I) 10g2(r +
kfit + (k - I)J)).
212 C H A PTE R 5. SEA RCH, S PAC E I AN D TIM E

If pallet capacity at each station is e > 1 and a pallet moves from station i to station
i + 1 only when full, then we conceive of each job as inducing (k - I) Ie interactions for
pallet arrivals and (k - I)le for returns, so that, for example, heap insertion and deletion
times per job each are 0(((1 + 2le)k - 21e) log2(r + kin + (k - 1)/)).

UNCAPACITATED TRAVEL

If transit requires no pallet, cart, etc., the number of jobs in transit from station to
station is unbounded, and therefore our analysis uses a different approach.

5.5.1 VARIABLE TRANSIT TIMES

Assume that upon completing service at station j, a job travels to station j + 1 with
nonnegative transit time for j = 1, ... , k - 1. Two scenarios are possible, capacitated travel
and uncapacitated travel. Let L j (t") denote the number of jobs in transit at time t" from
station j to station j + 1. Then the arrival of a job at its destination constitutes an event and,
hence, must be scheduled. Accordingly, at simulated time t" the FES contains r + B (t" ) + L ( t" )
event notices, where, as before, r denotes the number of scheduled arrival event notices and

Scheduling an event now takes ga(r + B(t") + L(t")) average computing time, whereas selec-
tion takes gb (r + B (t") + L (t" )) average computing time, where, as before, each simulation
language's procedure for scheduling and selection determines the forms of ga(·) and gb(·).
Unlike the model of Section 5.1, this broader model puts no upper bound on the
number of event notices in the FES. Let l/Wtj denote mean transit time for a job from
station j to j + 1 for j = 1, ... , k - 1 and let

Wa
Vj:=-.
Wtj

For an arbitrarily selected time t", the mean number of jobs in transit in an uncapacitated
buffer system (ej = 00, j = 1, ... , r) is

E(t") = (k - I)j), (5.11)

where

1 k-l
- ~ I
v:= k-l ~ Vj.
J=l

Note that long transit times relative to interarrival times at state 1 imply a large mean number
of jobs in transit.
 5 .5 . BROAD E N I N G THE C LAS 5 0 F D E LAY MOD E L 5 213

Since jobs in transit from one station to the next are in an infinite-server system, we
can think of this extended model as a sequence of2k - 1 stations where the odd-numbered
stations each have finite numbers ofservers and the even-numbered ones each have an infinite
number of servers. At an arbitrarily chosen simulated time r, the FES for the extended model
has mean number of event notices

E[r + B(r) + L(r)] :::: r - i + k(m + v).

Moreover, each job passing through the system has 2k scheduling interactions and 2k
selection interactions with the FES.
For the single linear list approach to FES management in Section 5.4.1,

average scheduling time per job = O(2k(r - jj + k(m + v)))

= O(2kz(m + v) + 2k(r - v)),

whereas
average selection time per job = O(2k).
Since no competitive simulation language uses this data structure, these results are included
merely for purposes of comparison with the more commonly employed data structures.
For the multiple linear list FES of Section 5.4.2, we treat all transit arrivals at station
j as forming a separate class of events for j = 2, ... ,k. Therefore, the extended model has
2k event classes, one for arrivals, k for service completions, and k - 1 for station-to-station
transit completions. Then

average scheduling time per job = O(k(m + jj) + r - v),

whereas

average selection time per job = O(4k z),

since the first entries in all2k event lists must be compared for each job's 2k selections.
For an FES based on the binary heap concept of Section 5.4.3 and given the number
of busy servers B (r) and the number of jobs in transit L ( r ), a single scheduling takes
o (logz (r + B( r) + L( r)) average computing time, as does a single selection. The inequality
x ~ 0,

enables us to write

L(r) 1
log2(r + B(r) + L(r)) :::: logz(r + km) + - - - . -1- . (5.12)
r + km oge2
214 C H A PTE R 5. 5 EAR CH, 5 PAC E I AND TIM E

Therefore,

average scheduling time per job

= 0 ( 2klogz(r +km) + 2k(k -k I)ii 1)

. -1-
r+ m oge2

=0 ( 2klo (r+km)+
2kii
.---
2ii 1
.-- ,
1)
gz r / k + m loge 2 r / k + m loge 2

and

average selection time per job

= 0 ( 2k 10 (r + km) + 2kii
.-- - 1 2ii 1)
.-- .
gz r/ k + m loge 2 r/ k + m loge 2

For fixed r, m, and ii, these bounds grow as k logz k. As ii increases relative to r / k + m for
fixed r, k, and m, the bounds become increasingly larger than need be. Nevertheless, with
regard to average scheduling and selection times per job, the binary heap approach maintains
its edge over single and multiple linear list approaches.
Recall that empirical studies (Henriksen 1994) show that scheduling an event using
Henriksen's Algorithm (Section 5.4.4) when N event notices are in the FES takes average
computing time proportional to logz N. Also, selecting an event takes 0(1) time. Based on
these properties, applying Henriksen's Algorithm for managing the FES for our (2k - I)
station model has

average scheduling time per job

=0 ( 2klo (km+r)+
2kii
.---
ii 1
.-- ,
1)
gz r / k + m loge 2 r / k + m loge 2

whereas

average selection time per job = 0(2k).

These results continue to imply an advantage for binary heaps and Henriksen's Algorithm
for scheduling and selection times.

5.5.2 VARYING NUMBER OF TASKS PER JOB

An alternative extension ofthe model in Section 5.1 assumes that the number ofstations
that a job visits varies during its passage through the system. At least three modalities exist
for this behavior:

1. Jobs make one pass through the system but skip some stations.
 5 .5 . BROAD E N I N G THE C LAS 5 0 F D E LAY MOD E L 5 215

2. Jobs may visit some stations more than once.

3. For some or all stations j, failure of quality-assurance testing following completion at

the station directs the job back to the station for rework.

Since Modality 1 induces k or fewer interactions per job with the FES, the results of Sections
5.4.1 through 5.4.4 continue to apply.
For Modality 2, let

Wij/ := probability that a job of type i visits

station j a total of I times, 1= 0,1,2, ....

If for a class of problems that incorporate Modality 2

k 00

LL1Wij/ = O(k)
j=l 1=0

as the number of stations grows, then the earlier results again continue to apply. In particular,
the assumption holds if Wijl = 0 for I greater than a specified integer.
Modality 3 requires more thought. Assume that quality-assurance testing follows each
station's service. Thus the extended model has 2k stations where mj is the total number of
testers after station j, each with mean testing time 1jw;j' such that wajmjw;j < 1. As a
consequence, the FES has at most k(iii + iii') + r event notices, where

_,
m := k1 (m ,1 + ... + mk') .

For convenience of exposition, assume that every job that completes service at station
j has probability p of failing the subsequent quality-assurance test, independent of type
and the number of times it has previously failed the test. Then the mean numbers of passes
through station j and through its ensuing test are each 1j (1 - p). Therefore, the mean
number of interactions per job with the FES is 1 + 2kj(l - p).
For the multiple list approach to FES management in Section 5.4.2,

average scheduling time per job = OCr + (iii + iii')j(l- p)),

whereas

average selection time per job = 0((2k + 1)2).

For an FES based on the binary heap structure or on Henriksen's Algorithm,

average scheduling time per job = 0((2kj(l - p) + 1) log2(k(iii + iii') + r)).

216 C HAP TE R 5. SEA RCH, SPA eE, AND TIM E

Average selection time per job for binary heaps has the same time complexity, but Henriksen's
Algorithm has O(2k + 1) average selection per job.

5.6 LESSONS LEARNED

• Space management and list processing consume a relatively large proportion of the
time expended in executing a discrete-event simulation program.

• Every simulation language incorporates a means for automatically managing space.

Some rely on dynamic space allocation on demand. Others use reserved block-array
structures that need to be resized and initialized as the demand for space increases.

• Every simulation language provides a timing routine for maintaining its future-event
set (FES), the list of records of scheduled events to be executed in the future.
• In a highly congested system, space needs for the FES increase linearly with the number
of stations in series and linearly with the average number of series per station.

• GPSS/H and SLAM use Henriksen's Algorithim (Section 5.4.4) to maintain their FESs.
SIMAN uses an algorithim based on a heap concept (Section 5.4.3). SIMSCRIPT U.S
uses multiple linear lists (Section 5.4.2).
• When jobs of different types have i.i.d. interarrival times, an opportunity may exist
for reducing the space the FES requires. This is especially true if the inter arrival times
have the exponential distribution.
• A job passing through all k stations in series in a simulation has interactions with the
FES that take O(k log k) time for a heap-like structure and for Henriksen's Algorithm.

5.7 EXERCISES

TERMINAL PAD CONTROLLER MODEL (EXERCISE 2.5)

rfi? 5.1 If one generates a character interarrival time separately for each of the r input
buffers, the FES necessarily contains at least r event notices.

a. Using the exponential property, devise an alternative interarrival scheme that requires
a single event notice in the FES to discount for these events. Justify your proposal.
b. For the two methods, what is the difference in the numbers of event notices?

rfi? 5.2 If w denotes the mean number of characters per packet, then direct sampling from
EO/A) to generate interarrival times implies that an average of w event notices are filed and
 5.8. REFERENCES 217

removed from the FES to fill the packet and move it from its input buffer to the output buffer.
The same average holds for the method ofExcercise 5.1. Devise a sampling procedure that
reduces the number of insertions and deletions from the FES to one each per packet while
preserving the i.i.d. exponential property of the interarrival times and that allows sampling
in time independent of w, as w becomes large. Hint: See Sections 8.13 and 8.19.

5.8 REFERENCES

Arguelles, M.e., and G .S. Fishman (1997). Reducing the frequency of future event set search
in manufacturing simulation, Operations Research Department, University of North
Carolina at Chapel Hill, in preparation.
Chung, K., J. Sang, and V. Rego (1993). A performance comparison of event calendar
algorithms: an empirical approach, Software-Practice and Experience, 23, 1107-1138.
Evans, J.B. (1988). Structure ofDiscrete Event Simulation, Ellis Horwood Limited, Chichester,
England.
Henriksen, J.O. (1977). An improved events list algorithm, Proceedings of the 1977 Winter
Simulation Conference, H.J. Highland, R.G. Sargent, and J.w. Schmidt, editors, 547-
557.
Henriksen, J.O. (1983). Event list management, a tutorial, Proc. Winter Simulation Conference,
IEEE and SCS, Piscataway, N.J., 543-551.
Henriksen, J.O. (1994). Personal communication.
Jones, D.W. (1986). An empirical comparison of priority-queue and event-set implementa-
tions, Comm. ACM, 29, 300-310.
Kingston, J.H. (1986a). Analysis of Henriksen's algorithm for the simulation event set, SIAM
~ Comput, 15,887-902.
Kingston, J.H. (1986b). The amortized complexity of Henriksen's algorithm, Bit, 26,156-
163.
Knuth, D. (1973). The Art ofComputer Programming: Sorting and Searching, Addison-Wesley,
Reading, MA.
Kollerstrom, J. (1974). Heavy traffic theory for queues with several servers. I, J. Appl. Prob.,
11, 544-552.
McCormack, W.M. and R.G. Sargent (1981). Analysis of future event set algorithms for
discrete event simulation, Comm. ACM, 24, 801-812.
Williams, J.w.J. (1964)."Algorithm 232:' Comm. ACM, 7, 347-348.
 CHAPTER

G. S. Fishman, Discrete-Event Simulation

© Springer Science+Business Media New York 2001
Output Analysis

Every discrete-event simulation experiment with random input generates random sample
paths as output. Each path usually consists of a sequence of dependent observations that
serve as the raw material for estimating the true values of one or more long-run performance
measures, hereafter called parameters. Two phenomena, sampling error and systematic error,
influence how well an estimate approximates the true value of a parameter. Random input
induces sampling error, and the dependence among observations often enhances its severity.
Systematic error arises from the dependence of the observations on the initially chosen state
or conditions in the simulation. The presence of random and systematic errors implies that
all estimates need to be interpreted with qualification.
This chapter addresses both types of error. It describes how to assess systematic error
in point estimates and how to reduce its influence. It also shows how to gauge the extent
of sampling error in the point estimates, principally, by constructing asymptotically valid
confidence intervals for the corresponding true values of the parameters. A confidence interval
is an interva~ rather than a point, estimate of a performance measure. An asymptotically valid
100 x (1 - 8) percent confidence interval is one that includes the true parameter value with
probability 1 - 8 for 0 < 8 < 1 as the sample-path length t -+ 00.
220 CHAPTER 6. OUTPUT ANALYSIS

While the width of a confidence interval indicates how well the corresponding point
estimate approximates the true value, this interpretation has a justifiable statistical basis only
ifsystematic error is negligible and the confidence interval is computed in a manner consistent
with achieving the property of asymptotic validity. Accomplishing these objectives calls for
considerably more effort than merely assessing error when the data consist of independent
and identically distributed (i.i.d.) observations. As a consequence, simulation results are
often reported with little if any supporting analysis, thus undermining conclusions based on
the values of point estimates. This need not be so.
To make it easier for a simulationist to assess systematic and sampling error, this
chapter focuses on easily implementable procedures, but grounded in statistical theory. This
is especially so for confidence intervals, for which we describe readily accessible software,
LABATCH.2, that performs the computation with minimal user effort. We also describe the
automatic procedure used within Arena.

6.1 WARM-UP INTERVAL

Three factors influence the extent to which initial conditions affect how well an average
based on sample-path data of finite length approximates its corresponding long-term system
average. They are s, the state assigned to the system at the beginning of a simulation; k, the
point on the sample path at which data collection for estimation begins; and t, the length of
the sample path, used to compute the finite sample average.
In principle, selecting an s that induces convergence of the sample path to steady-state
behavior for as small a k as possible is desirable. Since little is known a priori about the
underlying stochastic structure of the sample path, this is rarely possible. Customarily, s is
chosen for computational convenience. For example, beginning a simulation with an arrival
to an empty and idle system relieves the simulation user of the need to compute initial queue
levels, times to the completion of service, and scheduled execution times, as described in §4.
Suppose a simulation generates a sample path Xl. ... , Xt' of length t'. Since the
influence ofinitial conditions on X j tends to decrease in importance as j increases, discarding
data XI, ... , Xk-I for some k :::: t' at the beginning ofthe path and using the remaining
t := t' - k + 1 observations Xt, ... , XI' for estimation usually reduces systematic error. In
principle, the more one discards of the sample path, beginning at its origin, the smaller is
the systematic error in estimates based on the remaining data. This observation encourages
substantial truncation; but doing so can be unnecessarily costly unless tempered by the need
to reduce sampling error as well. In particular, since the width of a confidence interval tends
to decrease as t- I / 2 , one is encouraged to use a substantial portion of the sample path to
reduce sampling error.
This last action runs counter to the tendency for excessive truncation. For a fixed total
path length, this discussion makes clear the need to choose a warm-up interval k - 1 in a
 6 ,1 , WAR M - U PiN TE RV A L 221

way that makes systematic error negligible while providing a valid basis for computing a,
hopefully, narrow confidence interval.
In a discrete-event simulation, the sequence {XI, ' , . , X t ,} may represent one of many
phenomena. For example, for i = 1, ... , f', Xi maybe equal to:

• Wi := the waiting time of the i th customer

• Q«i -1)Ll, iLl) = Ll- I ft~~l)D. Q(u)du

• = time-integrated queue-length average over ((i - I)Ll, iLll

• Q Ci Ll) := queue length at time iLl, for some Ll > 0

• B«i - 1)Ll, iLl) = Ll- I ft~~I)D. B(u )du

• = time-integrated average resource utilization over ((i - I)Ll, i Lll

• BCi Ll) := number of busy servers attime iLl.

Alternatively, Xi may denote a function of one or more of these random quantities. For
example, {Xi := I(w,oo)(Wi ); i = 1" .. t'} denotes a 0-1 sequence, where Xi = 0 if
Wi :::: w and Xi = 1 if Wi > W; or

{ Xi = f:
q=1
I(q-l,q](Q((i -1)Ll»I(q-l,q](QCiLl))); i = 1, ... , t'}
is a 0-1 sequence, where Xi = 0 if queue lengths differ at times iLl and Ci + 1) Ll, whereas
Xi = 1 if they do not.
To illustrate the problem of initial conditions, we focus on a sample path of waiting
times generated in the SIMSCRIPT 11.5 airline reservation example in Chapter 4, where Xi
denotes the waiting time of the i th customer to begin service. Figure 6.1 a displays the first
100 waiting times (in black) for a single sample path. Since the simulation began with all
m = 4 reservationists idle and no callers waiting, the first four waiting times are necessarily
zero. To include these data in the estimating procedure would induce a downward systematic
error or bias in the sample waiting time time average as an approximation to the long-run
waiting time time average. Figure 6.1a also displays (in bold black) the sequence of truncated
time averages

- t'-J'+I :=
XJ'I 1 L Xi,
t'
1 :::: j :::: f', (6.1)
, f' - " + 1 '=J
, ,

where XjV - HI omits the firstj - 1 waiting times. This graph shows a tendency for
Xj I ,t'-H I initially to increase as j
increases, reflecting the reduction in systematic error that
truncation induces. However, as j continues to increase, XjV - HI fails to stabilize in the
222 CHAPTER 6. OUTPUT ANALYSIS

a. n=1, 1' =100

4 ·

Jth Custom.r to begin HrYIce

4 -

3 -

W.l1Ing .
Hmo .
2 ·

1 •

20 40 60 60 100

ith Customer to begin MrVlce

C. n=1000, t'=100

4 ·

3-

Waiting
Hmo
2·

1·

0 20 100
40 60 80
tth Customer to begin ..me.

Figure 6.1 Waiting times for airline simulation (black Xjni> bold black
Xjn,t'-H]')
 6.1. WAR M - U PiN TE RV A L 223

neighborhood of any particular value. This behavior for large j is a consequence of the fact
that random error in the time average Xj1 ,t'-j+l has variance proportional to l/(t' - j + 1)
and, therefore, exhibits increasing variation as j approaches t'.
As Figure 6.1a shows, the presence of sampling error limits one's ability to detect
systematic error from a single sample path. To reduce this sampling error, we can resort to
multiple independent replications. Suppose that we generate a second run of the simulation
with the same initial conditions but employ a pseudorandom number sequence disjoint
from that used on the first sample path. This strategy allows us to regard the sample paths
as statistically independent. Using the last pseudorandom numbers listed in Figure 4.3 as
the seeds for the new sample path accomplishes this. Let us call each such run a replication
and observe that one can generate n independent replications each oflength t' by employing
the last pseudorandom numbers on replication i as the seeds for replication i + 1 for
i = 1, ... , n - l.
To help us understand the value of multiple replications, let

S := set of all possible initial conditions for starting the simulation,

XY) := jth observation on replication i,

and

S( r) := state of the system at time r.

For the moment, we take xji) to be the j th waiting time on the i th of n replications each of
length t', which are the rows in Table 6.1. Suppose that each replication begins in the same
initial state s E S. Then

- 1 ~ (i)
X jn1 := - L-t Xj , 1 :::: j :::: t', (6.2)
n ;=1

define the column averages;

- (i)
X.,.
J,t - J+l
= 1
t' _ k + 1
L. X
t'
(i)
I'
1 :::: i :::: n, (6.3)
I=J

define the row averages based on observations j, ... , t'; and

-
X· , . '= -1
J,n,t -J+l .
L X.,.
n
-(i)
J,t -J+l
= ,
1
. +1
L X-
f

Inl (6.4)
n ;=1 t - ] I=j

defines the grand truncated average, which omits observations 1, ... , j - 1 on each replica-
tion. As noted in Chapter 3, the distribution of Xjil converges to an equilibrium distribution
224 CHAPTER 6. OUTPUT ANALYSIS

Table 6.1 n replications oflength t '

Replication Step j

2 t'

X(I) (I) (I) - (I)

I , X 2 , ••• ,X" Xj."_i+1

X(2) (2) (2) - (2)

2 I , X 2 , ••• ,X" Xj."-i+1

X(n) X~n) , ... , X;~) -en)

n I ' Xj."_i+1

X ln !> K2nl , ... , K"nl Kjn."-i+1

as j -+ 00, say, with mean J.L and variance (12 independent of sand j. However for finite
j ~ I:

J.LSJ· := E(X~l) I S(O) = s) = E(X~2) I S(O) = s) = ... = E(X<.n) I S(O) = s) (6.5)

J J J

and

SJ := var(X~l)
(12. J
I S(O) = s) = var(X~2)
J
I S(O) = s) = ... = var(X<.n)
J
I S(O) = s), (6.6)

where J.Lsj (# J.L) and (1~ (# (12) denote the conditional mean and variance, respectively, of
waiting time j, given the initial state S (0) = s. Moreover, each column average in Table 6.1
has conditional mean

E(Xjn1 I S(O) = s) = J.Lsj'

because of identical initial states, and conditional variance

2
- (1sj
var(X jn1 I S(O) = s) = - ,
n

because of identical initial states and n independent replications.

Since J.Lsj -+ J.L as j -+ 00 for aIls E S, the function {1J.Lsj - J.LI, j ~ I} characterizes
systematic error due to initial conditions, and a graph of {X jn1 , 1 ::::: j ::::: t'} provides a
way of assessing the extent of this error. Since E[XjndS(O) = s 1is unaffected by n but
the conditional standard error Jvar(Xjn1IS(0) =
s) decreases with increasing n, multiple
replications enhance our ability to detect the presence of systematic error. Figure 6.1 b based
on n = 10 replications and t' = 100 illustrates the improvement over n = 1. It indicates
 6.1. WARM-UP INTERVAL 225

the presence of systematic error for all j :::: 20. Figure 6.Ic based on n = 1000 reveals
even more. It shows that systematic error decreases monotonically as j increases from 1 to
100, suggesting that a not inconsequential systematic error may exist for initial path length
t'> 100.
Figure 6.2 presents comparable information for the longer initial path length t ' =
1000. For n = 1, Figure 6.2a offers little benefit over Figure 6.Ia. However, for n = 10,
Figure 6.2b suggests that systematic error becomes relatively unimportant in Xjn1 for j ::::
200. Figure 6.2c confirms this for n = 1000. In practice, a warm-up analysis rarely employs
more than 10 replications. Our use of n = 1000 is merely for illustration.
Figures 6.1 and 6.2 also reveal that the ability to identify a warm-up interval k - 1
such that systematic error is relatively incidental for j :::: k depends crucially on taking a
sufficiently long initial sample path length t'.
How does a simulation user choose suitable nand t ' prior to simulation for a warm-up
analysis? Occasionally, characteristics of the problem indicate the extent of variation across
replications for a given j and the extent of dependence along a sample path. Since this is
rarely so, an iterative solution often is the only alternative. One starts with an initial (n, t ' )
2-tuple and iteratively increases nand t ' until the succession of graphs, as in Figures 6.1
and 6.2, suggests a warm-up interval k - 1 such that for j :::: k, Xjn1 appears relatively
uninfluenced by initial conditions.
We now summarize several important observations regarding k, n, s, and t':

• As the warm -up interval k increases for a fixed t ' , systematic error decreases. However,
random error increases, especially as k approaches t'.

• As t ' increases for a fixed k, systematic and random errors decrease.

• Averages based on n independent replications {X jnl, 1 :::: j :::: t'} provide a basis for
discerning the extent of systematic error when all replications start in the same state.

• Random fluctuations in these averages decrease with increasing number of replications

n.

• Systematic errors in these averages remain unaffected by increasing n.

• Starting the simulation in a state S E S for which the conditional mean {/lsj, j :::: I}
is monotone in j enhances our ability to distinguish systematic error from sampling
error in analyzing {X jn1 , 1 :::: j :::: t'}. Although there is no guarantee that a state
exists with this property, it is known that the empty and idle state induces the property
for the conditional means of queue length and waiting time in many single-server
queueing problems.
226 CHAPTER 6. OUTPUT ANALYSIS

•. n=l , 1'=1000
30

20'

W.~ l ng 15 .
time

10 ·

5 ·

200 400 600 600 1000

j1h Customer '0 ~In HMce

b. n=10, 1'=1000
10

200 400 600 800 1000

jth Customer to begin "Mea

C. n=looo, 1'=1000
10

8 -

6 ·
Wolting
tim.
4 -

2 -

0
200 400 600 800 1000

1111 Cuot..".,.,. beg in ..rvI. .

Figure 6.2 Waiting times for airline simulation (black Xjnl> bold black
Xjn ,t' -j+l)
 6.1. WAR M - U PiN TE RV A L 227

6.1.1 MULTIPLE INITIAL STATES

Starting all n replications in the same initial state S E S enables us to draw inferences
about an appropriate warm-up interval k - 1 from a sequence of estimates {Xjnd of the
conditional mean function {JLsj}. This approach can be especially revealing when {JLsj} is
monotone in j, particularly if a priori information enables a simulation user to select a state
S that induces this monotonicity and that favors rapid convergence, relative to that from

other possible initial states, to equilibrium.

When the monotone property is absent, the analyst faces the additional burden of
distinguishing between fluctuations in {X jnh j ~ I} attributable to local minima and
maxima in {JLsj} and fluctuations due to sampling error. In particular, some practitioners
are reluctant to initialize each replication to a single state S for fear of perpetuating a large
systematic error in each of them. Since X jnl has mean-square error

2
-
MSE(X jn1 ; S, .•• , S) = -ansj + (JLsj - 2
JL) ,

an uninformed choice of S can, for fixed j, result in a large contribution (JLsj - JL)2 to the
mean-square error that remains regardless of how large n becomes. To limit this possibility,
these practitioners resort to an alternative approach.
Suppose the i th independent replication begins in state Si E S for i = 1, ... , n, where
Sl, ••• ,Sn are distinct and presumably span the state space S. For example, replication i of
the airline reservation simulation might begin with an arrival to a system with qi callers
waiting for or in service, where qi < qi+l for i = 1, ... ,n - 1 and where Xjil denotes the
waiting time of the (qi + j)th caller on replication i for j ~ 1. Then Xjn1 has expectation
nl "L..d=l
,n . l",n 2 d
JLsij, vanance 1i2 L...i=l a sij ' an mean-square error

Although the contribution ~ L:7=1 (JLsij - JL)2 is again independent of n for fixed j,
it is the average of n squared deviations, presumably some small and some large, which
hopefully make a smaller contribution than (JLsj - JL)2 when this latter quantity is large.
While this rationale has an inituitive appeal, this alternative has an advantage for
detecting systematic error only if MSE(X jn1 ; Sl, ..• , sn) < MSE(X jn1 ; s, ... , s) and the
graphical depiction to which it leads gives a more definitive picture of conditional mean
behavior as a function of j. In practice, the latter property is less likely to materialize. If S
induces a monotone {JLsj} but at least one of the Si'S does not, then the analyst faces the
additional problem of distinguishing between changes in direction in {Xjnd attributable to
nonmonotonicity and changes due to sampling fluctuations.
There are systems for which a single initial state for all replications may limit the
analyst's capacity to perform a comprehensive warm-up analysis: Suppose that the state
228 CHAPTER 6. OUTPUT ANALYSIS

space S is the union of two disjoint subsets SI and S2 such that a simulation that begins in a
state s in Si tends to remain in Si for an extraordinarily long simulated time before moving
to a state in Si(mod 2) + 1 for i = 1,2. For example, a system with equipment whose times
between failures and repair times are large relative to their processing times might have states
with all equipment operating in SI and states with at least one inoperative machine in S2.
Unless t ' is sufficiently long, starting all n replications with a state in SI is likely to induce a
warm-up analysis that fails to account for transition to S2.

6.1.2 HOW MANY WARM-UP ANALYSES?

Most discrete-event simulation runs generate sample paths on three generic types of
stochastic processes, waiting time {Wi, i ::: I}, queue length {Q(r), r ::: O}, and resource
utilization {B( r), r ::: O}. If a warm-up analysis based on the waiting-time data concludes
that deleting WI, ... , Wk-I from the data makes Wk, Wk+ I, ... relatively free of the influence
of initial conditions, the analyst customarily deletes {Q (r), 0 :s r :s rk-I} and {B (r), 0 :s
r :s rk-d from their respective sample paths and treats all Q(r) and B(r) for r > rk-I as
also independent of these initial conditions, where Section 3.4 defines r j.
Alternatively, if a warm-up analysis based on the queue-length data concludes that
deleting {Q (r), 0 :s r :s r*} from the data makes Q (r) for r > r* relatively free of the
influence of initial conditions, then the analyst customarily deletes {B ( r), 0 :s r :s r*} and
{Wi; i = 1, ... , N(O, r*)} from their respective sample paths and treats B(r) for r > r*
and Wi for i > N(O, r*) as also independent of initial conditions.
Justification for these procedures comes from the implicit assumption that from an
arbitrarily selected initial state the resource utilization and waiting-time (queue-length)
processes exhibit convergence rates to equilibrium no smaller than that for the queue-length
(waiting-time) process. If an equilibrium exceedance probability pr[ Q(.) > q 1as well as a
mean queue length I-LQ are to be estimated, it also implicitly assumes that starting from an
arbitrarily selected state, I(q.oo) (Q( r)) converges at a rate atleast as great as that for Q( r) as
r ~ 00. Similar assumptions hold when exceedance probabilities for resource utilization
and waiting time are to be estimated. Theoretical results based on spectral structure for
single-server queues support these assumptions.

6.1.3 STEPS FOR ESTIMATIt04G THE LEt04GTH OF THE WARM-UP It04TERVAL

We now summarize the steps to estimate the length of a suitable warm-up interval:

1. Choose the preliminary sample path length t'.

2. Choose the number of independent replications n.
3. Choose a set of initial pseudorandom number seeds.
4. Execute the simulation n times and collect {XY), 1 :s j :s t'} on replications i for
i = 1, ... ,n. On the first replication, use the initally chosen pseudorandom number
 6.1. WARM-UP INTERVAL 229

seeds. On each subsequent replication, use the final seeds of the previous replication as
the starting seed.

5. Compute and plot {X jn1 , 1 :::: j :::: t'l and {Xjn,tl-Hl, 1 :::: j :::: t'l.
6. If the graphs reveal a suitable warm-up interval k -1, use that choice in a new independent
run to collect a larger sample path to assess sampling error. See Sections 6.6 through 6.8.

7. If the graph fails to reveal a suitable warm-up interval, increase t', either continue execu-
tion and data collection of the first n independent replications or simply execute n new
independent replications with the new length, and go to step 5.

As illustration, an amended version of the SIMSCRIPT 11.5 AIR-EV.sim program in

Figure 4.2 can generate N independent sequences of waiting times where each replication
begins in the empty and idle state. To save space, we list only the modifications. They are:

Step 1. preamble
Between lines 33 and 34, insert:
define N as a variable
, , N := no. of independent replications
Step 2. main
Between lines 15 and 16, insert:
readN
for I = 1 to N do
Between lines 19 and 20, insert:
loop

Step 3. ACCOUNTING
In line 13, delete:
stop
Between lines 14 and 15, insert:
ifNUM >= K and NUM < = K+T-l call WARM_UP...ANALYSIS
always

Step 4. Merge WARM_UP ...ANALYSIS in Figure 6.3 with AIR-EV.sim in Figure 4.2.

The output in SIMUlS will consist of n = N tableaus as in Figure 4.3 followed by the
summary data {X jn1 , 1 :::: j :::: t'l and {Xjn,tl-Hh 1 :::: j :::: n}.
In addition to generating the data of interest, the WARM_UP ...ANALYSIS routine in
Figure 6.3 resets the state of the system back to the empty and idle state at the end of each
replication. It does this by removing and destroying each event notice in the event lists and
removing and destroying all CALLERs waiting in the QUEUE. Note that f.ev.s(·) denotes the
first event notice in event list ev.s(·). To guarantee nonoverlapping pseudorandom number
streams, the routine assigns the final pseudorandom numbers on the last replication as the
230 C HAP TE R 6. 0 U T PUT A N A L Y SIS

seeds for the next replication. As currently written, the amended program produces a tableau
as in Figure 4.3 for each of the N replications. We leave as an exercise the task of aggregating
the N tableaus into a single comprehensive summary.

Figure 6.3 WARM_UP .-ANALYSIS program for airline simulation

, '(April 1998)

"Computes {XBAR(J,N,1); J=K,K+1, ... ,K+T-1}

and
{XBAR(J,N,K+T-J); J=K,K+1, ... K+T-1}.

define Z as a 1-dim double saved array

define COUNT as a saved variable
COUNT := currently executing replication
define Y as 1-dim double array
define FIRST as an integer variable
i f COUNT=O
let COUNT = 1
reserve Z(*) as T
always
i f NUM >= K and NUM <= K+T-1
add WAITING. TIME to Z(NUM - K + 1)
always
if NUM < K + T -1 return
else

Empty the future event sets

let FIRST = f.ev.s(i.CALL)

cancel the CALL called FIRST
destroy the CALL called FIRST
NJ =n.ev.s(i.END_MESSAGE)
for J = 1 to NJ do
let FIRST = f.ev.s(i.END_MESSAGE)
cancel the END_MESSAGE called FIRST
destroy the END_MESSAGE called FIRST
loop

NJ = n.ev.s(i.COMPLETION)
for J = 1 to NJ do
let FIRST = f.ev.s(i.COMPLETION)
cancel the COMPLETION called FIRST
destroy the COMPLETION called FIRST
loop

Idle all RESERVATIONISTs

for J = 1 to n.RESERVATIONIST
 6.1. WARM-UP INTERVAL 231

Figure 6.3 (cont.)

let STATUS(J) =IDLE

Empty the QUEUE

NJ = n.QUEUE
for J = 1 to NJ do
remove the first PERSON from QUEUE
destroy the CUSTOMER called PERSON
loop

Assign the final pseudorandom numbers from this replication as the initial
seeds for the next replication.

for J = 1 to 4 let SEEDS(J) = seed.v(J)

Summarize sample path data for N independent replications

if COUNT = N
reserve Y(*) as T

Write {XBAR(J,N,l); J=K,K+l, ... K+T-l} to unit 15.

for J = K to K+T-l write J,Z(J-K+l)/N as i 7, e(16,6)

let yeT) = Z(T)/N
for J = 2 to T do
let yeT - J + 1) = yeT - J + 2) + Z(T - J + l)/N
let yeT - J + 2) = yeT - J + 2)/(J -1)
loop
let Y(l) = Y(l)/T

Write {XBAR(J,N,K+T-J); J=K, ... K+T-l} to unit 15.

for J=K to K+T-l write J,Y(J-K+l) as i 7, e(16,6)

always
add 1 to COUNT
return end

Arena/SIMAN provides a capability for automatically creating graphs for a warm-

up analysis. Recall in Figure 4.l3 that the TALLIES element declared that waiting times
be collected. By specifying, say, 10 replications in the dialog box for the REPLICATIONS
element, the simulationist instructs Arena to collect 10 independent waiting-time sample
sequences. After simulation execution terminates, the "Output Analyzer" on the Tools menu
allows the simulationist to plot all 10 sample paths on the same graph or to lump them
together into a single plot.
The output in SIMU15 will consist of n = N tableaus as in Figure 4.3 followed by the
summary data {X jnl, 1 :::: j :::: t'} and {X jn,r'- HI,! :::: j :::: n}.
232 C HAP TE R 6. 0 U T PUT A N A LY 5 I 5

6.2 ASSESSING THE ACCURACY OF THE SAMPLE AVERAGE

After selecting a warm-up interval k, a simulator has at least three options:

A. Use Xkn,t'-k+l as a point estimate ofJL andkn,t'-k+l (8) as an approximating 100 x (1-8)
percent confidence interval for JL, where

(6.7)

(6.8)

and in-l (1 - 8/2) denotes the 1 - 8/2 critical value of Student's t distribution with n - 1
degrees of freedom.

B. Select t > t', restart the nth simulation in the state prevailing when x~~) was collected,
collect k + t - t' - 1 additional observations X~~l' ... , Xk~t-l' and compute the trun-
cated sample average Xk7) (see expression (6.3» as a point estimate of JL, compute an
•
estImate v.(n) f
t
X- (n) • th etruncated sampIe-path d ata X(n)
0 tvar kt usmg k '
X(n)
k+i"'"
X(n)
k+t-l'
and compute

(6.9)

as an approximating 100 x (1 - 8) percent confidence interval for JL, where the degrees
of freedom I (t) - 1 depend on the method employed for computing V,tn) .
C. Start a new independent simulation run, n + 1, in the same initial state as the preceding
·
n replicatIOns, co11ect X(n+I)
k ' ••• ,
X(n+l) ft .
k+t-l a er a warm-up mterv
al k -1, use X-kt
(n+l) as a
point estimate of JL andC k~~N(8) as an approximating 100 x (1- 8) percent confidence
interval for JL.

Computational and statistical properties determine the relative desirabilities of each

option. Since Option B has no need to rerun the warm-up interval, it offers an advan-
tage over Option C with regard to computational efficiency. However, if programming the
simulation to restart in the last state presents too much of a challenge, then Option C
becomes more appealing. Since the statistical properties of (xt), v/n ) , C k~1(t)(8») and
(x- (n+l) ' v.(n+l)
ktl(t) t C (n+l)
'ktl(t) (0») are 1' d
0
' al,we hereafter suppress the superscnpts
entlC . n and
()
(n + 1) unless required for clarity. Moreover, any discussion of Option B applies equally to
Option C.
Because ofi.i.d. replications, Option A easily computes an estimate, S~t/n, ofvarXkt.
By contrast, computing Vt as an estimate of t varXkt with desirable statistical properties calls
 6.3. FORMALISMS 233

for considerably more demanding computational procedures. Section 6.6 describes these
procedures, and Section 6.7 describes software, available on the Internet, for implementing
the procedures with minimal effort for the simulationist.
Since the graphical method for selecting a warm-up interval k contains no formal
statistical testing procedure for deciding whether or not systematic error remains after trun-
cating k - 1 observations, it is clearly a subjective approach. Most importantly, picking k too
small affects Options A and B in different ways. For Option B, the influence of any residual
warm-up error diminishes as t ~ 00. By contrast, this error remains constant under Op-
tion A as n ~ 00, implying that Option B is more forgiving than Option A with regard to
understating the warm-up interval.
Regardless of computational convenience, any assertions that a simulationist makes
regarding the statistical errors in Xknt or Xkt are supportable only if the methodology for
deriving Aknt(8) and Cktl(t) (8) rests on well-established statistical theory. For fixed t, the
interval estimate A knt (8) is asymptotically valid if and only if

pr[JL E A knt (8)] ~ 1- 8 as n ~ 00. (6.10)

For fixed k, Ckt1(t) (8) is asymptotically valid if and only if

pr[JL E Ckt l(t)(8)] ~ 1- 8 as t ~ 00. (6.11)

To address these and related issues, we need to introduce several formalisms.

6.3 FORMALISMS

Although the analysis in Section 6.1 relies exclusively on sample data for a particular sim-
ulation model, it offers an invaluable introduction to the assessment of systematic error.
However, the perspective is a limited one. The next several sections describe a more formal
setting for assessing error. The formalisms apply to data generated by many different types
of discrete-event simulation, and this high level of generality overcomes the limitation of
Section 6.1 by providing the reader with a considerably more comprehensive basis for error
analysis.
Again consider a single sample path Xl> ... , Xk+t-l with finite time average

_ 1 k+t-l
X kt := -
t
L
i=k
Xi (6.12)

as an approximation to the long-run average JL. Let X denote the sample space for the X j
and let {Fj(xls), x E X} denote the conditional d.f. of Xi given that the simulation starts
234 CHAPTER 6. OUTPUT ANALYSIS

in state S(O) = s E S (Section 6.1), so that expressions (6.5) and (6.6) take the equivalent
forms

/-isj = Ix xdFj (x Is)

and

Suppose that Xi has the equilibrium dJ. {F(x), x EX} and thatthe long-run average
/-i equates to the equilibrium mean of Xi; that is, /-i = Ix
x dF(x). For expression (6.12) to
be a useful basis for inferring the true value of the unknown /-i, the underlying stochastic
process must have several properties. These include:

• For every initial state s E S, /-isj -+ /-i < 00 as j -+ 00.

• Let

- 1 k+t-l
/-iskt := E(XktIS(O) = s) = - L 1 xdFj(xls). (6.13)
t j=k x

For every initial state s E S and each fixed integer k :::: 1,

/-is kt -+ /-i as t -+ 00. (6.14)

• Let
2 -
a skt := var(XktIS(O) = s).

For every s E S and each fixed t :::: 1, there exists a positive constant at2 independent
of sand k such that

(6.15)

and there exists a positive constant a~ such that

(6.16)

• For every s E S and each fixed k :::: 1,

as t -+ 00. (6.17)
 6.4. INDEPENDENT REPLICATIONS 235

Although convergence as in expressions (6.14) through (6.17) is essential, it is not suf-

ficient for us to develop a distributional theory as the basis for computing an asymptotically
valid confidence interval for IL. Speed of convergence is also an issue. For example, for

pr [Iikt - ILsktl :::: z S(O) = s] = pr [ a&, Ii kt - IL + IL -ILskt :::: zIS(O) = s]

r;;;-
VUskt
ta}kt Ja&,/t Ja&,/t

~ pr (
likt - ILl
r::'2I;:::: z) as t ~ 00 (6.18)
"a&,/t

if and only if ILskt ~ IL at a rate proportional to t- I / 2 or faster. For many stochastic processes
encountered in discrete-event simulation,

ILsj = IL + O(fJ j) as j ~ 00 (6.19)

for some fJ E [0, 1), so that

ILskt = IL + O(fJ kIt) as k ~ 00 and t ~ 00, (6.20)

which satisfies the requirement for convergence to IL.

6.4 INDEPENDENT REPLICATIONS

Our objective is to assess how well i knt in Option A and defined in expression (6.4) approx-
imates the unknown equilibrium mean IL. A confidence interval provides a basis for doing
this. Four parameters affect the validity of this approach. They are the initial state s, the
warm-up interval k - 1, the length of sample path t used for estimating IL, and the number
of independent replications n. This section describes an approach based on independent
replications for deriving confidence intervals that are asymptotically valid (as n ~ (0).
Multiple independent replications allow us to employ standard, and relatively simple, sta-
tistical theory applicable to i.i.d. observations to compute confidence intervals. However,
the approach does have limitations, as we show. Later, Section 6.5 describes an alternative
approach that principally relies for its validity on t ~ 00 for fixed k and n.

Theorem 6.1.
Let N «(1\, (12) denote the normal distribution with mean (11 and variance (12. Let
{xji) , 1 :::: j :::: t' = k + t - I} for 1 :::: i :::: n denote n independent realizations or
236 C HAP TE R 6. 0 U T PUT A N A L Y SIS

sample paths of the process {X j} each of length k + t - 1. Suppose that expression (6.20) holds
for the conditional mean and that the conditional variance has the representation

(6.21)

where

(6.22)

Then

1. Forfixedk:?: 1 andt:?: 1, (Xknt - tLskt)/jas2kt /n ~ N (0,1) asn -+ 00.

u. Forfixedt :?: 1, (X knt - tL)/Ja?!n ~ N (0,1) ifk/lnn -+ 00 ask -+ 00 andn -+ 00.

PROOF. Part i follows from the classical Central Limit Theorem for i.i.d. random variables.
For part ii, let

X knt - tL
Aknt:= J a/ /n

and

tL - tLskt
Bknt := r::2I::' (6.23)
valin

so that

A knt -_ Xknt - tLskt (6.24)

j a;kt/ n

Substituting (6.20), (6.21), and (6.22) into (6.23) and (6.24) shows that the condition
kiln n -+ 00 as k -+ 00 and n -+ 00 suffices for Bknt -+ 0. Also, a;kt -+ a/ as k -+ 00.
- ~
Therefore, A knt converges to (X knt - tLskt)/ Vaskt/n w.p.I, which converges to a normal
random variable from N (0, 1). 0

Although k is supposedly chosen to remove the ostensible influence ofinitial conditions

on Xb Xk+I, ... , it is nevertheless the case that tLskt i= tL for fixed k and t; moreover,
Theorem 6.Ii implies that the interval
 6.4 . I N D E PEN DEN T REP Lie A T ION S 237

covers JLskt> not JL, with approximate probability 1 - 8 for large n, where <1>-1(1 - 8/2)
denotes the 1 - 8/2 point of the standard normal distribution N (0, 1). To achieve this same
coverage rate for JL, Theorem 6.1 ii asserts that the warm -up interval k must grow faster than
In n as the number of replications n increases.
Expressions (6.20), (6.21), and (6.22) characterize the conditional means and variances
of the sample means of many stochastic processes encountered in discrete-event simula-
tion. This generality encourages us to adopt this characterization as we learn more about
computing a valid confidence interval.
Since the variance a/ is unknown, Theorem 6.1 is of academic but not of operational
interest. The availability of multiple independent replications proves especially helpful in
making it operational.

Corollary 6.2.
For the specification in Theorem 6.1, fixed t ::: 1, and sint as defined in expression (6.8),
(i knt - JL)/ Jsint/ n ~ N (0, 1) ifk/In n ~ 00 as k ~ 00 and n ~ 00.

PROOF.
- r::;-;:
Let Zknt := (Xknt - JLskt)/ VSknt/n and note that

(6.25)

Since Sfnt/as2kt ~ 1 as n ~ 00 w.p.l and a t2/ as~t ~ 1 as k ~ 00, then as in Theorem 6.1
kiln n ~ 00, as k ~ 00 and n ~ 00, ensure that Zknt converges to Aknt, which has
N (0, 1) as its limiting distribution. 0

Corollary 6.2, implies that

(6.26)

provides an approximating 100 x (1 - 8) percent normal confidence interval for JL that

is asymptotically valid for k /In n ~ 00 as k ~ 00 and n ~ 00 for fixed t ::: 1. Since
Ln-1 (1 - 8/2) > Ln(1 - 8/2) > <1>-1(1 - 8/2) for each fixed n, common practice uses the
confidence interval (6.7), rather than (6.26), to compensate for the finiteness of n.
The most notable parametric dependence remains that between k and n. In particular,
these results assert that as one takes more independent replications presumably to shorten
the confidence interval, the warm-up interval k must increase faster than In n in order for
the desired limit to hold. This tie between k and n is inconvenient to implement in practice.
238 CHAPTER 6. OUTPUT ANALYSIS

RECURSIVE COMPUTATION

The algebraic equivalences (e.g., Fishman 1996, p. 68)

( 1) - 1- i Xk,i-I,t
1- (i)
+ iXkt' 1 .::: i .::: n,
Xkit ={
0, i = 0,

and

i- 2 2 1 - (i) - 2
2 { -
. -1sk,l-I,t
· + -. (kt
X -X·
k,l-I,t) , 2'::: i .::: n,
Skit = I - I
0, i = 1

allow recursive computation of Xkit and Sfit' respectively, without the need to store Xk:) , ... ,
Xk7). This facility is especially useful when space is limited.
Since the recursion for sL
is the sum of two nonnegative quantities of roughly the
same magnitude and both are independent of n, it induces a smaller numerical round-off
error than the more commonly encountered space-saving formula (Neely 1966)

2
Sknt = --=-
n
1[~-
1
(i) 2
~(Xkt)
i=1
- 2 ]
- nX knt .

This representation is the difference of two quantities each of which increases with nand
thus can exhibit substantial round-off error when the two quantities are close in value.

6.4.1 ALLOWING t TO GROW

For Option A to generate an asymptotically valid confidence interval A knt (8), as in

expression (6.7) for fixed t, its computing time necessarily increases faster than n In n, prin-
cipally due to the dependence of k on n. Under specified conditions, allowing t to increase
for fixed n can reduce this truncation cost in relative terms. To do this effectively calls for an
understanding of the distributional behavior of (Xk~) - J-L) / O'&,/t as t --+ 00. J

BROWNIAN MOTION

Let {Z ( r), r :::: O} denote a stochastic process in ffi with continuous parameter rand
with the properties:

i. pr [Z(O) = zol = 1
 6 .4 . I N D E PEN DEN T REP L I C A TI 0 N 5 239

iii. For 0 ::::: s < r, the increment Z(r) - Z(s) has the normal distribution with mean
01 (r - s) and variance 02( r - s).

Then {Z(O) = Zo; Z(r), r > O} is called Brownian motion or a Wiener process starting
at zoo Most importantly, {Zj := Z(n - Z(j - 1); j = 1,2, ... } is a sequence ofi.i.d.
normal random variables with mean 01 and variance O2 • If 01 = 0 and O2 = 1, then {Z (0) =
Zo; Z(r), r > O} is called standard Brownian motion starting at zoo

ASSUMPTION OF WEAK APPROXIMATION

If {X j , j ::: O} is a stochastic sequence and there exist a standard Brownian motion

{Z(r), r ::: O} and finite constants a > 0 and d such that t 1/ 2 (X t - d)/a converges
in distribution to Z(t)/t 1/ 2 , then {X j , j ::: O} is said to satisfy the assumption of weak
approximation (AWA).
This property provides an essential ingredient for deriving asymptotically valid
confidence intervals as the sample-path length t increases.

Theorem 6.3.
For each i = 1, ... , n and each fixed k :::: 1, suppose that the process {Xii 21+ j' j :::: I}
with Xyl as in Theorem 6.1 satisfiesAWA witha 2 = O'~ andd = f,1,. Thenforfixedn :::: 1 and
k ::: 1 as t ~ 00,

i. v'ili(Xknt - f,1,)/~ ~ N(O, 1).

ii. (n - 1)sint/O'~ ~ X;-p where X;-l denotes the chi-squared distribution with n - 1
degrees offreedom.

iii. (Xknt - f,1,)/Jsint/n ~ rn-I> where rn-l denotes Student's t distribution with n - 1
degrees offreedom.

SKETCH OF PROOF. As a consequence of AWA, .Jt(Xi!l - f,1,) /~, ... , .Jt(Xi;l - f,1,) / ~
converge w.p.l to n independent random variables each from N(O, 1). The limits in parts i,
ii, and iii follow from this property. D

As a consequence of Theorem 6.3iii, pr[f,1, E Aknt(8)] ~ 1 - 8 as t ~ 00 for fixed n

and k and Aknt (8) in definition (6. 7). Simple in concept, and asymptotically valid as t ~ 00,
240 CHAPTER 6. OUTPUT ANALYSIS

this approach to confidence interval computation has a natural appeal, once one selects the
number of replications n and the warm-up interval k.
For completeness, we describe the necessary and sufficient conditions that must exist
between nand t for attaining desirable limiting behavior as these two quantities increase
simultaneously. We also describe the necessary and sufficient conditions that must exist
among k, n, and t as all three increase.

Theorem 6.4.
Assume {xjl) , 1':::: j .:::: k + t - I}, ... , {xjn), 1.:::: j .:::: k + t - I} as in Theorem 6.1.

I. Iffor fixed k 2: 1, t / n -+ 00 as n -+ 00 and t -+ 00, then

II. Ifk / In(n / t) -+ 00 as k -+ 00, n -+ 00, and t -+ 00, then

PROOF. For fixed k, t must increase faster then n is required to make

Itskt - It

J a}kt/ n

vanish. For increasing k, this occurs if and only if k In(n/t) -+ 00. D

Although Theorems 6.3 and 6.4 collectively offer a method of generating an asymp-
totically valid 100 x (1 - 8) percent confidence interval for It, its implementation in practice
is problematic. Given an upper bound on total computing time, any attempt to realize the
asymptotic properties that accrue as t increases can only come at the expense of having
n diminish. Conversely, any attempt to realize the asymptotic properties that accrue as n
grows large can only come at the expense of having t diminish. If we were to adopt the latter
course, we would have to ensure that the truncation interval k also grows faster than In n,
so that Corollary 6.2 holds. If we adopted the former course (Option B), we would need
an alternative way of estimating a ~ that ensure an asymptotically valid confidence interval
for It. Indeed, Sections 3.3.4 and 3.5 have already raised this issue for estimating long-run
average utilization, queue length, and waiting time. In the current context, if one has an
estimator Vt based on a single sample path of length t such that

Vt -+ a~ as t -+ 00 w.p.I, (6.27)

then for n = 1 and fixed k, the distribution of (X kt - It)/ y'Vr/t converges to N(O,l) as
t -+ 00. Recall that a Vt satisfying expression (6.27) is called strongly consistent.
 6.5. STRICTLY STATIONARY STOCHASTIC PROCESSES 241

Many alternative methods exist for computing a strongly consistent estimator of Vt

for Option B. All are less than direct methodologically and for large t can themselves pose
a not inconsequential computing burden. Among these the batch-means method (Section
6.6) lends itself to relatively simple interpretation and has an algorithmic implementation,
LABATCH.2 in Section 6.7, that induces O(logzt) storage space and OCt) computing time,
while producing a strongly consistent estimator of a~ and thus an asymptotically valid
confidence interval for f..i. To the potential simulation user, the cost is to acquaint oneself
with more advanced concepts of statistics than those associated with i.i.d. observations.
The benefit is the need principally to concern oneself with only two design parameters, the
warm-up interval k and the sample path length t, while fixing the number of replications,
n, at unity.

6.5 STRICTLY STATIONARY STOCHASTIC PROCESSES

To understand the motivation that underlies the batch-means method for estimating
a~, it is necessary to become acquainted with the implications of dependence among
Xl, Xz, X 3 , •••• Let us momentarily assume that a warm-up interval k has been deter-
mined so that the corresponding error of approximation is negligible in regarding each
Xb Xk+l, ... as having the desired equilibrium distribution. Interest then focuses on as-
sessing the statistical accuracy of Xkt as an estimator of f..i. This section begins this process
by characterizing the structure of a? = varXkt. It does this within the context of a strictly
stationary stochastic process and, in particular, describes how dependence along the sample
path affects a tz.
Consider a stochastic process {X j ,j 2: o} with the property that for all positive integers
i and for each positive integer I the i-tuples (X jj , X j" ... , Xj;) and (X h +l , Xj,+l, ... ,
Xj;+l) have the same i-dimensional probability distribution. This property, called strict
stationarity, immediately implies that for each j 2: 0, X j has the same marginal distribution.
Moreover, for j 2: i 2: 1 the covariance between X; and X j is

cov(X;, X j ) = Ex;,xj [(X; - f..i)(X j - f..i)]

= Ex; [(X; - f..i)ExjIX;(Xj - f..i)]

= Ex; {(Xi - f..i)[ExjIX;(Xj) - f..i]}

(6.28)

revealing the conditional mean function {ExjIX; (X j), j 2: I} and the autocovariance function
{R j - i := cov(Xi , X j ); i, j = 1,2, ... } to be functions of the index difference j - i, but
not of the absolute locations i and j in the index set. Moreover, expression (6.28) shows that
an intimate relationship exists between this conditional mean function and the covariance
242 C HAP TE R 6. 0 U T PUT A N A LY SIS

structure of {X j' j ::: I}. In particular, if for fixed i < j

(6.29)

then R j ~ 0, as j ~ 00, revealing that the covariance between Xi and X i +j vanishes as

the index difference increases without bound.
Since cov( X 1> X j+ 1) = cov( Xi, Xi+ j) for all i, it suffices for us to examine the prop-
erties of the autocovariance function {R j, j = 0, ± 1, ... } for a strictly stationary process.
Note that

varXi = Ro, (6.30)

R j = R_ j , (6.31)

IRjl ~ R o, j = 0, ±1, ±2, ... (6.32)

The correlation between Xi and Xj is

corr(
Xi, )
Xj = Pi-j := -R;-j
-,
Ro
i, j = 0,1, ... , (6.33)

where Po = 1, Pj = P_ j , and, as a consequence of expression (6.32), IPj I ~ 1. Here

{P j ; j = 0, ±1, ±2, ... } denotes the autocorrelation function of {X;}. Figure 6.4 shows the
autocorrelation function, {Pj = a Ui }, for a = .90 and -.90. As j increases one displays
monotonically decreasing behavior and the other, alternating positive and negative values.
Suppose we observe the process in the steady state and collect a sample record or
realization Xl, ... , X t for the purpose of estimating JL • Since data collection begins in the
steady state, the sample mean, Xt = (Xl + ... + Xt)/t, is an unbiased estimator of JL. Also,

2 - - 2
at := varX t = E(Xt - JL)

1
LR
t
="2 i- j
t i,j=l

1 t-l
= 2" L
t j=l-t
(t -ljI)R j • (6.34)
 6.5. S T RIC TL Y S TAT ION A RY S Toe HAS TIC PRO C E SSE S 243

------------------a. ----------~----~~

0.8 . • a • .90
•
•
~
0.6 . •
1. •
1> •
0.4 .
•
••
••
0.2 .
• ••
•• .....
............... __. . . . . . . . . . A-

0 10 20 30 40 50
j

b.

0.75 a a ·.90

0.5

. • ... • .....
. • • • • • . . • . ............. ---
0.25
~
1. 0
1>
-0.25

-0.5

-0.75

0 10 20 30 40 50

Figure6.4 Autocorrelation function Pj := R j / Ro = a 1jl

By the symmetry property (6.31),

2
LO- jjt)R
/-1
Ro
+ - j • (6.35)
t t j=1
'-v-'
due to variance
due to correlation
Ro from each
between observations
observation
244 CHAPTER 6. OUTPUT ANALYSIS

This partition reveals two distinct contributions to at2 • If Xl. X 2 , ••• were i.i.d., then
the correlations between observations would be zero and a,2 = Rolt. However, congestion
generally induces dependence in waiting time, queue length, and utilization sequences,
and this dependence frequently manifests itself as positive correlation. As a consequence,
a,2 > Rolt for most queueing-related processes. Moreover, as a system becomes more
congested, this positive correlation grows, usually increasing the relative contribution to a,2
made by the underlying correlation structure.
The condition limHoo ta,2 = a~ < 00 in expression (6.16) implies that

t-l
Ro L (l-Ijllt)pj ~ a~ as t ~ 00. (6.36)
j=l-t

For this to hold, limj-+oo Pj = 0 is necessary but not sufficient. However, the condition
t-l

Ro L Pj ~ a~ < 00 (6.37)
j=l-t

is both necessary and sufficient. This property implies that, in addition to the correlation
between Xi and X i+j converging to zero as j ~ 00, it must dissipate sufficiently fast so that
the summation in expression (6.37) converges. An equivalent characterization is possible
in terms of the conditional mean. See expression (6.28). Hereafter, we occasionally refer to
a~1 Ro as variance inflation.
To illustrate the implications of the limit (6.37), consider the autocorrelation function

pj=a UI , -1<a<l, j = 0, ±1, ... , (6.38)

for which

(6.39)

and

a~ l+a 1 + PI
= = (6.40)
Ro 1- a 1 - PI

Table 6.2 shows how varXt becomes inflated as a increases. For example, a nearest-
neighbor correlation of a = .10 results in a 22 percent increase in the variance of X t over that
to be expected if the XI, X 2, .•. were i.i.d. More generally, the entries in Table 6.2 reveal that
treating X t as the average of i.i.d. data when it is not can result in a substantial overstatement
of the accuracy of Xt •
Since a~1 Ro < 1 for a < 0, var Xt is less than would occur if the data were i.i.d.
 6 .5 . S T RIC TL Y S TAT ION A RY S Toe HAS TIC PRO C E SSE S 245

Table 6.2 Variance inflation

a00
2 l+a
a
Ro I-a
.10 1.222
.20 1.500
.50 3.000
.75 7.000
.80 9.000
.85 12.333
.90 19.000
.95 39.000
.99 199.000

Although autocorrelation functions for utilization, queue length, and waiting time are
rarely, if ever, literally of this geometric form, they often have geometric bounds. For example,
a waiting time sequence, Xl, X 2 , ••• for the stationary M/M/I queueing model with arrival
rate).. and service rate w (> )..) has traffic intensity v := )..jw, mean /-t = vjw(1 - v),
variance Ro = v(2 - v)jui(1 - V)2, and autocorrelation function (Blomqvist 1967)

00 [ V Ji (2i - 3 ) ! . . ..
Pi = i~3 (1 + V)2 i!Ci _ 2)! (I - ] - 1)(1 - ] - 2), j = 0, I, ... , (6.41)

with tightest possible geometric upper bound

Pi ~
[
(1
4v
+ V)2
Ji '
j = 0, I, ... . (6.42)

For example, v = .90 and .95 give 4vj(1 + V)2 = .9972 and .9993, respectively, leading to
relatively slow decreases in the bound (6.42) as the lag j between two waiting times increases.
Moreover,

· t varX- t =
11m 2
0'00 = 2(
V
)4
(
V
3
- 4v
2
+ 5v + 2), (6.43)
t-'>oo W 1- v

revealing a variance inflation

1 3 2
( )(
2-v I-v
)2 (v - 4v + 5v + 2) (6.44)

that increases as 1j (1 - V)2 as v increases.

246 CHAPTER 6. OUTPUT ANALYSIS

6.6 BATCH-MEANS METHOD

To assess how well Xt approximates fL, we need an estimate of a&,. The batch-means method
offers one option. Let bet) denote a positive integer « t), let let) := Lt/b(t)J, and let
aCt) := l(t)b(t). Our version of the batch-means method fits within the context of Option B
in Section 6.2. It partitions the sequence Xl, ... , Xa(t) into let) nonoverlapping batches
each of size b(t),

(6.45)

computes the batch averages

Yjb(t)
1
:= bet) 8
b(t)
X(j-l)b(t)+i, j = 1, ... , let), (6.46)

and an estimate ofvar Yjb(t)

1 I(t)

W1(t)b(t) := let) _ 1 'L...'/Yjb(t)

" - -
Xa(t)) 2, (6.47)
j=l

and uses Va(t) := b(t) WI(t)b(t) as an estimate of a~. Then

(6.48)

as in expression (6.9), with subscripts k and t suppressed, provides an approximating 100 x

(1 - 8) percent confidence interval for fL. The challenge is to choose the batching sequence
((let), bet)), t :::: I} in a way that ensures asymptotic validity,

pr[fL E C1(t) (8) 1 -+ 1- 8 as t -+ 00,

and, among all such choices, to select one that makes the width of the interval relatively
narrow and that is computationally efficient. We first describe an application of the batch-
means method. For a simulation of the MIMII queueing model begun in the steady state with
interarrival rate v = .90, service rate unity, for t = 107 customers, and 8 = .01, Figure 6.5a
shows{(X tj , C1(tj)(8)): tl = 35, tHl := 2tj; j = 1, ... , 18} U {(Xt, C1(t)(8))} for mean
waiting time in queue (fL = 9), and Figure 6.5b shows the corresponding statistics for the
probability of waiting (fL := pr(Wi > 0) = .90). These statistics were computed using LA-
BATCH.2, an implementation of the batch-means method. It is available in C, FORTRAN,
and SIMSCRIPT II.5 on the Internet at http://www.or.unc.edu/~gfish/labatch.2.html.LA­
BATCH.2 can be applied either interactively to data generated during a simulation or after
simulation termination to stored sample sequences generated by the simulation.
 6.6. BATCH-MEANS METHOD 247

(a) Series 1

20 -

p-
15 -

Mean
waiting

-1- -1--1-- H -H
time 10 -

--j-- --....---
5 -

o 0 5 10 15 20
Interim review I

(a) Series 1

20 -

15 -

-p--I--I-+-H+-'-
Mean
waiting
time 10 -

--j- -1- -.--- •

5 -

o 0 5 10 15 20
Interim review I

Figure 6.5 LABATCH.2 sample means and 99% confidence intervals for ABATCH
rule; simulation starts in steady state

The graphs allow us to assess how well the point estimates {X t " ••• , X/" , Xt } approx-
imate f.-L. For small j the Xt j are unstable. However, the sequences stabilize with increasing
j. At the same time, the confidence intervals first widen with j and then decrease. By j = 14
in each figure, stabilization of Xt j and the width of the confidence intervals suggest that the
error of approximation is relatively small. We return shortly to a discussion of this initial
widening of confidence intervals.
248 CHAPTER 6. OUTPUT ANALYSIS

6.6.1 LABATCH.2 TABLEAUS

Figure 6.6 shows LABATCH.2-generated tableaus that provided the data for the graphs
in Figure 6.5. For each series, the "Final Tableau" presents results computed at the end of
the simulation; in particular, the point estimate Xof J1 based on all t = 10 7 observations,
the estimated standard error of X, JB*W(L, B)/t, where the numerator is based on the
first t'(107) = 9,175,040 observations, a confidence interval [Lower, Upper] based on
Cz(I) ( . 0 1) in expression (6.48), and a rough measure of relative error (Upper-Lower) I IXI for
X. Section 6.6.3 explains why W(L,B) is based on t' (107 ) rather than 107 observations.
Figure 6.6 reveals an estimated error of ± 100 x .0302/2 = ± 1.51 percent relative
to the sample mean waiting time and an error of ± 100 x .002416/2 = ± .1208 percent
relative to the sample exceedance probability. As a rule of thumb, a simulationist may regard
an entry under IUpper - Lowerl/lXI ::: .20 as implying relatively high accuracy.
Readers concerned principally with results as in the Final Tableau may skip to Sec-
tion 6.7 for a description of how to implement LABATCH.2 or to Section 6.6.5 for an account
of how Arena/SIMAN uses the batch-means method. The remainder of this section together
with Sections 6.6.2 through 6.6.9 describes details that influence the extent to which an
application of the batch-means methods leads to asymptotically valid confidence intervals.
For each series, an "Interim Review Tableau" in Figure 6.6 lists results for each review j
based on tj = L * B observations, where L denotes number of batches, B denotes batch size,
and tHI = 2tj. Section 6.6.5 describes the mechanism for increasing Land B. The entries
under Sqrt[B*W(L,B)] provide a sequence of estimates A of (Yoo that become increasingly
stable as j increases. Figures 6.7a and 6.7b show the graphs for these entries for each series.
Most notable is the initial growth followed by fluctuation about the theoretically known
limits (Yoo = 189.5 and (Yoo = 1.308 in Figures 6.7a and 6.7b respectively (Blomqvist 1967).
This initial growth pattern is responsible for the widening of confidence intervals for J1 for
successive small j. Section 6.8 provides an additional way of assessing convergence in Figure
6.12b.

6.6.2 SYSTEMATIC VARIANCE ERROR

Why do the estimates of (Yoo behave as in Figures 6.7a and 6.7b? To the potential user of
the batch-means method, this behavior and the initial widening of the confidence intervals
can be puzzling. A description of the sampling properties of VI explains these phenomena.
When XI,"" X t are i.i.d., with varX i < 00, the assignment bet) = 1 and let) = t makes
CI (I)(8) an approximating confidence interval, based on t - 1 degrees of freedom, whose
error of approximation is generally negligible, say, for t ::: 100, regardless of the parent
distribution of the Xi. When {Xi} is a dependent sequence, as commonly occurs in discrete-
event simulation, no comparable rule of thumb applies, since two properties now affect the
error of approximation. As before, one is the marginal distribution of the Xi; the other is
the correlation between Xi and X j for all i #- j in the sample. An analysis of the properties
of VI reveals how these errors arise.
 6.6. BATCH-MEANS METHOD 249

Ca)
Final Tableau
Mean Estimation
***************
(t = 10000000 )

99.0%
Standard Error Coni idence Interval
Series Sqrt [B*W(L.B) It] Lower Upper (Upper-Lower) I I X I

0.89490+01 0.56810-01 0.88020+01 0.90970+01 0.32900-01

0.89950+00 0.42000-03 0.89840+00 0.90060+00 0.24160-02

X is based on all t observations.

WeL.B) is based on first 91.75% of the t observations.

Cb)
Interim Review Tableau
ABATeH Oata Analysis for Series 1

99.0%
Confidence Interval
Review L*B Lower Upper Sqrt [B*Iri(L.B) ] p-value

35 7 0.16600+02 0.13790+02 0.19400+02 0.44750+01 0.3804

70 10 0.18100+02 0.15750+02 0.20450+02 0.60470+01 0.1109
140 14 10 0.19140+020.16930+020.21350+02 0.86800+01 0.1524
280 20 14 0.18860+020.16620+020.21110+02 0.13130+02 0.0194
560 20 28 0.11110+02 0.57480+01 0.16460+02 0.44320+02 0.0000
1120 20 56 0.72540+01 0.27410+01 0.11770+02 0.52780+02 0.0001
2240 20 112 0.67060+01 0.29600+01 0.10450+02 0.61960+02 0.0005
4480 20 224 0.75560+01 0.39960+01 0.11120+02 0.83280+02 0.0358
8960 20 448 0.67470+01 0.41110+01 0.93830+01 0.87210+02 0.7293
10 17920 28 640 0.78170+01 0.54780+01 0.10160+02 0.11300+03 0.7095
11 35840 40 896 0.95130+01 0.62800+01 0.12750+02 0.22600+03 0.0655
12 71680 40 1792 0.96680+01 0.66930+01 0.12640+02 0.29410+03 0.8172
13 143360 56 2560 0.90730+01 0.74490+01 0.10700+02 0.23040+03 0.7200
14 286720 80 3584 0.88830+01 0.80120+01 0.97540+01 0.17670+03 0.2993
15 573440 112 5120 0.92480+01 0.85110+01 0.99850+01 0.21280+03 0.7163
16 1146880 160 7168 0.91260+01 0.86530+01 0.96000+01 0.19450+03 0.0993
17 2293760 160 14336 0.91380+01 0.88090+01 0.94680+01 0.19130+03 0.1852
18 4587520 224 20480 0.90320+01 0.88050+01 0.92590+01 0.18740+03 0.7882
19 9175040 320 28672 0.89490+01 0.88020+01 0.90970+01 0.17960+03 0.2362

If data are independent:

10000000 10000000 1 0.89490+01 0.89410+01 0.89570+01 0.98210+01 0.0000

0.10 significance level for independence testing.

Review 19 used the first 91.751. of the t observations for W(L.B).

Interim Review Tableau ABATCH Oata Analysis for Series 2

99.0%
Confidence Interval
Review L*B Lower Upper Sqrt[B*W(L,B)] p-value

35 5 0.10000+01 0.10000+01 0.10000+01 0.00000+00 0.0000

70 10 0.10000+01 0.10000+01 0.10000+01 0.00000+00 0.0000
140 20 0.10000+01 0.10000+01 0.10000+01 0.00000+00 0.0000
280 40 0.10000+01 0.10000+01 0.10000+01 0.00000+00 0.0000
560 80 0.92500+000.78380+000.10660+01 0.90140+00 0.0067
1120 160 0.88120+00 0.76970+00 0.99280+00 0.10070+01 0.0140
2240 320 0.88620+00 0.75030+00 0.10220+01 0.17340+01 0.1002
4480 10 448 0.88300+00 0.79750+00 0.96860+00 0.17620+01 0.8214
8960 14 640 0.87700+00 0.83600+00 0.91800+00 0.12890+01 0.9589
10 17920 20 896 0.88880+00 0.86070+00 0.91690+00 0.13160+01 0.9648
11 35840 28 1280 0.90010+00 0.87970+00 0.92060+00 0.13980+01 0.3029
12 71680 40 1792 0.89730+00 0.88130+00 0.91330+00 0.15810+01 0.9235
13 143360 56 2560 0.89670+00 0.88790+00 0.90550+00 0.12500+01 0.6376
14 286720 80 3584. 0.89760+00 0.89160+00 0.90360+00 0.12200+01 0.5008
15 573440 112 5120 0.90010+00 0.89560+00 0.90450+00 0.12800+01 0.1352
16 1146880 160 7168 0.89960+00 0.89630+00 0.90290+00 0.13740+01 0.5547
17 2293760 224 10240 0.89920+00 0.89680+00 0.90160+00 0.13810+01 0.6289
18 4587520 320 14336 0.89950+00 0.89790+00 0.90110+00 0.13130+01 0.7435
19 9175040 448 20480 0.89950+00 0.89840+00 0.90060+00 0.13280+01 0.6766

If data are independent:

10000000 10000000 1 0.89950+00 0.89920+00 0.89970+00 0.30070+00 0.0000

0.10 significance level for independence testing.

Review 19 used the first 91.751. of the t observations for WCL.B).

Figure 6.6 LABATCH.2 output for MIMII queueing simulation (Series 1: waiting
time in queue, Series 2: 1:=wait, O:=no wait)
250 CHAPTER 6. OUTPUT ANALYSIS

(a) Series 1,0'00 = 189.5

300

2SO .

200 •

~
J
flSO :
~
rr -
fIl 100 ·

SO ·

0
0 5 10 15 20
Interim review J

(b) Series 2, 0'00 = 1.308

2

1.75 :

1.5 :

1.25 :
~

t
1:

~O.75 :
rr .
fIl
0.5 :

0.25 :

0
0 5 10 15 20
Interim review J

Figure 6.7 LABATCH.2 ABATCH estimates, ,JBW(L, B), of 0'00

Observe that for a(t) = t, b := b(t), and 1:= l(t),

t 2 2
EVt = - - ( ab - at ), (6.49)
1- 1

which for positively auto correlated sequences is usually negative. IfE(Xi - f.1.)12 < 00 and
{Xd is ¢-mixing with ¢i = 0(i-9), then expression (6.49) takes the form (Goldsman and
 6.6. BATCH-MEANS METHOD 251

Meketon 1986, Song and Schmeiser 1995)

EVt = a~ + y(1 + 1)/t + oO/b), (6.50)

where

I> cov(X
00

y := -2 1, Xl+i)'
i=1

The sequence {Xd is said to be ¢-mixing if there exists a sequence of real numbers
{¢z} with ¢z --* 00 as 1 --* 00 such that for all subsets A and B <; X and all i, j 2: 1

The property implies that Xi and X j become independent as Ii - j I --* 00.

In addition to the expectation (6.50), ¢-mixing leads to (Chien et al. 1997)

var V. = 2a!(l + 1) + 0(1/lb l / 4 ) + 0(1/12) (6.51 )

t (l - 1)2 '

so that Vt has mean-square error

with the property that MSE(Vt ) --* 0 if and only if I(t) --* 00 and b(t) --* 00 as t --* 00.
This property is called mean-square convergence.
Expression (6.49) leads to the representation

Vt -a~ = tal2 - a~ ta 2 (1 -1ba; /ta 2 )

- bit t
+ Et (6.52)
---.- I '-v-'
error due to error due to
error due to ignoring cor- random
finite t
relation between batches sampling

where Et has mean zero and variance (6.51). Hereafter, we collectively refer to the errors due
to finite t and to ignoring correlation as systematic variance error. Expression (6.50) shows
that systematic variance error behaves as O( (l + 1) / t) = O( 1/ b), whereas expression (6.51)
shows that JvarE t behaves as 00/ 11/2), revealing the tradeoffbetween the two types of error
that a choice of 1 and b induces. LABATCH.2 makes this choice automatically, thus relieving
the user of a major burden. Sections 6.6.3 through 6.6.5 describe why this choice leads to a
strongly consistent estimator of a~ and an asymptotically valid confidence interval for th.
Since ba; < tal2 < a~ for a substantial number of stochastic processes, systematic
error variance tends to induce a downward bias in Vt, as an estimator of a~, that dissipates
252 C HAP TE R 6. 0 U T PUT A N A LY 5 I 5

as t increases; hence the behavior in Figures 6.7a and 6.7h. Notice that after the systematic
variance error becomes negligible, the remaining fluctuations in each graph tend to vary
around some constant level and diminish in magnitude. This behavior signals that systematic
variance error is negligible. Reduction of the magnitude of fluctuations reflects the var Et =
0(11 l) property induced by the ABATCH rule (Section 6.6.5) that LABATCH.2 employed
when choosing number of batches and batch size on each review.
Mean-square error convergence implies convergence in probability of Vt to a~. In the
current context, this means that for given E > 0

MSE(Vt ) ~ 0 as t ~ 00 => pr(!Vt - a!1 > E) ~ 0 as t ~ 00.

We then say that Vt is a weakly consistent estimator of a~. However, this property does not
ensure that C1(t)(8) is an asymptotically valid confidence interval for /1-. For this to be so, Vt
must be a strongly consistent estimator of a~. To achieve this property, we must impose
additional restrictions on the evolution of ((l(t), bet))}.
Assume that there exists a constant A. E (0, !) such that

where {Z(s), s ::: O} denotes standard Brownian motion (Section 6.4). This is called the
Assumption of Strong Approximation (ASA). A A. close to ! signifies a marginal distribution
for the Xi close to the standard normal and low correlation between Xi and Xj for all i i= j.
Conversely, A. close to zero implies the absence of one or both of these properties. See Philipp
and Stout (1975).

Theorem 6.5 (Damerdji 1994).

Assume that ASA holds. If {bet)} and {let)} are deterministic non decreasing functions
in t, bet) ~ 00 and let) ~ 00 as t ~ 00, t l - 2A (logt)lb(t) ~ 0 as t ~ 00, and there
exists a positive integer q < 00 such that L:::l [I (i) ]-q < 00, then Vt is a strongly consistent
. - d
estimatorofa~ and(X t -/1-)I..jVr/t ~ N(O, 1) ast ~ 00.

Theorem 6.5 implies that b(t) must grow at least as fast as t 8 , where e > 1 - 2A..
Conversely, let) must grow sufficiently fast to ensure that the summation L:i=dl(i)]-q
converges. Since A. is usually unknown in practice, we are unable to confirm that our choice
of ((let), bet)} satisfies the requirements. Nevertheless, the guidance is unequivocal. If Xi
has a highly nonnormal distribution or Xl, X 2 , ••• have high dependence (A. close to zero),
batch size must grow faster than would be required if the distribution of Xi were close to
the normal and Xl, X 2 , ••• exhibited little dependence (A. close to!).
 6 .6 . BAT C H - MEA N S MET HOD 253

6.6.3 INTERIM REVIEW

If a simulationist computes only one estimate, Wl(t)b(t) , based on the total sample
record of length t, then, regardless of the batch size assignment rule, the analysis pro-
vides no information about the extent to which the desired asymptotic properties hold
for Vt = bet) Wl(t)b(t) as an approximation to a~. As Figures 6.6 and 6.7 show, a sequence
of interim reviews overcomes this limitation by computing and displaying results for over-
lapping segments of the sample record of successively increasing lengths tl < t2 < ... .:::: t.
The approach has considerable value when the cost per observation is high and there is
doubt, before experimentation, about how many observations to collect. Moreover, choos-
ing tHI = 2tj for j = 1,2, ... with given tl allows LABATCH.2 to compute the sequence
of estimates Wl(tl)b(t.), Wl(t,)b(t,), ... , Wl(tj)b(tj)' Wl(t)b(t) for each series in OCt) time. If LA-
BATCH.2 is implemented dynamically, the space requirement is O(log2 t). Maintaining an
O(t) computing time is essential for the single sample path approach to achieve greater sta-
tistical efficiency than the multiple independent replications approach. If generating the data
takes O(t) time, then no increase in computational time complexity arises. The O(log2 t)
space complexity is particularly appealing when t is large and interim reviews are desired
for estimates of more than one mean.
To understand the statistical implications of our choice of {tj}, we consider more gen-
eral schemes tl < t2 < .... For a sequence ofi.i.d. random variables Xl, X 2, ... , this choice
implies corr(Xtj Xt,+ j ) = J t j I ti+ j for i ::: 0 and j ::: l. More generally, if corr( Xl, X 1+ j) =
Ijl • - - •
a , for some -1 < a < I, then hmj-->oo corr(X tj , Xt,+j) = hmj-->oo Jtjlti+j. These
observations suggest that we choose e := tHdtj sufficiently large so that estimates on
successive reviews avoid substantial redundancy. We choose e = 2, which implies corr
(.X t , Xt . ) = 2- i / 2 in the independent and asymptotic (as j ---+ 00) cases. For example,
} I+}

corr(X J Xt}+l ) = -1212 = .7071 (in the i.i.d. case). Any growth factor less than 2 would
I ,

induce higher correlation. Moreover, any factor other than 2 would substantially change the
space complexity log2 t.

6.6.4 FNB RULE

Let lj := l(tj) and bj := b(tj) and recall that systematic variance error dissipates as
O(llbj ). This implies that choosing b j ex tj would diminish this error most rapidly. We
illustrate the benefit and the limitation of this approach using the FNB rule to define the
batching sequence {(I j, b j), j = 1,2, ... }. Given (11, bd, this rule fixes Ij = 11 for all j and
doubles the batch size, b HI = 2bj , on successive reviews; hence the title, FNB, for a fixed
number of batches.
Appendix A describes the principal steps for implementing the rule with tHI := 2tj
for j = 1, ... , J(t), where

J(t): = 1 + Llog(tlllbd/log2J
254 CHAPTER 6. OUTPUT ANALYSIS

= total number of interim reviews.

Under relatively weak conditions,

pr[JL E Cl, (8)] ---+ 1 - 8 as t ---+ 00, (6.53)

implying that Cl, (8) is an asymptotically valid confidence interval for JL. However,

as t ---+ 00, (6.54)

implying that Vt](I) does not converge in mean-square as t ---+ 00 and, therefore, lacks
the desired statistical property of consistency. Appendix A graphically illustrates how the
absence of mean-square convergence results in a sequence of estimates of a~ whose sampling
fluctuations fail to dampen as t ---+ 00.

6.6.5 ARENA'S AUTOMATIC USE OF THE BATCH-MEANS METHOD

Recall that Arena/SlMAN classifies sample-path data as time persistent or discrete

change. Queue length and number of busy servers exemplify time-persistent phenomena,
whereas waiting time exemplifies a discrete-change phenomenon. Upon termination of a
simulation run, Arena automatically applies the batch-means method in an attempt to
compute approximating 95 percent confidence intervals for the means of all time-persistent
and discrete-change phenomena declared in DSTAT and TALLIES elements (Kelton et al.
1997, p. 232). The results appear in the half-width column in its output tableau (e.g., Figure
4.14).
For the discrete-change case based on number of observations t < 320, Arena enters
(Insuf) in the column, signifying too small a sample size to implement its analysis. For
t ::: 320, it dynamically maintains

l(r) = Lr/16Lr/320JJ (6.55)

batches, after collecting observation r for 320 :::: r :::: t with

b(r) = Lr/l(r)J = Lr/Lr/16Lr/320JJJ (6.56)

observations per batch. As a consequence, I ( r) periodically cycles through the values

20,21, ... , 39 as r increases, so thatthe number of batches always remains bounded. There-
fore, the resulting Va(t) is not a consistent estimator of a~ (Section 6.62 and Appendix
A.l).
Byway ofcomparison, columns "Lower" and "Upper" in the LABATCH.2 final-tableau
display the lower (L) and upper (U) limits of the confidence interval (e.g., Figure 6.6),
 6 .6 . BAT C H - MEA N 5 MET HOD 255

whereas column "HalfWidth" in the Arena final tableau displays (U - L) /2 (e.g., Figure 4.14).
Although the latter quantity is usually called the half-length in statistical analysis, we retain
the half-width notation merely for consistency with Arena labeling.
At the end of a run of t observations, Arena tests the batches for independence using
the procedure in Section 6.6.8. If rejected, Arena enters (Corr) in the Half Width column,
signifying that the batches are correlated. If accepted, Arena computes and enters the 95
percent half-width rZ(tJ-l(.975)jVa(t)/t, where aCt) := l(t)b(t) denotes the number of
observations used to compute Va(t). Note that the computation uses all the data if r ::: 320
and r /16 is integer. For t ::: 320, aCt) / t > .955; for t > 1920, a(t) / t > .99.
Let M ( T ) denote the number of value changes in a time-persistent phenomenon such
as queue length, in a simulated time interval (0, r]. For a sample path of t simulated time
units, Arena enters (Insuf) in the Half Width column if M (t) < 320. Otherwise, it maintains

I(M(r)) = LM(r)/16LM(T)/320JJ (6.57)

batches at simulated time r ::: inf[z ::: 0 : M(z) ::: 320] with

b(M(r)) = LM(r)/LM(r)/16LM(T)/320JJJ (6.58)

value changes per batch. IfH is rejected, it enters (Corr) in the HalfWidth column. Otherwise,
it enters rZ(t)-1 (.975)jVa(t)/ t, where a(t) = LI(M(t) )b(M(t))J is the number of observa-
tions used to compute Va(t). As before, 20 :.s I(M(r)) :.s 39, and t'(M(t))/M(t) ::: .955
for M(t) ::: 320 and::: .99 for M(t) ::: 1920.
For the bounded number of batches in the Arena method, expressions (6.50) and
(6.51) imply that Va(t) has coefficient of variation

jvarVa(t) 2[I(t) + 1]
y(Va(t)):= EVa(t) --+ [l(t) - 1]2 as t --+ 00,

where

2[I(t) + 1]
.2354:.s [let) _ 1]2 :.s .3411.

This suggests persistent and relatively substantial fluctuations in Va(t) regardless of how
large t is. An analogous result holds for Vt'(M(t)). Section 6.9 describes how an Arena user can
specify batch size and confidence level and perform a post-simulation batch-means analysis
on stored sample data. It also discusses the differences between Arena's and LABATCH.2's
use of the batch-means method.
256 CHAPTER 6. OUTPUT ANALYSIS

6.6.6 SQRT RULE

Within LABATCH.2, the SQRT rule offers an alternative that ensures convergence for
Vt.ltdefines {Uj' bj), j ~ I} such that lj+l/lj == ..tiandbj+Ifbj == ..ti.Morespecifically,
for given II and bl> it sets

lz = II := L..tih + .5J, (6.59)

~ b, ,~{
ifb l = 1,
b, :"ru" + .5J ifb l > 1,
(6.60)

lj+1 = 2Ij -i>

bj+1 = 2b j _ 1 j = 2, 3, ....

Choosing (/1, bl)from 13 in Table 6.3 ensures that 2l l b l = llbl> so that tj = ljb j = 2j - l l l b l
and, therefore, t j+ If t j = 2, as desired.
Appendix A.2 describes the principal steps for the SQRT rule for j = 1, ... , J(t).
Under this rule, Vt ](,) --+ 0 as t --+ 00, ensuring the desired mean-square convergence. Also,
the rule has an optimal property with regard to the standardized statistic

X t - jL
(6.61)
Zlb:= JWlb/ 1

with d.f. Fib. Appendix A.2 shows that l(t) ()( t l / 2 and b(t) ()( t l / 2 ensures the fastest
convergence of Flb(Z) to 4>(z) for each Z E (-00,00). Since for large t we use the normal
d.f. 4>, in place of the unknown d.f. Fib, to compute a confidence interval for jL, this last
property implies that the actual coverage rate for confidence intervals converges to the desired
theoretical coverage rate 1 - 8 as rapidly as possible.
AppendixA.2 also illustrates how the rule works for the MIMll example and compares
its performance with that of the FNB rule. The comparison especially in Figure 6.8a reveals
that the FNB rule offers an initial benefit but the SQRT rule ultimately offers the advantage
of mean-square convergence.
If the specified sample path length is t = 201 - IZl b l for ot > 1 and (/1, bd as in Table
6.3, then a(t) = t. Recall that a(t) is the number of observations used by the batch-means
method (Section 6.6). More generally, for any t ~ 500, LABATCH.2 automatically selects
(/1, b l ) such that a(t)/t is maximal. This maximum always exceeds .89. See Appendix A.3.

6.6.7 LBATCH AND ABATCH RULES

The FNB and SQRT rules each have desirable and undesirable features. By employing
a test for independence to choose between them on successive reviews, LABATCH.2 exploits
the better properties of each and avoids their most serious limitations. It does this by means
of two hybrid rules LBATCH and ABATCH.
 6.6. BATCH-MEANS METHOD 257

Table 6.3 B := {(iI, b1 ) : 1 :s b 1 :s 11 :s 100, 2/ 1b1 = f1hd"

b1 11 b1 11

! ~ 38 54
2 3,6 39 55
3 4 41 58,87
4 6 43 61
5 Z,14,21,28 44 62,93
Z 10, 15,20,25,30,35 46 65
~ 11 48 51,68,85
9 13 49 69
10 14,21,28 50 71
12 17,34,51,68,85 51 60,72,84,96
14 15,20,25,30,35 53 75
15 21,28 54 57
17 24,36,48,60,72,84,96 55 78
19 27,54 56 79
20 21,28 58 82
21 25,30,35 60 68,85
22 n,62,93 61 86
24 34,51,68,85 62 66
25 28 63 89
26 37 65 92
27 38,57 66 93
28 30,35 67 95
29 41,82 68 72,84,96
31 44,66,88 70 99
32 45 72 85
33 47 82 87
34 36,48,60,72,84,96 84 85
36 51,68,85 85 96
37 52 88 93
af1 is defined in expression (6.59) and hI in expression (6.60).
For underlined quantities, see Section 6.7.4.

Let Ho denote the hypothesis: On review j) the I j batches Y 1bj , ... , Yljb j are i.i.d. The
LBATCH and ABATCH rules both use the outcome of a test of Ho to switch between the
FNB and SQRT rules on successive reviews. The effect is to retain the desirable properties
of each basic rule while reducing the influence of their limitations. The principal features of
these hybrid rules are:

LBATCH RULE

• Start with the FNB rule on review 1.

• For j :::: I, ifHo is rejected on review j, use the FNB rule on review j + 1.
• Once Ho is accepted on review j, use the SQRT rule on reviews, j + 1, j + 2, ....
258 C HAP TE R 6. 0 U T PUT A N A L Y SIS

Ca) Wa iting time in queue, (500 = 189.5

350
... ,
I
..
\
300 . \
\

250 ·

-------------------------------- ~

100 -
•
50 -

o ~~~~=-----------------------------~
o 5 10 15 20
Interim review j

(b) Probability of waiting, (500 = 130.8

2

1.75

1.5

1.25 ~
~
J
fm 1
EC"
In 0.75 ~

0.5 ~

0.25 :

0
0 5 10 15 20
Interim review J

Figure 6.8 LABATCH.2 estimates of a oo (ABATCH rule: heavy line, FNB rule: thin
dashed line, SQRT rule: thin line, a oo = 189.5: heavy dashed line)
 6 .6 . BAT C H - MEA N S MET HOD 259

and:

ABATCH RULE

• Start with the FNB rule on review 1.

• For j 2: 1, ifHo is rejected on review j, use the FNB rule on review j + l.

• For j 2: 1, ifHo is accepted on review j, use the SQRT rule on review j + l.

By initially fixing Z, the LBATCH rule allows batch size, b, to increase at the maximal
rate when Ho is rejected, thus dissipating systematic error in Vlj as fast as possible. Once
Ho is accepted, the rule switches to the SQRT rule to dissipate the error in coverage rate as
rapidly as possible. By testing Ho on every review, the ABATCH rule takes into consideration
the possibility of Type II errors on successive replications. In the present context, Type II
error denotes the random acceptance of Ho when it is false. The LBATCH rule ignores this
source of error, whereas the ABATCH rule allows the procedure to correct itself. Because of
this self-correcting property, we favor the ABATCH rule.
As a consequence of this testing, batch size and number of batches on review j (> 1)
are random. To acknowledge this property, we denote them by B j and L j respectively. Let
K j denote the number of test rejections of Ho on reviews 1, ... , j. Then the LBATCH and
ABATCH rules induce

2(j-K j )/2Z I

~
if j - K j is even,
Lj { 2(j-Kj -I)/2f[ (6.62)
if j - K j is odd,

and

2(j+K j -2)/2 b i

~
if j - K j is even,
Bj { 2(j+K j -I)/2b I (6.63)
if j - K j is odd,

so thattj = L jBj = 2 j - I ZI b i for j = 1,2, ....

Since {( L j, B j ), j 2: I} is a random rather than a deterministic batching sequence,
Theorem 6.5 does not apply directly. We need additional qualifications to achieve the desired
limiting results:

Theorem 6.6 (Yarberry 1993, Fishman and Yarberry 1997).

If to'} ~ a';' as t ~ 00 and ASA holds, 2Z I b i = l[bI> and there exist constants r < 1
and () > 1 - 4A such that L~I pr(Kj Ij > r) < 00 and K j Ij ~ () as j ~ 00 w.p.i, then
260 CHAPTER 6. OUTPUT ANALYSIS

and

asj ~ 00.

Recall that aCt) = tJ(t), where J(t) is defined in Section 6.6.4. Provided that its
conditions are met, Theorem 6.6 gives a strongly consistent estimator of O'~ and the basis
for the asymptotic validity of the confidence interval

(6.64)

We conjecture that there is a large class of problems that satisfy these conditions.
On review J(t), LABATCH.2 actually reports the confidence interval eLJ(t) (8) in
expression (6.48), which differs from VLJ(t) (8) principally by substituting Xt , based on
all the data, in place of Xa(t), based on the first aCt) observations. If aCt) = t, then
C LJ (t)(8) = V LJ (t)(8) and Theorem 6.6 applies. If t > aCt), then we need to reconcile
the difference between CLJ(r) (8) and VLJ(t) (8) with regard to asymptotic validity. We do this
in Appendix A.3.
For purposes of comparison, Figure 6.8 displays {J
B j W LjBj ; j = 1, ... , J(107)}
for the ABATCH, FNB, and the SQRT rules. The dissipation of systematic error under the
ABATCH rule mirrors that under the FNB rule, but its sampling fluctuations are considerably
more attenuated.

6.6.8 TESTING Ho

Because of stationarity (Section 6.5), Y1b, ... , Ylb have the autocorrelation function

To test Ho on the batch averages Y1b, ... , Ylb, we use the von Neumann ratio (von
Neumann 1941, Young 1941, Fishman 1978, 1996)

(6.65)

i=1
 6.6. BATCH-MEANS METHOD 261

where, because of stationarity (Section 6.5),

estimates the first lagged autocorrelation WI. Inspection reveals that Cz is itself an estimator of
WI adjusted for end effects that diminish in importance as the number of batches I increases.
If {Xi, i ::: O} has a monotone nonincreasing autocorrelation function (e.g., Figure 6.4a),
then in the present setting WI is positive and decreases monotonically to zero as batch size b
increases. If for some I and b, Ho is true for Y lb , ... , Y Zb , then WI = 0, and more generally,
the monotone property implies corr(Yib , Y jb ) = 0 for all i =f j.
If Y1b-fL, ••• , Ylb-fL are i.i.d. from N(O, 1), then under Ho, Cz has mean zero, vari-
Ub lIb
ance (I - 2)1(12 - 1), and a distribution that is remarkably close to normal for I as small
as eight. For i.i.d. nonnormal Y lb , ... , YZb under H o, Cz has mean zero. As I increases,
[(12 - 1)/(1 - 2)] x var Cz --+ 1 and the skewness, E(Cz - Ecd I[E(C z - ECZ)2j3/2, and
excess kurtosis, E( Cz - ECZ)4 1[E( Cz - ECz )2]2 - 3, converge to zero.
Under the assumption of a monotone nonincreasing autocorrelation function for
{Xi, i ::: OJ, a one-sided test of size f3 is in order. In particular, if Cz ::: <1>-1 (1 -
(3)J(I- 2)/([2 - 1), Ho is accepted. Otherwise, it is rejected. The one-sided test protects
against accepting Ho when the sample autocorrelation, Cz, between Y ib and Yjb for all
Ii - j I = 1 is "too large" to be attributed to sampling variation.
For the ABATCH rule, Figure 6.9a shows {L j; j = 1, ... , l(l07)} and Figure 6.9b
displays

j=I, ... ,19,

which we call the p-value sequence. In particular, Pj provides a credibility index on review
j by estimating the probability of observing a C L j' under H o, larger than the one computed
from Y IBj ' ... , Y LjBj • A small P j implies low credibility for Ho. LABATCH.2 makes its
decision on review j as follows. If Pj < f3 on review j, for example for a user-specified
f3 = .10, then LABATCH.2 rejects Ho and uses the FNB rule on review j + 1. This implies
Lj+1 = L j , as Figure 6.9a shows for j = 4 through 8,11, and 16. If Pj ::: f3, then
LABATCH.2 uses the SQRT rule on review j + 1, so that L j+t! L j == ../2. Figure 6.9a
illustrates this behavior for j = 1,2,3,9,10,12 through 15, 17, and 18. The long run
of successes beginning on review 12 suggests that batch sizes greater than El2 = 10240
make the residual correlation between batches negligible. When the LBATCH rule is used,
LABATCH.2 makes its acceptanceirejection decisions according to that rule.
Suppose that {Xd has an autocorrelation function that exhibits damped harmonic
behavior around the zero axis (e.g., Figure 6.4b), or that {Xd is an i.i.d. sequence. In the
damped harmonic case, the one-sided test accepts Ho when negative correlation is present.
262 C HAP TE R 6. 0 U T PUT A N A LY 5 I 5

(a) Number of batches

300 ·

250 ·

200 .
Number
of
batches 150 ·

100 .

50 ·

0
0 5 10 15 20
Interim review J

(b) p-value for testing H Q , p= .10

0.8 ·

0.6 ·
p·value
lor Ho
0.4 ·

0.2 ·

0'0~-------=~----~--~1~0'---------'1~
5 ----------~2~
0

Interim review J

Figure 6.9 LABATCH.2 output for Series 1 using the ABATCH rule

In the ij.d. case, review J (t) produces final results with B (t) > 1 when a unit batch size
suffices and gives a statistically better estimate of a ~. LABATCH.2 provides information that
enables a simulationist to infer the presence of either of these cases and to take advantage of
this knowledge.
 6.6. BATCH-MEANS METHOD 263

Once LABATCH.2 completes its last review J(t), it computes the p-value

P: := 1- <I>( V~
t=:lC; ) ,

where

W:. 1 := --~1 ~ (Xi -Xt

- )2 ,
t - 1 i=1

for the hypothesis Ho applied to the original observations XJ, ... , Xt. It also computes an
approximating 100 x (1 - 8) percent confidence interval

(6.66)

and sample standard deviation Jw:. 1 and displays them in the Interim Review Tableau for
a series below the title "If data are independent:' For example, see Figure 6.6, where the
p-value for each series is 0.0000, thereby rejecting Ho.
Whenever this final row displays P:
> {3, the specified significance level for testing
H o, three possibilities arise:

• {Xd is an i.i.d. sequence

• {Xi} has a damped harmonic autocorrelation function
• This p-value merely exhibits a random aberration.
To investigate these possibilities we proceed as follows. If P: > {3, and the sequence
{JBjWLjB j ; j = 1, ... , J(t)} of estimates a oo , listed under Sqrt[B*W(L, B)], fluctu-
ates around JWt •1 displaying no discernible trend, then it is not unreasonable to assume
that {Xd is ani.i.d. sequence, to H o, and to use the confidence interval (6.66). This approach
offers a statistically better estimate of a oo and greater degrees of freedom.
If P: > {3 but {J Bj WLjB j ; j = 1, ... , J(t)} tends to decrease with increasing j
and, in particular, BJ(t) WLJ(t)BJ(t) » W:. 1 , then there is reason to believe that {Xi} has an
autocorrelation function that fluctuates around zero, implying the presence of negative as
well as positive correlation. In this case, using the confidence interval for review J (t) to reap
the potential benefits of the negative correlation (e.g., see Section 6.5), usually inducing a
shorter interval width.
If P: > {3 but {J"'B=-jW;;-;-L-jB-j ; j = 1, ... , J (t ) } tends to increase with increasing j and,
in particular, BJ(t) WLJ(t)BJ(t) » Wt.1> then there is reason to believe that the p-value reflects
a random aberration.
264 CHAPTER 6. OUTPUT ANALYSIS

In summary, while LABATCH.2 automatically provides its Final Tableau based on the
concept of asymptotically valid confidence intervals, a perusal of the entries in the row below
"If the data are independent" for each series offers an opportunity for obtaining a tighter
confidence interval.

6.6.9 ASSESSING THE ADEQUACY OF THE WARM-UP INTERVAL

LABATCH.2 also provides an assessment of the extent of warm-up bias in Xt • We

again use the M/M/l simulation, but this time with an arrival to an empty and idle system
and data collection beginning with X I := waiting time of the first arrival. For this scenario,
XI = 0 w.p.1 and XI, X 2 , ••• , X t is stationary, only asymptotically (as t --+ (0). This is a
more biased environment than one would expect to encounter when data collection begins
after a user-specified warm-up interval.
Figure 6.10 displays the 99 percent confidence intervals for mean waiting time in queue,
taken from the LABATCH.2 interim review tableau for Series 1 for this run. It suggests little
bias after review 11, which corresponds to 211 x LI X BI = 2048 x 7 x 5 = 71,680
observations.
Suppose the user had specified t = 4480 for the sample-path length. Then LABATCH.2
would have generated the same sample averages and confidence intervals for LI = 7 and
B I = 5, but only forthe first seven interim reviews. Moreover, a display of these results would
have aroused considerably more concern about the dissipation of bias in Xt • Interestingly, had
we taken t = 4480 in our first simulation, which began in an equilibrium state, we might
equally be suspicious of Xt based on the path it displays in Figure 6.5 for j = 1, ... ,7.

20 ·

15 -

. ····l·· TT ··rt+·
Mean
waiting
time 10

··I··t·t.H..•..•...
5

o ~~------~-------------- ________________~_
o 5 10 15 20
Interim review j

Figure 6.10 LABATCH.2 Sample means and 99 percent confidence intervals for
Series 1 using the ABATCH rule; simulation starts in empty and idle state
 6.7. LABATCH.2265

However, this observation in no way mitigates the value of the assessment when we know
for a fact that Xt may contain systematic bias as a result of starting in an arbitrarily chosen
state and possibly choosing an inadequate warm-up interval.

6.7 LABATCH.2

LABATCH.2 is a collection of computer programs that perform statistical analyses on sam-

ple sequences collected on strictly stationary stochastic processes. It offers two modes of
implementation: One integrates LABATCH.2 into an executing data-generating program
(e.g., simulation) to analyze the evolving data on repeated subroutine calls; the other takes
data from an existing file as input. In addition to minimizing user effort, the first option
considerably reduces space requirements. It also allows user interaction with the executing
program via screen displays of interim estimates. The second option permits statistical anal-
ysis of stored data, regardless of source and date of generation, thereby making LABATCH.2
applicable in a considerably wider range of data-generating environments.
LABATCH.2 is a revision ofLABATCH (Fishman 1996, Fishman and Yarberry 1997)
that considerably simplifies its implementation and use. A user merely inserts a single subrou-
tine call statement in her/his main program and assigns values to several control arguments
of the subroutine.
This section discusses several major features ofLABATCH.2. C, FORTRAN, and SIM-
SCRIPT 11.5 implementations of LABATCH.2 are obtainable by anonymous file transfer
protocol (ftp) at http://www.or.unc.edu/~gfish/labatch.2.html.
For any path length t :::: 20, LABATCH.2 automatically computes the number of
batches, LI> and the batch size, BI> to be used in its first review. For example, it chose LI = 7
and BI = 5 for t = 107 for the MIM/l example.
C
A call from a FORTRAN main program to
SIMSCRIPT u.s

BATCH~EANS(IN_UNIT,OUT _UNIT,T,S~UM,PSLVECTOR, (6.67)

DELTA,RULE, BETA,LUPPER,SCREEN)

implements LABATCH.2, where Table 6.4 defines its arguments. As an example, suppose
they assume the values

IN-UNIT = 0,
OULUNIT = 15,
T = 1000,
S_NUM = 2,
PSLVECTOR = pointer to data vector,
DELTA = .01, (6.68)
266 CHAPTER 6. OUTPUT ANALYSIS

Table 6.4 LABATCH.2 arguments

User-specified Definition Notation

input in Chapter

IN-UNIT := 0 if data are repeatedly to be

transferred from a main program

:= 30 if data are to be read one vector

c.30
at a time from input file fort.30
SIMU30
(30 is merely an example)

OUT _UNIT := designated unit for writing output

T := total number of observations

S_NUM := number of sample series

PSLVECTOR := pointer to vector of S_ NUM sample values

DELTA := desired significance level for

confidence intervals (.01 or .05 suggested)

RULE := 1 if ABATCH rule is used

:= 0 if LBATCH rule is used

BETA := significance level for testing for independence

(.10 suggested)
If BETA = 1, each review uses the FNB rule
= -1, each review uses the SQRT rule

LUPPER := upper bound on initial number of batches

(3 <= LUPPER <= 100)

SCREEN := 1 if interim review estimates are to be

displayed on the screen
:= 0 otherwise

RULE = 1,
BETA = .10,
LUPPER= 30,
SCREEN = O.

Then LABATCH.2 processes S~UM=2 sequences in T = 1000 iterative calls from

the user's main program, computes 100 x (1 - DELTA}=99 percent confidence inter-
c.lS
vals for the sample averages, and writes the output to a file called fort.lS (IN_UNIT=O,
SIMU1S
OUT_UNIT=IS). RULE=1 causes LABATCH.2 to employ the ABATCH rule to determine
batch size on each review, and BETA = .10 causes it to test for independence of batch av-
 6.7. LABATCH.2267

erages at the .10 signifiance level. LABATCH.2 begins its first iteration with the number of
batches no greater than LUPPER=30. SCREEN=O suppresses the interactive screen feature.
In practice, all but the values of T and S_NUM can be set once and the subroutine used
repeatedly in different settings.
Testing (Section 6.6.7) determines whether the batch size on review j + 1 increases
by a factor of 2 (rejection) or approximately -Ji (success). A doubling of batch size aims at
reducing any residual systematic variance error detected on review j as fast as possible. A -Ji
increase signals that the number of batches is also increasing approximately by a -Ji factor.
This growth in both batch size and number of batches as the sample path length grows is
a necessary condition for obtaining a strongly consistent estimator of a~. Our subsequent
description ofLABATCH.2 uses the definitions in Table 6.4.

6.7.1 TWO MODALITIES

As already mentioned, LABATCH.2 provides two ways of accessing data for statis-
tical analysis. One requires the user to insert a call statement into the data-generating
program that executes the call each time it produces a new data vector. Calling and exe-
cuting BATCH_MEANS each ofT times that a data vector with S_NUM entries is generated
results in O(S_NUMxT) computing time and O(S_NUMx log2 T) space being used to gen-
erate the LABATCH.2 output. Both complexities arise from choosing rules that cause either
Bj+1 = 2Bj or Bj+1 ~ -JiBj on successive reviews j = 1,2, .... The space bound is par-
ticularly appealing when T is large. Yarberry (1993) and Alexopoulos et al. (1997) describe
the basis for these complexities.
The other option allows LABATCH.2 to read its data from a file, giving the software a
considerably broader range of application than merely for in-line generated sample records.
We illustrate how the in-line option works in the context of the M/M/1 example, but stress
the applicability of the approach to stored sample data.
If IN_UNIT = 30 and OUT _UNIT = 15 (Table 6.4), then a main program needs
to call BATCH-11EANS just once to cause LABATCH.2 to read its sample data from a file
c.30 c.IS
called fort.30 and to write its output to for1.lS • While the O(S_NUMxT) time and
SIMU30 SIMUIS
O(S_NUMx log2 T) space bounds remain for LABATCH.2, they do not tell the whole story.
In particular, the input file requires space proportional to S_NUM x T, which for S_NUM as
small as 1 can be substantial for sufficiently large T.
Programs written in C, FORTRAN, or SIMSCRIPT II.s can implement the second
option without qualification. Any program that provides access to and execution of a C,
FORTRAN, or SIMSCRIPT II.S subroutine can take advantage of the first option. In a sim-
ulation environment, a user-written program in a language that provides standard linkages
for incorporating a subroutine generally consumes less time calling BATCH-11EANS and
analyzing data than a program generated at the icon level in a point-and-click environment.
268 CHAPTER 6. OUTPUT ANALYSIS

6.7.2 INTERACTING WITH THE ANALYSIS

In queueing simulations, O'~ usually increases superlinearly with traffic intensity. In

this setting, LABATCH.2 offers a way ofviewing interim results that enables the user to assess
the quality of the O'~ estimates during a run. If the interim results suggest that an acceptable
level of accuracy has been achieved, then LABATCH.2 allows the user to terminate the
sampling experiment at that point, thus saving computing time.
If SCREEN = 0 in expression (6.67), then LABATCH.2 performs as already described.
However, if SCREEN = 1, then after executing interim review j, LABATCH.2 displays on
the screen Xt}, the sample coefficient of variation of Xt}, sometimes called the relative error,
and JBj W(L j , B j ), for Series i in column i + 1 for i = 1, ... , min(6, S-.NUM), where
screen size dictates the upper limit on i. LABATCH.2 then asks the user whether he/she
wishes to continue. Recall that JBj W(lj, Bj) is an estimate of 0'00'
Figure 6.11 displays the tableau for the steady-state M/M/l simulation, where column 2
shows the sequence of estimates for Series 1 and column 3 does likewise for Series 2. Ifthe user
J
concludes that B j W (L j, B j) has stabilized in each column, so that the systematic errors
have become negligible, and that the coefficients of variation C.V.(Xt }) are sufficiently small,
then he/she may terminate the simulation by typing "n". This action causes LABATCH.2 to
compute confidence intervals for the S_NUM and to write both the final and interim review
tableaus (Figure 6.7) to file OUT _UNIT. If this action occurs immediately after the on -screen
display, for review j, then the final tableau, as well as the interim review tableaus, use L j B j
observations. If the screen display suggests that systematic error remains, then, provided
that L j B j < t, typing "y" causes LABATCH.2 to collect additional data, to perform the
next review, to display JBj+l W(L j +1 , Bj+d on screen, and to ask the user whether he/she
wishes to continue. If L j B j = a (t) and the user types "y'~ then the simulation goes to
completion and LABATCH.2 uses all t = T observations for the sample averages and the
first aCt) observations for the final BJ(t) W(LJ(t), BJ(t»)'s.
As illustration of how this procedure works in practice, consider review 11 in Fig-
ure 6.11, where C.V.(X..BAR) = .1255 for mean waiting time (Series 1) and C.V.(X..BAR) =
.008205 for probability of waiting (Series 2) may encourage one to conclude that the sample
averages (X..BAR) are statistically reliable. By constrast, the variability in the Sqrt [B*W(L,B) 1
estimates of 0'00 for reviews 9 through 11 encourages continuation of the experiment; hence
"y" in response to the query produces the output on review 12.
By way of qualification we encourage the reader to recognize that this approach differs
substantially from the sequential mean estimation approach of Chow and Robbins (1965),
as described in Fishman (1978) and Law and Kelton (1991). Whereas the objective there is
to estimate tL to within a specified accuracy with specified probability, the objective of the
interactive approach here is to allow a user to assess the quality of variance estimates while
the sampling experiment is executing.
Kelton et al. (1998, p. 233) assert that a check is possible on half-width during Arena
execution using the THALF(·) system function. However, the cited Section 11.5 and, in
particular, Subsection 11.5.2 describe a different but related problem.
 6 .7. LAB ATe H . 2 269

X_BAR, C.V. (X_BAR) , and Sqrt[B*W(L,B)] for Series 1 Through 2

**************************************************************
No. of
Reviev Obs. 1 2

1 35 X_BAR 0.1660D+020.1000D+01
C. V. (X_BAR) 0.4557D-01 O.OOOOD+OO
Sqrt [B*W(L,B)] 0.4475D+01 O.OOOOD+OO
continue[y/n]? y
2 70 X_BAR 0.1810D+020.1000D+01
C. V. (X_BAR) 0.3993D-01 O.OOOOD+OO
Sqrt[B*W(L,B)] 0.6047D+01 O.OOOOD+OO
continue[y/n]? y
3 140 X_BAR 0.1914D+02 0.1000D+01
C. V. (X_BAR) 0.3833D-01 O.OOOOD+OO
Sqrt[B*W(L,B)] 0.8680D+01 O.OOOOD+OO
continue[y/n]? y
4 280 X_BAR 0.1886D+02 0.1000D+01
C. V. (X_BAR) 0.4158D-01 O.OOOOD+OO
Sqrt[B*W(L,B)] 0.1313D+02 O.OOOOD+OO
continue[y/n]? y
5 560 X_BAR 0.1111D+020.9250D+00
C. V. (X_BAR) 0.1686D+00 0.4118D-01
Sqrt[B*W(L,B)] 0.4432D+02 0.9014D+00
continue[y/n]? y
6 1120 X_BAR 0.7254D+010.8812D+00
C. V. (X_BAR) 0.2174D+00 0.3415D-01
Sqrt [B*W(L,B)] 0.5278D+02 0.1007D+01
continue[y/n]? y
7 2240 X_BAR 0.6706D+01 0.8862D+00
C.V.(X_BAR) O.1952D+OO O.4135D-Ol
Sqrt[B*W(L,B)] 0.6196D+02 0.1734D+01
continue [y/n] ? y
8 4480 X_BAR 0.7556D+01 0.8830D+00
C. V. (X_BAR) 0.1647D+00 0.2981D-01
Sqrt[B*W(L,B)] 0.8328D+02 0.1762D+01
continue[y/n]? y
9 8960 X_BAR 0.6747D+01 0.8770D+00
C. V. (X_BAR) 0.1366D+00 0.1553D-Ol
Sqrt[B*W(L,B)] 0.8721D+02 0.1289D+01
continue[y/n]? y
10 17920 X_BAR 0.7817D+01 0.8888D+00
C. V. (X_BAR) 0.1080D+00 0.1106D-01
Sqrt [B*W(L,B)] 0.1130D+03 0.1316D+01
continue[y/n]? y
11 35840 X_BAR 0.9513D+01 0.9001D+00
C. V. (X_BAR) 0.1255D+00 0.8205D-02
Sqrt[B*W(L,B)] 0.2260D+030.1398D+01
continue[y/n]? y
12 71680 X_BAR O.9668D+01 0.8973D+00
C. V. (X_BAR) 0.1136D+00 0.6581D-02
Sqrt [B*W (L, B)] 0.2941D+03 0.1581D+01
continue [y/n] ?

Figure 6.11 LABATCH.2 screen display when SCREEN = 1

270 C HAP TE R 6. 0 U T PUT A N A L Y SIS

6.7.3 IMPLEMENTATION

As mentioned in the introduction, C, FORTRAN, and SIMSCRIPT 11.5 versions of

LABATCH.2 are available on the Internet. The statement

BATCH~EANS(IN_UNIT,OUT _UNIT,S_NUM,T,DATA,DELTA,RULE,BETA,

LUPPER,SCREEN);

in a C program calls BATCH~EANS.1t requires that a previously defined double S_NUM xl

array, DATA, contain the latest data entries. Table 6.4 defines all remaining arguments. If nu-
merical values replace IN_UNIT, OULUNIT, S_NUM, T, DELTA, RULE, BETA, LUPPER,
and SCREEN, then no need exists to define these variables in the calling program. This also
applies to the FORTRAN and SIMSCRIPT 11.5 calls.
The statement

call BATCH~EANS(IN _UNIT,OUT _UNIT,S_NUM,T,DATA,

DELTA,RULE,BETA,LUPPER,SCREEN)

in a FORTRAN program calls BATCH~EANS. It requires that a previously defined double

precision S_NUM x 1 array, DATA, contain the latest data entries. The character @ in column
6 indicates a continuation.
The statement

call BATCH~EANS(IN_UNIT,OUT _UNIT,T,S_NUM,DATA(*),

DELTA,RULE,BETA,LUPPER,SCREEN)

in a SIMSCRIPT 11.5 program calls BATCH~EANS. In addition to reserving space for the
double S_NUM x 1 array DATA in the main program, a user must also include the statements

define DATA as I-dim double array

define PHI as double function
define PHLQUANTILE as double function
define STUDENT_T_QUANTILE as double function

in the preamble. See CACI (1983, p. 91).

6.7.4 USER-SPECIFIED II AND b l

LABATCH.2 also allows a user to choose any underlined (II, b l ) in Table 6.3 as the initial
number of batches and the initial batch size by specifying a sample path length T = 2a l l b l
for the selected II and bl and where a is a positive integer. In this case, LABATCH.2 uses 100
percent of the data to compute the final BW(L,B).
 6.8 . A I RLIN ERE 5 E RV A T ION PRO B L EM 271

6.7.5 CODING

As mentioned earlier, LABATCH.2 is a revision of the LABATCH statistical analysis

package. The coding of LABATCH provides users with a considerable number of options.
For example, it allows a user to specify whether headings are to be printed, and for each series
it allows a user to specify different significance levels for testing H and different confidence
levels for the means.
To reduce the burden of decision making that a potential user faces, these options have
been severely curtailed in LABATCH.2. However, a perusal of the code makes clear that the
code and data structures that support these wider choices remain in place, but are suppressed.
This was done to reduce the possibility of error in modifying the code and to leave structures
in place that may become useful once again in subsequent revisions of LABATCH.2.

6.8 AIRLINE RESERVATION PROBLEM

We now apply LABATCH.2 to the airline reservation problem. Our interest is in estimating
the mean and 11 ordinates of the distribution function of waiting time. Hereafter, we refer
to each sample sequence for estimating a parameter as a series. The 12 series of interest are:

Xi
Parameter Datum for
Series to be estimated ith observation

J-Lw Wi
2 pr(W > 0) 1(0.00) (Wi)
3 pr(W> 1) 1(1.00) (Wi)
4 pr(W > 2) 1(2.00) (Wi )
5 pr(W> 3) 1(3.00) (Wi )
6 pr(W > 4) 1(4.00) (Wi)
7 pr(W > 5) 1(5,00) (Wi)
8 pr(W > 6) 1(6,00) (Wi)
9 pr(W > 7) 1(7,00) (Wi)
10 pr(W> 8) 1(8,00) (Wi)
11 pr(W > 9) 1(9,00) (Wi)
12 pr(W> 10) 1(10.00) (Wi ).

To make the LABATCH.2 software operational within the AIKEV.sim program in

Figure 4.2, one modifies the program with:

Step 1. preamble
Between lines 33 and 34, insert:
"Vector for data transfer to batch_means
"
272 C HAP TE R 6. 0 U T PUT A N A LY 5 I 5

define DATA as I-dim double array

"Declarations for functions called by BATCH~EANS

define PHI as double function

define PHLQUANTILE as double function
define STUDENT_T_QUANTILE as double function

Step 2. main
Between lines 1 and 2, insert:
reserve DATA(*) as 12

Step 3. ACCOUNTING
Between lines 9 and 10, insert:

for 1=1 to 12 DATA(I)=O.

DATA(I)= WAITING. TIME
ifDATA(l) > 0
DATA(2) = 1
endif
ifDATA(l) > 1
DATA(3) = 1
endif
ifDATA(l) > 2
DATA(4) = 1
endif
ifDATA(l) > 3
DATA(S) =1
endif
ifDATA(l) > 4
DATA(6) = 1
endif
if DATA(l) > 5
DATA(7) = 1
endif
ifDATA(l) > 6
DATA(8) = 1
endif
ifDATA(l) > 7
DATA(9) = 1
endif
ifDATA(l) > 8
DATA(lO) = 1
endif
ifDATA(l) > 9
 6 .9 . BAT C H MEA N 5 I N ARE N A 273

DATA(ll) = 1
endif
if DATA(1) > 10
DATA(12) = 1
endif

call BATCH~EANS(O,15,T,12,DATA(*),.0I,I,.10,30,O)

The call to the BATCH~EANS routine transfers the pointer to the DATA vector with 12
entries for analysis.
Figure 6.12a displays the final tableau for a run with warm-up interval K = 200 and
sample path length T = 2 X 106 . This choice ofT induced LABATCH.2 to choose 11 = 13
and b 1 = 9 and to produce results on J(T) = 15 interim reviews. Figure 6.13 shows graphs
for JBW(L,B) based on the interim review tableaus for Series 1 through 12. All graphs
reveal the dissipation of systematic variance error well before review 15. Figure 6.14 shows
the final point estimates and corresponding confidence intervals for {pr(W > w); w =
0, 1, ... , 10 minutes}.
Figure 6.12b offers a particularly revealing illustration of how convergence occurs for
Xtj = X and [W(L(tj)' B(tj))f /2 = [W(L,B)j1/ 2 for j = 6, ... ,15 based on entries in the
LABATCH.2 tableau for waiting time (series O. It shows how both quantities converge to a
relatively small neighborhood as sample-path length increases. Note that t6 = 9 x 13 X 25 =
3744, whereas tIS = 9 x 13 X 214 = 1,916,928. We began at review 6, rather than review
I, to focus on the ultimate convergence.
Our inclusion of Figures 6.12 through 6.14 here and Figures 6.5 through 6.8 in Section
6.6.1 is intended to inform the reader about the potential uses of LABATCH.2 output.
Since Practical considerations may limit how much output for each data sequence can be
presented and digested, we recommend, as a minimum, a display as in Figure 6.12a and, for
each sequence, a display as in Figure 6.12b.

6.9 BATCH MEANS IN ARENA

Section 6.6.5 describes how Arena automatically employs the batch-means method with
Student's t distribution to compute an approximating 95 percent half-width for each mean
of interest. For t = 2 x 106 waiting times, execution of the Arena airline reservation program
in Figures 4.12 and 4.13 yielded a sample average waiting time of 4.229 and 95 percent half-
width of .1309. For queue length it gave a sample average of 4.1189 and 95 percent half-width
of .1295. For this t, expression (6.55) implies only l(t) = 20 batches but b(t) = 100,000
observations per batch.
Arena also provides an option for a post-simulation batch-means analysis for user-
specified batch size and confidence level. As illustration, consider the WaiLTime.DAT file
274 CHAPTER 6. OUTPUT ANALYSIS

declared in Figure 4.13. At the end of a simulation run, a user clicks on "Output" on the Tools
menu, on "Analyze;' and on "Batch/Truncate Obs'vns ... ," and then enters WaiLTime.DAT,
as the file name, the desired number of observations to be truncated, the desired batch size,
and the name of the file, suffixed by .fit, in which the batch averages are to be stored. The
user again clicks on Analyze and then on Conf. Interval on Mean, and on Classical ... , and
then enters the name of the .stl file and the selected confidence level.

Final Tableau
Mean Estimation
***************
(t = 2000000 )

99.0%
Standard Error Confidence Interval
Series X sqrt[B*W(L,B)/t] Lower Upper (Upper-Lower)/IXI

1 +4. 251E+000 +4. 960E-002 +4. 122E+000 +4. 380E+000 +6.066E-002

2 +7. 339E-001 +1. 559E-003 +7. 298E-001 +7.379E-001 +1. 099E-002

3 +6.264E-001 +2.067E-003 +6. 211E-001 +6.318E-001 +1.707E-002

4 +5.282E-00l +2.454E-003 +5. 219E-001 +5. 346E-001 +2.405E-002

5 +4. 426E-001 +2. 717E-003 +4. 355E-00l +4. 496E-001 +3. 178E-002

6 +3.708E-001 +2. 849E-003 +3.635E-001 +3. 782E-001 +3. 976E-002

7 +3.106E-001 +2.909E-003 +3.030E-00l +3.181E-001 +4. 848E-002

8 +2.603E-001 +2.895E-003 +2. 528E-00l +2. 677E-001 +5. 756E-002

9 +2.178E-00l +2.844E-003 +2.104E-001 +2. 251E-001 +6. 758E-002

10 +1.829E-00l +2.762E-003 +1.757E-001 +1.900E-001 +7. 816E-002

11 +1.534E-00l +2.633E-003 +1.466E-00l +1.602E-001 +8. 884E-002

12 +1. 288E-001 +2. 498E-003 +1. 223E-001 +1.352E-001 +1.004E-001

X(t) is based on all t observations.

W(L,B) is based on first 95.85% of the t observations.

Figure 6.12 (a) LABATCH.2 final estimates for the airline reservation simulation
using the ABATCH Rule
 6 .9 . BAT C H MEA N 5 I N ARE N A 275

Since the approximating 99 percent confidence interval for waiting time (Series 1) in
Figure 6.12a is based on an LABATCH.2 determined batch size of 4608 for the SIMSCRIPT
U.s airline reservation model, the batch-means analysis for the Arena data was performed
for a batch size of b(t) = 5000, implying l(t) = 400 batches. It yielded an approximating
99 percent half-width of .1370 for waiting time, which compares to the LABATCH.2 half-
width of .1290 in Figre 6.12a for Series 1. Although based on two independent sample paths
generated by two different simulation programs, the estimates are reasonably close. However,
LABATCH.2 automatically determined its batch size, whereas a user must provide a batch
size for the Arena option.
Given the WaiLTime.DAT file, a user can overcome this limitation on knowing an
appropriate batch size by using LABATCH.2. The WaiLTime.DAT file has binary mode.
Arena provides a capability for converting these data to scientific notation and exporting
them to a new file that can serve as input to LABATCH.2. To make this conversion, a user
clicks on Tools, Output Analyzer, File, Data File, and Export, and then enters the name of
the .DAT file and name of the receiving file.
For these data with maximal initial number of batches LUPPER = 30 (Section 6.7),
LABATCH.2 computed a final 99 percent confidence interval [4.039,4.359], which gave
a half-width of .130, based on L(t) = 208 batches and B(t) = 9216 observations per
batch. The larger L(t) compared to l(t) = 20 in the automatically generated Arena 95

80
12
t ) · = 13x9x 2 j - 1
78

14
76

74
S 13
~
d. 72
~ 9
II
S 70

~ IS
CQ
,...:: 68

::r
-~

If 66

64
10

62

60
4 4.2 4 .4 4 .6 4.8 5.2 5.4

-
X,=X
J

Figure 6.12 (b) Convergencefor series 1

276 CHAPTER 6. OUTPUT ANALYSIS

(a) and (b)

r..--.
.
;,.
L
. " .. . . ..
--, .. ..

~,

i",. L
(c) and (d) ,.

--I --I

(e) and (f)

i',.

--, .. " ..

(g) and (h)

r ,.
".
L
'.
--, .. . ..
--,

(i) and (j)

i",.
,.
r- ,.

--I M .. ..
--,

(k) and (I)

r:,.
r ,.

.. ..
--, --, "

Figure 6.13 LABATCH.2 estimates of 0'00 for Series 1 through 12 in AIR-EY.sim

simulation using the ABATCH rule
 6 .9 . BAT C H MEA N 5 I N ARE N A 277

percent analysis implies that in addition to accounting for the dependence among batches,
the LABATCH.2 estimate of ta~ has smaller variance.
The .fIt file requires about 16t bytes of space to store a record of observations containing
t waiting times. This excessive space need arises because Arena also stores the times at which
each observation was collected and retains eight-byte double-precision accuracy for each
entry. In the present case 16t = 2 x 8 x 2 X 106 = 32 megabytes. In comparison, LABATCH.2
requires O(810g 2 (t» space to perform its analysis on t waiting times in double precision.
Had we also wanted to do an Arena analysis for queue length, at the .99 significance
level, the additionally required space would have substantially exceeded that used for the
waiting times because of the need to store data each time the time-persistent queue-length
variable changes value. Therefore, we do this by using the Arena automatically generated
queue-length statistics. Recall that these were a sample-average queue length 4.119 and a 95
percent half-width .1259. Multiplying the automatically displayed half-width by <1(1)-1 (1 -
8/2)/<1(1)-1 (.975) converts a 95 percent half-width to a 100 x (1 - 8) percent half-width,
where equation (6.55) defines l(t), the number of batches, and <1(1)-1 (1 - 8/2) denotes the
1 - 8/2 critical value of Student's t distribution with l (t) - 1 degrees of freedom. In the
present case, 8 = .01 gives a 99 percent half-width .1771, but based on only let) = 20
batches.
Several additional differences distinguish the Arena and LABATCH.2 results with re-
gard to interpretation. For each series, LABATCH.2 provides a Final Tableau, inclusive of all
interim reviews, from which an analyst can assess the stability of the final estimate, BW(L,B),
of a~ and therefore the stability of the corresponding half-width. Arena's approach provides

Individual 99% Confidence Intervals using ABATCH

0.8 -

0.6 -
pr(W > w)

0.4 -

0.2 -

o -O~
' ------~
2 ------~
4 ------~
6------~8--------
10
-----

Figure6.14 Estimate of {pr(W > w); w = 0, 1, ... , 1O} in AIR_EV.sim using the
ABATCHrule
278 C HAP TE R 6. 0 U T PUT A N A LY 5 I 5

no comparable basis for assessment. For reasons of space, the Final Tableaus corresponding
to the series in Figure 6.12 have been omitted here. However, the Final Tableaus in Figure
6.6 for the MIMll model, and their corresponding plots in Figure 6.7, attest to their value
for assessing stability.
A second difference relates to ratio estimators. In a simulation, such as the airline
reservation problem, executed for a fixed number of waiting times, the sample average
queue length has a random denominator and thus is a ratio estimator. LABATCH.2 can be
used to produce a confidence interval that accounts for this property (Section 6.11). Arena
produces a half-width that does not account for this property.

6.10 PREBATCHING

The LABATCH.2 with IN_UNIT = 0 requires a call to the BATCH~EANS subroutine after
each vector of observations is collected. An alternative design can be formulated that would
pass segments of each individual series at the moments of interim review, thereby reducing
the number of calls to BATCH~EANS substantially. However, this alternative requires a
space allocation to store the segment for each series between reviews. A second alternative
has more appeal. It reduces computing cost by prebatching the data. Let

c
Xj = c- 1 L Xc(j-l)+i, c = integer::: 1, j = 1,2, .... (6.69)
i=1

Suppose that Xl, X 2, ••• is the input sequence to LABATCH.2 in place of Xl, X 2, •••• Then
LABATCH.2 is executed no more than t / c times, thereby reducing computing time with no
increase in required space. However, prebatching may induce a loss of statistical efficiency in
estimating a! by inducing larger batches than the correlation, structure in {X j } warrants.
Therefore, this approach has its greatest appeal in the presence of high correlation, which,
regardless of prebatching, would inevitably induce large batches. Prebatching can reduce
cost significantly when multiple sequences are being analyzed.

6.11 RATIO ESTIMATION

Recall Option B in Sections 3.4 and 3.5, where data collection begins when job M completes
service and ends when job M + N completes service. Then expression (3.51) with s = rM
and r = rM+N - rM gives the sample average waiting time

(6.70)
 6.11. RAT I 0 EST I MAT ION 279

where N = N( T:M+N) - N( T:M), and expression (3.49) gives the sample average queue length
as

(6.71)

Since M and N are fixed, W( T:M, T:M+N) is the ratio of a random quantity and a
deterministic quantity, and LABATCH.2, operating on WM+l, ... , WM+N, provides an as-
sessment of how well W(T:M, T:M+N) approximates the long-run average waiting time JLw.
However, Q(T:M, T:M+N ) under Option B is the ratio of two random quantities, J,';+N Q (u )du
and T:M+N - T:M. Therefore, direct application of the batch-means method does not provide
an assessment of how well Q( T:M, T:M+N) approximates the long-run average queue length
JLQ. However, an indirect application ofLABATCH.2 can provide the ingredients for making
this assessment.
Several indirect methods exist for assessing accuracy in the present case, and we de-
scribe them in the order of the increasing effort they require. First, there is Little's law (Section
3.5.1), which provides an alternative for estimation JLQ using Series 1 in Figure 6.12a. Since
A = 1, that approach yields the point estimate 4.251 and the interval estimate [4.122, 4.380].
Relatively little effort is required to compute these quantities. However, Little's law provides
no alternative if, for example, the objective is to estimate pr [Q (T:) > q] for some q > o.
The remaining indirect methods make use of the observation that expression (6.70)
has the equivalent form

1 M+N
- L !,'i Q(u)du
Q(T:M, T:M+N) = _N_i=_M_+_l_'_-l_ __ (6.72)
1 M+N '
- L (T:i - T:i-d
N i=M+l

which allows us to regard the numerator as the sample average of the N random variables
r'M+l Q(u)du, ... , J'tM+N-l
JT:M
r'M+N Q(u)du and the denominator as the sample average of the N
random variables T:M+l - T:M, ... , rM+N - T:M+N-l. To exploit this representation, we resort
to a more general formulation.
Let Xl, •.• , X t and Y1 , ••• , Yt denote sample records collected on two phenomena of
interest during a simulation. Let Xt and Yt denote their respective sample means and Rt :=
Yr/X t their sample ratio. Assume JLx ,eO. With reference to Option B, Yi = JrM+I_l
r'M+i Q(u)du
and Xi = T:M+i - T:M+i-l for i = 1, ... , t := N. Our objective is to assess how well Rt
approximates R := JL Y/ JL x .
Under relatively mild conditions,

2aoo (X,Y)
t varR-t --+ hxy := R z [a~(x)
--z- +a~(x)]
-- as t --+ 00,
JLy JLxJLY JL~
280 CHAPTER 6. OUTPUT ANALYSIS

where

2 • -
aooeX) := hm t varXr,
t-+oo

Clearly, accounting for randomness in the denominator Xt as well as in the numerator

Yt adds a burden that one would prefer to avoid. Under certain conditions, this possibility
exists. Let

2 a~eX)
Yx:= --2-'
f..Lx
2 a~eY)
Yy:= --2-'
f..Ly
aooex, Y)
Pxy := aooeX)aooeY)'

so that h xy has the equivalent representation

( 1-2pxy -Yx + 2Yi) .

yY Yy

Therefore, a~ey)ltf..Li approximates varR t with relatively small error for large t if

- - -yy ) «PXy« -1 (YX

-1 (YX - + -yy ) , e6.73)
2 yy Yx 2 yy Yx

where « denotes "much less than." For example, yy Iyx = 10 implies -4.95 «pxy «
5.05, whereas yY Iyx = 100 implies -49.995 « PXy « 50.005. Since PXy E e-1, 1),
the conditions are easily satisfied for these examples. In short, if the numerator Yt has
a considerably larger coefficient of variation than the denominator Xt, then the error of
approximation is relatively small in approximating var Rt by a~ey)ltf..Li. If a~eY) denote
the batch-means estimate of a~eY), then a~ey)ltX; is our approximation to var Rt . For
the queue length example, tXt = iM+N -iM, sothattheestimateist a~ey)/eiM+N - i101)2.
The photocopying problem in Section 1.4 illustrates ratio estimation in practice. For
each scenario a simulation was executed for t = 1000 weeks, about 20 years of five-day
weeks. Here we use data for Group 3 eTable 1.1) and for Scenario 1, the current operating
 6.11. RATIO ESTIMATION 281

policy. To estimate long-run average waiting two quantities were computed,

Y lOOO : sample average waiting time per week

= 676.0,
X1000 : sample average number of users per week
124.9,

along with Ja;)Y)lt = 39.96 and Ja'&:,(X)lt = .3876 via LABATCH.2, as estimates of
JvarYt and JvarX t , respectively. Since

A A 39.96 124.0
yyjYx = - - x - - = 19.05,
676.0 .3876

a negligible error arises by ignoring randomness in X1000.

We see that yY » Yx implies that ignoring the randomness in the denominator
introduces relatively little error. But what about moderate and small yy I yx? If yY = Yx,
then inequality (6.73) becomes 0 « PXy « 1, and the error vanishes for PXy = .50. If
yY Iyx < ..fi - 1 == .4142, the inequality is

1 < -1 (YX
- - -yy) «PXy« -1 (YX
- + -yy) ,
2yy Yx 2yY Yx

which cannot be satisfied. In summary, the approximation works best for yY » Yx. The
batch-means inplementation in Arena implicitly uses this variance estimate in computing
the half~length for sample averages that are ratio estimates.
A third indirect method of coping with randomness in denominator as well as
numerator requires more effort but does not rely on condition (6.73). LABATCH.2-
generated tableaus for the three input series Xl, ... ,Xt ; Yl. ... , Yt ; and Zl, ... , Zt, where
Zi := Xi - Yi , provide the basis for this assessment.
Assume that {Xd and{Yd obeyASA (Section 6.6.2). Thenast ---+ 00, Gt := Yt-RX t
has the properties

G t ---+ 0 w.p.I,
t varG t ---+ a~(G) := R2a~(X) - 2Raoo (X, Y) + a~(Y)
= R2a~(X) - R[a~(Z) - a~(X) - a~(Y)l + a~(Y)
< 00,

Gt d
---+ N (0, 1),
-ja'&:,(G)lt
282 CHAPTER 6. OUTPUT ANALYSIS

where

For large t (Fieller 1954),

pr{ IYt - RXtl ::: <1>-1(1 - 8/2)

x J{R2a~(X) + R[a~(Z) - a~(X) - a~(Y)] + a~(Y)}/t}

~ 1-8,

or, equivalently, with approximate probability 1 - 8,

J{2X t Yt -e2[a~(X) + a~(Y)-a~(Z)l/tF-4[xl-e2a~(X)/t] [y?-e2a~(Y)/t]]

2[xl-e2a~(X)/t] ,
(6.74)

where e := <1>-1(1 - 8/2) and where the argument of the square root is nonnegative as
t -+ 00 w.p.1. Most importantly, this confidence interval remains asymptotically valid for
R when strongly consistent estimates replace a~(X), a~(Y), and a~(Z). The tableaus
in LABATCH.2 for the three series XI, ... , X t; Y1 , ••• , Yt ; and Z I, ... , Zt provide these
estimates.
Figure 6.15 shows the final tableau for Series 13 (numerator), Series 14 (denomi-
nator), and Series 15 (difference) for the airline reservation simulation, whose remaining
output appears in Figure 6.12a. The average queue length /l-Q has point estimate Rt =
4.142 = 2.2876 x 10- 3 /(6.944 x 10- 4 ) and 99 percent approximating confidence interval
[4.103,4.270] based on the interval (6.74), where (3.362 x 10-5 )2, (5.072 x 10-7 )2, and
(3.386 x 10-5 )2 are the estimates ofa~(Y)/t, a~(X)/t, anda~(Z)/t, respectively. The rel-
ative length of the confidence interval is (4.270- 4.013)/4.142 = .0621. Also, LU-I (1-8/2)
replaces <1>-1(1 - 8/2), where L' denotes the minimum of the final number of batches in
estimating a~(X), a~(Y), and a~(Z).
For our first indirect method based on Little's law, recall the confidence interval [4.122,
4.30]. Since yy = 16.53 and Yx = 1.033 based on entries in Figure 6.15, our second indirect
method seems worth applying. It leads to the confidence interval

2.789 2.963]
[ .6944' .6944 = [ 4.016, 4.267].
 6.12. LESSONS LEARNED 283

Final Tableau
Mean Estimation
***************
(t = 2000000)

99.0%
Standard Error Confidence Interval
Series X sqrt [B*W (L, B) It] Lower Upper (Upper-Lower)/IXI

13 +2. 876E-003 +3. 362E-005 +2.789E-003 +2.963E-003 +6.051E-002

14 +6.944E-004 +5.072E-007 +6.931E-004 +6.958E-004 +3.771E-003

15 +2. 181E-003 +3. 386E-005 +2.094E-003 +2.269E-003 +8.035E-002

X(t) is based on all t observations.

W(L,B) is based on first 95.85% of the t observations.

Figure 6.15 LABATCH.2 output for ratio estimation for airline reservation problem

In summary, all three methods give relatively similar confidence intervals in this case of mean
queue-length estimation.

6.12 LESSONS LEARNED

• A simulation run ofa delay system tends to generate sample paths made up of sequences
of dependent events. The dependence increases with the level of congestion or delay
in the system.

• As a consequence of this dependence, the initial conditions, or state S(O), that prevail
at the begining of a simulation run influence the course of the sample path that evolves
as simulated time r evolves. However, the influence of these initial conditions S(O) on
the state of the system S( r) at time r tends to diminish as r increases.

• To generate sample-path data relatively free of the influence of initial conditions, a

simulationist needs to identify a warm-up interval oflength k - 1 that is allowed to
elapse before data collection begins. The simulationist also needs to choose the length
of the sample path t over which data are to be collected and the number of independent
replications n to be run.
284 CHAPTER 6. OUTPUT ANALYSIS

• Multiple replications are helpful in choosing an adequate k. Starting with the same ini-
tial conditions on each simplifies the analysis. Also, choosing an initial state that causes
the influence of the initial state to diminish monotonically simplifies the analysis.

• If sample averages on n independent replications are used to estimate a long-run aver-

age, then the relatively elementary methods of statistical inference apply to assessing
the error of approximation, provided that the warm-up interval oflength k - 1 has
elapsed on each replication before data collection begins.

• If statistical inference is based on a single (n = 1) sample path oflength t, the estimation

of the variance of each sample average needs to account for the dependence among the
t observations on the path. Failure to do so can lead to an overly optimistic assessment
of accuracy. The single sample-path approach requires that one warm-up interval
elapse before data collection begins. This contrasts with the need to let a warm-up
interval elapse on each of n independent replications.

• The batch-means method provides one technique for incorporating this dependence.

• LABATCH.2 contains implementations of the batch-means method in C, FORTRAN,

and SIMSCRIPT II.S that provide a strongly consistent estimator of the variance of
the sample mean and an asymptotically valid (as t --+ 00) confidence interval for the
true long-run average of interest. In practice, it can be employed dynamically during
a simulation to produce these results. Alternatively, it can process stored data aft,er a
simulation run terminates. Most importantly, LABATCH.2 provides the wherewithal
for determining how well the estimating procedure has accounted for dependence
among observations. It is available on the Internet.

• LABATCH.2 also can be used to infer the accuracy of ratio estimates (Section 3.4). See
Section 6.11.

• Arena automatically provides a 9S percent confidence interval for each sample average.
For a single sample path (n = 1) it uses another variant of the batch-means method.
However, its assessment of the adequacy of the estimate of the variance of the sample
average is more limited than in LABATCH.2.

6.13 EXERCISES
You now have at least one working simulation program for one or more of the four problems
described in the Exercise section of Chapter 2. The program relies on either the event-
scheduling or process-interaction approach. The exercises described below are designed
to estimate a suitable warm-up interval k - 1 and to assess how well finite-sample aver-
ages approximate corresponding unknown long-run averages. For the latter assessment, use
LABATCH.2, if possible, either in-line or operating on data files after the simulation's exe-
 6.13. EXERCISES 285

cution has terminated. If an alternative procedure is used, it must provide an assessment of

the stability of the variance estimate. See, for example, Exercise 6.2f.
The exercises specify sample-path lengths t' for the warm-up analysis and t for com-
puting estimates.· If either or both of these prove inadequate, increase them. Give supporting
information in your reports as to why these changes were necessary.

QUARRY-STONE CRUSHING (EXERCISES 2.1, 2.2, 4.2, 4.3, 4.5, 4.6)

The objective is to estimate Amax> the arrival rate for external trucks that maximizes
profit as defined in Exercise 2.2. This calls for a simulation program developed in Exercise
4.2 and a sequence of simulation runs executed for different values of A in (0, A*), where
A* = ~ trucks per minute denotes the maximal allowable arrival rate at the crusher for
external 50-ton trucks.
~ 6.1 Initial Conditions. Each distinct value of A introduces a different level of conges-
tion and, hence, a different interval for dissipating the influence of initial conditions. Since
congestion increases with A, the length of a suitable warm-up interval should also increase
with A and be largest for A** := maximal arrival rate used in experiments. Take A** = .9A*
to determine this maximal warm-up interval.

a. Execute n = 10 independent replications each for t' = 100 hours with A = A** and each
starting with the same initial conditions and parametric values, specified in Exercises
2.1,2.2,4.2,4.3,4.5, and 4.6. For each stream j, use its last pseudorandom number on
replication i as the seed for stream j on replication i + 1 for i = I, ... , n - 1.

b. Using the methodology of Section 6.1, prepare graphs of sample average throughput per
five minutes and crusher queue length and use the graphs to choose the k that makes the
influence of initial conditions negligible. Write a short report justifying your choice of k.
Use tables and graphs to support the choice. Label each table and graph with appropriate
titles and identify all variables used therein.

~ 6.2 Estimating Long-Run Average Profit. Carry out a sequence of experiments to

estimate the value of Amax. Use the estimated warm-up interval k - 1 of Exercise 6.1 in all
runs. For each distinct value of A considered, generate a sample path oflength k + t - 1 with
t = 1000 hours.

a. On each run, compute the quantities listed in Exercise 4.2e and the sample average profit
apart from the unknown factor A in Exercise 2.2.

b. On each run, compute 99 percent confidence intervals for long-run average profit and
crusher queue length and relative assessments of the accuracy of their sample averages.
286 CHAPTER 6. OUTPUT ANALYSIS

c. To identify the neighborhood of Amax, use nine runs with

i-I
Ai = - - A** for i = 1, ... ,9.
8

d. Let [A', A"] denote the interval that your results to part c suggest contains Amax. Continue
to make runs for selected values in [A', A"] to refine your estimate of Amax. As guidance
for when to stop, use your point and interval estimates oflong-run average profit.

e. Prepare a report identifying your choice of Amax using graphs and tables to support your
choice.

f. In your report, describe the extent to which your estimates of a~ are stable for sample
average profit and crusher queue length for each A.

LOADING TANKERS (EXERCISES 2.3, 4.7, 4.8)

To compete with other oil-loading ports, the port authority plans to institute a rebate
program based on total port time. This is the summation of waiting, towing, and loading
times. If a tanker's port time exceeds 48 but is no more than 72 hours, the authority will pay
its owner a rebate of A dollars. If it exceeds 72 but is no more than 96 hours, the rebate is
3A dollars. In general, if it exceeds 24i but is no more than 24(i + 1) hours, the rebate is
(2i - I)A dollars. From Exercise 2.3, A = 10,000 dollars.

~ 6.3 Initial Conditions. This exercise is designed to estimate a suitable truncation

point k - 1 for the port-time sequence based on multiple independent replications. In what
follows, use your own pseudorandom number seeds and whichever of your two programs
you wish.

a. Execute n = 10 independent replications each for 1000 tankers and each starting with
initial conditions specified in Exercises 4.7 and 4.8. Use your orignal run as replication
1. For each stream j, use its last pseudorandom number on replication i as the seed for
stream j on replication i + 1 for i = 1, ... , n - 1.

b. Using the methodology of Section 6.1, prepare graphs of sample averages of port times
for tanker i for i = 1, ... ,1000 and use the graphs to choose the warm-up interval k
that makes the influence of initial conditions "negligible:' Write a short report justifying
your choice of k. Use tables as well as graphs to support your choice. Label each graph
and table with appropriate titles and identify all variables used therein.

~ 6.4 Estimating Long-Run Average Rebate. The purpose of this exercise is to estimate
f..Lr := sample average rebate per tanker and f..L~ := sample average rebate per tanker that
received a rebate (Exercise 2.3).
 6.13. Ex ERe I S E S 287

a. Using the warm-up interval determined in Exercise 6.3, generate a sample path oflength
k + t - 1 = 20, 000 tanker departures.

b. Compute point estimates of Mr and M~.

c. Compute a 99 percent confidence interval for Mr.

d. Using the methodology of Section 6.11, compute a 99 percent confidence interval for M~.

e. Prepare a report describing your findings and their statistical quality.

f. Your report should also describe the extent to which estimates of a~ are stable for each
sample sequence.

TERMINAL PAD CONTROLLER MODEL (EXERCISES 2.5, 4.12, 4.13)

The purpose of Exercises 6.5 and 6.6 is to generate sample-path data using the
simulation program developed in either Exercise 4.12 or Exercise 4.13 to estimate the
minimal mean delay M(A, m) in definition (2.6) and the A at which it occurs for each
m E M := {8, 16, 24, 32}. Recall that A denotes character-arrival rate to an input buffer
and m denotes maximal packet size. Recall that.30 :s r A :s 1 and m E {8, 9, ... , 32}, where
r = 10 denotes the number of input buffers.

iff? 6.5 Initial Conditions. For each fixed m, delay time varies with A, but not necessarily
in a monotone way. In particular, small Aimplies that some characters wait a long time in their
input buffers until being transferred to the relatively uncongested output buffer. Conversely,
large A implies that characters have a relatively short wait in their input buffers, but, because
of congestion, have a long wait in the output buffer. While small Ainduces dependence among
no more than m characters in an input buffer, the low congestion in the output buffer induces
little dependence among characters from different input buffers. However, congestion in the
output buffer for large A does create dependence among characters in different input buffers
and needs to be taken into consideration when deciding on a warm-up interval.
Therefore, for each fixed m, the length of a suitable warm-up interval k m - 1 can be
expected to increase with A and be largest for A* (m), the maximal character-arrival rate used
in experiments. Take A*(m) = 0.95 x poir, where Po is determined in Exercise 2.sb. Since
monotone behavior for delay time is not apparent for varying m but fixed A, the task of
identifying a suitable warm-up interval becomes more challenging. To resolve this problem:

a. For each m E M, execute n = 10 independent replications each for t' = 1,000 trans-
mitted nonoverhead characters with (A = A* (m)) and each starting with the same initial
conditions and parametric values as in Exercises 2.5 and 4.12. For each stream j, use its
last pseudorandom number on replication i + 1 for i = 1, ... , n - 1.

b. Using the methodology of Section 6.1, prepare graphs of sample average delay time
separately for each m E M and use them to estimate suitable truncation points {k m -
288 CHAPTER 6. OUTPUT ANALYSIS

1, m EM}. Write a report justifying your choices. Use tables and graphs to support your
choice. Label each table and graph with appropriate titles and identify all variables used
therein.

c. Comment about how km varies with m.

4f? 6.6 Estimating Long-Run Average Delay Time. For each m E M, carry out a se-
quence of experiments to estimate the function (fL(A, m), 0 < A < polr} where Po is
derived in Exercise 2.5b. Use the estimated warm-up interval k m - 1 of Exercise 6.5 for all
runs with size m input buffers. For each run, generate a sample path of length k m + t .- 1
where t = 20, 000 transmitted nonoverhead characters.

a. Compute estimates of f.L(A, m) for each m EM and A = Ai. where

Ai := (0.05 + O.li)polr i = 1, ... ,10.

b. Compute an approximating 99 percent confidence interval for each f.L(A, m).

c. If the objective is to compute a point estimate, [l(A, m) that is within ± 10 percent of the
unknown f.L(A, m), how well have you succeeded? Use quantitative evidence to justify
your answer.

d. Use the results in part a to identify an interval [a(m), b(m)] s:; (0, polr) for A in which
you estimate min). f.L(A, m) is located. Then for each m E M, compute estimates of
f.L(A, m) for each

Ai := {a(m) + O.li [b(m) - a(m)]}polr i = 1, ... , 10.

e. Write a report identifying your point estimates of min). f.L(A, m) and of

Amin(m) := A at which f.L(A, m) has its minimum

for each m E M. Use graphs and tables to support your choices, especially those
{Amin(m), m EM}.

f. Provide evidence that your choices of (T~ for each (A, m) combination is stable. If some
are not, explain why.

g. For the data generated in part d, plot estimates of (Too versus Ai = {a(m) + O.li[b(m) -
a (m)]}po/rfor i = 1, ... , 10 and eachm E M. Explain the behavior in (Too as functions
of A and m.
 6.13. Ex ERe I 5 E 5 289

MOBILE-TELEPHONE SERVICE (EXERCISES 2.5, 4.14, 4.15)

The purpose of Exercises 6.7 and 6.8 is to generate sample-path data using the
simulation program developed in either Exercises 4.14 or 4.15 and for each c in C =
{.Ii; i = 1, ... , 9} to use these data to estimate the minimal long-run average total cost
(2.7) as a function of m, the total number of available lines, and r, the number of lines
dedicated to handed-off calls. Since Ah = 4, As = 2, and w = 1 calls per minute, m must
exceed 6 and m - r must exceed 2 in order for the mean number oflost calls per minute to
remain bounded.

,if? 6.7 Initial Conditions. For m transmission lines, r of which are dedicated to handed-
off mobile calls, let

B (r) := number of busy lines at time r

and

fJ,(m, r) := long-run proportion of busy lines.

Hereafter, we call fJ,(m, r) utilization. For fixed m, utilization decreases monotonically as r

increases and, for fixed r, decreases monotonically as m increases. For given m, r, and initial
conditions S(O) = s, we have

E [B(r) I S(O) = s] -+ mfJ,(m, r) as r -+ 00.

It is anticipated that the slowest convergence rate occurs for m = 7 and r = O.

Let

Y; = I 0
if call i is a mobile call and it is handed off,
otherwise,

and

Z; = I 0
if call i is a standard call and it is denied service,
otherwise.
(6.75)

Then in equilibrium

and
290 C HAP TE R 6. 0 U T PUT A N A LY 5 I 5

a. Execute n = 10 independent replications each for t' = 1000 calls with m = 7, r =: 0,

and other parametric values specified in Exercises 2.6 and 4.14. For each stream j, use
its last pseudorandom number on replication i as the seed for stream j on replication
i + 1, for i = 1, ... , n - 1.
b. Using the methodology of Section 6.1, prepare graphs of the sample average number of
busy lines over five-minute intervals and use the graphs to determine a time at which
to begin data collection for estimating (3s(m, r) and (3h(m, r). Write a report justifying
your choice of warm-up interval k - 1 for m = 7 and r = O.

c. Examine the extentto which this choice of k in part b suffices for all r E {O, 1, ... , m -- 2}
and m E {7, 9, 11, 13, IS}. If you conclude that this choice is inadequate for these other
(m, r) 2-tuples, perform additional warm -up analyses to identify more suitable intervals.
Provide evidence to support your choices.

1f? 6.8 Estimating Loss Rates. Carry out a sequence of independent sampling experi-
ments to estimate the loss rates (3s(m, r) and (3h(m, r) for each r E {O, 1, ... , m - 2} and
m E {7, 9,11,13, IS}. Each experiment is to be run for t = 10,000 calls, regardless of type
after the warm-up interval k -1 chosen in Exercise 6.7b or 6.7c has elapsed. There are several
available ways of estimating C(m, r, c) in expression (2.7). One collects the data

if a mobile call is handed off,

if a standard call is denied service,
otherwise.

Then

_ 1
L
~ k+t-l
C(m, r, c) = X kt := - Xi (6.76)
t i=k

provides an estimate of C(m, r, c). To estimate this quantity with these data requires 432
sample paths, one for each (m, r, c) triplet.
An alternative approach reduces the required number of sample paths. It relies on the
Yi and Zi defined above. Since Xi = Yi + Zi, expression (6.76) is algebraically equivalent
to

(6.77)

where

_ 1 k+t-l
Ykt =-
t
L
i=k
Yi
 6.13. Ex ERe I S E S 291

and

_ 1 k+t-l
Zkt =-
t
L
i=k
Zi.

However, for each fixed (m, r), {C(m, r, c), c E c} can be computed from a single
replication, thus reducing the number of replications to 48.
If we merely want point estimates, the second data collection scheme would be un-
equivocally superior. However, we also want to assess the statistical error in each C(m, r, c).
That can be done straightforwardly using LABATCH.2, for example, for each of the 432
runs. However, with a moderate amount of additional programming, it can also be done via
LABATCH.2 for the second 48 sample-path data collection scheme. Since

and since

C(m, r, c) - C(m, r, c) C(m, r, c) - C(m, r, c) -! N (0,1)

as t -+ 00,
jvarX kt jO'&,(X)lt

where

O'!(X) := lim tvarXkt

t--+oo
= O'!(y) + 2 cO'oo(Y, Z) + c2 O'!(Z),
2 • -
O'oo(Y) := hm tvarYkt,
t--+oo

we need estimates of O'~(Y), O'~(Z), and O'oo(Y, Z). For example, using the data
{(Yi , Zi, Yi - Zi), k ::=: i ::=: k + t - I} as input on each call to LABATCH.2 produces
estimates of a~(Y), a~(Z), and a~(Y - Z), where the latter is an estimate of

Therefore,

provides an estimate ofO'~(Y, Z), and [a~(Y) + 2caoo (Y, Z) + c2a~(Z)] It provides the
needed variance estimate. If L(Y), L(Z), and L(Y - Z) are the numbers of batches used on
292 CHAPTER 6. OUTPUT ANALYSIS

the final interim reviews in LABATCH.2, then a conservative approach to interval estimation
takes min[L(Y), L(Z), L(Y, Z)] - 1 as the degrees of freedom.

a. On each run, compute estimates of C(m, r, c) in expression (2.7) for each c E C.

b. On each run, compute an approximating 99 percent confidence interval for each

C(m, r, c).

c. For each fixed m and c, identify your choice of r that minimizes C (m, r, c).

d. To what extent do the confidence intervals support your choice? Be specific in providing
justification.

6.14 REFERENCES

Alexopoulos, c., G.S. Fishman, and A.T. Seila (1997). Computational experience with
the batch means method, Proceedings of the 1997 Winter Simulation Conference, s.
Andrad6ttir, K.J. Healy, D.H. Withers, and B.L. Nelson, eds., IEEE, Piscataway, NJ.

Blomqvist, N. (1967). The covariance function of the M/G/I queueing system, Skandinavisk
AktuarietidskriJt,50,157-174.
CACI, Inc. (1983). SIMSCRIPT II.S Reference Handbook, CACI Products Company, La Jolla,
California.

Chien, C.H., D. Goldsman, and B. Melamed (1997). Large-sample results for batch means,
Management Science, 43, 1288-1295.
Chow, Y.S., and H. Robbins (1965). On the asymptotic theory of fixed-width sequential
confidence intervals for the mean, Ann. Math. Statist., 36, 457-462.

Damerdji, H. (1994). Strong consistency of the variance estimator in steady-state simulation

output analysis, Math. Oper. Res., 19,494-512.

Fieller, E.C. (1954). Some problems in interval estimation, J. Roy. Statist. Soc., Series B, 2,
175-185.

Fishman, G.S. (1978). Principles of Discrete Event Simulation, Wiley, New York.

Fishman, G.S. (1996). Monte Carlo: Concepts, Algorithms, and Applications, Springer Verlag,
New York.

Fishman, G.S., and L.S. Yarberry (1997). An implementation of the batch means method,
INFORMS J. on Comput., 9, 296-310.
 6.14. REF ERE NeE 5 293

Law, A.M. and W.D. Kelton (1991). Simulation Modeling and Analysis, McGraw-Hill, New
York.
Goldsman, D., and M.S. Meketon (1986). A comparison of several variance estimators for
stationary increment stochastic process, Technical Report, School of Industrial and
Systems Engineering, Georgia Institute of Technology.
Kelton, W.D., R.P. Sadowski, and D.A. Sadowski (1998). Simulation with Arena, McGraw-Hill,
New York.
Philipp, W. and W. Stout (1975). Almost sure invariance principles for partial sums of weakly
dependent random variables. Mem. Am. Math. Soc., 2.
Song, W.T. and B.W. Schmeiser (1995), Optimal mean-squared-error batch sizes, Man. Sci.,
41,110-123.

von Neumann, J. (1941). The mean square difference, Ann. Math. Statist., 12, 153-162.
Yarberry, 1.S. (1993). Incorporating a dynamic batch size selection mechanism in a
fixed-sample-size batch means procedure, unpublished Ph.D. dissertation, Dept. of
Operations Research, University of North Carolina, Chapel Hill.
Young, 1.c. (1941). On randomness of order sequences, Ann. Math. Statist., 12,293-300.
 CHAPTER 7

G. S. Fishman, Discrete-Event Simulation

© Springer Science+Business Media New York 2001
Making Sense
of Output and
Increasing
Efficiency

A simulation run customarily generates sample-path data on user-specified dynamic pro-

cesses that evolve as simulated time elapses. These often include delay time, queue length,
and resource utilization. Sample averages summarize these data and usually become the
focal point of reports and presentations to management. Since they merely approximate
corresponding long-run averages that would be observed if the simulation were run for an
infinite rather than a finite amount of time, some measure of error should accompany each
sample average. Confidence intervals, standard errors, and relative errors provide three alter-
native means of assessing how well sample averages approximate true long-run averages. As
illustration, the Final Tableau in Figure 6.12a displays 99 percent confidence intervals under
the headings Lower and Upper, standard error under sqrt[B*W(L, B)/tj, and relative error
under (Upper - Lower)/IXI. A report that omits an error assessment can lead to serious
misconceptions of how well sample averages characterize unknown long-run averages.
Providing an error assessment with each sample average facilitates informed decision
making. Additional analyses, that focus on the particular problem to be resolved, can further
enhance the quality of decision making. This chapter describes techniques for performing
several of these analyses, in particular, ones that compare simulated system behavior for
296 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

different operating scenarios. Recall that a scenario denotes a set of specific values assigned
to input parameters and a collection oflogical system-operating rules.
As illustration, consider the airline reservation problem and the waiting-time results in
Section 4.22 for m = 4 and 5 reservationists with first-come-first-served queueing discipline.
Suppose management is willing to increase the number of reservations from four to five if
that action reduces mean waiting time by at least three minutes and if the proportion of
customers who wait is cut by .30. To make this an informed decision, an analysis needs to
assess how credible each difference is individually and how credible they are when taken
simultaneously. Section 7.1 describes the details.
This example illustrates issues that arise in many studies that examine simulated be-
havior for different values of input parameters or different operating rules. The issues are the
comparison of results generated on different experiments and the simultaneity of multiple
confidence statements made about these results. This chapter describes statistical procedures
for making these comparisions and simultaneous confidence statements. It presents them in
the special context in which a simulationist may wish to use them when analyzing simula-
tion output. Collectively, we refer to these procedures as credibility analysis, an elusion to the
frequent need, in making a decision, to assess how plausible it is to assume that a difference
of a particular magnitude actually exists.
Although space limitations restrict our discussion to the most basic of these issues, the
exposition addresses problems that a simulationist frequently encounters when preparing a
report for a management choosing among alternative resource configurations or operating
modes. Within the considerable statistical literature on these topics, Bechhofer et al. (1995)
provides a comprehensive account of the technical issues with special emphasis on selecting
the best among alternative systems. Matejcik and Nelson (1995a, 1995b) give one typl~ of
analysis of multiple comparisons within the special context of discrete-event simulation.
When a simulation study focuses on the difference between two sample averages each
generated under a different scenario, Section 4.22 shows how common pseudorandom num-
bers (cpn) can induce a higher level of statistical accuracy than would arise for independent
runs. The discussion there concentrates on the mechanics of implementing cpn. Section 7.7
and, in particular, 7.7.1 return to this subject, focusing on how some differences in input
scenarios tend to favor substantial variance reductions whereas others do not.
Sections 7.8 through 7.10 then introduce the reader to a second variance-reducing
technique. It is based on optimal-sample-path length (opt) for independent runs. The ac-
count reveals when opt induces a smaller variance for the difference of sample averages than
cpn and when the reverse occurs. In particular, a large disparity in individual sample-average
variances tends to favor opt over cpn.
Many other methods exist for reducing variance, including control variates, stratified
sampling, and antithetic variates; see, e.g., Fishman (1996) and Law and Kelton (1991). The
present account discusses only cpn and opt for several reasons. First, they are the two that
address differences in averages. Second, they each require a modest amount of effort to
implement. Third, space considerations limit what can be covered under this topic.
 7.1. SIMULTANEOUS CONFIDENCE STATEMENTS 297

In what follows, we repeatedly make use of d.f.s and inverse d.f.s. Recall that <I> (. )
denotes the d.f. of a random variable from N(O, 1), and <1>-1(.) denotes its inverse. Also,
define G d (.) as the d.f. of a random variable from Student's t distribution with d degrees of
freedom and Td (.) as its inverse.

7.1 SIMULTANEOUS CONFIDENCE STATEMENTS

We begin with the concept of a multiple confidence statement and how it relates to individual
confidence statements. Let A I, ... , Ar denote r independent random events. If A I, ... , Ar
are independent, then the probability that they occur simultaneously is

(7.1)

As an example, suppose that r parameters ILl, ... , ILr have been estimated by confidence
intervals. If Ai denotes the event that the ith confidence interval includes the ith unknown
ILi with probability

pr(Ai) = 1 - a,

then

To achieve a 1 - 8 confidence level for simultaneous coverage, a needs to be chosen as

(7.2)

How does simultaneity affect the lengths of the confidence intervals? For large numbers
of degrees of freedom (e.g., Section 6.6), interval length is proportional to <1>-10 - a/2).
Then

<1>-1 (0 + 0 - 8)I/r)/2)
v(8 r) .- ------'------,---------'-- (7.3)
,.- <1>-1 (1 - 8/2)

gives the ratio of simultaneous to individual interval lengths. For r = 2, v (.01, 2) = 1.089
and v(.05, 2) = 1.14l. For r = 11, v(.OI, 11) = 1.287 and v(.05, 11) = l.444. These
numbers indicate that claiming simultaneous coverage without correcting interval lengths
results in an overestimate of accuracy that increases with r for a given 8. For example,
v(.OI, 11) = l.287 implies that the simultaneous interval lengths should be 28.7 percent
longer.
298 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

When two or more confidence intervals are computed from data generated on the same
simulation run, they are rarely independent, and expression (7.1) does not apply. Section
6.8 illustrates this case, where Figure 6.12a displays a 99 percent confidence interval for each
of the 11. exceedance probabilities, pre W > w), for waiting time for the airline reservation
problem using Series = w + 2 for w = 0,1, ... , 10 minutes. While characterizng the
quality of each corresponding point estimate separately, these intervals do not make a single
statement regarding all 11 point estimates taken together. Moreover, since they are based on
the same data, the intervals are dependent. To address this issue, we resort to simultaneous
confidence intervals based on Bonferroni's inequality.
Let Ai denote the event that Ai does not occur. In the present case, we may think of
Al as the event that the confidence interval for Series 2 on review 15 in Figure 6.12a covers
pr(W > 0), A2 as the event that the confidence interval for Series 3 on review 15 covers
pr(W > 1), etc. From the law of total probability,

and from the observation thatpr(Al' A 2)+pr(A 1 , A 2) = pr(A 2) and pr(A 1 , A 2) :::: pr(,4 1 ),
we have

This is a lower bound on the probability that both Al and A2 occur simultaneously.
Let B2 := Al n A 2. Then

pr(A 3, B2) + pr(A3, B2) + pr(A3' B2) + pr(A3' B2) = 1,

pr(A3' B2) + pr(A3' B2) = pr(B2 ),

and

so that

pr(A3, B 2) ::: 1 - pr(B2 ) - pr(A3)

= 1 - 1 + pr(B2) - pr(A3)
= 1 - pr(A 1 ) - pr(A 2) - pr(A 3).

More generally, for each j and B j := n {=l Ai,

j
pr(B j ) ::: 1 - Lpr(A i ), j = 1, ... , r, (7.4)
i=l
 7.1. SIMULTANEOUS CONFIDENCE STATEMENTS 299

which is called Bonferroni's inequality. Since pr(Br ) = pr(A 1 , ••• , Ar), we have a lower
bound for all r events occurring simultaneously.
Again, let Ai denotes the event that the ith confidence interval includes the ith un-
known mean f.1,i with pr(Ai) = 1 - a. Then inequality (7.4) implies that simultaneous
coverage for j = r is at least 1 - ra. To achieve a 1 - 8 simultaneous confidence level, we
choose a as

a = 8/r, I (7.5)

leading to an asymptotic ratio of simultaneous and individual interval lengths

v'(8, r) := cp-1(1- 8/2r)/cp-1(1- 8/2).

For r = 2, v'(.OI, 2) = 1.090 and v'(.OS, 2) = 1.144. For r = 11, v'(.OI, 11) = 1.288 and
v'(.OS, 11) = 1.448. The ratios reveal that the interval lengths increase at roughly the same
relative rates as their corresponding ratios in the independent case. However, it is easily seen
that cp-1(1 - 8/2r) - cp ((1 + (1- 8)1/r)/2) increases with r.
For the 11 confidence intervals for exceedance probabilities in Figure 6.12a, pr(A i ) =
.01, so that

pr(Bu) := probability that all 11 intervals simultaneously

cover their respective true probabilities ::: .89.

If a simulationist wants a 1 - 8 = .99 simultaneous confidence for the r = 11 ordinates of

the above example, one would set a = .01/11 = .0009091 in the original runs. As a final
point, note that for 0 ::::: a < 1/ r, (1 - a Y / (1 - ra) decreases to its lower bound of unity
as a decreases.
Returning to the issue, raised in the introduction, of increasing m from four to five
reservationists in the airline problem, we recall that management is amenable to this change
if mean waiting time is reduced by three minutes and the proportion of customers who
must wait decreases by .30. Section 4.22 estimates the reduction in mean waiting time as
Z = 3.337 minutes with standard error Ja'&,(Z)/t = .1687. It also estimates the reduction
in the proportion who wait as Z' = .3S97 with standard error Ja'&,(Z')/t = 3.628 x 10-3 •
The point estimates Z and Z' support the decision to increase the number of reserva-
tionists. However, these are estimates of the unobserved true changes, and hence a decision
maker would benefit from knowing how credible these changes are. The standardized
statistics

- Z- 8
(7.6)
Z(8):= Ja'&,(Z)/t'
- Z'-cp
Z' (A.) .- ~=o===;:= (7.7)
'I-' .- /a'&,(Z')/t'
300 C HAP TE R 7. M A KIN G SEN 5 E 0 F 0 U T PUT AND INC REA 5 I N G E FFie lEN C Y

with e := EZ and cp := EZ', each have degrees of freedom in excess of 100 (determined by
LABATCH.2), so that treating each as from N (0, 1) introduces negligible error.
Under the hypothesis HI(e') : e = e', the probability of the event

Al (e') := event of observing aN (0, 1) random variate no less than Z(e') in value

IS

pr[AI(e')] = <I>(Z(e')).

Under the hypothesis Hz(cp') : cp = cp', the probability of the event

Az(cp') := event of observing aN (0,1) random variate no less than Z'(cp') in value

IS

pr[Az(cp')] = <I>(Z'(cp')).
Then for the joint event Bonferroni's inequality gives the lower bound

pr [AI(e'), Az(cp')] 2: <I>(Z(e')) + <I>(Z'(cp')) - 1.

For e' = 3 and cp' = .30,

pr [A I (3), A z(.30)] 2: .9772 + 1.000 - 1 = .9772.

This lower bound gives considerable credibility to HI (3) n Hz (.30). However, suppose
that management is willing to increase the number of reservationists by one only if e' = 3
and cp' = .36. Then

pr [A I (3), A z(.30)] 2: .9772 + .4670 - 1 = .4442,

which gives considerably less support for the joint hypotheses HI (3) n Hz (.36). Most impor-
tantly, it can alert management to the greater risk of error in assuming e' = 3 and cp' = .36
than assuming e' = 3 and cp' = .30.
The next section extends this form of credibility analysis as a function of increasing
mean difference.

7.2 COMPARING TWO MEANS

The airline example of the last section compares mean waiting times for four and five
reservationists. The illustration takes advantage of the dependence induced by common
 7 .2. COM PAR I N G Tw 0 MEA N S 301

pseudorandom numbers to reduce standard errors. This section addresses the more general
case of independence of results across experiments and replications. Section 7.5 further
extends these ideas to the case of single replications under each scenario. Section 7.7 returns
to the approach based on common pseudorandom numbers.
Recall that Section 6.1 defines Xt~l as the sample average of tl observations collected
sequentially on sample path i after deleting kl - 1 observations to reduce the influence
of initial conditions. Moreover, for nl independent replications expression (6.4) defines
X := Xklnltl as the arithmetic average of Xk~;l' ... , xt:l) , and expression (6.8) defines
S2(~) := sflnltl as the sample variance of an Xt~l' so that s2(X)/nl is an estimate of
varXklnltl'
By analogy, let Y denote the grand sample average and S2 (Y) the sample variance of the
sample averages Yk~I;" ... , Yi;t~) collected on n2 independent replications of the simulation
generated under a different scenario than for the first nl replications. Note that the first
set of replications each delete kl - 1 observations and use tl observations to compute each
Xt~l ' whereas the second set deletes k2 - 1 observations and thereafter uses t2 observations
to compute each Yi:;,. Presumably, kl and k2 reflect differences in the rates at which the
influences of initial conditions dissipate under the two different scenarios, and tl and t2
reflect differences associated with computing costs and possibly differences due to sampling
variation along sample paths. Assume that tl and t2 are sufficiently large so that we may
regard each Xt~l and each yi:;, as approximately normally distributed.
Comparing the sample averages X and Y is a commonly encountered objective in
simulation studies. Recall that X and Yare estimates of long-run averages /Lx and /LY,
respectively, and it is ofinterest to assess the extent to which /L x and /L Y differ as a consequence
of different scenarios in each experiment. An approximating 100 x (1- 8) percent confidence
interval for /Lx - /LY is

(7.8)

where

d = d(c, n, n2):=
[f;2 (1-
-- + ---
f;?]-I (7.9)
nl - 1 n2 - 1

with

gives the degrees of freedom for the critical value of Student's t distribution.
If varXi/tl = varYi;;, and nl = n2, then (X - Y)/ [s2(X)/nl + s2(Y)/n2r/2 has
Students's t distribution with nl + n2 - 2 degrees of freedom, and no issue of approximation
arises other than the assumption of normality. When either or both of these conditions
302 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

are absent, this statistic does not have Student's t distribution with nl + n2 - 2 degrees
of freedom. Then, expressions (7.8 ) and (7.9 ) comprise Welch's sug~ested approach to
approximating the distribution of eX - Y)j [s2(X)jnl + s2(Y)jn2]1/ . See for example,
Kendall and Stuart (1961, vol. 2) and Scheffe (1970). Section 7.3 describes the basis for this
approximation.

o ExAMPLE 7.1

As illustration, consider two variants of a queueing system, the first ofwhich produces a
sample average waiting time X = 12.94 hours with sample variance S2 (X) = 30.43 based on
n 1 = 15 independent replications and the second ofwhich produces a sample average waiting
time Y = 8.734 hours with sample variance S2(y) = 15.07 based on n2 = 10 independent
replications. Then X - Y = 4.206 hours, and expression (7.8) yields an approximating 99
percent confidence interval [-1.094,9.506] for f,tx - f,ty based on d = 22.89 degrees of
freedom.
Cost considerations, external to the simulation study, lead management to conclude
that the operating environment that generated Y~21i" ... , Y~;t~) would be preferable to the one
that generated Xk:~2' ... , xt:! if f,tx ~ f,ty :::: 2. Therefore, management wants to know the
extent to which the simulation output provides evidence that this inequality indeed holds.
We first study this issue in a more general setting.
Let

and take the Student t distribution with d = 22.89 degrees of freedom as the approximating
distribution for Z(f,tx - f,ty). Given X, Y, s2(X)jnl> and s2(Y)jn2' suppose it is of interest
to determine how credible it is to assume that f,tx - f,ty :::: v for some specified v. Under the
hypothesis H(O) : f,tx - f,ty = 0, let

A(O) := event of observing a Student t random variate, with d degrees of freedom

no greater than Z(O).

Then

pr [A(O)] = Gd(Z(O)),

where Gd(Z(O)) gives the probability of observing a standardized statistic no greater than
Z(O). Moreover, Gd(Z(O)) decreases monotonically with increasing O. Hence, GAZ(O))
provides a measure of credibility for f,t x - f,t y :::: O. The graph of this function in Figure
7.1 is revealing. It indicates Gd(O) = .87 for 0 = 2 hours, thus providing support for
 7.3. RA T ION ALE FOR DE G RE E S 0 F FRE E D 0 M 303

management's action. However, if the decision point had been 8 = 4 hours, Figure 7.1
would offer considerably less support for the scenario that generated yi:;" ... , Yk~~~)'
To evaluate the credibility of {Lx - {Ly :s 8, we use 1 - Gd(Z(8)), the probabil-
ity of observing a standardized statistic no less than Z(8) under H(8). This probability
monotonically decreases with decreasing 8.
D

Let us now reflect on this form of credibility analysis. Graphically, it provides consid-
erably more food for thoughtthan a report that merely displays point estimates and, possibly,
standard errors. In particular, it allows management to assess the plausibility of conjectured
differences in means that are pivotal to its decision making. Most importantly, the graphics
are easily created with application software such as Microsoft Excel.

7.3 RATIONALE FOR DEGREES OF FREEDOM

Expression (7.9) for degrees of freedom may well arouse the reader's curiosity. This section
describes its rationale. A reader willing to accept these dJ.s may proceed directly to Section
7.4.
Since we assume that tl and t2 are sufficiently large so that xk:l j , and
••• , xt:!
Y2!', ... ,YL~!) are normally distributed, X and s2(X) are statistically independent, (nl -
1)s2(X)/varXk:lj has the chi-squared distribution with nl - 1 degrees of freedom, denoted
by x 2(nl -1), and (X - {Lx)/ Js2(X)/nl has Student's t distribution withnl -1 degrees of
freedom, denoted by T(nl - 1). Likewise, y, S2(Y) are independent, (n2 - 1)s2(Y)/varYk~;2
is from x2(n2 - 1), and (Y - {LY)/Js2(Y)/n2 from T(n2 - 1).
But what about the standardized difference

If varXk:~j = varyi:l, and nl = n2, it is from T(nl + n2 - 2). However, this equality of
variances rarely arises in simulation practice. Welch's solution to this problem relies on
properties of the chi-squared distribution; in particular, a chi-squared variate Z with 1
degrees of freedom has EZ = 1 and var Z = 2/.
Therefore,

2E 2 Z
1--- (7.10)
- varZ'
304 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

d= 22.89
0.8 +----------'~-----------__l

0.6

0.4

0.2

a
a 2 3 4 5 6

e
Figure 7.1 Credibility analysis for J-Ll - J-Lll > () in the photocopying problem
(Section 1.4)

Welch employs this equivalence to derive approximate degrees of freedom. Let a := var xt~l
and b := var Y~::2' Since

and

expression (7.lO) gives

d(c,nb n 2) = [ --
C2
+
(1- C)2]-1 ,
nl - 1 n2 - 1
._ a/nl
c.- ,
a/nl + b/n2
and d(c, n, n2) in expression (7.11 ) is an estimate of this quantity with S2(X) and s2(y)
replacing a and b, respectively.
 7 .4 . RAT loA N A LY 5 I 5 305

7 .4 RATIO ANALYSIS

Assume that X and f are independent and normally distributed with variances a/n] and
b/n2, respectively. Occasionally, it is of interest to estimate the relative difference R :=
f.L yj f.L x, assuming f.L x i= O. Since f - R X is normally distributed with mean zero and
variance b/n2 + R 2a/nJ> we have by analogy with the development in Section 6.11

~ 1-8, (7.11)

where v := <1>-]0 - 8/2).

This result has meaning only if Xb/n2 + fa/n] - v 2ab/n]n2 ::: v and X2 - v 2a/n] >
0.1t can be shown that both conditions hold asymptotically w.p.1 as min(n], n2) ~ 00.
Since a and b are unknown, we replace them by S2(X) and S2(Y), respectively, and
<1>-]0 - 8/2) byrdO- 8/2), where d = d(c, n], n2) and

(7.12)

and compute an approximating 100 x (1- 8) percent confidence interval for R. The rationale
for this choice of d comes from the observation that

E [S2(Y) + R 2S2(X)] = .!!...- + R 2a

n2 n] n2 n]

and

which, by analogy with the equivalence (7.10), leads to an approximating number of degrees
of freedom

b2 a2
4 +----;-4----
f.Ly(n2 - l)n2 f.Lx(n] - l)n]

which we estimate byd = fCc, n]n2)'

306 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

7.5 COMPARISON OF MEANS ON TWO SAMPLE PATHS

For single replications (nl = n2 = 1), the comparison of experiments proceeds like that for
multiple replications but with several differences. We assume that tl and t2 are sufficiently
large so that varXkll/1 ~ a~(X)/tl and varl\,1t, ~ a~(Y)/t2 (Section 6.3). Then we
employ LABATCH.2 (Section 6.7) to compute a strongly consistent estimate a~(X) =
BJ(ld WL](tliB]ltli for a~(X) from the sample record Xk l , ••• , Xkl+/I-I' For a~(Y) we again
use LABATCH.2 to compute a a~(Y) = BJ(/,) WL]it,)B](t,) based on the sample-path data
Yk".·., Yk,+I,-I.
Recall that L J(/I) denotes the estimated number ofindependent batches used in the final
review of Xkp ... , Xkl+/I-1 and LJ(/,), the estimated number of independent batches used
in the final review of Yk" ... , Yk, +I, -I. By analogy with the case ofindependent replications,
this suggests the approximating 100 x (1 - 8) percent confidence interval

where

d = d(c, LJ(/d - 1, LJ(/,) - 1) (7.13)

and

.. a~(X)/tl
c := a~(X)/tl + a~(Y)/t2'
and expression (7.9) defines d(·, ., .).
As before, d varies with c, from a minimum ofmin(LJ(/,), LJ(/,) - 1 to a maximum
of LJ(/,) + LJ(/,) - 2. Setting d = min(LJ(/d, LJ(/,) - 1 would again lead to a wider
confidence interval and thus militate against any tendency to induce a coverage rate less than
1 - 8 because of the inherent approximations in the degrees of freedom.

7.6 MULTIPLE COMPARISONS

Occasionally, a simulation study addresses differences between corresponding performance
measures that materialize in a system run under more than two different scenarios. The
purpose of the study is to make inferences about the relative desirability of each scenario
based on the estimates of long-run averages and their statistical errors. The simulationist
then has the task of reporting these inferences to decision makers. In addition to being
scientifically defensible, the collection of assertions in the report needs to be organized into
a way that weaves a story that makes sense of the many results and can act as a guide for
 7.6 . M U L TIP LEe 0 M PAR I SON 5 307

decision making. When two scenarios are being compared, the accounts in Sections 7.2 and
7.4 provide a basis for accomplishing this. When there are more than two, organizing the
results becomes a greater challenge.
Formalisms exist in the statistical literature for making these multiple comparisons.
Banks et al. (1996, Chapter 13) and Law and Kelton (1991, Chapter 10) address these issues
within the special context of discrete-event simulation. The present account takes a more
illustrative approach, focusing on a particular example to describe concepts. The presump-
tion is that the particular issues raised in the example carryover to other problems studied
by discrete-event simulation. Our account is based on the concepts of simultaneity (Section
7.1) and the analysis of pairwise differences (Sections 7.2 and 7.4).

7.6.1 LIBRARY PHOTOCOPYING

Recall that the purpose of the library photocopying simulation study in Section 1.4
was to assess the extent to which alternative operating policies affect long-run patron waiting
time. At current demand rates, these included:

SCENARIO

Current operating system

3 Current operating system with no copier breakdowns

5 Scenario 1 plus one additional copier on the third floor

7 Scenario 5 plus one additional copier on the fourth floor

9 Move the copier on the fifth floor to the fourth floor

11 Replace two copiers on the third floor and two on the fourth floor with new copiers.
The replaced copiers have the highest failure rates.

The study also examined these scenarios with a fifty percent increase in demand (these
are Scenarios 2, 4, 6,8, 10, 12). Here we address only the current-demand case. Moreover,
we focus on Group 3 (Table 1.1), the time interval from 11 a.m. to 4 p.m., the busiest on a
daily basis.
Let f..ti denote long-run patron waiting time for Scenario i, so that interest focuses on
estimating f..tl, f..t3, f..ts, f..t7, f..t9, f..tll and comparing their point estimates. For each scenario,
a single run of t = 1000 weeks was made. This corresponds roughly to 20 years of weekday
operation. For each Scenario 2i -1 E {I, 3, 5, 7, 9, Il}, a sample average waiting time P,Zi-l
was estimated. Because the elapsed run time was fixed, the number of sample waiting times
in this average was random, so that P,Zi-l was a ratio estimate (Section 3.4). However, its
denominator had a coefficient of variation considerably smaller than that of its numerator,
and hence ignoring the randomness in the denominator introduced negligible error. This
308 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

was true for all scenarios. See Section 6.11 for details of ratio estimation and, in particular,
on how to assess the importance of randomness in a denominator.
Table 7.la displays summary results for 1l2i-1 for i = I, ... ,6. The quantity &~ 2i-1
denotes an estimate of t varll2i-1 computed by LABATCH.2 and degrees of freedom are
one less than the number of batches determined by LABATCH.2. See Section 6.6. Although
the run length of 1000 weeks resulted in relative errors considerably larger than the .20
that Section 6.6.1 regards as desirable, the large number of runs dictated this limitation on
length. For visual comparison, Figure 7.2 displays the approximating 99 percent confidence
intervals.

NATURAL ORDERING

The structure of a particular problem often allows a simulationist to anticipate a natural

ordering among some, if not all, long-run averages prior to output analysis. In the present
study, the scenarios imply

f.l-I > f.l-ll > f.l-3,

f.l-I > f.l-s > f.l-7,

merely based on the knowledge that waiting time diminishes with decreasing failure rate
and/or adding resources. Therefore, the analysis principally focused on how large the
difference f.l-I - f.l-j is for f.l-I > f.l-j.
Table 7.lb gives estimates of differences between long-run average waiting on the
baseline Scenario I and the remaining five scenarios (Section 7.4). Of principal interest is
f.l-I - f.l-3, the greatest possible reduction in long-run average waiting time that can materialize
without adding or reassigning resources. Ofcourse, this is an ideal, since all copiers eventually
break down. Nevertheless, it is instructive to note that III - 113 = 3.635 minutes with an
approximating 99 percent confidence interval [2.656,4.613].
Table 7.1 b also reveals that reassigning old copiers and adding a single new copier to
the fifth floor (Scenario 7) induced an estimated reduction of III - 117 = 2.092 minutes
with approximating 99 percent confidence interval [.4242,3.759]. Also, adding two new
copiers on the third and also on the fourth floor (Scenario 11) induced a reduction of
III - Illl = 2.828 minutes with approximating 99 percent confidence interval [1.806, 3.849].
Recall that the structure of the problem allows no a priori inference regarding Scenarios 7
and II. Although the point estimates seem to favor Scenario 11 over Scenario 7, the overlap
in their confidence intervals raises a question.
The entries in Table 7.1 c for 117 - III I suggest that with current sample sizes a significant
difference between f.l-7 and f.l-ll is not supportable. We return to this issue presently. First, we
note that Table 7.ld estimates the difference f.l-3 - f.l-ll by 113 - Illl = -.8089 minutes with
approximating 99 percent interval [-1.221, -.3923], suggesting that it is not unreasonable
to expect Scenario II to reduce long-run average waiting time by at least 2 of the estimated
3.635 minute reduction achievable under the perfect-copier Scenario 3.
 7.6. MULTIPLE COMPARISONS 309

Q) 6
E
:;:::;
g>5
:;:::;
.(tj
~ 4
Q)
en
as
....

I
Q)
3
~
c::
~ 2
en
c::
0
...J
1 -

0
3 5 7 9 11

Scenario

Figure 7.2 Waiting time for photocopying problem in Section 1.4

Implementing Scenario 7 would result in a smaller immediate capital cost than Sce-
nario 11. However, there is an issue of how much better it is than Scenario 1 and how much
benefit one gives up by not choosing Scenario 11. To shed light on this problem, consider
two hypotheses:

HI : J-ti - J-t7 ~ 2 minutes;

H2 : J-t7 - J-tll ::: 1 minute.

If HI and H2 were true, then Scenario 7 would induce a reduction of at least two minutes in
long-run average waiting time and would be within one minute of the average that would
obtain with Scenario 11. We now examine the credibility of these hypotheses.
For (i, j) = (1, 7) and (7, 11), let

Z.).. (J-t.. _ J-t).-

.)._ f.ti - f.t j - J-ti + J-t j ,
/(a!, i + a!, )/t

whose approximating distribution is Student's t distribution with dij degrees of freedom

computed as in expression (7.13) using the output in Table 7.1.
310 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

Table 7.1 Long-run average waiting-time analysis for photocopying problem in Section 1.4

(time in minutes)
(a)
Relative error
Sample Standard 99%
Upper-Lower
mean error Confidence interval lil,l
fii .,j8-"!x,dt dJ. Lower Upper

5.412 .3199 12 4.435 6.389 .3611

3 l.778 .01559 25 l.734 l.821 .04893

5 4.676 .3280 17 3.725 5.626 .4066

7 3.321 .5097 17 l.844 4.798 .8896

9 4.784 .4645 12 3.365 6.203 .8023

11 2.585 .1478 25 2.173 2.997 .3190

(b)
Upper-Lower
fil - fii .,j8-"!x,1 + 8-"!x,i)/t c lil,-ilil
3 3.635 .3203 .9976 12.06 2.656 4.613 .5384

5 .7363 .4582 .4876 28.37 -.5298 2.002 3.439

7 2.092 .6018 .2827 27.08 .4242 3.759 l.594

9 .6282 .5640 .3218 21.29 -.9688 2.225 5.663

11 2.828 .3524 .3524 17.29 18.06 3.849 .7224

(c)
Upper-Lower
fi7 - fill J(8-"!x,7 + 8-"!x,.II)/t lil,-illli

.3618 .5307 .8642 22.38 -1.134 l.858 8.270

(d)

fi3 - fill J8-"!x,3 + 8-"!x,.II)/t

-.8069 .1487 .01099 25.56 -1.221 -.3925 l.027
 7.6. MULTIPLE COMPARISONS 311

Al (v) := event of observing a standardized statistic no greater than ZI.7(O)

under the assumption that ILl - 1L7 = 0

and

A 2(v') := event of observing a standardized statistic no less than Z7.11 (<p)

under the assumption that 1L7 - 1L11 = <p'.

Then

and

and using Bonferroni's inequality,

Table 7.2 gives these probabilities for selected 0 and <p. While there is some support
for ILl - 1L7 ~ 2 and 1L7 - 1L11 ::::; 1, there is considerably more for ILl - 1L7 ~ 1.5 and
1L7 - 1L11 ::::; 1.

Table 7.2 Credibility analysis for photocopier problem for long-run average waiting
time (minutes: seconds)

G27.0S (ZI.7(B» - G22.3S (Z7. 11 (1/»)

B\I/> .30 45 1:00 G27.0S(ZI.7(B»
1:30 .4345 .5969 .7121 .8329

1:45 .3141 .4765 .5917 .7125

2:00 .1614 .3238 .4390 .5599

1 - G 22.3S (Z7, 11 (1/») .6015 .7640 .8791
312 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

7.7 COMMON PSEUDORANDOM NUMBERS REVISITED

Recall that Section 4.22 describes how the technique of common pseudorandom num-
bers (cpn) applied to two simulation runs, each based on a different scenario, can induce
var(X - Y) < varX + varf. Since confidence-interval length tends to be proportional to
standard error, this reduction in variance implies a shorter interval length for the difference
of two means than would arise for independent runs. This property motivates us to identify
circumstances in which a simulationist may wisely employ cpn. We begin with a review and
extension of the formalisms in Section 4.22. Section 7.7.1 then describes which differences
in scenarios can be expected to reduce var(X - Y) and which are less likely, if at all, to do
so. Section 7.8 then compares the benefits of cpn with those of another variance-reducing
technique, based on optimal path lengths, which has particular appeal when one scenario
induces considerably larger sampling fluctuations than the other.
Recall from Section 4.22 that using cpn leads to

Vepn := [O'!(X) - 2p(X, Y)O'oo(X)O'oo(Y) + O'!(Y)] It,

where p(X, Y) = corr(X, Y). Hereafter, we use the more concise notation
0'00 (X)
h := - (-) and p := p(X, Y). (7.14)
0'00 Y

The induced variance reduction is

variance of X - Y with cpn

(7.15)
VRcpn := variance of X - Y for independent runs
= 1 - .,----2--,-P"""'C'C'"
h + 1/ h'

revealing that max (h, 1/ h) and p determine the level of variance reduction. In particular, for
given p, YRcpn increases monotonically as max(h, 1/ h) increases, thus limiting the appeal
of cpn. If p < 0, then variance inflation occurs, and cpn is to be avoided. Hereafter, we
assume p 2: O.
The quantity 100 x (1 - .JVRcpn) gives the percentage reduction in the length of
a confidence interval when cpn, rather than independent runs with tl = t2, is used. Table
7.3 displays 1 - JVRcpn as a function of selected hand p. For equal asymptotic variances
(h = 1), it reveals a 68 percent reduction for p = .90. However, max(h, 1/ h) = 10 leads
to only a 9.3 percent reduction for this same p. For p = .50 the reductions are 29.3 and 5.1
percent for max(h, 1/ h) = 1 and 10, respectively.
Recall that cpn in Section 4.22 estimates tVcpn = O'&,(Z) by &&,(Z), computed by
LABATCH.2 using the combined sample-path data Zi := Xi - Yi for i = 1, ... , t. Using
the sequence of differences in this way captures the correlations between XI, ... , XI and
Y I , •.. , Yt in a single variance correlation. However, expression (7.13) no longer applies for
 7.7. COMMON PSEUDORANDOM NUMBERS REVISITED 313

degrees of freedom. In this case treating (X - Y)/ J(i~(Z)/t as a Student t variate with
degrees offreedom one less than the number of batches used to compute a~(Z) allows one
to compute an asymptotically valid 100 x (1- 8) percent confidence interval for /Lx -/LY.

7.7.1 HOW SCENARIOS AFFECT CORRELATIO~

Although it falls to the simulationist to match pseudorandom number streams and

seeds on the two runs, it is the properties induced by the two scenarios that ultimately dictate
the correlation p and the ratio h. This section addresses the way in which scenarios affect p.
The next addresses the way in which h affects variance reduction in cpn and how one can
take advantage of a rough estimate of h to reduce variance in yet another way.
Recall that a scenario consists of a set of numerical values assigned to input parameters
and a collection oflogical operating rules. For example, the parameters for the airline reser-
vation problem include the intercall rate A, the service rate w for a single destination, the
proportion of customers P who plan multidestination trips, and the number of reservation-
ists m. The set also includes distinct seeds for the four sources of pseudorandom variation
and the number of customers t to be simulated. The problem has a single operating rule:
Calls are serviced in the order they arrive.
The first example in Section 4.22 shows how cpn works when m = 4 and 5 and all other
parameters and rules remain the same. By assigning separate pseudorandom number streams
to each source of variation in that problem, customer i was assigned the same intercall time
and the same service time on both runs for each i = 1, ... , t, thus inducing the sample
correlation p(X, Y) = .7033 for the sample average waiting time and p(X, Y) = .8906 for
the sample exceedance probability.
The second example keeps m = 4 but considers the effect of varying P from PI = .75
to P2 = .90. This assignment results in customer i being assigned the same intercall time
on both runs but not necessarily the same service times. However, by following the strategy
described in Section 4.22, 100 [min(1 - PI, 1 - P2) + min(PI, P2)] = 85 percent of the
customers can be expected to be assigned the same service times on both runs. With regard
to exceedance probability, the (m = 4, PI = .75) and (m = 4, P2 = .90) assignments
induced a sample correlation

A( )
p X Y =
a~(X) + a~(Y) - a~(Z) = .8656,
, 2Ja~(X)a~(Y)

a relatively small loss from .8906. Note that in this particular problem at least 50 percent of
the service times match regardless of the value of PI and P2.
Suppose the two runs were made with the same values for w, p, and m but two different
values of A, namely Al and A2. Then customer i has the same service time on each run, but
its intercall times differ. The extent to which this difference militates against a variance
reduction depends on the particular distribution being sampled and on the method used to
generate intercall times. Recall that intercall times in the example come from £(1/A).
314 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

Let VI and V2 denote customer i's intercall times on the runs with Al and A2, respec-
tively. If the inverse transform method (Section 8.1) were used to generate these quantities
with the same uniform deviate U E (0, 1), then

1
VI = - Al In(I - U),
1
V2 =- A2 In(I - U),

so that

implying

corr(VI , V2 ) = 1.
We see that corresponding intercall times differ on the two runs but are nevertheless perfectly
correlated. While not as favorable as inducing the same intercall times, perfect positive
correlation contributes to a positive correlation between the output on the two runs, thus
continuing to give a benefit.
But what if a method other than the inverse transform method were used to generate
intercall times? With the exponential distribution, we are in luck. Recall from Section 8.9 that
all algorithms for sampling from E(I/A) actually sample from E(1) and then multiply the
result by 1/ A, thus ensuring perfect correlation. More generally, perfect correlation between
VI and V2 arises regardless of distribution if the number of uniform deviates required to
generate VI is independent of the intercall time parameters and V2 is computed as V2 =
a VI +b forsomea > Oandb. This includes samplingfromE(I/A) withvaryingA, N (fl, 0'2)
with varying fl and 0'2, gca, f3) with varying f3 but the same a, and W(a, f3) with varying
f3 but the same a. It also applies to the inverted Gamma and inverse Weibull distributions
for varying f3.
Suppose intercall times are drawn from Q(al, f3) and Q(a2, f3) on the two runs using
Algorithm GKM3 (Section 8.13) based on the ratio-of-uniforms methods (Section 8.6).
Recall that this method uses a random number of uniform deviates to generate samples. If
al = a2 and the same seed is used on the two runs, then both use the same pseudorandom
number deviates, and the seeds for the next generation are identical. However, if al =f:. a2,
generation of the two runs may require different numbers of uniform deviates so that the
seeds for the next generation differ. Then synchronization of pseudorandom numbers, so
vital to inducing positive correlation, is lost. In practice, it is ill advised to use the same seed
in this case, lest the practice unintentionally induce a negative correlation.
What about different logical operating rules? These include job routing through a
system and job selection for service. For example, suppose one run takes jobs for service in
 7.S. OPTIMAL SAMPLE-PATH LENGTHS 315

the order of arrival, whereas the second run takes them in reverse order. As a consequence, we
anticipate that the waiting times for customer i on the two runs are negatively correlated, so
that sample average waiting times are negatively correlated, and therefore their difference has
larger variance than would arise for independent runs. Interestingly, the long-run average
waiting times are identical for both queueing disciplines.
Unless the implications for induced correlation are known and favorable, it is gener-
ally advisable to use independent runs, rather than cpn, when comparing different logical
operating rules.

7.7.2. MULTIPLE COMPARISONS

What if we are comparing several sample averages as in Section 7.6? For pairwise
comparisons, the discussion in these sections continues to apply with the variance of each
pairwise difference being estimated as in Section 4.22 and simultaneity entering as in Section
7.l.

7.8 OPTIMAL SAMPLE-PATH LENGTHS

Other variance-reducing techniques exist that do not require correlation between sample
averages. This section describes one for which variance reduction improves as max(h, 1/ h)
increases.
Let us examine the approach of Section 7.7 more closely for independent runs gener-
ating X and Y based on sample-path lengths tl and t2 not necessarily equal but satisfying
tl + t2 = 2t. Then executing two independent runs with sample-path lengths

2th 2t
and
1 +h
t ---
1-
t ---
2 - 1 +h
(7.16)

induces minimal variance

Vopt
-
:= var(X - Y)
-
= [O'oo(X) + O'oo(Y)] 2 /2t
and variance reduction

var(X - Y)
VRopt := ----=----=- (7.17)
varX+varY

Table 7.3 shows the reduction in interval length that opt induces for selected
max(h, 1/ h). Clearly, there is a crossover point at which opt offers more of a reduction
than cpn. In particular,

Vepn ::; Vopt if max(h, 1/ h) ::; 1 + 2p + 2-/ p(l + p),

316 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

Vcpn > Vopt otherwise.

As illustrations,

P 1 + 2p + 2J p(1 + p)
0 1
.25 2.618
.50 3.297
.75 4.791
.90 5.415
.95 5.622
1.0 5.828.

In particular, note that for all p E (0, 1) opt induces smaller variance than cpn if
max(h, 1/ h) > 5.828. Also, ! :::
YRopt ::: 1 with

~->{
1 asmax(h, l/h) --+ 1,
1
asmax(h, l/h) --+ 00,
2

whereas 0 ::: VRq,n ::: 1 with

I-p asmax(h, 1/ h --+ 1

VRcpn --+ {
0 as max(h, 1/ h) --+ 00.

Clearly, cpn with t1 = t2 = t has its greatest advantage when variances are equal,
!
whereas opt, with t1 + t2 = 2t, achieves its best variance reduction of when a~(X) and
a~(Y) differ substantially. The opt variance-reducing technique does have the limitation
that h is unknown and, in principle, must be estimated in some way. We defer discussion of
this issue until Section 7.10
Because of the independence of the sample paths X I> ... , XI, and YI> ... , Y12 , expres-
sion (7.13) can be used to estimate degrees of freedom for (X -y)/ Ju'&,(X)/t1 + u'&,(Y)/t2,
and an asymptotically valid 100 x (1 - 8) percent confidence interval can be computed for
JLx - JLy.

7.9 UNEQUAL SAMPLING COSTS

The foregoing discussion assumes that generating costs per observation are identical, or close
to identical, on each scenario. Cost is customarily measured in terms of cpu time expended
per simulation observation. If these costs differ substanially, then we need to reformulate
the opt approach and its comparison to cpn.
 7.9. UNEQUAL SAMPLING COSTS 317

Table 7.3 Proportional reduction in confidence interval

length for cpn and opt

max(h, 1/ h) 1 - ylVRopt p 1 - ylVRpn

0 0 0
.25 .l340
.50 .2929
.75 .5000
.90 .6838
.95 .7764
1.0 1.0

2 .05l32 0 0
.25 .1056
.50 .2254
.90 .4709
.75 .3675
.95 .5101
1.0 .5528

4 .1425 0 0
.25 .06066
.50 .1255
.75 .1956
.90 .2407
.95 .2564
1.0 .2724

10 .2260 0 0
.25 .02507
.50 .05080
.75 .07724
.90 .09348
.95 .09896
1.0 .1045
318 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

Let b l and b2 be the unit costs on runs 1 and 2, respectively, and suppose a total budget
constraint

exists. Then the variance-minimizing (tl, t2) is

and (7.18)

with

_ b
Vopt -
1 [ b 1/2 O"oo(X) + b21O"oo(Y)
/2]2 ,
l

whereas cpn with tl = t2 = b/(b l + b 2) induces

As immediate consequences,

1 + bl as h-+O i.
b2

1 + b2 as h -+ 00 ii.
bl
bl
h 2 - 2ph +1 as - -+ 0
b2
iii.
vcpn
--+ bl
Vopt 1-2p/h+l/h 2 as - -+ 00 iv.
b2
(h 2 + 1)(1 + bI/b2)
as p -+ 0 v.
(hb~/2 /by2 + 1)2
(h - 1)2(1 + bI/b2)
as p-+l. vi.
(hb~/2 /b~/2 + 1)2

Limits i, ii, and v indicate that opt induces smaller variance than cpn. Limit iii shows
that opt leads to a smaller variance for p < h /2 and a larger variance for p > h /2. Therefore,
! !
limit iii favors opt for all h > 2. Conversely, limit iv favors opt for p < h and cpn for p > h.
This implies cpn for all h < 2.
Limit vi gives cpn its greatest potential advantage with regard to correlation.
Nevertheless, opt induces a smaller variance if

max(h/g, g/ h) > 1 + g + l/g + ..j(g + l/g)2 + 2(g + l/g),

 7.10. ERRORS IN OPTIMALITY 319

where

More generally, opt induces smaller variance if

max(h/g, g/h) > 1 + (g + l/g)p + J(g + l/g)2pz + 2(g + l/g)p, (7.19)

and cpn induces smaller variance otherwise.

As mentioned before, variances tend to increase as congestion increases. Also, the
computing time to generate an observation tends to increase, but usually at a slower rate
than variance. Hence, for a~(X) > a~(Y) it is not unusual to expect h ~ g. Figure 7.3
displays the bound (7.19) for this case. If h Z lies in the region above the line corresponding
to a given p, then opt gives the smaller variance. Indeed, for p = 1 and gZ ::: 1, we see that
opt gives the smaller variance for all hZ ~ (3 + 2..{i? = 33.97.
In summary,

• For fixed p and g, opt gives the greater variance reduction for sufficiently large h.
• For fixed hand p, cpn gives the greater variance reduction for sufficiently large g.
• For fixed hand g, cpn gives the greater variance reduction if h > 1 + g + g1/Z +
J(l + g)Z + 2g1/z(l + g) for all p E [0,1].
• For fixed h and g, opt gives the greater variance reduction if h > g1/Z as p ~ o.

7.10 ERRORS IN OPTIMALITY

To become operational, opt requires values for hand g. Since these are unknown, a sim-
ulationist customarily substitutes estimates for them. If preliminary runs make available
rough estimates ofaoo(X) and aoo(Y), then these can serve to estimate h. The question now
becomes, How rough can these estimates be and yet lead to a variance reduction?
For the moment, assume that g is known and that ah is the estimate of h, where a
is a positive quantity. If ah replaces h in computing tl and tz in expression (7.18), then n
analogous to the ratio (7.17) we have

(a) ._ (gh/a + l)(gha + 1)

"VRopt .- (h Z + l)(gZ + 1) .

A variance reduction occurs (i.e., VRopt(a) ::: 1) for all

(7.20)
320 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

200 --------------------- ---

P =1
180
p=.95
160 P =.90
140
P =.75
120

a;,(X) 100
a;,(Y)
80
P =.50
60

40
p=.25
20
p =0
0
0 0.5 1.5 2 2.5 3 3.5 4

Figure 7.3 cpn vs. opt

For example, if in reality h = 10 and r = 2, then a variance reduction occurs if one

overestimates h by a factor no greater than a = 5 or if one underestimates h by a factor
no smaller than a = k.
More generally, inequality (7.20) reveals that variance reduction is
possible even with large errors of approximation.
Although the unit costs b l and b2 rarely are known exactly, timings for preliminary de-
bugging runs usually can provide good approximations. Therefore, degradation in variance
reduction due to replacing b l and b2 with estimates should be negligible.

7.11 OPT AND MULTIPLE REPLICATIONS

To extend opt to multiple comparisons, we focus on a special but commonly encountered
case. Assume r scenarios with long-run averages ILl, ... , ILr. The principal objective is to
compare ILl with each of the remaining ILi. Let ti denote sample-path length under Scenario
i,

2
(foci:=
1.1m tivafl.Li,
A

ti~OO
 7.12. L E S SON S LEA RNED 321

and

i = 2, ... , r.

If cost per observation is the same on all r runs, then for fixed rt = t] + ... + t" choosing
rt
t] = ,
1 + 1/ h2 + ... + 1/ hr
t]
tj = - j = 2, ... , r,
hj

minimizes each var(!L] - !Li) for i = 2, ... , r. See expression (7.16).

Suppose unit costs b], ... , br are unequal and let

gj := ~, j = 2, ... , r.

Then for a fixed cost b = bIt] + ... + brt" choosing

and

t]g]
tj = h-' j = 2, ... , r,
J

minimizes all var(!L] - !Li) for i = 2, ... , r. See expression (7.18). The commentary in
Section 7.11 regarding estimation of g and h continues to apply here.

7.12 LESSONS LEARNED

• Bonferroni's inequality provides a means for computing a lower bound on the
probability that several random events occur simultaneously.

• In computing a confidence interval for the difference of means based on two inde-
pendent runs made with different input scenarios, Welch's approximation provides a
means for computing degrees of freedom.

• Simultaneous confidence statements about pairwise differences for multiple runs, each
with a diferent scenario, provide a basis for weaving a coherent story about what the
data show.
322 CHAPTER 7. MAKING SENSE OF OUTPUT AND INCREASING EFFICIENCY

• Common pseudorandom numbers (cpn) can reduce the variance of the difference
of sample averages computed from simulation runs executed under different input
scenarios.

• The potential variance reduction of cpn decreases as the variances of the individual
sample averages diverge.

• cpn also apply to multiple comparisons of pairwise differences of means.

• For independent runs, opt is a variance-reducing technique that chooses sample-path

lengths to minimize the variance of the difference of two sample averages.

• opt takes into consideration differences in sample-average variances and differences

in unit data-collection costs on each run.

• opt also applies to multiple comparisons of pairwise differences in means where one
mean is common to all differences.

7.13 EXERCISES
QUARRY-STONE CRUSHING (EXERCISE 6.2)

~ 7.1 Exercise 6.2 provides estimates oflong-run average profit as a function of A, the
arrival rate of external deliveries to the crusher. Presumably, these estimates have allowed you
to estimate Amax> the rate that maximizes long-run average profit. For institutional reasons,
management is inclined to choose A = .15 as the permissible arrival rate, provided that this
action results in no more than a 1.5 cent reduction in profit per ton.

a. Let p(A) denote long-run average profit at rate A. Determine the credibility of the
hypotheses

P(AmaxJ - p(.15) ::: d for all d E [1,2].

b. Interpret the results of part a in light of management's qualification of no more than

a 1.5 cent loss in profit per ton.

LOADING TANKERS (EXERCISE 6.4)

~ 7.2 The port authority wants to reduce ILr, the average rebate per tanker. One plan to
do so replaces the tug by a new one that takes £ ( ~) time to berth an empty tanker, and £ ( ~ )
time to deberth a full tanker, and £ ( ~) to deadhead in either direction. As before, times are
in hours. Let ILr(old) and ILr,(old) denote the quantities estimated in Exercise 6.4 with the
old tug and let ILr (new) and ILr' (new) be the average rebates with the new tug.
 7.13. Ex ERe I S E S 323

a. Rerun Exercise 6.4 with these new times to estimate J.Lr(new) and J.Lr' (new). Since the
system now has less congestion, the warm-up interval estimated in Exercise 6.3 should
suffice.

b. Perform an analysis to determine the credibility of J.Lr(old) - J.Lr (new)::::: dJ.Lr( old) for
each dE [.2, .75].

c. Perform an analysis to estimate how large management can expect the reduction in
J.Lr,(old) - J.Lr,(new) to be. What does this analysis reveal about waiting patterns for
the old and new tugs?

TERMINAL PAD CONTROLLER (EXERCISE 6.6)

~ 7.3 This exercise provides an opportunity to evaluate the benefits of common pseu-
dorandom number streams for the case of rn = 8 maximal characters in a packet. Part a
establishes a baseline against which the results in part b, based on common pseudorandom
numbers, can be compared. Let ~min denote your estimate of A that minimizes mean delay
J.L(A, 8) in Exercise 6.6.

a. Use the results of the runs in Exercise 6.6 to compute point and 99 percent interval
estimates for the difference in mean delay J.L(A, 8) - J.L(~min, 8) for each (A =f=. ~min)
used for rn = 8.

b. Repeat the simulation runs for all A with the seeds used for the run with A = ~min
in Exercise 6.6 being the common pseudorandom numbers seeds on all runs. As a
consequence, results for the different A are correlated. Use these data to compute
point and 99 percent interval estimates of J.L(A, 8) - J.L().min, 8). Then for each fixed Ai
compare the statistical accuracies of those differences with those computed in part a.

MOBILE-TELEPHONE SERVICE (EXERCISE 6.8)

~ 7.4 The set C = {.Ii; i = 1, ... , 9} of cost evaluation points in Exercise 6.7 is in-
tended merely to provide a perspective on the best choice of r for each rn. In reality, cost c takes
some value in (0, 1) but is unknown. Suppose management has the prior Beta distribution
Be(3, 1.11) (Section 8.14) for c.

a. For each (rn, r) in Exercise 6.7, compute a point estimate C(rn, r) of

C(rn, r) := expected long-run average cost

for c distributed as Be(3, 1.11).

b. For each (rn, r), compute an approximating 99 percent confidence interval for each
C(rn, r) and for the function {CCrn, r), r = 0,1, ... , rn - 2}.
324 C HAP TE R 7. M A KIN G SEN 5 E 0 F 0 U T PUT AND INC REA 5 I N G EF Fie lEN C Y

c. For each m, estimate

Cmin(m):= min C(m, r)

O:Y~m-2

and

rmin(m) := the minimizing r.

One approach, called Alternative A, to system specification for fixed m chooses r min (m ).
An alternative approach, called Alternative B, makes use of the piecewise constant
function {rmin(m, c), C E C}, where for each r E to, 1, ... ,m - 2}

rmin(m, c) := r that minimizes C(m, r, c).

Since C changes randomly every 12 months, management is considering a policy that

replaces rmin(m, Cold) with rmin(m, cnew) each time Cchanges, where cold denotes the
old cost and Cnew the new.

d. For each m, compute the expected cost of Alternative R Hint: Start by computing the
probabilities of choosing each r for r E to, 1, ... ,m - 2}.

e. Since there is an expense to changing r annually, management wants some assurance

that any savings that Alternative B offers is at least 20 percent of Cmin (m) for each m.
Evaluate the credibility of this statement.

7.14 REFERENCES

Banks, J., J.S. Carson, III, and R L. Nelson (1996). Discrete-Event System Simulation, second
edition, Prentice-Hall, Upper Saddle River, New Jersey.

Bechhofer, R.E., T.J. Santner, and D. Goldman (1995). Design and Analysis for Statistical
Selection, Screening and Multiple Comparisons, Wiley, New York.
Fishman, G.S. (1996). Monte Carlo: Concepts, Algorithms, and Applications, Springer-Verlag,
New York.
Kendall, M.G., and A. Stuart (1961). The Advanced Theory ofStatistics, 2. Hafner, New York.
Law, A.M., and W. D. Kelton (1991). Simulation Modeling and Analysis, second edition,
McGraw-Hill, New York.

Matejcik, F.J., and RL. Nelson (1995a). Two-stage multiple comparisons with the best for
computer simulation, Operations Research, 43, 633-640.
 7.14. REF ERE NeE s 325

Matejcik, EJ., and B.L. Nelson (l995b). Using common random numbers for indifference-
zone selection and multiple comparisons in simulation, Management Science, 41,
1935-1945.
Scheffe, H. (1970). Practical solutions of the Behrens-Fisher problem, J. Amer. Statist. Assoc.,
65, 1501-1508.
 CHAPTER

G. S. Fishman, Discrete-Event Simulation

© Springer Science+Business Media New York 2001
Sampling from
Probability
Distributions

Virtually every commercially available product for performing discrete-event simulation

incorporates software for sampling from diverse probability distributions. Often, this incor-
poration is relatively seamless, requiring the user merely to pull down a menu of options,
select a distribution, and specify its parameters. This major convenience relieves the user of
the need to write her or his code to effect sampling.
Occasionally, a simulation requires samples from a distribution not available in the
user's selected software. To remove this limitation, this chapter describes the theory un-
derlying procedures and gives algorithms for generating samples from many commonly
encountered distributions. Although several alternative algorithms may exist for produc-
ing a sample from a particular distribution, the chapter repeatedly focuses on those that
guarantee a fixed mean computing time, or an upper bound on mean computing time that
holds uniformly, regardless of the values that the parameters of the distribution assume.
Moreover, for each distribution considered, the account concentrates on algorithms whose
computing times compare favorably with the computing times of other proposed algorithms
for generating a sample from the same distribution.
Expertise in generating samples from diverse distributions has grown constantly since
the feasibility of performing Monte Carlo experiments became a reality during World War II.
328 C HAP TE R 8. SAM PLIN G FRO M PRO BA B I LIT Y DIS T RIB UTI 0 N 5

Milestones in this development include the demonstrations in von Neumann (1951) of how
the principles of conditional probability could be exploited for this purpose, in Marsaglia
et al. (1964) of how a synthesis of probabilistic and computer-scientific considerations can
lead to highly efficient generating procedures, and in Ahrens and Dieter (1974a,b) of how a
bounded mean computing time could be realized for an arbitrary distribution.
Although the chapter describes procedures that are theoretically exact, the limitations
of working on a finite word-size computer inevitably produce results that differ in practice
from their theoretical ideals. The account notes where serious differences are most likely to
occur and emphasizes the need to employ methods that keep potential errors within tolerable
bounds.
Remarkably, all generating methods produce samples drawn from a specified distri-
bution by transforming a sequence of independent uniform deviates each randomly drawn
from U (0, 1). In practice, a sequence of (normalized) pseudorandom numbers (Chapter 9)
takes the place of the idealized sequence of uniform deviates, and inevitably, this substitution
induces an error of approximation. Chapter 9 describes in considerable detail how to control
the extent to which the distribution of k-tuples of these normalized pseudorandom num-
bers departs from the uniform distribution on the k-dimensional unit hypercube for each
k :::: 1. However, a more insidious source of error arises when the theoretically exact meth-
ods of this chapter use these pseudorandom numbers. Given several alternative methods for
transforming uniform deviates into a sample drawn from a particular distribution, some
are considerably more sensitive than others to the error of approximation that replacing
the deviates by pseudorandom numbers induces. In particular, a growing body of evidence
suggests that this sensitivity increases with the number of pseudorandom numbers that a
method requires to generate a single value from a specified distribution (e.g., Afflerbach and
Hormann 1990 and Hormann and Derflinger 1993).
In practice, this sobering discussion of error encourages two strategies. First, one
should always employ a pseudorandom generator whose error of approximation is no more
serious than the errors of approximation of available competing generators. Chapter 9 de-
scribes how to make this assessment and lists several generators that perform well according
to it. Secondly, when selecting a method to generate samples from a particular distribu-
tion, one should prefer those that require few pseudorandom numbers per sample. This last
stricture favors the inverse transform method (Section 8.1).
Today, many online computer program libraries provide programs for generating
samples from a wide range of distributions. Using these programs can save considerable
time and effort and, when appropriately implemented, can reduce the potential for incurring
serious numerical errors. The conscientious user of any such program will want to assure
herself or himself that the method on which it is based is technically sound and that its
implementation keeps the inevitable error within an acceptable limit. This chapter aims
at assisting the reader in making this evaluation by acquainting her or him with the exact
methodologies for generating samples and with their potential sources of error. Also, it aims
at providing the reader with the capacity to write a generating program, if necessary.
 8 .1. I N V E R5 E TR AN 5 FOR M MET HOD 329

This chapter assumes the availability of a relatively inexhaustible source of pseudo-

random numbers and Chapter 9 describes the detailed considerations that play roles in the
design and implementation of a pseudorandom number generator. This particular organi-
zation gives more continuity to the topics that immediately concern the potential user of
the Monte Carlo method. However, we emphasize that an understanding of issues related to
pseudorandom number generation is essential for one knowledgeably to assess the suitability
of computer library programs that claim to effect this generation.
Four principal methods exist for transforming uniform deviates drawn from U (0, 1)
into samples drawn from a specified distribution. They are the inverse transform method, the
composition method, the acceptance-rejection method, and the ratio-of-uniforms method.
Sections 8.1 through 8.6 describe them in detail, emphasizing that sampling from particular
distributions may require one to employ a combination of these methods. Of the remaining
available methods, Section 8.7 describes the exact-approximate method, which illustrates
how careful transformation of variates can reap benefits. Devroye (1986) describes many of
the remaining methods.

8.1 INVERSE TRANSFORM METHOD

The inverse transform method provides the most direct route to generating a sample.

Theorem 8.1.
Let {F(z), a .::: z .::: b} denote a distribution function with inverse distribution function

F- 1 (u) := inf{z E la, b]: F(z) 2: u, 0:::: u :::: I}. (8.1)

LetU denote a random variablefromU (0, 1). ThenZ = F- 1 (U) hasthed.f. F.

The proof follows from the observation that pr(Z .::: z) = pr[F- 1 (U) .::: z] = pr[U <
F(z)] = F(z).

This result immediately suggests the following generating scheme:

ALGORITHM ITM

Purpose: To generate Z randomly from {F(z), a .::: z .::: b}.

Input: Capabilityforevaluating{F- 1 (u), 0:::: u.::: I}.
Output: Z.
Method:
Randomly generate U from U (0, 1).
Z +- F- 1 (U).
Return Z.
330 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

8.1.1 CONTINUOUS DISTRIBUTIONS

Table 8.1 lists selected continuous distributions and their inverse d.f.s. Note that using
the modified form allows one to gain computational efficiency while preserving the distri-
bution of Z. For those continuous distributions whose inverse dJ.s have closed forms, the
inverse transform method offers the most direct method of sampling. If no closed form is
available but F(z) can be easily computed, an iterative method can facilitate sampling. For
finite a and b,Algorithm BISECTION with k(z) = F(z) - U,Zl = a andZ 2 = b determines
Z=z.
If Z is unbounded with fez) > 0, then the Newton-Raphson method, as described
in Algorithm N-R with k(z) = F(z) - U, produces a solution. If {fez)} is bounded and
unimodal, then the modal value provides a reasonable initial value zoo In addition to the
numerical error inherent in working on any finite word-size computer, the bisection and
Newton-Raphson methods both induce an additional error based on the specified error
tolerances. Since making v smaller inevitably increases the number ofiterations until success,
one must inevitably balance cost and accuracy considerations here. See Devroye (1986) for
a more complete discussion of this issue.

ALGORITHM BISECTION

Purpose: To compute the solution z to k(z) = 0 for Z E [Zl> z21.

Input: Function {k(z), ZI :::; Z :::; Z2} and error tolerances Vlo V2 > o.
Output: Z and k(z).
Method: 1. UntillZ L - zul :::; VI or Ik(z)1 :::: V2 :
a. Z = (ZL + Zu )/2.
b. Compute k(z).
c. If k(z) < 0, set ZL +- z; otherwise, set Zu +- z.
2. Return with Z and k(z).

ALGORITHM N-R

Purpose: To compute the solution z to k(z) ~ o.

Input: Functions {k(z), -00 < z < oo} and {k'(z) = ak/az, -00 < z < oo}, initial
value zo, and error tolerance v > o.
Output: z.
Method: y +- Zoo
Repeat
x+-y
compute k(x) and k' (x)
y +-x-k(x)/k'(x)
untilly - xl < v.
Return z +- y.
 distribution p.d.f. inverse dJ. preferred form ~
<:r
~
?C
.-
uniform
U>
U (a, 13) 1/(13 - a) a~z~f3 a + (13 - a)u
~
(')
....
beta p..
'"
az,,-l (')
Be(a,ll a> 0, 0 ~ z ~ 1 u l /" 0
i:l
beta
g.
~
13(1 - Z)/l-l 1 - (1 - U)l//l 1 - ul//l 0
Be(1,f3) 13 > 0, 0 ~ z ~ 1 ~
'"p..
exponential In'
....
..,
1 -'//l
-e 6'
E (13) 13 > 0, Z :::: 0 -13 In (1 - u) -13 In u ~
13 ;::to
0 00
i:l
logistic '" -'
e-(z-OI)//l
£ (a, 13) 13 > 0, -00 < Z < 00 a + f3ln[u/(l - u)]
13 [1 + e-(z-,,)//l F
non-central Cauchy z
13 <
rn
C(a, 13) 13 > 0, -00 < z < 00 a + 13 tan n(u - 1/2) a + f3/tan nu
(f)
n [13 2 + (z - a)2] ""
rn
normal
(2n(J2) -1/2 e-(Z-/L)' /2([' -i
N(JL,(J2) (J2 > 0, -00 < Z< 00 Mora (1995) ""
>
z
Pareto (f)

Pa(a, 13) af3" /ZOl+1 z :::: 13 > 0, a> 0 13/(1 - U)l/OI 13 /u 1/01 0

Weibull
s:""
W(a, 13) (a/f3 01 )zOl-le-(z/!W a,f3 > 0, Z :::: 0 f3[-ln(1- U)Jl/Oi f3( -In U)l/OI 3:
rn
-I
I
0
0

W
W
.....
332 C HAP TE R 8. SAM P LIN G FRO M PRO BA B I LIT Y DIS T RIB UTI 0 N 5

Although ease of implementation gives the inverse transform method its appeal, it
is not necessarily the most efficient method of sampling from a particular distribution. Its
principal limitation is its heavy reliance on functional transformation. In particular, sam-
pling from Be(a, 1), Be (1 , /3), and Pa(a, /3) call for two transformations per sample. For
example, a sample from Be( a, 1) has the form exp( a-I In u). Whereas computers that incor-
porate algorithms for evaluating these transformations into their hardware offer the fastest
time implementation, the more common case is to have them evaluated by software with cor-
respondingly longer times. Later sections describe alternative sampling methods designed
to reduce the frequency of performing transformations and thus reducing computing time.

8.1.2 RESTRICTED SAMPLU-4G

The inverse transform method offers one convenience that no other generating method
has. It allows one to generate a sample from the modified distribution

- F(z) - F(a')
F(z):= F(b') _ F(a') , a':::: a' .:::: z .:::: b' .:::: b, (8.2)

with the same ease as one generates a sample from the parent distribution {F(z),a .:::: z .:::: b}.
We refer to this alternative as restricted sampling. Replacing U by U' = F(a') + [F(b') -
F (a') 1U as the argument of the inverse dJ. accomplishes this result. Except for the additional
computation of F(a'), F(b'), and U', the times to generate Z from {F(z)} and {F(z)} are
the same.

8.1.3 DISCRETE DISTRIBUTIONS

The inverse transform method also applies to the case of discrete Z. Suppose that

F(z) = I qz,
0,
z = a, a
otherwise,
+ 1, ... , b,
(8.3)

where

qa-I = 0,
qz<qz+l, z=a,a+1, ... ,b-1,

Then an immediate result of Theorem 8.1 is that

Z := min{z : qz-I < U .:::: qz, U '" U (0, 1)}

has the requisite distribution. Algorithm DI describes the steps.

 8.1. INVERSE TRANSFORM METHOD 333

ALGORITHM DI

Purpose: To generate Z randomly from the distribution {O < qa < qa+ I < ... < qb = I}.
Input: {qa, ... ,qb}·
Output: Z.
Method:
Randomly generate U from U(O, I} and set Z +- a.
While U > qz : Z +- Z + 1.
Return Z.

Observe that the principal work here is a sequential search in which the mean number
of comparisons until success is

revealing that the mean generation time grows linearly with E Z.

ALGORITHM ITR

Purpose: To generate Z randomly from the p.m.f. {Po, PI, ... }.

Input: Po and {c(z + l}} from Table 8.2.
Output: Z.
Method:
P +- Po, q +- Po and Z +- O.
Randomly generate U from U (0, I).
While U > q : Z +- Z + 1; P +- pc(Z}; q +- q + p.
Return Z.

Implementation of this method can take at least two forms. Table 8.2 lists several
commonly encountered discrete distributions and shows how to evaluate each successive
probability Pz+I recursively from pz given Po. Then Algorithm ITR shows how to generate Z
from anyone of these distributions merely by specifying Po and c(z + I}. The discrete uniform
distribution Ud (a, fJ) is the exception here, merely requiring Z +- La + (fJ - a + l) U J. If
repeated generation occurs from the same distribution with the same Po, then computing
this quantity before generation begins, storing it, and reusing it as input save a major part of
the setup cost in computing Po. For example, computing Po for the binomial distribution
B(r, p} requires the operations Po +- exp[r In(l- p}].Aslater sections show, more efficient
generating procedures exist for each of these distributions in both the cases of varying and
constant input parameter values.
334 C HAP TE R 8. SAM P LIN G FRO M PRO B A B I LIT Y DIS TR I BUT ION 5

The discrete inverse transform method also allows one to generate Z from the restricted
distribution {(qz-qa'-l)/(qb'-qa,-d};z = a',a'+I, ... , b'},wherea :s a' < b':s bmerely
byinitiallysettingp +--- Pa', q +--- qa', and Z +--- a', and replacing U byqa'-l +U(qb' -qa,-d
in Algorithm ITR. Also, the method preserves the monotone relationship between U and
Z, which allows one to induce negative correlation by using U to generate Zl and 1 - U to
generate Z2. Table 3.3 illustrates the effect for the Poisson distribution.

ACCUMULATED ROUND-OFF ERROR

In practice, the repeated updating q +--- q + P in Algorithm ITR creates a source of

numerical error that increases with b - a. At each step, q represents the accumulation of all
previously computed p's, each of which contains round-off error due to the finite word-size
of the computer employed. The larger b - a is, the greater is the potential number of additions
that form q, and consequently, the greater is the accumulated round-off error it contains. If,
for example, this round-off error produces a q that converges to 1 - E, for E > 0, and one
generates U such that 1 - E < U < 1, then the sequential search for Z continues forever.
While such an event is rare, it can occur in practice. Using extended-precision arithmetic to
compute P and q reduces the frequency with which such an event occurs.

8.2 CUTPOINT METHOD

If repeated sampling is to occur from a specified discrete distribution with finite support,
one can exploit other features of the inverse transform method to improve efficiency. Most
notable is the cutpoint method as described in Chen and Asau (1974). The present discus-
sion follows Fishman and Moore (1984), who give a detailed analysis of the efficiency of this
method. The procedure uses sequential search to determine Z, but begins the search at a point
in {a,a+ 1, ... , b} considerably closer to the solution Z on average than the point used by Al-
gorithms DI and ITR. It achieves this result by computing and storing m cutpoints or pointers

._ { min{i: qi > (j - I)/m, a :s i :s b}, j = 1, ... ,m,

Ij .- (8.4)
b, j == m + 1,

beforehand. Then each generation randomly selects an integer L = rm Ul and starts the
search at h. Algorithm CM describes the essential steps.
Since
 mean no. of ~
distribution p.m.f. Po e(z + 1) = Pz+l/Pz comparisons ~
co
N
binomial en
per - z) ~
B(r, p) G)pz(1- py-z z=O,I, ... r (1- py 1 +rp (')
(1 - p)(z + 1) ...
(1)
0.-
geometric &:
(')
Qe(p) (1- p)pZ Z = 0,1, ... I-p p 1/(1- p) '"
(1)
......
(1)

hypergeometric &:
a, f3, n, z integer ...~
e)C~z) f3!(a + f3 - n)! (a - z)(n - z) S'
H (a, f3, n) a+f3~n>O 1 + na/(a + f3) t::
(":Il) (f3 - n)!(a + f3)! (z + 1)(f3 - n + Z + 1) a.
0
::I
max(O, n - f3 - 1)
'"
8
:::0 z :::0 min(a, n)
1\
negative binomial '1::!
1\
i(r + z) 1 r z per + z)
NB(r, p) r > 0, Z = 0, 1, ... (1- py l+rp/(1-p)
l(r)l(z + 1) ( - p) p Z+1 00
Poisson 1'0
AZe- A
peA) A > 0, z = 0, 1, ... e- A A/(Z + 1) I+A
z!
(")
uniform C
-I
1
Ud(a, f3) -- a<f3 z = a + L(f3 - a)JLJ "0
f3-a
Z
z = a, a + 1, ... , f3 -I
a and f3 integer
~
m
-I
I
0
0

w
w
U'I
336 C HAP TE R 8. SAM P LIN G FRO M PRO BA B I LIT Y DIS T RIB UTI 0 N S

the maximal number of comparisons needed to determine Z is h+1 - h + 1, and the

expected maximal number of comparisons has the upper bound

1 1 1
em := -(lz - h + 1) + -(h - lz + 1) + ... + -(/m+1 - 1m + 1)
m m m
= (/m+1 - h + m)/m. (8.5)

ALGORITHM eM

Purpose: To generate Z randomly from the distribution

{O < qa < qa+1 < ... < qb = I}.
Input: Distribution {qa •...• qb}, and pointers {h, ... , 1m} as in expression (8.4).
Output: Z.
Source: Fishman and Moore (1984).
Method: Randomly generate U from U (0, 1).
L +- fmUl
Z +-h.
While U > qz : Z +- Z + 1.
Return Z.

The algorithm requires a total of b - a + m + 1 locations: b - a + 1 to store {qa, ... ,qb}

and m to store h, ... , 1m.
Observe that em :::: 2 if m :::: b - a, ensuring that no more than two comparisons are
required on average to determine Z. Also, observe that

lm:= 1 + Z - h

gives the total number of comparisons, so that

1
L
m
Elm = 1 + EZ - Eh = 1 + EZ - - Ij • (8.6)
m j=1

Table 8.3 lists em and Elm using the eight-point distribution in Fishman (1978, p. 459). As
an illustration, m = 4 leads to II = 1, lz = 4, h = 5, and 14 = 7. Also, EZ = 5.31 and
Eh = 17/4 = 4.25, so that E14 = 2.06. If space is not an issue, one may regard Elm - 1 as
the cost of sampling. Algorithm CMSET shows how to compute the points h ..... 1m in a
presampling step.
 8 .2 . CUT POI NT MET HOD 337

ALGORITHM CMSET

Purpose: To compute pointers for use in Algorithm CM.

Input: Distribution {qa, ... , qb} and number of pointers m.
Output: II, ... , 1m as in expression (8.4).
Source: Fishman and Moore (1984).
Method:
i +- 0, j +- a-I and A +- 0.
Whilei < m:
While A ::: i :
j +-j+l.
A+- mqj.
i +-i+l.
Ii +- j.
Return h, ... , 1m.

For repeated sampling from a continuous distribution, Ahrens (1993) describes a

procedure that merges the cutpoint method with the acceptance-rejection method and
whose computing time is negligibly influenced by the form of the distribution.

8.2.1 RESTRICTED SAMPLING

The cutpoint method offers several additional features that make it attractive in prac-
tice. Of these, restricted sampling is one of the most important. Suppose that one wants
to generate a sample from the modified distribution {qi = (qi - qa,-d/(qb' - qa,-d;
a ::: a' ::: i ::: b' ::: b}. Generating U from U (qa'-I> qb') instead of from U (0, l) in Algo-
rithm CM achieves this result while preserving the efficiency of the method. In particular,
once the original tables of {qa, ... ,qb} and {II> ... , 1m} are computed and stored, one can
sample from them with varying a' < b' without incurring any new cost. No other method
of sampling from tables offers this advantage.

Table 8.3 Example of cutpoint method

Pi qi m m+b em Elm
.01 .01 9 8.00 5.31
2 .04 .05 2 10 4.50 3.31
3 .07 .12 3 11 3.33 2.31
4 .15 .27 4 12 2.75 2.06
5 .28 .55 5 13 2.40 1.71
6 .19 .74 6 14 2.17 1.64
7 .21 .95 7 15 2.00 1.45
8 .05 1.00 8 18 1.88 1.44
16 24 1.44 1.19
338 C H A PT E R 8. SAM P LIN G FRO M PRO 8 A 8 I LIT Y DIS T RIB U,T ION S

If the p.m.f. of interest changes after one generates each sample, then neither the
cutpoint method nor the alias method (Section 8.4) can guarantee an 0(1) generating
time. For a commonly encountered form of changing k-cell p.m.f., Fishman and Yarberry
(1997) describe a generating procedure that takes O(log2 k) time. Also, see Fishman (1996,
Section 3.31).

8.3 COMPOSITION METHOD

This approach to sampling relies on mixtures of random variables to achieve its effectiveness.
Let Z denote a random variable on [a, b1with d.f. F that has the decomposition

F(z) = Lw;F;(z),
;=1
where
0< W; < 1, i = 1, ... , r,
r
Lw;=l,
;=1

and {FI (z), a :s al :s z :s b l :s b}, ... , {Fr (z), a :s a r :s z :s br :s b} also denote d.f.s. To
generate Z, one performs the steps in Algorithm COMPM.

ALGORITHM COMPM

Purpose: To generate Z randomly from the distribution {F(z), a :s z :s b}.

Input: {wi;i = 1, ... , r} and {F;(z); a; :s z :s b;} for i = 1, ... , r.
Method:
Randomly generate I from the mixing distribution {WI, ... , Wr }.
Randomly generate Z from the distribution {F[(z), a[ :s z :s b[}.
Return Z.

At least two motivations exist for using this method. First, generation directly from
F may be difficult and expensive, whereas generation from {WI, ... , w r } and the chosen F;
may be simple and inexpensive. For example, suppose that

F(z) = a(1 - e- ZIP1 ) + (1 - a)(1 - e- zIP,)

= 1 - ae- ZIPl - (1 - a)e- zIP" 0 < a < 1, th, th > 0, z::: o.

 8 .3 . ( 0 M PO SIT ION MET HOD 339

While the inverse transform method does not apply easily here, the composition
method, with

WI := ex, Wz := 1 - ex,
FI(z) := 1 - e- z/ f\ Fz(z):= 1- e-z/1h,

does.
The second motivation relates solely to the relative generating cost. A particularly good
example arises in randomly sampling from the exponential distribution E (1). Recall that the
inverse transform method (Table 8.1) requires the evaluation of a logarithm. If a computer
contains hardware for effecting this evaluation, then the inverse transform method is usually
the most efficient approach. If software, by way of a library subroutine, effects the evaluation,
the computation time increases. In this case, the rectangle-wedge-tail method in MacLaren
et al. (1964), based on the composition method, usually can generate an exponential sample
in less time. Define A > 0, r = 2k + 1, and

f(z) = e- z = wdl (z) + ... + WZk+dZk+1 (z), z ::: 0,

Wj = Ae- P"

with

h(z) = I/A (j - 1)A .::: z .::: jA, j = 1, ... , k,

Wj = e-U-k)A(eA - 1 - ).,),

h(z) = (e-Z+(j-k)A -1)/(e A - 1 - A) (j - k - l)A .::: z .::: (j - k)A,

j = k + 1, ... , 2k,
-kA
WZk+1 = e ,
izk+1 (z) = e- z+k", z ::: kA.

Figure 3.1 shows the decomposition. Here {f(z)} has 2k + 1 components of which the
first k, fr, ... , fb are rectangular (uniform) p.d.f.s on [0, Aj, [A, 2A], ... , [(k - 1)A, kAj,
respectively. Moreover, the probability of sampling from one of these is

(8.7)

Components k + 1, ... , 2k are the wedge-shaped p.d.f.s fk+l, ... , izb from which one
can sample by the acceptance-rejection technique, which Section 8.5 describes in detail.
Although the cost of this sampling is higher than the cost for fl, ... , fb the frequency with
340 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

1.0 - - . - - - - - - - - - - - - - - - - - - - - - - - - ,

0.9
0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0.0 -+----1-I---I---f---jf----1---I---t--j-l---I---l----I=t==t=A=F=F;==r=..,.-..,......j
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0
z

Figure S.l Exponential generation using the rectangle-wedge-tail method

which this more expensive sampling occurs is

(8.8)

which can be made small relative to Wi + ... + Wk by an appropriate choice of)" and k.
If the tail component 2k + 1 is chosen, then Z ?: k)" with probability 1. Since

one can proceed as in Algorithm RWTE. In particular, one iterates until J ::::: 2k before
generating Z. The mean number ofiterations is 1/(1 - W2k+d. Actual implementation of
this algorithm depends heavily on the features of the computer to be employed.

ALGORITHM RWTE

Purpose: To generate Z from randomly £ (1) using the rectangle-wedge-tail method.

Input: )", {Wi> ••• , W2k+d, {fk+j(z), (j - 1))" ::::: Z ::::: j),,} for j = 1, ... ,k.
Method:
1+---1.
Repeat
randomly generate J from the distribution {Wi, ••• , W2k+ i }
 8.4. ALIAS METHOD 341

1+-1+1
until J ~ 2k.
IfJ~k:
Randomly generate U from U (0, 1).
Z +- [kl + (J - 1) + U]A.
Return Z.
Otherwise:
Randomly generate Y from the distribution {fJ(z), (J - k - 1)1.. ~ Z
~ (J - k)A}.
Z +- kIA + Y.
Return Z.

This technique requires one to set two parameters k and A. As k increases for a fixed A,
the probability of iteration W2k+ I decreases, but the size of the tables needed for choosing J
from {WI. ••• , W2k+ d increases. As A increases, WI + ... + Wk in (8.7) eventually decreases,
which is undesirable from the viewpoint of computational efficiency. As an illustration of
the consequences of two choices, k = 80 and A = .05 leads to

WI+ ... + WSO = .9573,

WSI + ... + Wl60 = .0244,
Wl61 = .0183,

revealing the dominance of sampling from one of the 80 rectangular distributions. Also,
note that the mean number of iterations is 1/(1 - e-sox.OS) = 1.0187. The choice k = 160
and 1..= .05 leads to

WI + ... + Wl60 = .9749,

Wl61 + ... + W320 = .0248,
W321 = .0003,

with a mean number of iterations = 1/(1 - e-160x.OS) = 1.0003. For a detailed account of
how to prepare a highly efficient computer program to execute this algorithm, see Ahrens
and Dieter (1974a).

8.4 ALIAS METHOD

The composition method of Section 8.3 also offers an alternative technique for generating a
sample from a table. The procedure requires exactly one comparison, regardless of the size
342 C HAP TE R 8. SAM PLIN G FRO M PRO BA B I LIT Y DIS T RIB UTI 0 N S

of the table (Walker 1974, 1977). This implausible, but nevertheless true, result makes this
approach at least as appealing as the cutpoint method from the viewpoint of computational
efficiency. Moreover, it is an application of the composition method of Section 8.3. Let Z
denote a random variable that assumes value i in T = {a, a + 1, ... , b} with probability
Pi. To generate a Z, the alias method first decomposes the support T into b - a + 1 2-tuples
(A a , Ba), (Aa+l> B a+ I ), .•. , (Ab, B b) such that

and assigns probabilities P(Ai) > °to Ai and P(Bi ) = 1- P(Ai) to Bi such that

1 b
Pi = b-a+1 L h(i),
J=a
(8.9)

where {h(i) i = a, a + 1, ... , b} is a p.m.f. with

ifi = Aj ,
ifi = B j ,
otherwise.

Expression (8.9) implies that selecting a 2-tuple with probability l/(b - a + 1) and then
choosing between Al and BI by Bernoulli sampling produces the desired result. Algorithm
ALIAS describes the procedure.

ALGORITHM ALIAS

Purpose: To generate Z randomly from the p.m.f. {Pa, ... , Pb}.

Input: {A a , ... , A b }, {B a , ... , Bb}, {P(A a ), ... , P(Ab)}.
Method:
Randomly generate UI from U (0,1).
Y +- a + L(b - a + 1)Ud.
Randomly generate U2 from U (0, 1).
IfU2 :::: P(A y ); return Z +- Ay; otherwise, return Z +- By.

There remains the issue of establishing that a decomposition satisfying expression

(8.9) exists and of describing a procedure for computing this decomposition. Theorem 8.2
provides the proof for a = 1, which, without loss of generality, applies for a > 1 with n
replaced by b - a + 1.

Theorem 8.2.
Any discrete distribution with a finite number n of mass points can be represented as an
equiprobable mixture ofn distributions, each of which has, at most, two mass points.
 8 .4 . A LI A S MET HOD 343

For the proof, see Kronmal and Peterson (1979), Dieter (1982), or Fishman (1996).

ALGORITHM ALSET

Purpose: To compute the setup for the alias method.

Input: {Pi;i = a, a + 1, ... , b}.
Output: {Ai, B i , P(A;); i = a, a + 1, ... , b}.
Source: Kronmal and Peterson (1979).
Method: k +- (b - a + 1), S +- 0, 9 +- 0, and i +- a.
While i :'S b :
ri +- kpi.
Ifri < I, S +- S + {i}; otherwise, 9 +- 9 + {i}.
i+-i+1.
i +- a.
While i :'S b :
Select and remove j from S.
Ai +-j;P(A i ) +-rj.
1£9 # 0:
Select and remove l from g.
Bi +-l;rz +-rz-(1-rj)'
If rz :'S I, S +- S + {f}; otherwise, 9 +- 9 + {f}.
i+-i+1.

8.4.1 SETIING UP THE TABLES

Algorithm ALSET describes how to compute {Ai, B i , P (Ai ); i = 1, ... , b - a + I}.

To illustrate the procedure, consider the eight-point distribution in Table 8.3 for which
Algorithm ALSET initially give:

r1 +- .08,
r2 +- .32,
r3 +- .56,
r4 +- 1.20,

r5 +- 2.24,
r6 +- 1.52,
r7 +- 1.68,
r8 +- .40,
S +- {I, 2, 3, 8}, 9 +- {4, 5, 6, 7}.

Then
344 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

Step 1. Pick 1 and 4,

Al +- 1, P(Ad +- .08, BI +- 4, r4 +- 1.20 - (1 - .08) = .28,
= =
S {2, 3, 4, 8}, g {5, 6, 7}.
Step 2. Pick 2 and 5,
A2 +- 2, P(A 2) +- .32, B2 +- 5, rs +- 2.24 - (1 - .32) = 1.56,
= =
S {3, 4, 8}, g {5, 6, 7}.
Step 3. Pick 3 and 5,
A3 +- 3, P(A3) +- .56, B3 +- 5, rs +- 1.56 - (1. - 56) = 1.12,
S = {4, 8}, g = {5, 6, 7}.
Step 4. Pick 4 and 5,
A4 +- 4, P(A4) +- .28, B4 +- 5, rs +- 1.12 - (1 - .28) = .40,
S = {5, 8}, g = {6, 7}.
Step 5. Pick 5 and 6,
As +- 5, P(As) +- .40, Bs +- 6, r6 +- 1.52 - (1 - .40) = .92,
S = {6, 8}, g = {7}.
Step 6. Pick 6 and 7,
A6 +- 6, P(A 6) +- .92, B6 +- 7, r7 +- 1.68 - (1 - .92) = 1.60,
S = {8}, g = {7}.
Step 7. Pick 8 and 7,
A7 +- 8, P(A 7) +- .40, B7 +- 7, r7 +- 1.60 - (1 - .40) :;::: 1.00,
S = {7},g = 0.

Step 8. As +- 7, P(As) +- 1.00,

S=0,g=0.

Table 8.4 displays the corresponding probabilities where the row averages give the original
p.m.f. {PI, ... , Ps} and the column entries sum to unity for each stratum 1, ... , 8.

Table8.4 Example of the alias methodt P(A j ) and P(B j )

i/j 1 2 3 4 5 6 7 8 Pi
Al = 1 .08 .01
A2 = 3 .32 .04
A3 = 3 .56 .07
BI =A4 =4 .92 .28 .15
B2 = B3 = B4 = As =5 .68 .44 .72 .40 .28
Bs = A6 = 6 .60 .92 .19
B6 = B7 = As = 7 .08 .60 1.00 .21
A7 = 8 .40 .05
P(A j ) + P(B j ) 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00

t P(A j ) = first entry in column j; P(B j ) = second entry in column j; see expression (8.9) for
computation of {Pi> ... , ps}.
 8.5 . Ace E PTA NeE - RE J E C T ION MET HOD 345

8.5 ACCEPTANCE-REJECTION METHOD

Although every d.f. has a unique inverse, this inverse may not exist in a form convenient for
efficient computation. For example, the Erlang p.d.f.

a = integer> 0, y 2: 0,

leads to the d.f. (Feller 1971, p. 11)

F(z) =
1 1 z a-I
ya-Ie-Ydy = 1 - e- z - e- z 2:)Z' Ii!),
.
z 2: O. (8.lOa)
(a - I)! 0 i=1

Although the bisection and Newton-Raphson methods can provide a solution, their com-
puting times grow at least linearly with a. A more direct approach employs the expansion
of the inverse d.f. (Fisher and Cornish 1960)

F-I(u) = 2a + 2s-a l / 2 + 2(S-2 - 1)/3 + (S-3 - 7n/18a l / 2

- (6S- 4 + 14S-2 - 32)/8lOa + (9S- 5 + 256S- 3 - 433n/19440a 3/ 2
+ 02S-6 - 243s- 4 - 923s- 2 + 1472)/102060a 2 (8.lOb)
- (3753S-7 + 4353S-5 - 289517S 3 - 289717S)/73483200a 5/ 2
+ OO/a 3 ) as a -+ 00, O:::s u :::s 1,

where

Nevertheless, the time to compute a truncated version of expression (8.lOb) and

to discount negative values, which rarely occur, is not inconsequential. The acceptance-
rejection method allows one to sidestep this issue by exploiting the properties of conditional
probability.

Theorem 8.3 (von Neumann 1951).

Let (f(z), a :::s z :::s b} denote a p.d.f with factorization

f(z) = cg(z)h(z),

where

h(z) 2: 0, lb h(z)dz = 1, (8.11)

346 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

and

e = sup[f(z)1 h(z)] < 00,

z
o ::::: g(z) ::::: 1.

LetZ denote a random variablewithp.d.f. {h(z)} andletU befromU (0, 1). ItU ::::: g(Z),
then Z has the p.d.f. {f(z)}.

PROOF. The random variables U and Z have the joint p.d.f.

fu,z(u, z) = h(z), o : : : u ::::: 1, a::::: Z ::::: h.

Then Z has the conditional p.d.f.

rg(z) fi (u )du
h ( IU < (Z» = )0 u,Z, Z
Z Z - g pr[U ::::: g(Z)] .

Observe that e = J: g(z)h(z)dz. Since

(g(z)
10 fu,z(u, z)du = h(z)g(z)

and

pr[U ::::: g(Z)] = lab h(z)g(z)dz = lie,

it follows, that

hz(zlU ::::: g(Z» = eg(z)h(z) = f(z),

which proves the theorem. o
Since g(z) = f(z)le h(z), the assignment e = supJf(z)1 h(z)] maximizes the
acceptance probability g(z) for each z E [a, h]. An analogous result holds for discrete
distributions.
The significance of this result becomes most apparent when one can generate Z from
{h(z)} efficiently and e is close to unity. Algorithm AR describes how to implement the
approach as a sequential sampling experiment that terminates with the first success (U :::::
g(Z». Since pr[U ::::: g(Z)] = lie is the success probability on any iteration or trial, and
 8.5. ACCEPTANCE-REJECTION METHOD 347

since all trials are independent, the number of trials until success is a geometric random
variable with mean c, which explains why a small c is desirable. This procedure describes
one of the first probabilistic algorithms used on a computer.

ALGORITHM AR

Purpose: To generate Z randomly from the distribution {f(z) = cg(z)h(z), a :::: z :::: b}.
Input: {g(z), h(z); a :::: z :::: b}.
Method:
Repeat
randomly generate Z from the distribution {h(z)}
randomly generate U from U (0, 1)
evaluate g(Z)
until U :::: g(Z).
Return Z.

o ExAMPLE 8.1

To illustrate the method, consider generating Z from the half-normal distribution

with p.d.f.

0:::: z < 00. (8.l2a)

Writing this density in the form

(8.l2b)

leads to the factorization

h(z) = e- z ,
g(z) = e- Cz - 1)2/ 2 , (8.l2c)

and

c = J2e/1i = 1.3155.
Figure 8.2a shows the acceptance region (u :::: g ( - In v» and the rejection region
(u > g(-ln v».
D
348 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

(a)
1.0

0.9

0.8 g(-lnv)

0.7

0.6

u 0.5

0.4

0.3

0.2

0.1

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
v

(b)
1.0

0.9

0.8

0.7

0.6

u 0.5

0.4

0.3

0.2

0.1

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
v

Figure 8.2 Half-normal generation using the acceptance-rejection method

 8 ,5 , Ace E PTA NeE - RE J E C T ION MET HOD 349

At least two implementations are possible. Version 1, which uses the inverse transform
method to generate the exponential deviate Z, requires a logarithmic evaluation for Z and
an exponential evaluation for g(Z). Alternatively, version 2 generates Y and Z from E (1)
and avoids the exponentiation. If a highly efficient algorithm is available for generating
exponential samples, then version 2 can be expected to be less costly.

VERSION 1

Repeat
randomly generate X and V from U (0, 1)
Z+---ln V
g(Z) +-- e-(Z-l)' /2

until X :::: g(Z).

Return Z.

VERSION 2

Repeat
randomly generate Y and Z from E (1)
until Y ::: (Z - 1)2/2,
Return Z.

Half-normal generation provides one method for producing a sample from N (0, 1).
Let W denote a random variable from U (0, 1). Then

-Z 1'f 1
W <-,
Z* = { -2
Z otherwise,

has the standard normal p.d.f. (Table 8.1).

8.5.1 SQUEEZE METHOD

On average, success in Version 1 of the half-normal example requires 2J2e/rr =

2.6310 function evaluation, one logarithmic evaluation, and one exponential evaluation per
trial. Avoiding these can lead to a significant reduction in computing time. The squeeze
method (Marsaglia 1977) allows one to do this. Suppose there exist simple functions {gL (z),
a:::: z:::: b} and {gu(z), a:::: z:::: b} such that
(8.13)

where expression (8.11) defines g(z). Here the modifier "simple" means that the time to
evaluate gL (z) and gu (z) is less than the time to evaluate g(z). Note that for some Z, gL (z)
350 C HAP TE R 8. SAM PLIN G FRO M PRO BA B I LIT Y DIS T RIB UTI 0 N 5

may assume negative values or gu (z) may exceed unity. Algorithm ARS shows how to use
these bounding functions to reduce computing time. On each trial, this algorithm leads to

pr[evaluatingg(Z)) = pr[gL(Z) < U:::: gu(Z)).

The checks U :::: gL (Z) and U > gu (Z) are sometimes called preliminary tests or pretests.

ALGORITHM ARS

Purpose: To generate Z randomly from the p.d.f. {fez), a :::: z :::: b} using the
acceptance-rejection and squeeze methods.
Input: p.d.f. {h(z),a :::: z :::: b}, functions {gLCz), g(z), gu(z); a:::: Z :::: b}.
Method:
Repeat
randomly generate X from U (0, 1)
randomly generate Z from {h(z)}
if X :::: gLCZ), accept
otherwise:
if X :::: gu (Z) : if X :::: g(Z), accept
otherwise: reject
until accept.
Return Z.

Let S denote the event X < g(Z), SL the event X :::: gL (Z), and Su the event
X :::: gu (Z). Then

pr(Sd = lb max [0, gL(z))h(z)dz,

= lb
(8.14)
pr(Su) min[l, gu(z))h(z)dz,

= lb
and, as before,
preS) g(z)h(z)dz = lie.

Then on a single trial

pr[evaluatingg(Z)) = pr(Su) - pr(SL). (8.15)

If N denotes the number of evaluations of g(Z) until success, then

- - 1 -
EN = pr(SL, SuIS)[-(-) - 1) + pr(SL, SuIS), (8.16)
pr S
 8.5. Ace E PTA NeE - RE J E C T ION MET HOD 351

where SL and S denote the events X > gL (Z) and X > g(Z), respectively. Since

r(S SIS) = pr(Su) - pr(S)

P L, U 1 _ pr(S)

and

r(S SIS) = pr(S) - pr(Sd (S.17)

p L, U pr(S)'

it follows that

pr(Su) - pr(SL)
EN= .
pr(S)

To reduce the frequency of exponential evaluation, the squeeze method often employs the
inequalities (Exercise 8.1S)

x ::: O. (S.lS)

To reduce the frequency oflogarithmic evaluation, it often employs (Exercise S.19)

x-I
--slnxS-1+x, x ::: 0, (S.19)
x

or (Exercise 8.20)

x -l/x S 2lnx S -3+4x _x 2 , O<XSl. (S.20)

Example S.2 illustrates the benefit of using inequalities (S.lS).

o EXAMPLE 8.2

For the half-normal distribution in Example 8.1 with g(z) = C(Z-I)'/2, inequalities
(S.lS) leads to the lower and upper bounds

(S.21)

and

gu (z) = 1 - (z - 1)2/2 + (z - 1)4/8

= gL(Z) + (z _1)4/S, (S.22)
352 C HAP TE R 8. SAM P LIN G FRO M PRO BA B I LIT Y DIS TR I BUT ION S

respectively. Moreover,

pr(SL} = 1/2 + (1 + .J2}e-(l+.J2) = .7159, (8.23a)

pr(Su} = 13/8 - 16e-3 = .8284, (8.23b)
pr[evaluating g(Z} on a trial] = pr(Su} - pr(Sd = .1125, (8.23c)

and the mean number of evaluations of g (Z) until success is EN = .1480. Figure 8.2b shows
{gL (z)}, {g(z)}, and {gu (z)} for z = -In v and 0 < v < 1. In summary, incorporating the
squeeze method into the acceptance-rejection method in this example reduces the mean
number of evaluations of g(Z} from 1.3155 to 0.1340. Of course, some of this saving is
expended in the two additional pretests for SLand Su.
o

8.5.2 THEORY AND PRACTICE

As with the inverse transform method, the acceptance-rejection method is a theo-

retically exact approach to sampling, provided that a source of i.i.d. samples are available
from U (0, 1). Since every Monte Carlo experiment relies on a sequence of pseudorandom
numbers to approximate that LLd. sequence from U (0, I), an error of approximation always
exists. This error has more serious implications for the acceptance-rejection method than
for the inverse transform method. Indeed, it is also more serious for the ratio-of-uniforms
method (Section 8.6) and any other procedure that demands more than one uniform devi-
ate per draw. Since a careful choice of pseudorandom number generator (PNG) can reduce
the severity of the problem, being familiar with the properties of one's chosen PNG is well
advised. Fishman (1996, Section 7.6.1) illustrates the character of the error.

8.6 RATIO-Of-UNifORMS METHOD

The acceptance-rejection method incurs four forms of work: generation of Z from {h(z}},
generation of U from U (0, 1), evaluation of g(Z}, and repeated generation until success
occurs. The squeeze method reduces the frequency ofevaluating g (Z). The ratio-of-uniforms
method replaces the generation of Z and U by the generation of Z = al + a2Y / X, where
(X, Y) is uniformly distributed over a bounded region where al> a2 and the shape of the
region depend on {f(z)}. In principle, the work to generate (X, Y) is considerably less,
thus potentially reducing the computing time required to obtain a sample from {f(z)}.
Theorem 8.4 represents a generalization of the original proposal of Kinderman and Monahan
(1977) for the ratio-of-uniforms method and is used extensively by Ahrens and Dieter (1991)
and Stadlober (1989,1991) for generation from discrete distributions.
 8 .6 . RAT I 0 - 0 F - U N I FOR M 5 MET HOD 353

Theorem 8.4.
Let {r(z), -00 < Z < oo} denote a nonnegative integrable function and let Q denote
the region

+ a2Y/x), -00 <

J
Q = Q(a], a2) = {(x, y) : 0.::: x.::: r2(a] a] < 00, a2 > OJ. (8.24)

Let (X, Y) be uniformly distributed on a bounded region P 2 Q and set W = X 2 and

Z = a] + a2 Y / X. IfW .::: r(Z), then Z has the distribution

fz(z) = r(z)/d, -00 < z < 00, (8.25)

where

(8.26)

PROOF. If (X, Y) E Q, then X and Y have the joint p.d.f.

fx,Y(x, y) = l/IQI, (x, y) E Q,

and

fw,z(w, z) = Jfx,Y(.JW, (z - ad.JW/a2), 0.::: w .::: r(z), -00 < z < 00,

where J denotes the Jacobian

ax ax 1
- 0
aw az 2y'w
J= ,
ay ay z - a] y'w 2a2
aw az 2a2y'w a2

so that

r(z) r(z)
fz(z) = 10 fw,z(w, z)dw = 2a21QI' -00 < Z< 00,

and

1 00

-00
fz(z)dz =
1
Z-IQI
a2-00
1 00
r(z)dz = l. D

The equations

= r2(Z)
J
x(z)
354 C HAP TE R 8. SAM P LIN G FRO M PRO BA B I LIT Y DIS T RIB UTI 0 N S

and

1
y(z) = (z - adX(z)/a2 = (z - al)r> (Z)/a2

define the boundary of Q, and for specified al and a2 every point in Q lies in the rectangle

P:= {(x, y): ° ~ x ~ x*, y* ~ y ~ y*}, (8.27a)

where
1
x* := sup r> (z), (8.27b)

(8.27c)

and
1
y* := sup[(z - adr> (z)/a2]' (8.27d)
z

In order for y* to be finite, {Z2 r (Z)} must be integrable.

The motivation behind the ratio-of-uniforms method is now clear. Suppose one sets
r(z) = df(z) and generates (X, Y) uniformly on P.IfX ~ r~(al +a2Y/X), then (X, Y) E
Q, and Z = al + a2Y / X has the desired p.d.f. {f(z)}. Algorithm ROU describes the steps.

ALGORITHM ROU

Purpose: To generate Z randomly from the p.d.f. {f(z), -infty < z < oo} using the
ratio-of-uniforms method.
Input: Function {r(z) = df(z), -00 < z < oo}, constants al E (-00, 00) and
a2 E (0,00).
Setup: x* +--supzr~(z),y* +--infz[(z-al)r!(z)/a2],and
y* +-- sUPz[(z - al)r~(z)/a2]'
Method:
Repeat
randomly generate X from U (0, x*)
randomly generate Y from U (y*, y*)
Z+--al+a2 Y / X
until X 2 ~ r(Z).
Return Z.

Sincepr(acceptZ) = pr[X ~ r!(al +a2Y/X)] = pr[(X, Y) E Q] = IQI/IPI, the mean

number of trials until success has the form

(8.28)
 8.6. RAT I 0 - 0 F - U N I FOR M S MET HOD 355

Generating (X, Y) from P takes little time compared to the potentially greater time to
generate Z from {h(z)} in expression (8.11) using the acceptance-rejection method. While
the inclusive region P may assume a shape other than rectangular in order to make the ratio
IPI/I QI closer to unity, one must balance this reduction against the potentially greater cost
of generating X and Y from a less simple region. Section 8.13 describes how an alternatively
shaped inclusive region does indeed payoff for Gamma generation. The original proposal
in Kinderman and Monahan (1977) for this method chooses al = 0 and a2 = 1, and it
is this assignment we use in the next several sections unless an alternative assignment is
explicitly made. Section 8.15 shows how the judicious choice of al and a2 allows one to
sample from a simple region to generate samples from discrete distributions with bounded
mean computing times.
The acceptance check X 2 .::: real +a2Y / X) in Algorithm ROU can consume consider-
able time just as the acceptance check U .::: g(Z) in Algorithm AR can. Again, appropriately
chosen squeeze functions can reduce this time significantly.

o EXAMPLE 8.3

An example based on the half-normal distribution (8.12a) illustrates the features of

the ratio-of-uniforms method. Here we take al = 0 and a2 = 1, so that

Q = {(x, y) : 0 .::: x .::: e-(Y/2X)'},

r(z) = e- Z' /2,

1 ('" 1
IQI = "2 Jo e- z /2dz = "2.jn/2,
2

x* = 1, y* = 0, y* =.j2je,
IPI/IQI = 4/Fe = 1.3688.

Figure 8.3a shows the acceptance region. The acceptance test in Algorithm ROU takes the
form

X .::: e-(Y/2X)',

which is equivalent to the event T,

y2 .::: -4X 2 1n X. (8.29)

The acceptance probability

pr(T) = IQI/IPI = Fe/4 = .7306 (8.30)

356 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

is close to the acceptanceprobabilitypr(S) = Jrr 12e = .7602 in Example S.l, but the ratio-
of-uniforms method requires no exponential deviate as the acceptance-rejection method in
Example S.l does.
D

Pretesting based on the inequalities (S.19) again improves efficiency. Let TL denote
the event

(S.31)

and Tu the event

y2 :::: 4X(1 - X). (S.32)

One accepts Z = Y I X if TL occurs on the first pretest, but rejects Z if u occurs on thet
second pretest, where

pr(Td = 1
~
",2/e
11 1
0 0
2X
v'I=X dydx
(S.33a)
= her (D / r G) = 4(2e)~ /15
= .621S

and

pr(Tu) = ~Jmin[2/e,4x(1-x)]dydx
",2/e
1 / sin-1(-JI-2/e)+rr/2 (S.33b)
= 2"Y 1 - 2/e + ----'--2-..jVe--=2=/e=----
= .S579.
Then

pr[evaluating r(Y I X)] = pr(Tu) - pr(Td = .236l.

If M denotes the number of evaluations of r(Y I X) until success, then

EM = pr(Tu) - pr(T) [_1_

1 - pr(T) pr(T)
-IJ + =--pr_(T_)_----=-pr_(_TL_)
pr(T)
(S.33c)
pr(Tu) - pr(Td
= pr(T) = .3232.
 8.6. RATIO-OF-UNIFORMS METHOD 357

(a)
1.0 - , - - - - - - - - - - - - - - - - - - - - - - - - - ,

0.9

0.8

0.7

0.6

Y = 2xv- 1nx
Y 0.5

0.4

0.3

0.2

0.1

0.0 +--.--.--.---.-----,-----,----,---,---,---,---,---,---,---,---,---,----,----,----,----1
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
x

(b)
1.0 -,-----------=--~o:----------,

Y = 2Jx(1 - x)
0.9
Y = 2xV- 1n x
0.8

0.7

0.6

Y 0.5

0.4

0.3

0.2

0.1

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
x

Figure 8.3 Half-normal generation using the ratio-of-uniforms method

358 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

This result follows in a manner analogous to that for EN in expression (8.16) for the
acceptance-rejection method. Figure 8.3b shows the relevant curves.
In summary, incorporating the squeeze method into the ratio-of-uniforms method
for generating a sample from the half-normal distribution reduces the mean number of
evaluations of r(Y / X) from 1.3688 to 0.3232. Again, the two pretests consume part of this
time saving. However, one should note that 100 x (I - 0.3232/1.3688) = 76.39 percent of
the time no logarithmic or exponential evaluations are required, whereas the acceptance-
rejection method in Example 8.1 requires one exponential deviate on every trial, regardless
of whether or not the squeeze method is used.

8.7 EXACT-APPROXIMATION METHOD

The exact-approximation method in (Marsaglia 1984) demonstrates how for a specified p.d.f.
one can identify a decomposition that allows generation from either a uniform or normal
distribution most of the time and generation from a residual distribution for the remainder
of the time. Since among all variate-generating procedures those for generating a uniform
deviate are the least costly and those for generating a normal deviate are only marginally
more costly, the substantial frequency with which the acceptance-rejection method makes
direct use of uniform or normal generation gives the exact-approximation method its appeal.
Let Z have p.d.f. f and d.f. F. The exact-approximation method employs an approx-
imation {m(x), 0 ::::; x ::::; I} to the corresponding inverse d.f. {F-1(x), 0 ::::; x ::::; I} that
allows one to generate a sample from the desired d.f. F. The result is exact. Assume that
{m'(x) = dm/dx} exists everywhere and is nonnegative. Let X denote a random variable
with p.d.f. {fx(x), 0::::; x ::::; I}. Then Z = m(X) has the p.d.f.

(8.34)

wherem- 1 denotes the inverse function ofm . Since the objective is to achieve !z(z) = fez),
one wants to determine the p.d.f. fx that effects this result, namely

fx(m-1(z)) = fx(x)
= !z(z)m'(m-1(z)) (8.35)
= f(m(x))m'(x).

If m closely approximates F- 1 , then {fx(x)} is close to being uniform in appearance,

suggesting the decomposition

fx(x) = pI[O,l] (x) + (I - p)f*(x), 0< p ::::; min fx(x),

O:::;x:::;l
(8.36)
 8 .7 . Ex ACT - A P PRO X I MAT ION MET HOD 359

where the upper bound on p ensures that f* is a p.d.f. Clearly, assigning p the value of its
upper bound offers the greatest benefit.
Algorithm EAM describes the method, making use of the fact that if U :s p, then
U / p is from U (0, 1). The motivation for large p is now clear. It reduces the frequency
with which one must generate X from f*, which Marsaglia recommends be done by the
acceptance-rejection method.

ALGORITHM EAM

Purpose: To generate Z randomly from the p.d.f. {fez)}.

Input: p, {m(x), 0 :s x :s I}, and {f*(x) = [fx(x) - pliO - p), 0 :s x :s I}.
Method:
Randomly generate U from U (0, 1).
If U :s p, X ~ U /p. Otherwise: randomly generate X from the p.d.f. {f*(x)}.
Z ~ m(X).
Return Z.

An example for the Cauchy p.d.f. (Section 8.12)

1
fz(z) = lTO + Z2)' -00 < Z< 00, (8.37)

denoted bye (0, 1), illustrates the method. From Table 8.1, the corresponding inverse of d.f.
is {p-l (x) = tan IT (x - 1/2),0 :s x :s I}. For a = 2/lT and b = IT - 8/lT, the function

m(x) = y( a
1/4 - Y
2 + b) , Y := x - -,
1
2
O:s x :s 1, (8.38)

approximates {p-l(X)} (Ahrens and Dieter 1988). Since

, 1/4 + y2
m(x)=a( 2)2+b, (8.39)
1/4 - y

substitution into expression (8.35) gives

I/lT [ 1/4 + y2 ]
jx(x) = 1 + m2(x) a 0/4 _ y2)2 + b .

The largest p that ensures that

* 1
j (x) = 1 _ p
{ 1/ IT [ 1/4 + y2
1 + m2(x) a 0/4 _ y2)2 +
bJ - pI[o.l) (x)
} ::: 0 (8.40)
360 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

is p = .997738.
Algorithm CA describes an implementation of the exact-approximation method for
sampling Z from the Cauchy distribution C (0, 1) without the use of special functions.
When sampling from f*, it employs an acceptance-rejection procedure with a mean of
3.859819 additional samples from U (0, 1). However, Algorithm CA executes this branch
with probability 1 - P = .002262, resulting in an overall mean of 1.006469 samples from
U (0, 1). Section 8.10 relies on Algorithm CA to provide a fast algorithm for generating
samples from the normal distribution in Algorithm NA.

ALGORITHM CA

Purpose: To generate Z randomly from C (0, 1).

Output: Z.
Source: Ahrens and Dieter (1988).
Setup: Constants

a = 0.63806313 6607 7803, b = 0.5959486060529070,

q = 0.9339962957603656, W = 0.2488702280083841,
A = 0.6366 197723675813, B = 0.5972 9975 9353 9963,
H = 0.0214949004570452, P = 4.912501395303 3204.
Method:
Randomly generate U from U (0, 1).
T *- U - !
and S *- W - T2.
If S > 0, return Z *- T(AjS + B).
Repeat:
generate U and U' from U (0, 1)
T *- U - !,
S *- ~ - T2 and Z *- TCajS + b)
until S2[(1 + Z2)(HU' + P) - qj + S::s !.
Return Z.

8.8 ALGORITHMS FOR SELECTED DISTRIBUTIONS

The remainder of this chapter describes procedures for generating samples from selected
continuous and discrete distributions. Throughout, the focus is on algorithms whose mean
generating times are constant or bounded from above for all possible distributional para-
metric values. Section 3.14 illustrates the protection that using such algorithms offers in
practice. The algorithms described here are not necessarily the computationally most effi-
cient procedures known. However, they do compare favorably with the most efficient ones,
which often depend on the preparation of extensive tables whose features are themselves
machine dependent. The motivating factor in selecting algorithms for presentation here was
their pedagogic value. Faced with the need to develop an algorithm for a highly specialized
 8.9. Ex P0 N E N T I A L DIS T RIB UTI 0 N 361

distribution, the reader of these sections should be able to craft a procedure that performs
acceptably.

8.9 EXPONENTIAL DISTRIBUTION

If Y is drawn from E (1), then Z = )"Y,)" > 0, is from E (1/),,). Table 8.1 provides the
transformation for the inverse transform approach that is the simplest and most direct
method available. However, algorithms exist that when implemented in machine language
generate an exponential sample in less time. These include the rectangle-wedge-tail approach
in MacLaren et al. (1964) (Section 8.3) and Algorithms SA and EA in Ahrens and Dieter
(1972, 1988). Of these, Algorithm EA is the fastest both in FORTRAN and assembler language
versions (Ahrens and Dieter 1988). It generates a sample K from the geometric distribution
ye(1/2) (Section 8.21) and a sample V from the truncated exponential distribution

o ::: v ::: In 2. (8.41)

Since

pr(K = k) = (1/2l+ 1 , k = 0,1, ... , (8.42)

Z = K In 2 + V has the p.d.f.

f(z) = pr(K = k)h(v) = e- z , Z = k In2+v, k = 0,1, ... , 0::: v ::: In 2, (8.43)

so that Z has the desired distribution.

To sample V, Algorithm EA uses the exact -approximation method (Section 8.7) with

m(x) = a/(b - x) + C, 0::: x::: 1, (8.44)

and constants a = (4+3,J2)ln2,b = 2+,J2,andc = -(1+,J2) In 2asanapproximation

to the inverse d.f.

F-1(x) = -In(1- x/2), 0::: x::: 1. (8.45)

For the decomposition (8.36), expression (8.44) implies

2e- m (x) b(b - 1) In 2

fx(x) = (b _ X)2 ,0::: x::: 1, (8.46)
362 C HAP TE R 8. SAM P LIN G FRO M PRO BA B I LIT Y DIS T RIB UTI 0 N S

for which p = ..ti In 2 = .980258 is the largest probability that ensures

1 [2e- m (X)b(b - 1) In 2 ]
rex) = 1_ P (b _ X)2 - pI[o.I](x) :::: O. (8.47)

To sample from f*, Algorithm EA uses an acceptance-rejection method that requires

a .:= 1. 511076 trials on average. Each trial uses two samples from U (0, 1) and an exponen-
tiation. Therefore, the mean number of exponentiations is 100 x a(1 - p) = 2.983144 per
100 samples. Ahrens and Dieter (1988) describe a procedure that eliminates this relatively
modest frequency.

ALGORITHM EA

Purpose: To generate a sample Z from £ (1).

Output: Z.
Source: Ahrens and Dieter (1988).
Setup: Constants
q = In 2 = 0.6931 471805599453, a = 5.7133631526454228,
b = 1.4142 135623730950, c = -1.6734 0532 4028 4925,
p = 0.9802581434685472, A = 5.6005 7075 6973 8080,
B = 3.3468 1064 8056 9850, h = 0.0026 1067 2360 2095,
D = 0.0857 8643 7626 9050.
Method:
Randomly generate U from U (0, 1).
K +-- c and U +-- U + U.
While U < 1, U +-- U + U and K +-- K + q.
U+--U-1.
If U ::: p, return Z +-- K + AI(E - U).
Repeat:
randomly generate U and U' fromU (0,1)
Y +-- al(b - U)
until (U' H + D)(b - U)2 < e-(Y+c).
Return Z +-- K + Y.

8.9.1 WEIBULL DISTRIBUTION

Recall from Table 8.1 that the Weibull distribution has p.d.f.

a-I
f( Z) -- ~ e-(ziP)" ,
{Ja
 8.9. EXPONENTIAL DISTRIBUTION 363

d.f.

F(z) = 1 - e-(zlfJ)a, ex, f3 > 0, z ~ 0,

and is denoted by W(ex, f3). If Y is from £(f3), then it is easily seen that Z = y1/ a is from
W(ex, f3).

To sample Z from W(ex, f3),

• Sample Y from £(f3) .

• Return Z +- yl/a.

These steps are more general than those in Table 8.1, allowing for a faster method of
exponential generation to be used if such an implementation is available.

8.9.2 INVERSE WEIBULL DISTRIBUTIO...

If Y '" W(ex, f3), then Z = 1/ Y has the p.d.f.

ex, f3 > 0, z ~ 0,

which is called the inverse Weibull distribution and is denoted by IW(ex, f3). Its mean,
variance, and mode are, respectively,

EZ = r(1- l/ex)/f3,
varZ = [r(1- 2/ex) - r2(1- l/ex)] /f3 2 ,

mode = ~ C: Yla ex

The existence of EZ requires ex ::: 1, and of varZ, ex > 2. Since prey < a)
pr(1/Z ::: a),

Fz(z; ex, f3) =1- Fy(1/z; ex, f3),

where Fz and Fy denote d.f.s of Z and y, respectively.

Whereas the p.d.f. of Y has a right tail that decreases as e-(YIP)a, the right tail of the
p.d.f. of Z decreases polynomially as 1/ za+ 1 . Chapter 10 discusses this feature with regard
to fitting data to a distribution.
To sample Z from TW(ex, f3), first sample Y from £(f3) and then take Z = yl/a.
364 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

8.10 NORMAL DISTRIBUTION

Let Z denote a random variable with the normal p.d.f.

f(z) = ~e-CZ-I-')'/2a', a 2 > 0, -00 < z< 00, (8.48)

2rra 2

denoted by N (/L, a 2 ). Properties of the distribution include:

1. If Y is fromN (0,1), then Z = /L + aY is fromN (/L, a 2 ).

2. If ZI, ... , Zn are independent from N (/LI, af), ... , N (/Ln, a;), respectively, then
Z = ZI + ... + Zn is fromN (/LI + ... + /Ln, af + ... + a;).

The first property makes clear that one need only concentrate on devising a procedure for
generating a sample from N (0, 1), since the relatively inexpensive transformation Z =
/L + a Y generalizes its application. The second property shows that only one generation
from N (0, 1) is necessary to generate Z = ZI + ... + Zn.
Moro (1995) describes numerical evaluation by the inverse transform method with
10- 9 accuracy. Examples 8.1 and 8.2 show how to generate a sample from the half-normal
distribution from the acceptance-rejection and ratio-of-uniform methods, respectively.
Multiplying the outcome by a randomly determined sign makes it a sample from N (0, 1).
The rectangle-wedge-tail method in Marsaglia et al. (1964) and its modification in
Algorithm RT in Ahrens and Dieter (1972) provide the fastest algorithms for sampling from
N(O, 1). Unfortunately, they rely on extensive tables the preparation of which may not
appeal to a potential programmer. Shortly, we describe Algorithm NA in Ahrens and Dieter
(1988), which is the fastest known table-free method in both FORTRAN and assembler
language versions. Nevertheless, we first describe a commonly employed "slow" method
that establishes the basis for Algorithm NA.

Theorem 8.5 (Box and Muller 1958).

Let U and V denote independent random variables fromU (0, 1) and £ (1), respectively.
Then

= (2V)l: cos 2rrU

1
X

and (8.49)

1
Y = (2V)l: sin 2rrU

are independent random variables from N (0, 1).

 8.10. NORMAL DISTRIBUTION 365

PROOF. Observe that 2V = X 2 + y2, tan 2rrU = Y 1 X, and

au av au av I
/x,y(x, y) = /u,v(u(x, y), vex, y)) Iax ay - ayax

-00 < x, y < 00, (8.50)

since

au -y cos2 2rru -y au cos 2 2rru x

ax 2rrx2
- -
4rrv
,
ay 2rrx
- -
4rrv
,
av av
-=x, -=y.
ax ay

Expression (8.50) gives the p.d.f. of two independent random variables, as required. 0

The simplicity of implementation is most responsible for the common use of this
approach. However, its reliance on square root, cosine, and sine evaluations makes it
uncompetitive. However, it does establish a basis for Algorithm NA.
Observe that tan 2rr U in Theorem 8.5 has the Cauchy distribution C (0, I). Therefore,
employing Algorithm EA to generate V and Algorithm CA to generate Wand taking X =
J2V 1(1 + W2) and Y = W J2V 1(1 + W2) accomplishes our purpose except for signs.
Assigning a random sign to X and assigning the sign of W to Y solves this problem. Algorithm
NA assumes that the value assigned to variable Y is saved between successive executions of
the algorithm. As presented here, Algorithm NA is a slight variant of the one in Ahrens and
Dieter (1988) that exploits the inevitable call to Algorithm CA.

ALGORITHM NA

Purpose: To generate Z from N (0, 1).

Output: Z.
Source: Ahrens and Dieter (1988).
Method:
At compilation time, a flag F is set to 1.
Execution
F +- -F. IfF> 0, return Z +- Y.
Randomly generate U from U (0, 1).
IfU < ~, B +- 0: otherwise, B +- 1.
Randomly generate V from £ (I) using Algorithm EA.
S +- V + V.
Randomly generate W from C (0, I) using Algorithm CA (Section 8,12).
Z +- JS/(1 + W2) and Y +- WZ.
If B = 0, return Z; otherwise, return - Z.
366 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

8.11 LOGNORMAL DISTRIBUTION

Let Y be from N (f.1" 0'2). Then Z = eY has the lognormal distribution with p.d.f.

°:s z < 00,

denoted by.eN (f.1" 0'2). Occasionally, the distribution is described in terms of its moments

and

Common practice generates X from N (0, 1), computes Y = f.1, + Xo', and takes
Z = eY as the sample. The marginal increment in cost over normal generation is due to the
exponential evaluation.

8.12 CAUCHY DISTRIBUTION

Suppose Z has the noncentral Cauchy p.d.f.

~
f(z)=1l'[~2+(z-a)21' -oo<a<oo, ~>o, -oo<z<oo,

denoted byC (a, ~). The distribution has the following properties:

1. If Y is from C (0, 1), then Z = a + ~Y is from C (a, ~).

2. If ZI, ... , Zn are independent from C (aI, ~), ... , C (an' ~), respectively, then Z =
ZI + ... + Zn is from C (al + ... + an, ~).

3. If X I and X2 are independent from N (0, 1), then Z = X I! X2 is from C(O, 1).
4. If (Y I , Y2 ) is uniformly distributed on the unit circle y~ + y? = ~, then Z = YI / Y2 is
from C (0, 1).

°
Without loss of generality, we take a = and ~ = 1. Algorithm CA in Section 8.7 generates
Z from C (0, 1) using the exact-approximation method.
Table 8.1 gives the inverse d.f. for Z, which requires a tangent evaluation in practice
and compares unfavorably in timing with Algorithm CA. To avoid this, one can generate
XI and X2 each from N (0,1) and use Z = XI! X 2 • Alternatively, one can use the result
 8.13. GAM MAD 1ST RIB UTI 0 N 367

based on uniform generation on the unit circle. In particular (Exercise 8.37), straightforward
application of the acceptance-rejection technique gives

Repeat
generate Y l and Yz each from U (- ~, ~) (8.51)
until YlZ + Yi :s ~.
Return YJ/ Yz.

where the mean number of trials is 4/n = 1.2732. Devroye (1986) suggests the pretest
IYll + IYzi :s 1, which reduces the frequency of evaluating the two multiplications by half.
However, one must add in the time required to evaluate the absolute value function twice.

8.13 GAMMA DISTRIBUTION

Let Z denote a random variable having the Gamma p.d.f.

denoted by 9 (a, (3). Then Z has the properties:

1. If Zl, ... , Zn are independent from 9 (aI, (3), ... , 9 (an, (3), respectively, then Z =
Zl + ... + Zn is from 9 (al + ... + an, (3).

2. If a = 1, then Z has the exponential distribution E (fJ).

3. Ifa is an integer, then Z is the sum ofa i.i.d. exponential random variables from £ (f3)
and its distribution is called the Erlang distribution with a phases.

4. If a is an integer and f3 = 2, then Z has the chi-squared distribution with 2a degrees

of freedom.

5. If Y is from 9 (a, 1), then Z = f3Y is from 9 (a, (3).

6. If a < 1, then U(z)} is unbounded at z = o.

Without loss of generality, we hereafter take f3 = 1.
Unlike exponential, normal, and Cauchy generating times, Gamma generating time
can vary with the shape parameter a. Erlang generation illustrates the potential danger. If
a is integer, then Z can be viewed as the sum Z = Zl + ... + Za' where Zl, ... , Za are
independent identically distributed random variables from £ (1). This suggests

Z +- 0 and i +- 1.
Repeat
368 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

randomly generate Zi from £ (1)

Z ~ Z+Zi
i ~i +1
until i > a.
Return Z.

which takes O( a) time. Ifone uses the inverse transform method for the exponential deviates,
then an opportunity to improve efficiency exists.
That is,

Y ~ 1 andi ~ 1.
Repeat
randomly generate Ui from U (0, 1)
Y~YUi
i ~ i +1
until i > a.
Return Z ~ -In Y.

Although only one logarithmic evaluation occurs, the generation of a uniform deviates
retains the O(a) time. Also, the required integrality of a limits the applicability of this
approach. The remainder of this section addresses the problem of Gamma generation for
a :s 1 and the problem of Gamma generation with bounded mean computing time as
a ---+ 00.

0: !: 1
In their Algorithm GS, Ahrens and Dieter (1974b) generate Z from Q(a, 1) with
o< a :s 1, using a combination of the acceptance-rejection, composition, and inverse
transform methods with

e a I a
h(z) = --I[o,I](z)az - + --I(1,oo)(z)e- Z
+1
, (8.52a)
a+e a+e
g(z) = 1[0,1] (z)e- Z + 1(1,00) (Z)Za-1 , Z ~ 0, (8.52b)

and

c(a) = (a + e)/aer(a), (8.52c)

where I!A! (z) denotes the indicator function of the set A and {h(z)}, {g(z)}, and c(a)
are defined in Theorem 8.3. Observe that f(z) = c(a)g(z)h(z) gives the Gamma p.dJ.,
sampling from the p.dJ. {h (z)} uses the composition method and the mean number of trials
until success c(a) has the upper bound c :s 1.39. The composition method determines
whether one generates Z in expression (8.52a) from the dJ. {za, 0 :s z :s I} or from the
dJ. {I - e- z+l , 1 < z < oo} by the inverse transform method. The acceptance-rejection
method determines whether or not Z is from Q (a, 1).
 8.13. GAM MAD 1ST RIB UTI 0 N 369

Algorithm GS* describes the essential steps. It differs from those in Algorithm GS in
Ahrens and Dieter (1974b) in minor ways. Observe the setup b +- (a + e)/e. If repeated
sampling is to occur for the same value of a, then it is advantageous to compute b once at the
beginning of the experiment and use a and b, instead of a alone, as input, thus avoiding the
repeated cost of computing b. Note that Algorithm GS* admits no simple pretest to exploit
the squeeze method. Best (1983) suggests several modifications that reduce generating time
by a relatively incidental factor for a close to zero and by one-half for a close to unity. As an
extension of a method for a > 1 to be described in the next section, Cheng and Feast (1980)
describe a competitive method for 1/ n ::::: a ::::: 1, where n is a fixed integer.

ALGORITHM GS*

Purpose: To generate Z randomly from 9 (a, 1) with 0 < a < 1.

Input: a.
Setup: b +- (a + e)/e.
Method:
Repeat
randomly generate U from U (0,1); Y +- bU
ifY:::::l:
Z +- ylla; randomly generate W from E (1)
if W 2: Z, success
otherwise:
Z +- -In((b - Y)/a)
randomly generate WI from U (0, 1); W +- WII/(a-1)
if W :::: Z, success
until success.
Return Z.

a >1

Many alternative algorithms exist for this case of which those in Ahrens and Di-
eter (1974a, 1974b, 1982a), Cheng and Feast (1979), Marsaglia (1984), Minh (1988), and
Schmeiser and Lal (1980) are demonstrably fast. The one due to Cheng and Feast uses the
ratio-of-uniforms method (Section 8.6) and illustrates many of the nuances implicit in this
approach.
The acceptance region is

(8.53)

implying that Q (a) lies in the rectangle P (a) := {O ::::: x ::::: x* := [(a - 1)/e](a-1)/2,
0::::: y ::::: y* := [(a + 1)/e](a+1)/2} (definitions 8.27b and 8.27d). Direct application of the
370 C HAP TE R 8. SAM P LIN G FRO M PRO B A B I LIT Y DIS T RIB UTI 0 N 5

(a)
1.0-,---------------r-----c~~-------~
a=2
0.9
0.8
0.7
0.6
y' 0.5
0.4
0.3
0.2
0.1
0.0 -'f-~,.-,-____r~_+_~;__"F==f=;:::.,--,L_,_.,____,,____,_____r~_,___-,-----,_J
-0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6
x'

(b)
1.0-,----------------------y-~-""""

a = 100
0.9

0.8

0.7

0.6

y' 0.5

0.4

0.3

0.2

0.1

0.0 +L-,.----/"'-,---f-_,_____~____r_,____,_____~____r_,____,_____~____r_,____,_____~---,---I
-0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1
x'

Figure 8.4 Gamma generation: ratio-of-uniforms method x' := xjx* and y'
yjy*.

ratio-of-uniforms approach leads to mean number of trials until success

c(a) = IP(a)ljIQ(a)1 = 2(a + 1)(a+l)/2(a -1)(a-l)/2jr(a)ea , (8.54a)

for which

c(a)j..j(i -+ 2j(21l')~ asa -+ 00, (8.54b)

revealing an O(a~) mean generating time. In addition, x* and y* need to be computed.

 8.13. GAMMA DISTRIBUTION 371

Table 8.5 Gamma variate generation:

ratio-of-uniforms methodt
Mean number of
trials until success
a c(a) c'(a) Ratio
1.5 1.331 2.019 .6592
2 1.406 1.848 .7608
2.5 1.499 1.761 .8561
3 1.593 1.709 .9321
4 1.773 1.647 1.077
5 1.941 1.612 1.204
6 2.096 1.590 1.318
7 2.241 1.574 1.424
8 2.378 1.562 1.522
9 2.507 1.553 1.614
10 2.631 1.545 1.703
15 3.177 1.524 2.085
20 3.643 1.514 2.406
25 4.056 1.507 2.691
30 4.431 1.503 2.948
35 4.777 1.500 3.185
40 5.099 1.498 3.404
45 5.402 1.496 3.611
50 5.689 1.495 3.805
00 00 1.482 00

t Expression (8.54a) defines c(a);

expression (8.55) defines c'(a).

To eliminate the asymptotic dependence on a, Cheng and Feast employ a parallelogram

P 'ea), as in Figure 8.4, to enclose Q(a). It leads to a mean number of trials until success

c'(a) = IP '(a)1 = 2x*y*(l + ./ITe), (8.55)

IQ(a)1 r(a),Ja

where

c'(a) ~ 1.482 as a ~ 00.

Table 8.5 reveals that the square P(a) has an advantage for a ~ 3, whereas for a :::: 4,
the parallelogram p'ea) has an ever-increasing advantage as a ~ 00. This suggests a
procedure that uses the square for a ~ 3 and the parallelogram for a :::: 4. However, an
additional issue plays a role.
Recall that (X = x*X', Y = y*Y') is uniformly distributed on P (a) if(X', Y') is
uniformly distributed in the unit square [0, IF. If generation from the same Q(a, 1) is to
occur repeatedly, then one can compute x* and y* once at the beginning, store them, and
use a, x*, and y* as input to the algorithm on each call. If many distinct a's are used, this
372 CHAPTER 8. SAMPliNG FROM PROBABILITY DISTRIBUTIONS

approach may not be feasible, and without modification one would repeatedly incur the
relatively high setup cost of computing x* and y*. A modification of the acceptance region
reduces this cost substantially at a relatively small increase in the mean number of trials until
success for small a.
Let (X', Y') be uniformly distributed on [0, IF. Let X = x*X' and Y = y**Y', where
y** :::: y*. Then (X, Y) is uniformly distributed on the rectangle

PI(a):= {(x, y): 0 ~ x ~ x*, 0 ~ y ~ y**}, (8.56)

where Q (a) ~ P (a) ~ PI (a). Let W = X 2 and Z = Y/ X. If W ~ r(Z), then Z is from

9 (a, 1). Observe that this inequality implies

a: 1 lnX' ~ In(bY'/X') - bY'/X' + 1, (8.57)

where b := y** /x*(a - 1), suggesting that one merely has to take b :::: y* /x*(a - 1) =
(a + 1)/(a - 1)(u+I)/2 /e and Z = (a - 1)bY' / X'. Cheng and Feast suggest

b := (a - 1/6a.)/(a - 1), (8.58)

which ensures b :::: y* /x*(a - 1) and eliminates the need to evaluate x*, y*, and y**.
Since W and Z have the joint p.d.f.

iw,z(w, z) = w/b(a - 1), o~ z ~ b(a - 1)/w, 0 ~ w ~ 1,

it follows that

pr[W ~ r(Z)] = IQ(a)I/IPI(a)1 = r(a)eU- I /2b(a -1)u (8.59)

= r(a)eU- I /2(a - 1/6a.)(a - 1)U-I,

and Z given W ~ r(Z) has the desired Gamma p.d.f. Since X' and Y' are uniformly
distributed over [0, 1F, the mean number of trials is 2( a /217:) 1/2 as a -+ 00, which agrees
with the limit (8.54b).
Since P '(a) n [0, IF C P(a), this alteration also applies to sampling from P '(a).
While preserving the validity of the ratio-of-uniforms approach to Gamma generation for
a > 1, the change merely scales c(a) and c'(a) by b(a - l)x* /y*, which decreases from
1.133 at a = 1 to unity as a -+ 00.
Algorithm GKMI describes the generating method when (X', Y') are randomly gen-
erated on the unit square P(a). Note the pretest mX' - d + V + V-I ~ 0, which is based
on the inequality In w ~ w - 1 for w > O. This pretest reduces the frequency oflogarithmic
evaluation (Exercise 8.42). Whenever possible, evaluating a, b, m, and d beforehand, storing
them, and using them as input in place of a on repeated generations for the same a saves time.
 8.13. GAMMA D I STRI B UTI 0 N 373

ALGORITHM GKM1

Purpose: To sample Z randomly from the Gamma distribution 9 (a, 1) with a > 1.
Input: a.
Setup: a +- a-I, b +- (a - 1/6a)/a, m +- 2/a, d +- m + 2.
Source: Cheng and Feast (1979).
Method:
Repeat
randomly generate X' and Y' fromU(O, 1)
V+- bY'/X'
if mX' - d + V + V-I :s: 0, accept (fast acceptance)
if m In X' - In V + V-I :s: 0, accept
until accept.
Return Z +- aVo

Algorithm GKM2 describes the steps when sampling (X', Y') from the parallelogram
P '(a). Whenever possible, evaluating the setup, a, b, m, d and especially f, beforehand,

°
storing them and using them as input on repeated generations for the same a saves time.
Note that Ne = .857764. It is left to the reader (Exercise 33) to show that if < X' < 1,
then (X', Y') is uniformly distributed on p'ea) n [0, IF and that on any trial

e+2
pr(O < X' < 1) = 1 - v'2 > .5328. (8.60)
2Jae(~+ 2)-

ALGORITHM GKM2

Purpose: To generate Z randomly from 9 (a, 1) for a > 1.

Input: a.
Setup: a +- a -1,b +- (a -1/6a)/a,m, +- 2/a,d +- m +2, f +-.j(i.
Source: Cheng and Feast (1979).
Method:
Repeat
Repeat
randomly generate X and Y' fromU (0,1); X' +- Y' + (1- 1.857764X)/f
until °< X' < 1.
V+- bY'/X'
if mX' - d + V + V-I :s: 0, accept (fast acceptance)
ifm In X' -In V + V-I :s: 0, accept
until accept.
Return Z +- aVo

The step f +- .j(i keeps Algorithm GKM2 from completely dominating Algorithm GKMI
timewise. However, the associated O( 1) evaluation time increasingly favors Algorithm GKM2
374 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

when compared to the O(a~) mean generating time for Algorithm GKMI as a ~ 00. Al-
gorithm GKM3 reconciles these two options. Cheng and Feast recommend ao = 2.5.

ALGORITHM GKM3

Purpose: To generate Z randomly from Q (a, 1) for a > 1.

Input: a.
Source: Cheng and Feast (1979).
Method:
If a ~ ao : use Algorithm GKMI.
Otherwise: use Algorithm GKM2.

8.13.1 INVERTED GAMMA DISTRIBUTION

If Y "'-' Q(a, fJ), then Z = 1/ Y has the p.d.f.

f( ) - 1 e-1/Pz R
z~ 0,
Z - r(a)fJaza+l ,a,,., > 0,

which is called the inverted Gamma distribution and denoted by TQ(a, f3). Its mean,
variance, and mode are, respectively,

1
EZ= ,
(a - l)fJ
1 .
varZ= ,
(a - 2)(a - I)2fJ2

and

1
mode = ( )
a+IfJ

The existence ofEZ requires a > 1, and ofvarZ, a > 2. Since pr(Y ~ a) = pr(1/Z ~ a),

Fz(z; a, fJ) =1- Fy (1/z; a, fJ),

where Fz and Fy denote the d.f.s of Z and Y, respectively.

Whereas the p.d.f. of Y has a right tail that decreases exponentially, the p.d.f. of Z
decreases polynomially as I/za+l. Chapter 10 discusses this feature with regard to fitting
data to a distribution.
To sample Z from IQ(a, fJ), first sample Y from Q(a, fJ) and then take Z = 1/ Y.
 8.14. BET A DIS T RIB UTI 0 N 375

8.14 BETA DISTRIBUTION

The Beta distribution has p.dJ.

where B denotes the beta function, and is denoted by l3e(a, (3). The distribution has the
following properties:

1. If X and Yare independent random variables from 9 (a, 1) and 9 (f3, 1), respectively,
then Z = X/eX + Y) has the Beta distribution l3e(a, (3).
2. Let denote independent random variables each from U (0,1) and let
U I , .•. , Un
U(l) < ... < U(n) denote the corresponding order statistics. Then U(k) has the Beta
distribution l3e(k, nk + 1) for k = 1, ... , n.

3. If Z is from l3e(a, (3), then 1 - Z is from l3e(f3, a).

Property 1 provides an immediate way to generate a Beta variate given the availability of
a procedure for gamma generation. However, faster Beta generation methods exist. Property
2 reveals how to generate order statistics for any parent distribution for which generation via
the inverse transform method applies. See Fishman (1996, Section 3.26). The third property
shows that any development for a ::=: f3 applies equally to f3 :::: a by simple transformation.
Table 8.1 gives the inverse dJ.s that allow Beta generation in 0(1) time for a = 1
or f3 = 1. However, the derivation of an algorithm with bounded mean computing time
for arbitrary a and f3 calls for considerably more attention. We first consider the cases
max(a, (3) < 1 and min(a, (3) > 1 separately and then consider the case of min(a, (3) < 1
and max(a, (3) > 1.

max(o: ,{3) < 1

Atkinson and Whittaker (1976, 1979) describe an especially fast Beta generating proce-
dure for this case. It combines the composition, inverse transform, and acceptance-rejection
methods in a manner similar to that in Section 8.13 (Ahrens and Dieter 1974b) for Gamma
generation with a < 1. For max(a, (3) < 1 and 0 :::: t :::: 1, let

az,,-I f3(1 - z)/i-l

h(z) = pI[O,tl(Z)~ + (1 - p)I(t,l](z) (1 _ t)fl ' (8.62a)

g(z) = 1- z)fl-I + I(t,l](z) (Z)"-I

I[o,tl(z) ( 1 _ t t ,0:::: z :::: 1,

p = f3t/[f3t +a(1 - t)],

376 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

and

c(a, fJ, t) = r(a + fJ + 1)[fJt + a(1 - t)]1 r(a)r(fJ)(a + fJ)t l - a (1 - t)I-f3. (8.62b)

Here {h(z)} is a p.dJ., 0 ::: g(z) ::: 1,

c(a, fJ, t) = II t
10
g(z)h(z)dz = sup [f(z)1 h(z)]'
O~z~1
and c(a, fJ, t)g(z)h(z) is identical with f(z) in expression (8.61). To generate a Beta variate,
one first determines whether 0 ::: Z ::: t with probability p or t < Z ::: 1 with probability
1 - p. If 0 ::: Z ::: t, then generation occurs from {a za-I I t a , 0 ::: z ::: t} using the inverse
transform method. If (t < Z ::: 1), then generation occurs from {fJ(l - z)f3- I /(1 - t)f3,
t < z ::: I}, again using the inverse transform method. In both cases, the acceptance-
rejection method determines whether or not Z is from Be(a, fJ). The assignment t = t*,
where

fJ(1 - fJ)]-1
(8.63)
a(1 -a) ,

minimizes the mean number of trials c(a, fJ, t). We leave to the reader the proof (Exercise
8.45) that c(a, fJ, t) is bounded above for all 0 ::: a, fJ ::: 1 and all 0::: t ::: l.
Algorithm AW describes the procedure with two preliminary tests that use the inequal-
ity e > x + 1, which holds for all x > O. If repeated generation for the same a and fJ occurs,
then evaluating t prior to the first generation, storing it, and using it as input with a and fJ
on each execution of the algorithm eliminates the repeated computation of the square root.
For repeated generation with many different a's and fJ's, storing all the computed optimal
t's may not be a viable option. An alternative is to set t = ~ as a parameter, which eliminates
the square root evaluation but increases the mean number of trials c(a, fJ, ~). However,
sUPO<a,f3~1 c(a, fJ, ~) < 00. Note that if U ::: p, then UI p is from U (0, 1).

ALGORITHM AW

Purpose: To generate Z randomly from Be(a, fJ) for max(a, fJ) < l.
Input: a andfJ.
<=---:::,..,-1
Setup: and p *- fJtl[fJt + a(1 - t)].
Method:
Repeat
randomly generate U from U (0, 1) and Y from £ (1)
ifU:::p:
Z *- t(Ulp)l/a
 8.14. BET A DIS T RIB UTI 0 N 377

ifY ~ (1 - (3)(t - Z)/(1 - t), accept (fast acceptance)

if Y ~ (1 - (3) In e;=-~), accept
otherwise:
I U)I/fJ
Z +- 1 - (1 - t) ( I=p
ifY ~ (1 - a)(Z/t - I), accept (fast acceptance)
ifY ~ (1 -a)ln(Z/t),accept
until accept.
Return Z.

min(o: ,{3) > 1

Cheng (1978) describes Algorithm BB, which has bounded mean computing time and
is known to be fast. Schmeiser and Babu (1980) and Zechner and Stadlober (1993) describe
alternative fast algorithms. The Cheng procedure relies on the acceptance-rejection method
and exploits the observation that if Y is a random variable with p.d.f.

y ~ 0, (8.64)

then Z = Y /(1 + Y) is from Be(a, (3). Therefore, one need only concentrate on sampling
Y using the acceptance-rejection approach with the factorization

fy(y) = c(a, (3)g(y )h(y),

where

h(y) = AfJ.l·-1/(fJ. + yA)2, Y ~ 0,

g(y) = ya-A /(fJ. + yA)2(a + (3)a+ fJ /4(1 + y)a+ fJ a a+Af3 fJ -'-,
c(a, (3) = 4a af3fJ /AB(a, (3)(a + (3)a+ fJ , (8.65)
min(a, (3) if min(a, (3) :::: 1,
{
A= [(2af3 _ a - (3)/(a + f3 - 2)1~ otherwise,

and

fJ. = (a/f3)A.

For a and f3 ~ 1 c(a, (3) :::: 4/e = °

1.47; for either a :::: 1 and f3 ~ or f3 :::: 1 and a ~ 0,
c(a, (3) = 4, which gives the relatively poor upper bound for a and f3 :::: 1.
Since

x ~ 0,
378 C HAP TE R 8. SAM P LIN G FRO M PRO B A B I LIT Y DIS T RIB UTI 0 N S

the inverse transform method allows one to generate a sample from {h (y )} in constant time.
Algorithm BB* corresponds to Algorithm BB in Cheng (1978) in content. It contains
several pretests that reduce the required number of logarithmic and exponential transfor-
mations. If repeated samples are to be taken with the same a and fJ, then evaluating the
quantities in the setup beforehand, storing them, and using them as input to successive exe-
cutions reduces computing time. There is no way to avoid the logarithmic and exponential
evaluation in evaluating V and W. Note in Algorithm BB* that we use VI VI in place of vf.
Since some programming languages compute vf as exp(2ln Vd, it is desirable to avoid this
expensive evaluation method wherever possible.

ALGORITHM BB*

Purpose: To generate Z randomly from Be(a, fJ) for min(a, fJ) > 1.
Input: a and fJ.
Setup: d l = min(a, fJ), d2 = max(a, fJ), d3 = d l + dz,
d4 = [(d 3 - 2)/(2d l d2 - d3 )] ~, d s = d l + 1/d4 •
Method:
Repeat
randomly generate VI and Vz from U (0, 1)
V +- d4 In[Vt!(1- VI)]; W +- dle v ; ZI +- VIVIVz
R +- d s V - 1.38629436; S +- d l + R - W
if S + 2.60943791 > 5ZI> accept (preliminary test 1 with In 4 =
1.38629436 and 1 + In 5 = 2.60943791)
otherwise:
T +-lnZI
if S 2: T, accept (preliminary test 2)
otherwise: if R + d3 In[d3 /(dz + W)] 2: T, accept
until accept.
If d l = a, return Z +- W /(d z + W); otherwise, return Z +- dz/(d2 + W).

For min(a, fJ) < 1 and max(a, fJ) > 1, see Fishman (1996, Section 3.15).

8.14.1 BETA-PRIME DISTRIBUTION

If Y '" Be(a, fJ) then Z = y-I - 1 has the p.d.f.

r(a)r(fJ) Z~-I
f(z) = r(a + fJ) (1 + z)aH '
a, fJ > 0, Z 2: 0,

sometimes called the Beta-prime distribution (Johnson et al. 1994). Note that Z has infinite
support and that f decreases as O(z-(l+a)) as Z --+ 00.
 8.15. REVISITING THE RATIO-OF-UNIFORMS METHOD 379

Observe that

I < ex.

Moreover, Z has infinite mean for ex ::::: 1. For 1 < ex ::::: 2, it has infinite variance.
To sample from this distribution, first sample Y from Be(ex, fJ) and then return Z =
y- 1 _ 1.

8.15 REVISITING THE RATIO-Of-UNifORMS METHOD

Section B.13 shows howal = 0, a2 = 1, and a bounded region P other than rect-
angular create a bounded-computing-time algorithm for Gamma generation using the
ratio-of-uniforms method. This section describes another variant of the method that is par-
ticularly well adapted for efficient sampling from Poisson, binomial, and hypergeometric
distributions.
As in Section B.13, assume that we wish to generate samples from a bounded p.d.f.
{f(z), -00 < z < oo}. Let

r(z) := f(z) (B.66)

SUPt f(t)

and recall that

Q(al, a2) := {(x, Y): x::::: r 1/ 2 (al + a2Y/x), -00 < a, < 00, a2 > OJ.

Theorem B.4 asserts that if (X, Y) is uniformly distributed on Q(al, a2), then Z = al +
a2Y / X has the p.d.f. f.
Let P be the rectangle

P:= {(x, y): 0::::: x ::::: 1, -1::::: Y ::::: I}. (B.67)

Expressions (B.27b) through (B.27d) imply P 2 Q(al, a2) if and only if

o ::::: r(z) ::::: 1, -00 < z< 00, (B.6B)

and

-00 < z< 00. (B.69)

380 C HAP TE R 8. SAM P LIN G FRO M PRO B A B I LIT Y DIS T RIB UTI 0 N S

Definition (8.66) ensures inequality (8.68). However, two cases need to be considered
for inequality (8.69). In the first, I y I jx =1 z - al I ja2 :::: 1, for which inequality (8.68)
implies inequality (8.69). In the second, I y I jx =1 z - al I ja2 :::: 1, which requires

a2
r(z) :::: ( 2)2 V Z E (-00, al - a2) U (al + a2, (0). (8.70)
z- al

Recall that

1
I Q(al, a2) 1= -2
a2
1 00

-00
r(z)dz = 2
a2 SUPt
1 f( )'
t
(8.71)

Then one chooses al and a2 to minimize the mean number of trials until success

IPI
I Q( aI, a2 ) I = 4a2 sup f(t)
t
(8.72)

subject to -00 < al < 00, a2 > 0, and inequalities (8.68) and (8.70).
The application of the ratio-of-uniforms method to generation from a discrete distri-
bution is now relatively straightforward. Suppose Z has the p.m.f. {Pi; i = a, a + 1, ... , b}.
Then one takes

f(z) = I~i if i ::::

otherwise,
z < i +1 for a :::: i :::: b,
(8.73)

and applies the theory of this section to find desirable values of a I and a2. The next several
sections illustrate the generality that the method offers in practice.

8.16 POISSON DISTRIBUTION

Let Z have the Poisson p.m.f.

).. > 0, i = 0, 1, ... , (8.74)

denoted by P()"). This distribution has the following properties:

1. If X I, X 2, ... are i.i.d. random variables from £ (1) and Z is the smallest nonnegative
integer such that XI + ... + X Z + I > ).., then Z has the Poisson distribution P()").

2. If V I, V 2 , ••• are i.i.d. random variables fromU (0, 1) and Z is the smallest nonnegative
integer such that nt=i l Vi > ).., then Z has the Poisson distribution P()").
 8.16. POlS SON DIS T RIB UTI 0 N 381

3. If Z is from P(A), then (Z - A)/,J). has a distribution that converges to N (0,1) as

A ---+ 00.

Algorithm ITR applies the inverse transform method with Po = e- Aand c(z) = A/Z,
z > O. Although Section 8.1 shows the mean computing time as O( A), its relatively small setup
cost allows this procedure to compare favorably timewise with more sophisticated algorithms
with bounded mean computing time for small A. An alternative algorithm, based on property
2, is often encountered in practice, but timing studies in Atkinson (1979) and Schmeiser
and Kachitvichyanukul (1981) show it to be more time-consuming that Algorithm ITR. No
doubt, the multiple generation of U I , U2 , ••• accounts for this excess time consumption.
Let Y be from N (0, 1). Property 3 implies that treating max(O, LA + 1/2 + y,J}.J) as
a Poisson variate from P(A) incurs an error that diminishes as A increases. The factor is 4
a continuity correction that reduces the error of approximation for every fixed A. Although
this constant-time approach is acceptable in problems in which this error is small relative
to errors from other sources, it is unacceptable in problems where results are sensitive to a
departure from the Poisson distribution. This may be especially so in applying the Monte
Carlo method to estimate the distributional character of a statistic that is a function of
Poisson random variables.
Atkinson (1979), Ahrens and Dieter (1982b), Devroye (1981), Schmeiser and Ka-
chitvichyanukul (1981), and Ahrens and Dieter (1991) all propose Poisson generating
algorithms with bounded mean computing times. Among these, Algorithm PD in Ahrens
and Dieter (1982) appears least time-consuming, but not simple. Since the 1991 proposal of
Ahrens and Dieter for A ~ 1, which exploits the ratio-of-uniforms method, is considerably
simpler and not too much more time-consuming, we describe it here.
We take

r(z):= f(z)/max f(x),

x
-00 < z< 00, (8.75)

where expression (8.73) defines {f (z)} for discrete distributions. The Poisson distribution
has its mode at LAJ, so that

r(z) = PLzj = ALzj-LAj LAJ!/LzJ! (8.76)

maxt PLtj

and the mean number of trials is

(8.77)

To simplify the minimization problem, Ahrens and Dieter recommend

(8.78)
382 C HAP TE R 8. SAM P LIN G FRO M PRO BA B I LIT Y DIS T RIB UTI 0 N S

for al. For this choice, the az that minimizes the ratio (8.77) subject to inequality (8.71b),
A ::: 1, and az > 0 is

(8.79a)

where k* takes one of the two values k* = La .;u;; J and k* = ra .;u;;l. Also,
l - l -

a;(1) =~ (8.79b)
2

and

lim A-1/Za;(A)
A---+oo
= me = 0.8578. (8.79c)

Since computing a; in line is relatively time consuming, Ahrens and Dieter recommend the
less costly

(8.79d)

which manifests the same limits (8.79c) as the optimal az. Then

(8.80a)

and, using Stirling's formula j! ~ ..[iiijj+I/Zexp(-j + e/I2j) for integer j > 1 and
0< e < 1,

(8.80b)

a
Computations in Stadlober (1989) illustrate how well al and z fare with respect to
the optimal al and az. For A = 5, al = 5.277 and az = 2.271 minimize the mean number
a
of trials (8.77). For the recommended l = 5.5 in expression (8.78), expression (8.79a)
gives a; az
= 2.425 as best possible, and expression (8.79d) gives = 2.461, the more easily
a a
computed bound. Therefore, the mean number of trials based on l and z is within 8.37
percent of the achievable minimum, and Stadlober shows that this percentage progressively
decreases to zero as A increases.
Algorithm PRUA* describes a modification of Algorithm PRUN in Stadlober (1989),
which is a modification of Algorithm PRUA in Ahrens and Dieter (1991). Note that the setup
calls for a precomputed table of Yk = In k! for k = 0, 1, ... , k'. Moreover, for k > k', In k!
can be computed by using a truncated form of the expansion (Hastings 1955)

Yk := In..fiir + (k + 1/2) In k - k + I/12k - I/360k 3 + I/1260k s - I/I680F as k -+ 00.

(8.81)
 8.16. POISSON DISTRIBUTION 383

Stadlober reports an absolute truncation error less than 7.9 x 10-9 when terms up to O( 1/ k 3 )
are retained. The choice of k' is at the user's discretion and, in principle, depends on space
availability. Making k' large reduces the need to compute d7 and Yk in line and considerably
improves the efficiency ofthe procedure. If Poisson generation with LAj :::: k' is contemplated,
one can omit the test in the setup entirely. The rejection test Z ::: dg, based on the precision
factor do, allows the algorithm to avoid computing insignificant probabilities. As before, if
repeated sampling is to occur with the same A, then one can compute the setup prior to
any generation, store d3 , d4 , ds, d6 , d7 , Yd7 , dg, and d9 , and use these as input on successive
executions.

ALGORITHM PRUA*

Purpose: To generate K randomly from peA) with A ::: 1.

Input: A and in-line function for computing Yk = In k!.
Setup: Precomputed table of Yk = In k! for k = 0, 1, ...• k'.
Constants do = 7 (for nine-decimal-digit precision), d 1 = 2Ne =
1.715527770, d 2 = 3 - 2Ne = 0.898916162.
d3 +- A + 1/2, d4 +- ,.,fd;, ds +- d1d4 + d2 • d6 +- In A, d7 +- LAj.
dg +- Ld3 + do(d4 + 1)j.
If d7 > k', compute Yd7 in line.
d9 +- d6d7 - Yd7'
Method:
Repeat
randomly generate X and Y from U (0, 1).
Z +- d 3 + ds(Y - 1/2)/ X
if Z < 0 or Z ::: dg, (preliminary test 1
fast rejection)
otherwise:
K +- LZj; if K > k', compute YK in line
T+-Kd6 - Y K - d9
ifT ::: X(4 - X) - 3, accept (preliminary test 2
fast acceptance)
otherwise:
if X(X - T) ::: 1, reject (preliminary test 3
fast rejection)
otherwise:
if2InX :::: T, accept
until accept.
Return K.

Preliminary test 1 accounts for the possibility of Z < O. Preliminary tests 2 and 3
use the squeeze method with the inequalities x-I / x :::: 2ln x :::: -3 + 4x - x 2 , 0 <
384 C HAP TE R 8. SAM PLIN G FRO M PRO B A B I LIT Y DIS T RIB UTI 0 N S

x :::: 1, (Section 8.5.1) to reduce the frequency of evaluating In X by 86 percent according

to Stadlober (1989), who also indicates that nine-digit precision in computation ensures no
loss of accuracy in computing T. For Yk taken exclusively from a stored table, Stadlober also
shows that this approach compares well with all other known bounded mean computing
time Poisson generation algorithms for A :::: 5, except when compared to Algorithm PD
of Ahrens and Dieter (1982b). However, the present approach offers considerably greater
simplicity in description and implementation than that algorithm does.
Stadlober (1989) shows that, with the exception of Algorithm PD in Ahrens and Di-
eter (1982b), all bounded mean computing time algorithms for Poisson generation consume
more time than Algorithm ITR does for small A. This remains true whether or not one in-
cludes the setup times. Also, using the relatively simple Algorithm ITR for Poisson generation,
for say A :::: 10, and Algorithm PRUA>I- otherwise, appears to be a judicious compromise. If
speed is the overriding consideration, then the less simple Algorithm PD deserves attention.

8.17 BINOMIAL DISTRIBUTION

Let Z have the binomial distribution

Pi = G)pi(l-Pt- i , O<p<l, ninteger>O, i=O,l, ... ,n, (8.82)

denoted by B(n, p). The distribution has the following properties:

1. If ZI, Z2, ... , Zn are independent Bernoulli random variables with parameter p, then
Z = ZI + ... + Zn is from B(n, p).
2. Let /-L = np. For fixed /-L, the distribution of Z converges to P(/-L) as n -+ 00 and
p -+ 0 simultaneously.

3. If Z is from B(n, p), then (Z - np)/Jnp(1- p) has a distribution that converges to

N(O, 1) asn -+ 00.

Algorithm ITR applies the inverse transform method with Po = (1 - p)n and c(z) =
(n - Z + 1)p/z(1 - p), which consumes O(np) time. If one takes Po = (1 - iW and
+
c(z) = (n - z l)p/z(1- p), where p = min(p, 1- p), and returns Z if p :::: andn - Z !
if p > !,
then the algorithm has O(n min(p, 1 - p)) time. Algorithm BER provides an
alternative approach that merely generates n independent Bernoulli samples and takes their
total as Z. Although this avoids the need for the logarithmic and exponential evaluations
implicit in (1 - p)n , it takes O(n) time. Kachitvichyanukul and Schmeiser (1988) report that
the inverse transform approach performs faster than the Bernoulli approach does.
 8.17. BINOMIAL DISTRIBUTION 385

ALGORITHM BER

Purpose: To generate Z randomly from B(n, p).

Input: n andp.
Method:
Z +-- 0 and i +-- 1.
While i ::: n :
Randomly generate U from U (0, 1).
Z+--Z+LU+pj.
i+--i+1.
Return Z.

Although asymptotic properties 2 and 3 may tempt one to employ Poisson or normal
generation to produce an approximate binomial Z, the resulting error may lead to seriously
misleading results in the Monte Carlo experiment.
Fishman (1979), Ahrens and Dieter (1980), Devroye and Naderasamani (1980),
Kachitvichyanukul and Schmeiser (1988), and Stadlober (1991) all describe binomial gen-
erating algorithms with bounded mean computing times. We describe Stadlober's approach
based on the ratio-of-uniforms method for p ::: np ::: 1, and!,
r(z):= PLzJ = L(n + 1)pj!(n - +
L(n 1)pj)! [p/(1- p)]LzJ-L(n+I)pJ. (8.83)
maxt PLtJ Lzj!(n - Lzj)!

The mean number of trials is

IPI/IQ(al, az)1 = 4az ( n ) pL(n+l)pJ (1- p)n-L(n+l)pJ. (8.84a)

L(n+1)pj

For at> Stadlober recommends

A 1
al :=np+-, (8.84b)
2

which is optimal for p = !. For a2,

*
a - a -k
A * '(
m. n - m . )' (-p- )k*-m m := L(n + 1)pj, (8.85)
z- ( I ) k*!(n - k*)! 1_ P ,

wherek* = lal - J2a l (1- p) Jork* = fa l - J2a l (l- p)l maximizesIQ(a l , a2)1flP I·
Also, for np = 1,
lim a;(n, p) = 1.5,
n~oo
(8.86a)
p-40
386 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

and for fixed p,

lim a~(n, p) = J2Te = 0.8578. (8.86b)

n ..... oo Jnp(1 - p)

For convenience of computation, Stadlober recommends

a2 = a2 =
A
2 np(1- p)
[ 1] 3 (3
+"2 Ie + "2 - V;' (8.87)

which achieves the limits (8.86a) and (8.86b). Also,

(8.88a)

and

(8.88b)

Algorithm BRUA* is a modification of Algorithm BRUA in Stadlober (1991). It allows

for a precomputed table of {Y. = Ink !; k = 1, ... , k'}, and clearly, if k' ~ n, then no
in-line function evaluation of Y. occurs. As with Algorithm PRUA*, a large table improves
the computing time considerably. If repeated binomial generation with the same nand p
is to occur, then computing the setup beforehand, storing it, and using it as input on each
successive execution reduces computing time. Proofs of the fast acceptance and fast rejection
tests are left to the reader (Exercises 8.51 and 8.52).

ALGORITHM BRUA*

Purpose: To generate Z randomly from B(n, p) with n min(p, 1 - p) ~ 1.

Input: n, p and in-line function for computing Yk = In k!
Setup: Precomputed table of {Y. = Ink!; k = 0, 1, ... , k'}.
Constants do = 7 (for nine decimal digit precision), d1 = 21f =
1.715527770, d 2 = 3 - 2/f = 0.898916162.
d 3 +- min(p, 1 - p), d 4 +- nd3 + !, ds +- 1 - d 3, d 6 +- Jr-n-d3-d-s-+-!
d 7 +- d 1d 6 + d 2 , d s +- d 3lds , d 9 +-In ds, dlO +- L(n + l)d3 J,
d ll +- min(n + 1, Ld4 + d od 6 J).
If dlO > k', compute Yd 10 in line.
If n - dlO > k', compute Y n - d 10 in line.
d12 +- Yd lO + Yn-d lO •
 8.18. H Y PER G EO MET RIC DIS T RIB UTI 0 N 387

Method:
Repeat
randomly generate X and Y from U (0, 1)
W +--d4 +d7 (Y- ~)/X
if W < 0 or W 2: d ll , reject (fast rejection)
otherwise:
Z+--LWJ
if Z > k', compute Yz in line
if n - Z > k', compute Yn - Z in line
T +-- (Z - d lO )d9 + dJ2 - Yz - Yn - Z
if X(4 - X) - 3 :s T, accept (fast acceptance)
otherwise:
if X(X - T) 2: 1, reject (fast rejection)
otherwise:
if 2 In X :s T, accept
until accept.
If P :s ~, return Z; otherwise, return n - Z.

For small n min(p, 1 - p), Algorithm ITR is less time-consuming than virtually all
bounded-time algorithms are for binomial generation. Accordingly, using Algorithm ITR for
n min(p, 1- p) :s 10, andAlgorithmBRUA* otherwise provides a reasonable compromise.

8.18 HYPERGEOMETRIC DISTRIBUTION

The hypergeometric distribution has the form

pz -- (az) (n f3_ z) / (a +n (3), a, f3 2: 0, a + f3 2: n 2: 1,

(8.89)
max(O, n - (3) :s z :s min(n, a),
and is denoted by '}{ (a, f3, n). In the study of production and quality control problems, it
applies to a finite population of items of size a + f3 of which a have a particular characteristic
and f3 do not. If one randomly draws n items, n :s (a + (3), from the population without
replacement and Z is the number of the drawn items that have the characteristic, then Z has
the p.m.f. (8.89).
Algorithm ITR with Po = f3!(a + f3 - n)!/(f3 - n)!(a + (3)! and e(z) = (a - z)(n -
z) I (z + 1) (f3 - n + z + 1), z 2: max (0, n - (3), provides the most direct generating method and
takesOC,7.B) time. Replacing a bymax(a, (3) andf3 bymin(a, (3) everywhere in Po ande(z)
and returning Z if a :s f3 and n - Z otherwise reduces the time to O(n min:(a, (3) I(a + (3)).
In the present case, the setup cost for Po is considerably larger than it is when Algorithm
388 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

ITR is applied to Poisson and binomial generation. Algorithm HYP provides an alternative
approach based on direct sampling without replacement and takes O(n) time, but avoids
the need to compute Po. However, it can require n uniform deviates, in contrast to one
deviate for Algorithm ITR. If repeated generation with the same a, fJ, and n is to occur,
then precomputing Po, storing it, and using it on successive executions of Algorithm ITR
makes this approach considerably more efficient than sampling without replacement as in
Algorithm HYP (Kachitvichyanukul and Schmeiser 1985).

ALGORITHM HYP

Purpose: To generate Z randomly from 1t (a, fJ, n).

Input: a,fJ, and n where a 2: 1, fJ 2: 1, and a + fJ 2: n 2: 1.
Method:
d l +- a + fJ - n, d2 +- min(a, fJ), Y +- d 2 and i +- n.
WhileYxi>O:
Randomly generate U fromU(O, O.
Y +- Y - LU + Y/(d l +i)J.
i+-i-1.
Z +- d2 - Y.
Ifa :::: fJ, return Z; otherwise, return Z +- n - Z.

Kachitvichyanukul and Schmeiser (1985) describe a bounded mean computing time al-
gorithm for generating a hypergeometric sample that employs the acceptance-rejection
method. Stadlober (1989) describes a bounded mean computing time algorithm that
employs the ratio-of-uniforms method of Section 8.15. We describe this last method here.
We have

PLzJ mICa - m)!(n - m)!(fJ - n + m)!

r(z) := = ------------ (8.90)
maxt PLtJ LzJ!(a - LzJ)!(n - LzJ)!(fJ - n + LzJ)!

and

m := L(n + l)(a + O/(a + fJ + 2)J

with the mean number of trials

(8.91)

For al and a2, Stadlober recommends

£21 := na/(a + fJ) + ~2 (8.92a)

 8.18. H Y PER G E 0 MET RIC DIS T RIB UTI 0 N 389

and

(8.92b)

where

2 a+f3-n a f3
a :=n· '--'--.
a+f3-1 a+f3 a+f3

An extensive numerical study in Stadlober (1989) shows that for al = al in expression

(8.92a), the choice of a2 = a2 in expression (8.92b) was always greater than the a2 that
minimizes the ratio (8.91), namely,

(8.93)

where

and

Yj := In j!,

and k* assumes the value

2al-f3-[1 - n/(a + f3)lJ or 2al-f3-[1 - n/(a + f3)ll.

a+f3 a+f3

Again, computing a2 instead of aI requires considerably less time.

Algorithm HRUA* is a modification of Algorithms HRUA and HRUN in Stadlober
(1989). Note that the setup cost is now considerable, and the admonition to precompute
the quantities in the setup, if possible, is now strongest. For k = Z, a + Z, n - Z, and
f3 - n + Z, the in-line function for 8z takes Yk from the table if k ::::: k* and computes Yk as in
(105) otherwise. Proof of the fast acceptance and rejection tests again are left to the reader
(Exercises 8.55 and 8.56).
Let J.,L = na / (a + f3). For small J.,L Algorithm HYP is considerably less time-consuming
than Algorithm HRUA* would be. Based on the timing study in Stadlober (1989) for algo-
rithms dose to HRUA*, using Algorithms HYP for J.,L ::::: 3 and HRUA* for J.,L > 0 provides a
reasonable compromise.
390 C HAP TE R 8. SAM P LIN G FRO M PRO B A B I LIT Y DIS TR I BUT ION 5

ALGORITHM HRUA*

Purpose: To generate Z randomly from 1i (a, {3, n).

Input: a, {3, n with 1 .:::: n .:::: (a + {3) /2; in-line function for Yk = In k!; in-line
function for lik = Y k + Yminla.PI-k + Y m- k + Ym.,la.PI-m+k'
Setup: Precomputed table of {Yk = In k!; k = 0, 1, ... , k'}.
Constants m *- min(n, a + {3 - n), do = 7 (for nine-decimal-digit precision),
d j = 2~ = 1.715527770,d2 = 3 - 2,J'3!e = 0.898916162.
d 3 *- min(a, {3), d 4 *- d 3 /(a + {3), d s *- 1 - d 4, d 6 *- md4 + 1,
d 7 *- __
J:~ 7nd4ds + 1, d g *- d d 7 + d 2 ,
j

d9 *- L(m + 1)(d3 + 1)/(a + {3 + 2)J,

dlO *- lid9, dB *- min(min(m, d 3) + 1, Ld6 + d od 7J).
Method:
Repeat
randomly generate X and Y from U(O, 1)
W *- d6 + dg(Y - 1/2)/ X
if W < 0 or W ::: dB, reject (fast rejection)
Z*-LWj.
otherwise:
T *- dlO - liz
if X(4 - X) - 3 .:::: T, accept (fast acceptance)
otherwise:
if X(X - T) ::: 1, reject (fast rejection)
otherwise:
if 2 In X .:::: T, accept
until accept.
Ifa .:::: {3, Z *- m - Z.
If m ::: n, return Z; otherwise, return Z *- a - Z.

8.19 GEOMETRIC DISTRIBUTION

Let Z have the geometric p.m.f.

Pi = (1 - p) pi, 0 < P < 1, i = 0, 1, ... , (8.94)

denoted by Qe(p). This distribution has the following properties:

1. Let Y j , Y2 ,
••• be a sequence of i.i.d. Bernoulli distributed random variables with

success probability 1 - p. Then

Z = min(i ::: 1 : Yj + ... + Yi = 1) - 1.

 8.20. NEG A T I V E BIN 0 M I A L DIS T RIB UTI 0 N 391

2. Let fJ = -1/ In P and let X denote a random variable from £ (fJ). Then LX J has the
p.m.f. (8.94).

Algorithm ITR with Po = 1 - P and c(z) = P, z > 0, provides a straightforward

generating method with an 00/0 - p)) mean computing time. Observe that the setup
cost for Po is relatively incidental when compared to the corresponding costs for Poisson,
binomial, and hypergeometric generation using Algorithm ITR. However, the O( 1/( 1 - p))
computing time limits the appeal of this approach as p increases. For example, p = .50 gives
1/0- p) = 2, whereas p = .90 gives I/O - p) = lO.
Algorithm GEO provides a bounded mean computing time algorithm by exploiting
property 2 relating to the exponential and geometric distributions. If repeated generation is
to occur with the same p, then computing fJ = -1/ In p beforehand, storing it, and using it
as input to successive executions reduces the unit generation cost. Also, the availability of an
efficient source of exponential samples benefits this approach. As p increases, one comes to
favor Algorithm GEO over Algorithm ITR. However, the exact p at which one is indifferent
as to which of these procedures one uses remains a function of the particular computer and
programming language being employed and the form of exponential generation.

ALGORITHM GEO

Purpose: To generate Z randomly from 9 e (p ).

Input: p.
Setup: fJ = -1/lnp.
Method:
Randomly generate Y from £ (1).
Z +- LfJY J.
Return Z.

8.20 NEGATIVE BINOMIAL DISTRIBUTION

Let Z have the negative binomial p.m.f.

r(r+i) i
Pi = ru + l)r(r) 0 -
r
p) p, o< p < 1, r > 0, i = 0, 1, ... , (8.95)

denoted by N13(r, p). This distribution has the following properties:

1. Let Y1 , Y2 , .•• be an infinite sequence of i.i.d. Bernoulli distributed random variables

with success probability 1 - p. For r integer,

Z = minU :::: 1 : Y1 + ... + Yi = r) - r

392 C HAP TE R 8. 5 AMP LIN G FRO M PRO BA B I LIT Y DIS TR I BUT ION S

has the p.m.f. (8.95).

2. For r integer, let ZJ, ... , Zr denote i.i.d. random variables from ge(p). Then Z =
ZI + ... + Zr has the p.m.f. (8.95).

3. Let X be from 9 (r, 1) and let Y given X be from P(XpI(l - p)). Then Y has the
p.m.f. (8.95).

In the case of r integer, {Pi} is sometimes called the Pascal distribution.

Algorithm ITR with Po = (l - p)' and c(z) = per + z - 1)lz, z ::: 1, provides a
generating method that takes OCr 1(1 - p)) mean computing time. If repeated generating
with the same rand p is to occur, then computing Po beforehand, storing it, and using it
as input to successive executions spreads the one-time setup cost over all generations, thus
reducing the unit cost. For r integer, applying Algorithm ITR with Po = 1- P and c(z) = p,
z ::: 1, r times and summing the results provides an alternative with O(rl(l - p)) mean
computing time that eliminates the setup cost entirely.
Algorithm NEG BIN describes a procedure that exploits property 3. Ifbounded mean
computing time algorithms such as GKM3 and PRUA* are employed for Gamma and Poisson
generation, respectively, and r ::: 1, then Algorithm NEGBIN, using those algorithms, has a
bounded mean computing time.

ALGORITHM NEGBIN

Purpose: To generate Z randomlyfromNB(r, p).

Input: r andp.
Method:
Randomly generate X from 9 (r, 1).
Randomly generate Z from P (Xpl(l- p)).
Return Z.

8.21 MULTINOMIAL DISTRIBUTION

Let Z = (Z J, ... , Zr) T have the multinomial distribution

!(zl, ... ,Zr)=n! T1~

r ( z;) (8.96)
i=1 Zi! '

where

0< Pi < 1,
PI + ... + Pr = 1,
 8.21. M U L TIN 0 M I A L DIS TR I BUT ION 393

Zi integer ::: 0,
ZI + ... + Zr = n,

and denoted by M (n, PI, ... , Pr). Observe that Zj has the binomial p.m.f.

and more importantly, Zj given ZI = ZI, ... , Zj_1 = Zj-I has the binomial p.m.f.

where

2 ::s J ::s r. (8.97)

Algorithm MULTN exploits this property. If used together with a bounded mean
computing time algorithm such as BRUA*, the worst-case mean computing time is OCr).
However, an opportunity does exist to improve efficiency further.
Let

Sj:=ZI+",+Zj, J = 1, ... , r,
So:= 0,

and let N denote the mean number of required binomial generations. Then

r
EN = I>
j=1
pr(Sj_1 < n, Sj = n)

r
= L[pr(Sj = n) - pr(Sj_1 = n)] (8.98)
j=1

= tJ [(tpi)n -(I:Pi)n]
J=I 1=1 1=1

Clearly, EN is minimized if PI ::: P2 ::: ... ::: PrJ suggesting that one rearrange cell proba-
bilities into descending order before executing Algorithm MULTN. The saving is especially
substantial if a single P j is considerably larger than the remaining ones.
394 C HAP TE R 8. 5 AMP LIN G FRO M PRO BA B I LIT Y DIS T RIB UTI 0 N S

ALGORITHM MULT....

Purpose: To generate Z = (Z I, •.• , Zr) T randomly from the multinomial distribution

M (n, PI,"" Pr)'
Input: n, r, PI,"" Pro
Method:
q +- 1; m +- n; j +- 1.
Until m . (r - j + 1) = 0 :
Randomly generate Z j from 13 (m, P j / q).
m +- m - Z j; q +- q - Pi; j +- j + 1.
ReturnZ = (Z\, ... , Zj_\, 0, ... , O)T.

8.22 BERNOULLI TRIALS

In principle, generating a sequence of n independent Bernoulli trials with success probability
P is straightforward. One generates UI , ••• , Un from U (0, 1) and forms Zi = LUi + PJ for
i = 1, ... , n. This takes O(n) time. If np is small, few successes occur, making an alternative
worth considering. Let X be from gee 1 - p), where one can regard T = 1 + X as the time
to the first success in an infinite Bernoulli sequence. Then Algorithm SKIP generates the set
of indices or times T\, ... , Tj _ 1 at which successive successes occur. The computing time
for Algorithm ITR with Po = P and c(z) = 1 - P, z > 0, is O(np/(l - p», which, when
P is small, more than compensates for the added cost of generating X from ge (1 - p).

ALGORITHM SKIP

Purpose: To generate random times T\, ... , Tj _ 1 at which Bernoulli trial

successes occur.
Input: Success probability P and number of successes n.
Method:
To +- 0; j +- O.
Until Tj > n :
j +-j+1.
Randomly generate X from ge(l- p).
Tj +- Tj _ 1 + X + 1.
If j = 1, return indicating no successes.
Otherwise, return T\, ... , Tj _ l •
 8.24. Ex ERe I S E S 395

8.23 LESSONS LEARNED

• All algorithms for generating samples from probability distributions rely on the avail-
ability of a procedure for generating a sequence of numbers with values in (0, 1) that
resembles a sequence of i.i.d. random variables from the uniform distribution on this
interval. Chapter 9 describes pseudorandom number generation, an essential ingredient
in creating these sequences.

• The principal algorithms for generating these samples are the inverse-transform, com-
position, acceptance-rejection, ratio-of-uniforms, and exact-approximation methods.
All are theoretically correct and apply to both continuous and discrete distributions.

• Random variate generating algorithms fall into three categories: those with computing
times that are independent of parameter values (1), those that tend to grow as functions
of the parameters of the distribution (2), and those that have an upper bound on
computing time (3). Whenever possible, it is preferable to use an algorithm of type
(1) or type (3).

• For continuous distributions, the inverse-transform method usually executes in time

independent of parameter values. Examples are exponential and normal variate
generation

• The cutpoint method, based on the inverse-transform approach, and the alias method,
based on the composition approach, allow sampling from a table in time independent
of its size.

• Virtually all competitive acceptance-rejection and ratio-of-uniforms algorithms have

upper bounds on computing time.

• The squeeze method (Section 8.5.1) provides a means of further reducing computing
times for the acceptance-rejection and ratio-of-uniforms techniques.

• Virtually all simulation languages provide library routines for sampling from com-
monly encountered distributions. These include the exponential, Gamma, Erlang,
normal, uniform, Weibull, binomial, Poisson, geometric, and negative binomial. Some
routines may be based on approximation rather than theoretically exact algorithms.
Also, some may have parameter-dependent computing times.

8.24 EXERCISES
INVERSE TRANSFORM METHOD

t? 8.1 Prove that if U is from U (0, 1), then V = 1- U is from U (0, 1).
396 C HAP TE R 8. SAM P LIN G FRO M PRO BA B I LIT Y DIS TR I BUT ION S

~ 8.2 Show that X = - fJ In U is from £ (fJ).

~ 8.3 For the trapezoidal p.d.f.

z-c
-- if c::: z::: a+c,
ab
1
fez) = - if a+c:::z:::b+c, (8.99)
b
a+b+c-z
if b + c ::: z ::: a + b + c, O<a:::b:
ab

a. Devise an algorithm for generating samples based on the inverse transform method.

b. Show that for appropriately chosen al> az, and a3, and UI and Uz independent of
U (0,1), Z = alU I + azUz + a3 has the trapezoidal p.d.f. in equation (8.1).

~ 8.4 Let VJ, ... , Vn be i.i.d. from U (a, b) and let V(l), V(Z), .•. , V(n) denote the order
statistics of VI, .•. , V n • That is, V(I) ::: V(Z) ::: ... ::: V(n)' It is known that V(j) has the p.d.f.

r(n + 1) . I .
f ·(v) = (v - a)J- (b - v)n-J(b - a)-n 0< a < v < b
J rU)r(n - j + 1) - - -

where r(k) = (k - 1)! for k integer 2: 1. Let XJ, ... , Xn denote U.d. random variables
that have the non-central Cauchy distribution

F(x) = -1
2
+ -1 tan- I
7r
(x - c)
--
d
d > 0, -00 < c< 00, -00 < x < 00.

a. Describe an algorithm for randomly generating Y = max(XJ, ... , Xn) using exactly
one uniform deviate U from U(O, 1).

b. Describe an algorithm for randomly generating Y = max(XJ, ... , Xn) and Z =

min(XJ, ... , Xn) using exactly two independent uniform deviates U and V from
U(O, 1). Hint: Conditioning plays a role.

~ 8.5 Apply the inverse transform method for generating a sample, X, from the d.f.

F(x) = (1- _2_CO_;_-_I_xy °: : x ::: 1.

~ 8.6 Let U and V be independent samples, each drawn from U(O, 1). Use U and V
together with the inverse transform and composition methods to generate a sample X from
 8.24. Ex ERe I 5 E 5 397

the d.f.

F(x) = 1 00
h(y)G(x, y)dy,

where

1
hey) = -e- y1jJ f3 > 0, y ~ 0
f3
and

1
G(x, y) = 1 + e- (x-y
)I
C
c > 0, -00 < x < 00.

e? 8.7 For the triangular p.d.f.

a. Devise an algorithm for generating random samples based on the inverse transform
method.

b. For the special case of m2 - ml = m3 - m2, devise a generating algorithm that relies
on a linear combination of independent VI and V 2 each from U (0,1).

CUTPOINT METHOD

e? 8.8 Derive the mean generating cost doEl;';, + (d l - do)E(l;';,IZ > b*)(l - qb*) for
Algorithm CM.

ALIAS METHOD

e? 8.9 Apply cutpoint method 2 for large b - a to the Poisson distribution with mean
A = 100 and using m = 25,50,100,200 cutpoints. For each m, compute:

a. Cutpoints II, ... , 1m

b. Mean number of comparisons

c. Total space requirements.

398 C HAP TE R 8. SAM P LIN G FRO M PRO BA B I LIT Y DIS T RIB UTI 0 N 5

<ff? 8.10 Prove that everyp.m.f. {PI, ... , Pn} can be expressed as an equiprobable mixture
of n - 1 two-point p.m.f.s.

<ff? 8.11 Define k = b - a + 1 and suppose that Pa < 1/ k and Pi ::: 1/ k, a + 1 ::::: i ::::: b.
Show that rl ::::: 1, VI E g, in step * in Algorithm ALSET. This property ensures that the
algorithm continues to iterate until all aliases are assigned.

ACCEPTANCE-REJECTION METHOD

<ff? 8.12 Consider the discrete p.m.f.

fez) = cg(z)h(z), z = a, a + 1, ... , b,

where

b
h(z) ~ 0, Lh(z) = 1,
z=a
c = max[f(z)/
z
h(z)]' 0::::: g(z) ::::: 1.

Suppose one samples Z from {h(z)} and U from U (0, 1). If U ::::: g(Z), show that Z has the
p.m.f. {fez)}.

<ff? 8.13 Let X denote a random variable with distribution function

a
x-a
F(x) = ( - -) Ct > 0, 0::::: a::::: x::::: b. (8.100)
b-a

Give an algorithm for sampling from (1) based on the acceptance-rejection method. Identify
any additional restrictions on Ct, a, and b and give the mean number of trials until success.

<ff? 8.14 Let X have bounded p.d.f. f over the finite domain [a, b] and define A =
maxx [f(x)]. Let U and Y be independent and fromU (0,1) andU (a, b), respectively, and
let S denote the event U ::::: f(y)/A.

a. Show that if S holds, then Y has the p.d.f. f.

b. Determine the success probability preS).

 8.24. Ex ERe I S E S 399

I(
c. Suppose one randomly generates X from the p.d.f.

e-x+1 - 1)/(e - 2), 0 < x < 1,

f(x) = - -:
0, otherwIse,

using the aforementioned acceptance-rejection scheme. Determine the corresponding

preS).

d. Describe an upper bound pretest that reduces the mean number of exponentiations
until success. Do not address the issue of a lower bound pretest. Hint: A picture may
be helpful.

!
e. Modify the generating method in part c to increase preS) by without introducing
any new nonlinear transformations to generate U and Y. Hint: The picture in part d
may again be helpful.

#f? 8.15 (Fishman 1976). Using the p.d.f.

v > 0, z ::: 0 :

a. Devise a factorization c(ex, v)g(z, ex, v)h(z, v) for the Gamma p.d.f.

for implementing the acceptance-rejection method.

b. Find the v that minimizes the mean number of trials.

c. Show analytically how the mean number of trials varies as a function of increasing ex.

d. Prepare an algorithm for Gamma generation that minimizes the mean number of
exponential and logarithmic function evaluations on each trial. Assume no pretest.

#f? 8.16 (Cheng 1977). Using the p.d.f.

z ::: 0,

where

f.1 = ex)..,
A = (2ex - 1)1/2,
400 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

a. Devise a factorization c(a)g(z, a)h(z, a) for the Gamma p.d.f.

a ::: 1, z ::: 0,

for implementing the acceptance-:-rejection method.

b. Show that lima --+ oo c(a) = 2/./ii.

c. Other choices of f.L and A. are possible. Explain the appeal of the choices made here.

d. Prepare an algorithm for Gamma generation that minimizes the number of

exponential and logarithmic function evaluations per trial. Assume no pretest.

~ 8.17 (Ahrens and Dieter 1974b). Using the truncated Cauchyp.d.f.

{3
h(z, {3, 71) = {hr[{32 + (z - 71)2]' {3 > 0, -00 < 71 < 00, Z ::: 0,

where

1 1 -1
() = - - - tan (-71/{3):
2 7r

a. Derive a factorization c(a, {3, 71)g(z, a, {3, 71)h(z, {3, 71) for the Gamma p.d.f.

a ::: 1, z ::: 0,

for implementing the acceptance-:-rejection method.

b. Show that 71 = a-I and {3 = (2a - 1)1/2 are acceptable choices and give a rationale
for using them.

c. With the choice of {3 and 71 in part b, show that liffia--+oo c(a, {3, 8) = 7r /2.
d. Prepare an algorithm for Gamma generation that minimizes the number of
exponential and logarithmic function evaluations per trial. Assume no pretest.

SQUEEZE METHOD

~ 8.18 Prove the inequalities

1 - x :::: e-x :::: 1- x + x 2 /2, x ::: 0.

 8.24. Ex ERe I 5 E 5 401

@J 8.19 Prove the inequalities

1-I/x:::: In x:::: -1 +x, x ::: o.

@J 8.20 Prove the inequalities

x - l/x :::: 2 In x :::: -3 + 4x - x2, O<x::::l.

@J 8.21 Prove the results for pr(Sd and pr(Su) in expressions (B.23a) and (B.23b),
respectively.

@J 8.22 Prove the result in expression (B.23c).

@J 8.23 For the algorithm developed in Exercise B.15:

a. Devise a pretest that reduces the mean number ofexponential and logarithmic function
evaluations.

b. Derive an expression for the reduced mean number of function evaluations in using
the pretest and compare it with the corresponding mean in Exercise B.15 as ex increases.

@J 8.24 Consider the inequality </Jz -In</J - 1 ::: lnz, \;f</J and z > O. For the algorithm
in Exercise B.16:

a. Devise a pretest that reduces the number of exponential and logarithmic function
evaluations.

b. Derive an expression for the mean number of function evaluations in using the pretest
and compare it with the corresponding mean in Exercise B.16 as ex increases.

@J 8.25 For the algorithm in Exercise B.17:

a. Devise a pretest that reduces the number of exponential and logarithmic function
evaluations.

b. Derive an expression for the reduced mean number of function evaluations and
compare it with the corresponding mean in Exercise B.17 as ex increases.
402 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

~ 8.26 Let X denote a continous random variable with p.d.f.

f(x) = ~sinx
2
0:::: x:::: n. (8.101)

a. Devise a factorization for applying the acceptance-rejection method to sample X from

{f(x)}.

b. Based on this factorization, describe an algorithm for generating a sample.

c. Give the probability of success on a trial and the mean number of trials until success.

d. Give the mean number of sine evaluations until success.

e. Using the squeeze method, devise a piecewise linearlower bound, {gdx)}, for {g(x)}
that reduces the mean number of sine evaluations until success.

f. Describe an algorithm that incorporates this lower bound into the acceptance-
rejection method.

g. Give the mean number of sine evaluations until success for the algorithm in part f.

RATIO-OF-UNIFORMS METHOD

~ 8.27 For the p.d.f. in Exercise 8.14:

a. Describe a procedure based on the ratio-of-uniforms method for randomly generating

a sample.

b. Determine the acceptance probabilitypr(T).

c. Devise a pretest that reduces the frequency of exponential and logarithmic function
evaluations.

d. Determine the mean number of exponential and logarithmic function evaluations

until success.

~ 8.28 Prove pr(Td and pr(Tu) in expressions (8.33a) and (8.33b), respectively.

~ 8.29 (Kinderman and Monahan 1980). Consider generating samples from the Gamma
p.d.f.

> 1, Z ::: 0,
 8.24. Ex ERe I S E S 403

@l 8.30 Using the ratio-of-uniforms method with al = a - I and az = 1.

a. Give {r(z)} and devise the bounds x*, y*, and y* in expression (S.27a).

b. Prepare an algorithm for generating random samples from Q(a, 1).

c. Derive an expression for the mean number of trials until success and characterize its
behavior as a -+ 00.

d. Define d = In[a/(a - 0] and W = (Z + 2ln X)/(a - O. Let TL denote the event

W .::: 2 - a / Z + d. Show that h provides an acceptance pretest that reduces the mean
number of exponential and logarithmic function evaluations.

e. Derive an expression for pr(Td and characterize its behavior as a -+ 00.

f. For d and W as defined in part d, let Tu denote the event W ~ Z/a + d. Show
that Tu provides a rejection pretest that reduces the mean number of exponential and
logarithmic function evaluations.

g. Derive an expression for pr(Tu) and characterize its behavior as a -+ 00.

h. Amend the algorithm of part b to incorporate the pretests of parts d and f.

I. Characterize the mean number of exponential and logarithmic function evaluations

asa -+ 00.

@l 8.31 Give an algorithm for sampling from (S.100) based on the ratio-of uniforms
method. Identify any additional restrictions on a, a, and b and give the mean number of
trials until success.

@l 8.32 For the p.d.f in Exercise S.26,

a) Devise a formulation for sampling X from {f (x)} using the ratio-of-uniforms method
with the enclosing P being the smallest rectangle containing Q.

b. Describe an algorithm for implementing the procedure.

c. Give the mean number of trials until success.

d. Devise a pretest for reducing the mean number of sine evaluations until success.

e. Amend the algorithm of part a to incorporate the pretest.

f. For the amended algorithm of part e, give the mean number of sine evaluations until
success.
404 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

~ 8.33 For the p.d.f in Exercise 8.26,

a. Devise a formulation for sampling X from {f (x ) } using the ratio-of-uniforms method

with the P being the smallest right triangle enclosing Q. For this alternative approach:

b. Describe an algorithm for implementing this procedure.

c. Give the mean number of trials until success.

d. Devise a pretest for reducing the mean number of sine evaluations until success.

e. Amend the algorithm of part a to incorporate the pretest.

f. For the amended algorithm of part e, give the mean number of sine evaluations until
success.

EXACT-APPROXIMATION METHOD

~ 8.34 (Marsaglia 1984). For the half-normal dJ. F(z) = .J2/rr Joz e- w2/2 dw, consider
the approximation

c > 0, Z ::: o.

a. Find {m(x), 0 ::s: x ::s: I}, the inverse of {M(z)}.

b. Show that Ux (x)} in expression (8.36) takes the form

O::s:x::S:l.

c. For the corresponding U* (x)} in expression (8.36) to be a p.dJ., one requires

O::S:P::S:l.

Determine the c that maximizes the left side of this inequality and then determine the
largest possible value for p.

EXPONENTIAL GENERATION

~ 8.35 Let X and Y be independent random variables from U (0, 1) and U (0, 2/e),
respectively.
 8.24. Ex ERe I S E S 405

a. Using the ratio-of-uniforms approach, describe an algorithm for exponential

generation.

b. Let W denote the event Y .::: -2X In X. Show that pr(W) = ej4 = 0.6796.

c. Using the inequalities (8.19), find a function {h*(x)} that minorizes -2x lnx and a
function {h*(x)} that majorizes -2x lnx.

d. Let WL denote the event Y .::: hAX) and Wu the event Y .::: h*(X). Derive expressions
for pr(Wd and pr(Wu ) and evaluate them.

e. Determine the mean required number of evaluations ofln X.

NORMAL GENERATION

t!? 8.36 Let U I and U 2 be independent random variables from U (- ~, ~). Show that if
Uf + ui .::: ~,then Y = ut/(Uf + Un l/2 has the same distribution as cos2nU and
Z = uz/(Uf + Un l/2 has the same distribution as sin 2nU, where U is from U (0, 1). See
Theorem 8.5.

CAUCHY GENERATION

t!? 8.37 Let X I and X 2 be independent random variables from U (- ~, ~).

a. Show that if Xi + X~ .::: 1, then Z = X t/ X 2 has the Cauchy distribution C(O, 1).
b. Determine pre acceptance).

GAMMA GENERATION

t!? 8.38 Show that if X is from 9 (a, 1), then Z = f3X is from 9 (a, f3).

t!? 8.39 Prove the correctness of Algorithm GS*.

t!? 8,40 Prove the result in expression (8.54b).

t!? 8,41 Prove the result in expression (8.57).

t!? 8,42 Determine the mean number oflogarithmic function evaluations for Algorithm
GKMI.
406 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

1jfF 8.43 For Algorithm GKM2:

a. Show that if 0 < X' < I,then (X', Y') isuniformlydistributedonP'(a) nI 2 , and
on any trial

e+2
pr(O < X' < 1) = 1 - ..fi
2J(re( vie + 2)

b. Determine the mean number oflogarithmic function evaluations.

1jfF 8.44 Determine the feasibility of employing the acceptance-rejection method for
sampling Z from 9 (a, 1), a::: I, for {h(z) = J2/rre- z2j2 , z ::: OJ.

BETA GENERATION

1jfF 8.45 Show that {c(a, f3, t), 0:::: t :::: I} in expression (8.62b) is bounded for all 0 < a,
f3 < 1. Find sUPO<a ..B::;:l c(a, f3, t*) for t* in expression (8.63).

1jfF 8.46 For c(a, f3) as in expression (8.65):

a. Show that c(a, f3) :::: 4/e for all a, f3 > O.

b. Show that limll-->o c(a, f3) = 4 for 0 < a :::: 1.

POISSON GENERATION

1jfF8.47 Derive an expression for the mean number of evaluations ofln X in Algorithm
PRUA*.

1jfF i,
8.48 For A. > 1.75 andal = A. + showthatai in definition (8.79a) withk* = ral -
~l minimizes the mean number of trials (8.77). Hint: First, show that the continuous
version of the problem has its optimal solution at Zo > ral - ",fiall.

1jfF 8.49 Prove the validity of preliminary test 2 in Algorithm PRUA*.

1jfF 8.50 Prove the validity of preliminary test 3 given that preliminary test 2 fails in
Algorithm PRUA*.

BINOMIAL GENERATION

1jfF 8.51 Prove the validity of the fast acceptance test in Algorithm BRUA*.

1jfF 8.52 Prove the validity of the second fast rejection test given the failure of the fast
acceptance test in Algorithm BRUA*.
 8.24. Ex ERe I S E S 407

g? 8.53 Derive an expression for the mean number of evaluations of In X in Algorithm

BRUA*.

g? 8.54 Derive an expression for the mean number of logarithmic evaluations in

Algorithm BRUA*.

HYPERGEOMETRIC GE .... ERATIO ....

g? 8.55 Prove the validity of the fast acceptance test in Algorithm HRUA*.

g? 8.56 Prove the validity of the second fast rejection test given the failure of the fast
acceptance test in Algorithm HRUA*.

g? 8.57 Derive an expression for the mean number of logarithmic evaluations in

Algorithm HRUA* .

.... EGATIVE BI .... OMIAL GE.... ERATIO ....

g? 8.58 ProvethatifXisfromQ(r, 1) and Y given X isfromP (Xp/(l-p)),O < p < 1,

then Y has the negative binomial distribution NB (r, p).

g? HANDS-ON-ExERCISES

These exercises provide opportunities to prepare and execute computer programs for sam-
pling on a computer and to analyze the results. Each requires a source of pseudorandom
numbers as input.

CUTPOI ....T METHOD

g? H.I For the distribution

Pi Pi
.001 11 .040
2 .008 12 .108
3 .010 13 .162
4 .011 14 .037
5 .081 15 .012
6 .049 16 .028
7 .048 17 .041
8 .059 18 .029
9 .034 19 .100
10 .142
408 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

a. Use Algorithm CMSET to determine m = 19 cutpoints.

b. Using these cutpoints as input to a computer program based on Algorithm CM, ran-
domly generate 10,000 samples from the distribution. Create a plot that compares the
histogram of these data with the distribution {Pi} and record the computing time per
sample (for comparison with the corresponding time for the alias method in Algorithm
ALIAS in Exercise S.H.2).

ALIAS METHOD

,ff? H.2 For the distribution in Exercise S.H.I:

a. Use Algorithm ALSET to determine the aliases Bb ... , B19 and the probabilities
P(A 1 ), •.. , P(A I9 ).

b. Using the output of Algorithm ALSET as input to a computer program based on

Algorithm ALIAS, randomly generate 10,000 samples from the distribution. Create a
plot that compares the histogram of the data with the distribution {Pi} and record the
computing time per sample.

c. Compare the computing time per sample here with that in Exercise S.H.I

GAMMA GENERATION

,ff? H.3 This exercise compares the performances of the acceptance-rejection and ratio-
of-uniforms methods for generating random samples from 9 (a, 1), 1 :::: a :::: 2.
a. Using the algorithm with the optimal choice of v and pretest devised in Exercises 3
and IS, prepare a computer program to generate samples from 9 (a, 1), a ::: l.

b. Prepare a computer program based on Algorithm GKM1.

c. For each program and each a E {l.2, 1.4, 1.6, I.S, 2}, generate 10,000 samples from
9 (a, 1) and record the computing time.
d. Compare the 10 recorded times and determine which algorithm is preferred for a E
[1,2].

e. Based on analytical considerations, give an argument as to why you would or would

not have anticipated these relative performances.

POISSON GENERATION

,ff? H.4 This exercise applies the ratio-of-uniform method to sampling from the Poisson
distribution. Section 8.15 contains the supplementary theory on which the method is based.
 8.24. EXERCISES 409

You are encouraged to study it and Section 8.16 carefully beore performing the computing
assignment. For A = 1,2,4, 8, 16,32, generate independent samples of size n = 100,000.

a. Using a class-adopted simulation programming language, write a program to gener-

ating samples from the Poisson distribution using its built-in-routine for generating
Poisson variates and then execute it. For each A, record the CPU generating time and
compute the sample average and sample variance of the n outcomes.

b. Write a program in a class-adopted programming language to generate samples from

the Poisson distribution using Algorithm PRUA* in Section 8.16 and execute it. For
each A, compute d3 , ••• , d9 within the program on each of the n calls to PRUA*. For
each A, record the CPU generating time and compute the sample average and sample
variance of the n outcomes. Also, for each A create and display the histogram of number
of trials until success, its sample average, and sample variane. In programming these
computations, avoid storing data for all n trials by computing the sample average and
sample variance recursively (e.g., Section 6.4. Recursive Computation) and, similarly, a
histogramwithordinates{l, 2, ... , 100}. The time to compute these quantities should
be relatively negligble.

c. For each A, compute d3 , ••• , d 9 and install the computed values in the amended
program of part b. Execute the program and again record the CPU generating time
and compute the sample average and sample variance of the n outcomes.

d. How do the three programs compare with regard to CPU generating time?

e. For each A, how do the sample averages of the number of trials compare with theory?

Note: If the simulation programming language selected for part a allows for programming
as in parts b and c and is also used there, the comparison will be more meaningful. If
n = 100, 000 does not suffice to get meaningful estimates of average generation time,
increase n.

~EGATIVE BI~OMIAL GE~ERATIO~

t!? H.5 This exercise applies the ration -of-uniform method to sampling from the negative
binomial distribution. Section 8.15 contains the supplementary theory on which the method
is based. You are encouraged to study it study it and Sections 8.13 (a > 1), 8.16, and 8.20 care-
fully before performing the computing assignment. For each A := i!p E {I, 2, 4,8, 16, 32}
and r E {I, 5, 2.5, 5, IO}, generate samples of size n = 100,000.

a. Using a class-adopted simulation programming language, write a program to generate

samples from the negative binomial distribution using its built-in routine for gener-
ating negative binomial variates and then execute it. If note exists, use Algorithms
NEGBIN with the language's built-in Poisson and Gamma routines. For each A and r,
410 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

record CPU generating time and compute the sample average and sample variance of
the n outcomes.

b. Write and execute a program in a class-adopted programming language to generate

samples from the negative binomial distribution using Property 1 or 2 of the Poisson
distribution (Section 8.16) and Algorithms NEGBIN, GKM3, and PRUA*. For each
A and r, compute d3 , ••• , d 9 within the program on each of the n calls to PRUA*.
For each A and r, record the CPU generating thime excute and comput the sample
average variance of the n outcomes. Also, for each A and r, create and display the
histogram of number of trials until success, its sample average, and its sample variance.
In programming these computations, avoid storing data for all n trials by computing
the sample mean and variance recursively (e.g., Section 6.4 Recursive Computation)
and, similarly, a histogram with ordinates {I, 2, ... , lOa}. The time to compute these
quantities should be relatively negligible.

c. How do the two programs compare with regard to CPU generating time?

d. For each A and r, how do the sample averages of number of trials compare?

Note: If the simulation programming language selected for part a allows for programming
as in part b and is also used there, the comparsion will be more meaningul. If n = 100, 000
does not suffice to get meaninful estimates of average generation time, increase n.

8.25 REFERENCES

Abramowitz, M., and I. Stegun (1964). Handbook of Mathematical Functions, Applied

Mathematics Series 55, National Bureau of Standards, Washington, D.C.

Afflerbach, L., and W. Hormann (1990). Nonuniform random numbers: a sensitivity analysis
for tranformation methods, Lecture Notes in Economics and Mathematical Systems, G.
Plug and U. Dieter, eds., Springer-Verlag, New York.

Agrawal, A., and A. Satyanarayana (1984). An 0(1£ I) time algorithm for computing the
reliability of a class of directed networks, Oper. Res., 32, 493-515.

Aho, A.V., J.E. Hopcroft, and J.D. Ullman (1974). The Design and Analysis of Computer
Algorithms, Addison-Wesley, Reading, MA.

Ahrens, J.H. (1989). How to avoid logarithms in comparisons with uniform random
variables, Computing, 41,163-166.

Ahrens, J.H. (1993). Sampling from general distributions by suboptimal division of domains,
Grazer Math. Berichte 319, Graz, Austria.
 8.25. REFERENCES 411

Ahrens, J.H., and U. Dieter (1972). Computer methods for sampling from the exponential
and normal distributions, Comm. ACM, 15, 873-882.

Ahrens, J.H., and U. Dieter (1974a). Non-Uniform Random-Numbers, Institut fur Math.
Statistik, Technische Hochschule in Graz, Austria.

Ahrens, J.H., and U. Dieter (1974b). Computer methods for sampling from Gamma Beta,
Poisson and binomial distributions, Computing, 12, 223-246.

Ahrens, J.H., and U. Dieter (1980a). Sampling from the binomial and Poisson distributions:
a method with bounded computation times, Computing, 25,193-208.

Ahrens, J.H., and U. Dieter (1982a). Generating Gamma variates by a modified rejection
technique, Comm. ACM, 25, 47-53.

Ahrens, J.H., and U. Dieter (1982b). Computer generation of Poisson deviates from modified
normal distributions, ACM Trans. Math. Software, 8,163-179.

Ahrens, J.H., and U. Dieter (1985). Sequential random sampling, ACM Transactions on
Mathematical Software, 11, 157-169.

Ahrens, J.H., and U. Dieter (1988). Efficient tablefree sampling methods for the exponential,
Cauchy and normal distributions, Comm, ACM, 31, l330-l337.

Ahrens, J.H., and U. Dieter (1991). A convenient sampling method with bounded com-
putation times for Poisson distributions, The Frontiers of Statistical Computation,
Simulation and Modeling. P.R. Nelson, E.J. Dudewicz, A.Oztiirk, and E.C. van der
Meulen, editors American Science Press, Syracuse, NY, l37-149.

Anderson, T.W. (1958). An Introduction to Multivariate Statistical Analysis, John Wiley and
Sons.

Atkinson, A.C. (1979). The computer generation of Poisson random variables, Appl. Statist.
28,29-35.

Atkinson, A.C., and J. Whittaker (1976). A switching algorithm for the generation of beta
random variables with at least one parameter less than one, J. Roy. Statist. Soc., Series
A, 139,462-467.

Best, D.J. (1978). A simple algorithm for the computer generation of random samples from
a Student's t or symmetric beta distribution, Proceedings of the 1978 COMPSTAT
Conference, Leiden, 341-347.

Best, D.J. (1983). A note on gamma variate generators with shape parameter less than unity,
Computing, 30, 185-188.

Box, G.E.P., and M.E. Muller (1958). A note on the generation of random normal deviates,
Ann. Math. Statist, 29, 610--611.
412 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

Broder, A.Z. (1989). Generating random spanning trees, Thirtieth Annual Symposium on
Foundations of Computer Science, 442-447.
Broder, A.Z., and A.R. Karlin (1989). Bounds on the cover time, J. Theoretical Probability, 2,
101-120.

Chen, H-C., and Y. Asau (1974). On generating random variates from an empirical
distribution, AIlE Trans., No.2, 6,163-166.

Cheng, R.C.H. (1977). The generation of Gamma variables with nonintegral shape
parameter, Appl. Stat., 26, 71-75.

Cheng, R.C.H. (1978). Generating beta variates with nonintegral shape parameters, Comm.
ACM, 21, 317-322.
Cheng, R.C.H., and G.M. Feast (1979). Some simple Gamma variate generators,Appl. Statist.,
28, 290-295.

Cheng, R.c.H., and G.M. Feast (1980). Gamma variate generators with increased shape
parameter range, Comm. ACM, 23, 389-394.

Devroye,1. (1981). The computer generation of Poisson random variables, Computing, 26,
197-207.

Devroye,1. (1986). Non-Uniform Random Variate Generation, Springer-Verlag, New York.

Devroye,1. and A. Naderisamani (1980). A binomial variate generator, Tech. Rep., School
of Computer Science, McGill University, Montreal.

Dieter, U. (1982). An alternate proof for the representation of discrete distributions by

equip rob able mixtures, J. Appl. Prob., 19, 869-872.

Feller, W. (1971). An Introduction to Probability Theory and Its Applications, Volume II,
second ed., Wiley, New York.

Fisher, R.A., and E.A. Cornish (1960). The percentile points of distributions having known
cumulants, Technometrics, 2, 209-225.

Fishman, G.S. (1976). Sampling from the gamma distribution on a computer, Comm. ACM,
19,407-409.

Fishman, G.S. (1978). Principles of Discrete Event Simulation, Wiley, New York.

Fishman, G.S. (1979). Sampling from the binomial distribution on a computer, J. Amer.
Statist. Assoc., 74,418-423.
Fishman, G.S. (1996). Monte Carlo: concepts, Algorithms and Applications, Springer-Verlag,
New York.

Fishman, G.S., and 1.R. Moore (1984). Sampling from a discrete distribution while
preserving monotonicity, Amer. Statist., 38, 219-223.
 8.25. REF ERE NeE 5 413

Fishman, G.S., and L.R. Moore (1986). An exhaustive analysis of multiplicative congruential
random number generators with modulus 231 -1, SIAM J. Sci. Stat. Comput., 7, 24-45.

Fishman, G.S., and L.S. Yarberry (1993). Generating a sample from a k-cell table with
changing probabilities in O(log2 k) time, ACM Trans. Math. Software, 19,257-261.

Gerontidis, I., and R.L. Smith ( 1982). Monte Carlo generation of order statistics from general
distributions, Appl. Statist, 31, 238-243.

Hastings, c., Jr. (1955). Approximations for Digital Computers, Princeton University Press,
Princeton, NJ.

Hoeffding, W. (1940). Masstabinvariante Korrelationstheorie, Schriften des Mathematischen

Instituts und des Instituts fur Angewandte Mathematik der Universitat Berlin, 5, 197-233.

Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables, J.

Amer. Statist. Assoc., 58,13-29.

Hormann, W., and G. Derflinger (1993). A portable random number generator well suited
for the rejection method, ACM Transactions on Mathematical Software, 19,489-495.

Johnson, N.L., S. Kotz, and N. Balakrishnan (1994). Continuous Univariate Distributions,

Volume II, second edition, Wiley, New York.

Kachitvichyanukul, v., and B. Schmeiser (1985). Computer generation of hypergeometric

random variates, J. Statist. Compo Simul., 22, 127-145.

Kachitvichyanukul, V., and B. Schmeiser (1988). Binomial random variate generation,

Comm. ACM, 31, 216-222.
Kinderman, A.J., and J.E Monahan (1977). Computer generation of random variables using
the ratio of uniform deviates, ACM Trans. Math. Software, 3, 257-260.

Kinderman, A.J., and J.E Monahan (1980). New methods for generating Student's t and
Gamma variables, Computing, 25,369-377.

Knuth, D. (1973). The Art ofComputer Programming: Sorting and Searching, Addison-Wesley,
Reading, MA.

Kronmal, Richard A., and A.v. Peterson, Jr. (1979). On the alias method for generating
random variables from a discrete distribution, TheAmer. Statist., 4, 214-218.

Kulkarni, V. G. (1990). Generating random combinatorial objects, J. Algorithms, 11, 185-207.

Lurie, D., and H.O. Hartley (1972). Machine-generation of order statistics for Monte Carlo
computations, TheAmer. Statist., 26, 26-27.

MacLaren, M.D., G. Marsaglia, and T.A. Bray (1964). A fast procedure for generating
exponential random variables, Comm. ACM, 7, 298-300.
414 CHAPTER 8. SAMPLING FROM PROBABILITY DISTRIBUTIONS

Marsaglia, G. (1984). The exact-approximation method for generating random variables in

a computer, J. Amer. Statist., 79, 218-22l.

Marsaglia, G., K. Ananthanarayanan, and N.J. Paul (1976). Improvements on fast methods
for generating normal random variables, Inf Proc. Letters, 5, 27-30.

Marsaglia, G., M.D. MacLaren, and T.A. Bray (1964). A fast procedure for generating normal
random variables, Comm. ACM, 7, 4-10.

Marsaglis, G., A. Zaman, and J. Marsaglia (1994) Rapid evaluation of the inverse of the
normal distribution function, Statistics and Probability Letters, 19, 259-266.

Minh, D.L. (1988). Generating gamma variates, ACM Trans. on Math. Software, 14, 261-266.
Moro, B. (1995). The full Monte, Risk, 8, 57-58.

Ross, S. and Z. Schechner (1986). Simulation uses of the exponential distribution, Stochastic
Programming, M. Lucertini, ed., in Lecture Notes in Control and Information Sciences,
Springer-Verlag, New York, 76, 41-52.

Schmeiser, B.W., and A.J.G. Babu (1980). Beta variate generation via exponential majorizing
functions, Oper. Res., 28, 917-926. Errata: Oper. Res., 31,1983,802.

Schmeiser, B.W., and R. Lal (1980). Squeeze methods for generating gamma variates, J. Amer.
Statist. Assoc. 75, 679-682.
Schmeiser, B.W., and V. Kachitvichyanukul (1981). Poisson random variate generation, Res.
Memo. 8184, School ofIndustrial Engineering, Purdue University.
Schreider, Y.A. (1964). Method of Statistical Testing, Elsevier, Amsterdam.
Schucany, W.R. (1972). Order statistics in simulation, J. Statist. Comput. and Simul., 1,
281-286.

Stadlober, E. (1982). Generating Student's t variates by a modified rejection method, Proba-

bility and Statistical Inference, W. Grossmann, G.Ch. Plug, and W. Wertz, eds., Reidel,
Dordrecht, Holland, 349-360.

Stadlober, E. (1989). Sampling from Poisson, binomial and hypergeometric distribu-

tions: ratio of uniforms as a simple and fast alternative, Math. Statist. Sektion 303,
Forschungsgesellschaft Joanneum, Graz, Austria.
Stadlober, E. (1991). Binomial random variate generation: a method based on ratio of un i-
forms, The Frontiers of Statistical Computation, Simulation, and Modeling, volume 1;
P.R. Nelson, E.J. Dudewicz, A. Oztiirk, E.c. van der Meulen, editors, American Sciences
Press, Syracuse, NY, 93-112.

von Neumann, J. (1951). Various techniques used in connection with random digits,
Monte Carlo Method, Applied Mathematics Series 12, National Bureau of Standards,
Washington, D.C.
 8.25. REFERENCES 415

Walker, A.J. (1974). New fast method for generating discrete random numbers with arbitrary
frequency distributions, Electronic Letters, 10, 127-128.
Walker, A.J. (1977). An efficient method for generating discrete random variables with
general distributions, ACM Trans. on Math. Software, 3, 253-256.

Wilks, S.S. (1962). Mathematical Statistics, Wiley, New York.

Wong, e.K. and M.e. Easton (1980). An efficient method for weighted sampling without
replacement, SIAM J. of Comput., 9,111-113.
Zechner, H., and E. Stadlober (1993). Generating Beta variates via patchwork rejection,
Computing, 50,1-18.
 CHAPTER

G. S. Fishman, Discrete-Event Simulation

© Springer Science+Business Media New York 2001
Pseudorandom
Number
Generation

Chapter 8 reveals that every algorithm that generates a sequence of i.i.d. random samples
from a probability distribution as output requires a sequence of i.i.d. random samples from
U(D, 1) as input. To meet this need, every discrete-event simulation programming language
provides a pseudorandom number generator that produces a sequence of nonnegative integers
ZI, Z2, ... with integer upper bound M > Zi Vi and then uses Ul> U2, ... , where Ui :=
Z;/ M, as an approximation to an i.i.d. sequence from U(D, 1).
Since Ui E M ~ Mo := {D, -h, ii, ... , MM1}, this generating procedure incurs a
discretization error that depends on how large M is and on how closely M resembles Mo.
Whereas an i.i.d. sequence from U(D, 1) is continuous, U 1 , U2 , ••• is a discrete sequence.
Moreover, the selected generating procedure determines how well the distribution of any
subset of U1 , U2, ... , namely Ujp Uh , ... , Ujk for arbitrary positive integers k, j, ... , A,
approximates the distribution of a k-tuple of i.i.d. uniform random variables on [D, 1). Most
importantly, there is the issue of how these discretization and distributional errors in the
generation of U1 , U2 , ••• propagate themselves in the sequence generated by anyone of the
algorithms in Chapter 8.
All generators described in this chapter exploit the concept of congruence. We say that
a is congruent to b modulo M if M divides a-b. In symbols, one writes a == b (mod M),
418 CHAPTER 9. PSEUDORANDOM NUMBER GENERATION

where M is called the modulus and == is called the congruence symbol. As examples, 7 == 1
(mod 6), 11 == 2 (mod 3), 338 == 8 (mod 11), and -5 == 1 (mod 6). Also, X = (B + C)
(mod M) denotes that X equals the quantity B + C reduced modulo M.

9.1 LINEAR CONGRUENTIAL GENERATORS

To generate Z 1, Zz, . .. virtually all discrete-event simulation programming languages
employ multiplicative congruential pseudorandom number generators of the form

Zi = AZi- 1 (mod M), i ~ 1, (9.1)

where Zo denotes the seed, A is a positive integer, called the multiplier, and M is the modulus.
Expression (9.1) is algebraically equivalent to

Zi = AZi- 1 - MK i , i ~ 1, (9.2)

where

Ki := LAZi-J/MJ.

To generate numbers from expression (9.1), one needs to prime the generator with a
seed. For example, to generate Zl, Zo acts as the seed; to generate Zz, Zl acts as the seed; to
generate Zn, Zn_l acts as the seed. The simplicity of priming this generator represents one
of its attractive features.
Reproducibility is another important feature. In designing a discrete-event simulation
experiment, it is not unusual to conduct a preliminary run to identify errors in program
logic and previously unexpected sample-path behavior. Such an observation often prompts
one to amend the program. By recording the seed for the first run, one can make a new run
with the modified program but using the same seed as input. Provided that the modified
program leaves the logic for assigning pseudorandom numbers to each source of variation
untouched, the sample-path behavior of the new run reflects only the code change and
contains no new variation due to sample generation. Therefore, the ability to reproduce
the exact same sampling variation on successive runs is a valuable tool in judging sample-
path changes that result from modified programs. All pseudorandom number generators
described in this chapter offer this feature of reproducibility.
Expression (9.1) makes it clear that the number of distinct values the Zi assume cannot
exceed M -1. Just how many distinct values are possible depends on the relationship between
 9.1. LINEAR (ONGRUENTIAL GENERATORS 419

A and M. To see how this occurs, first note that

ZI = AZo - MK I ,
Z2 = A2Zo - MK2 - MKIA,
Z3 = A 3Z 0 - MK3 - MK2A - MKIA2, (9.3)

Since 1 ::::: Zi < M for 1 ::::: i ::::: n, the quantity Kn + Kn-1A + ... + KIA n- 1 must
be the largest integer in An Zo / M, so that expression (9.1) has the alternative representation

Zn = An Zo (mod M). (9.4)

Observe that Zo completely determines the {Zi' i :::: I} integer sequence once A and
M are fixed. Therefore, our ability to make U I , U2 , ••• resemble a continuous-valued i.i.d.
sequence with elements fromU(O, 1) can, at best, depend on the choice of A and M.

9.1.1 PERIOD

Perhaps the most obvious property of the generator (9.1) is that once Zn = Zo for
some integer n > 0, then Zn+i = Zi for all i = 1, 2, .... We call

P(Zo, A, M) := min(n :::: 1 : Zn = Zo) (9.5)

the period of the generator for given Zo, A, M. Clearly, for given Zo and M we would like to
choose the multiplier A to maximize the period. In particular, we would like P (Zo, A, M) =
M - 1, which would make the period identical for all seeds Zo E Mo, and thus independent
of the starting seed. Recall from expressions (9.4) and (9.5) that Zn =
Zo implies

(An - I)Zo (mod M) = O. (9.6)

If n =M - 1 is the smallest integer satisfying the congruence (9.6), then it must hold
for all Zo E Mo, so that

(A M - I - I) (mod M) = O. (9.7)
420 CHAPTER 9. PSEUDORANDOM NUMBER GENERATION

Table9.1 Sequences for Zi = AZi - 1 (mod 11), i > 1; Zo =1

Zi
i \ A 2 3 4 5 6 7 8 9 10
0 1 1 1
2 3 4 5 6 7 8 9 10
2 4 9 5 3 3 5 9 4
3 8 5 9 4 7 2 6 3
4 5 4 3 9 9 3 4 5
5 10 10 10 10
6 9 5 4 3
7 7 8 6 2
8 3 4 9 5
9 6 2 8 7

9.1.2 PRIME MODULUS

If M is prime, there exist rjJ(M - 1) integers A E Mo for which n =M - 1 is the

smallest integer satisfying the congruence (9.7), where

(9.8)

and the product is taken over all distinct primes p dividing n. An A that satisfies expression
(9.7) for M prime is called a primitive root of M. Expression (9.8) defines the Euler totient
function.
As illustration, the largest integer that a 32-bit register in a computer can hold is
M = 231 - 1, which happens to be prime. Since M - 1 has the prime factorization M - 1 =
231 - 2 = 2 X 32 X 7 x 11 x 31 x 151 x 331, these exist rjJ(M - 1) = 534,600,000 primitive
roots of2 31 -1, each one of which guarantees a full period P = M - 1 for the multiplicative
congruential generator (9.1).
To illustrate the relationship between A and the period, Table 9.1 lists the sequences
that result for the prime-modulus generator Zi = AZi - 1 (mod 11). The table reveals
rjJ(11 - 1) = = =
10 x (1 - ~)(1 - ~) 4 primitive roots A 2, 6, 7, and 8. It also reveals that
= =
A 6 generates the same full period as A 2, but in reverse order. Moreover, A 7 and =
=
A 8 have a similar relationship. We next explain these relationships.

9.1.3 MULTIPLICATIVE INVERSE

Consider two generators

Xi = AXi - 1 (mod M)
 9.1. LIN EAR CON G RUE NT I A L G ENE RAT 0 R5 421

and

Yi - I = BYi (mod M)

with multipliers A and B and modulus M. We want to establish the condition on A and B
that guarantees Xi = Yi given X i - I = Yi - I for all i ::: 1. Since

Xi = AXi _ 1 - KM,
Yi - I = BYi - K'M,
for K := LA Xi-I! MJ and K' := LB Y;/ MJ, X i - I = Yi - I is equivalentto

Xi = A(BYi - K'M) - KM,

or, equivalently,

(ABYi - Xi) = (K + AK')M == 0 (mod M).

Therefore, Xi = Yi given X i- I = Yi - I if and only if

(AB - l)Xi == 0 (mod M),

or, equivalently, for prime M,

AB -1 == 0 (modM). (9.9)

If B satisfies this congruence for given M and A, we say that B is the multiplicative inverse
ofA modulo M.

9.1.4 FINDING PRIMITIVE ROOTS

If A is a primitive root of M, then B := A K (mod M) is also a primitive root of M if and

only if the greatest common divisor of K and M -1 is unity. Notationally, gcd( K, M -1) = 1.
This property implies that once a primitive root is known, others can be computed
by exponentiation and modulo reduction subject to the restriction on the exponent. As
illustration, 2 is the smallest (base) primitive root of 11, so that 2 3 (mod 11) = 8, 27 (mod 11)
= 7, and 29 (mod 11) = 6. For M = 231 - 1,7 is the base primitive root, so that any B =
7K (mod2 31 -1) isa primitive root, provided that K has no factors 2, 3, 7, 31,151, and331. For
example, 75 (mod 231 - 1) = 16807 is the multiplier found in the multiplicative congruential
generators in SLAM and SIMAN. Alternatively, GPSS/H uses B = 7K (mod 231 - 1) =
742938295, where K =981902737 can easily be shown to have no factors 2, 3, 7, 11,31,151,
and 331. SIMSCRIPTII.5uses B = 7 K (mod2 31 -l) = 630360016, where K = 1177652351
422 CHAPTER 9, PSEUDORANDOM NUMBER GENERATION

contains none of these factors. Section 9.5 rates these alternative multipliers with regard to
their randomness properties.

9.2 MODULUS 2,8

Multiplicative congruential generators with M = 2P for ,8 :::: 3 are also in common use,
although less so than prime modulus generators. The appeal of "powers-of-two" generators
stems from the computational efficiency they allow.
All modern-day computers perform computations in base 2 using either integer or
floating-point arithmetic. Integer arithmetic takes less computing time, and because of the
relatively slow speeds that characterized computers in the earlier days a natural preference
arose for using integer rather than floating-point arithmetic, whenever possible.
To evaluate Zi in expression (9.1) takes multiplication (AZi-d, division and trun-
cation (Ki = LAZi-dMJ), multiplication (MKi)' and subtraction (AZ i - 1 - MKi). Of
these, multiplication and division are the more time-consuming operations. By choosing
a modulus M = 2P, a pseudorandom-number-generating program can be written in the
machine language of the chosen computer that would use integer arithmetic and replace all
but the first multiplication (AZi-d by bit inspection and replacement.
As illustration, suppose that AZi - 1 has the binary representation BkBk-I. ... , Bl for
some positive integer k, where

Bj E {O, I}, j = I, . .. ,k,

Bk = I,

so that

k
AZi - l = '~
" 2j - 1 B j ,.
j=1

Then

if k ::: ,8,

2 j - 1 B j if k > ,8,

revealing that division, truncation, multiplication, and subtraction incurred in computing

min(k,p)
Zi = AZi - 1 (mod 2P) = L 2j - 1B j
j=1
 MOD U L U 5 2f3 423

can be replaced by the less costly steps of determining k by bit inspection and then replacing
bits minCk, fJ) + 1, ... ,k by zeros.
The desire to eliminate the multiplication AZ;_1 as well has led to a recommendation
for the form of A. However, Section 9.53 shows that this form introduces a disastrous
departure from randomness, especially in three dimensions.
Although integer arithmetic is faster, it imposes restrictions on word size that impose
other design constraints on pseudorandom number generation (png). Moreover, the speeds
of current-day computers are so much greater than in the past as to make the penalty
incurred in using floating-point arithmetic considerably less of a contentious issue. As a
consequence, many png programs rely on extended-precision floating-point arithmetic to
perform all computations.
Without its computational edge, one might have expected M = 2f3, as a multiplier,
to give way to M prime. Since this has not happened in total, we need to familiarize the
simulationists with the properties of powers-of-two moduli.
What does M = 2f3 imply about the sequences that the generator (9.1) produces?
If the seed Zi-1 is an even integer, then, for any positive integer A, AZi - 1 is even and
so is Z; = AZ;_1(mod2f3 ) = AZ;_1 - K;2 f3 , where K; = LAZi -d2 f3 J. Suppose that
Z;-1 = Y;-1 x 2Y for some odd integer Yi - 1 and integer y < fJ. Then

where

Yi := AYi - 1 (mod 2f3 - Y ).

Therefore, an even seed Zi-1 = Yi - 1 X 2Y induces a sequence {Zj, j 2: i} of even integers

with period no larger than 2f3 - Y - l.
If A and Zj -1 are odd integers, then {Z j = A Z j -1 (mod 2(3 ), j 2: l} is a sequence of
odd integers with period no greater than 2f3 - 1 • However, the upper bound is not achievable.
Fbr M = 2f3, there exists no multiplier A that admits a full period P(A, 2f3 , Zo) = 2f3 - l.
Indeed, the maximal achievable period is 2f3- 2 , which occurs for Zo odd and

A == 3 (mod 8)

and

A==5 (mod 8).

Equivalently, A E {3+8j; j = 0, 1, .. .,2 f3 - 3 } U {5+8j; j = 0,1, ... , 2f3 - 3 }. For M = 232 ,

this implies a period of 230 = 1,073,741,824 odd numbers on each of two disjoint cycles.
Moreover, this collection of values that A can assume implies that there are 230 maximal
period generators for M = 232 •
424 C HAP TE R 9. P5 E U DO RAN DO M N U M B ERG ENE RAT ION

Table 9.2 Zi = AZi - 1(mod 64)"

A=13 A=l1
i Zi Z(i) Zi Z(i) Zi Z(i) Zi Z(i)
0 1 1 3 3 1 1 7 5
1 13 5 39 7 11 3 13 7
2 41 9 59 11 57 9 15 13
3 21 13 63 15 51 11 37 15
4 17 17 51 19 49 17 23 21
5 29 21 23 23 27 19 61 23
6 57 25 43 27 41 25 31 29
7 37 29 47 31 3 27 21 31
8 33 33 35 35 33 33 39 37
9 45 37 7 39 43 35 45 39
10 9 41 27 43 25 41 47 45
11 53 45 31 47 19 43 5 47
12 49 49 19 51 17 49 55 53
13 61 53 55 55 59 51 29 55
14 25 57 11 59 9 57 63 61
15 5 61 15 63 35 59 53 63
16 1 3 1 7
"{Z(i); I = 0, 1, ... , 16} IS the ordered sequence of {Zi; I = 0, 1, ... , 16}.

Table 9.2 illustrates the sequences that these generators can produce. Here A = 13 == 5
(mod 8), whereas A = 11 == 3 (mod 8). Most notably, each A leads to two disjoint odd
sequences, depending on the starting Zoo Also, Z(;) - Z(;-I) = 4 for A = 13, whereas
Z(i) - Z(;_I) E {2, 6} for A = 11, implying that the latter sequences have less uniformity
than the former. More generally, maximal-period multiplicative congruential generators
with M = 2fi and f3 ::: 3 have the following properties (Jannson 1966):

Property Ml. For A == 5 (mod 8) and Zo == 1 (mod 4), {Z;, 1 < < P} is a
permutation of the integers 4j + 1; j = 0, 1, ... , 2fi - 2 - 1.

Property M2. For A == 5 (mod 8) and Zo == 3 (mod 4), {Z;, 1 < < P} is a
permutation of the integers 4j + 3; j = 0,1, ... , 2fi - 2 - 1.

Property M3. For A == 3 (mod 8) and Zo == 1 or 3 (mod 4), {Z;, 1 ::::: i ::::: P} is a
permutation of the integers 8j + 1 and 8j + 3; j = 0,1, ... , 2fi - 3 - 1.

PropertyM4. For A == 3 (mod 8) and Zo == 5 or 7 (mod 4), {Z;, 1 ::::: i ::::: P} is a

permutation of the integers 8j + 5 and 8j + 7; j = 0,1, ... , 2fi +3 - 1.

Here P := peA, M, Zo). Because of the inherent nonuniformity that A == 3 (mod 8)

induces, we hereafter focus exclusively on A == 5 (mod 8) for M = 2fi.
 MOD U L U S 2fi 425

9.2.1 MIXED CONGRUENTIAt GENERATORS

Consider the mixed generator

Zi = (AZi - 1 + C) (mod M), i :::: 1, (9.10)

which has the additional parameter C. For M = 2fi, C an odd integer, A == 1 ( mod 4), and
gcd(M, C) = 1, this generator has full period M consisting of all integers
{O}U Mo = {O, 1, ... , M - I}. Note that:

i. Zi assumes all values in {O, 1, ... , M - I} once per cycle.

ii. Any odd C satisfies gcd(M, C) = 1.

iii. Bit shifting suffices to implement modulo reduction.

In the early days of discrete-event simulation, the computational convenience of bit

shifting while preserving the full period P = M gave this generator considerable appeal.
With substantially enhanced speeds, computational convenience has become less of an issue.
While the full-period linear congruential generator (9.10) with M = 2fi may appear
to be different from the multiplicative congruential generator (9.1), elementary analysis
reveals the former merely to be an affine transformation of the latter (e.g., Overstreet 1974).
Assume {Zi = AZi - 1 (mod 2fi ), i :::: I} with A == 5 (mod 8) and Zo == 1 (mod 4), and let
Yi := LZi/4J and A' := LA/4J. Since Zi E {4v + 1; v = 0, 1, ... , 2fi - 2 }, Zi = 4Yi + 1.
Since A = 1 + 4(2K + 1) for some integer K, A = 4A' + 1. Therefore,

Zi = AZi - 1 (mod 2fi ), (9.11)

4Y; +1= A(4Yi - + 1)

1 (mod 2fi)

= 4AYi - 1 + 4A' + 1 (mod 2fi ),

so that

Yi = AYi - 1 + A'

where Yi E {O, 1, ... ,2fi - 2 }, thus proving our assertion. Since A == 5 (mod 8), A =
1 + 4(1 + 2K) for some integer K. Therefore, A == 1 (mod 4) and A' = Ll/4 + 1 + 2KJ
is odd, so that {Yi } has "full period" 2fi - 2 • Analogous results obtain for A == 5 (mod 8) and
Zo == 3 (mod 4) and for A == 3 (mod 8) and Zo == 1 (mod 2).
426 C HAP TE R 9. P5 E U DO RAN DO M N U M B ERG ENE RAT ION

In summary, we see that an analysis of the randomness properties of maximal-period

generators (9.1) with modulus 2P+2 suffices to characterize the randomness properties of
full-period generators (9.10) with modulus 2fJ.

9.3 IMPLEMENTATION AND PORTABILITY

As with any proposed numerical technique, implementing a pseudorandom number gen-
erator on a computer calls for considerable care. Moreover, an implementation appropriate
for one computer may produce a different (incorrect) sequence for the same seed when
employed on another computer with different internal architecture. Paying attention to cer-
tain features of a generator before attempting implementation can significantly reduce the
potential for such error.
All binary computers with a word size of a + 1 bits can perform integer arithmetic
and store the exact result, provided that it does not exceed 2(1 - 1 in absolute value. This
assumes that the leftmost bit contains the sign of the number. Likewise, an a-bit mantissa
in floating-point words ensures this exact storage in performing floating-point arithmetic.
In what follows we assume that 2(1 - 1 is the largest storable integer.
Consider an implementation of{Zi = AZi _1 (mod M), i 2: I} that executes the steps

(9.13)
K +-- LSjMJ,
Zi+--S-KM.

Computing S exactly and directly requires that

A(M - 1) ::'S 2(1 - 1

for a prime modulus generator (with period P = M - 1 < 2(1 - 1), and requires A x
(2 fJ -1) ::'S 2(1 - 1 (Property M2) for a multiplicative congruential generator with modulus
M = 2fJ and A == 5 (mod 8).
To illustrate the limitation that these bounds impose, consider the primitive root A
= 950706376 for M = 231 - 1, which leads to 260 < A(M - 1) < 261. Therefore, directly
performing integer arithmetic to compute AZi _ 1 would inevitably lead to an overflow error
for some i on 32-bit computers. Since floating-point representations on these computers
have 24-bit mantissas in single precision and at most 56-bit mantissas in double precision,
directly performing floating-point arithmetic on AZi - 1 also leads to an error for some i.
At least two solutions to this problem exist. The first uses extended-precision arith-
metic software and by specifying an adequate level of precision, eliminates the overflow
problem. However, this type of software is inevitably machine-dependent, thereby limiting
the portability of any pseudorandom number generation program that relies on it. A portable
 9 .4 . How RAN DO MAR E P5 E U DO RAN DO M N U M B E R 5 ? 427

computer program produces identical output on different computing platforms when given
identical input. In the context of discrete-event simulation experiments, portability is a
highly desirable feature.
The second alternative partitions arithmetic operations in a way that eliminates the
overflow problem on a wide range of computing platforms and therefore ensures portability.
For example, let Al := A (mod 216 ), A z := A - AI, Xi = AIZi _ 1 (mod 231 ), and Yi
AZZi - 1 (mod 231 ). Then expression (9.1) has the equivalent form

AIZi _ 1 .::: 247 - 1, and AZZi - 1 .::: 26Z - l. However, the least significant 16 bits of AZZi - 1
are identically zero. Therefore, this partition of AZi - 1 allows one to compute Zi with no
overflow, provided that a 47-bit mantissa is available.
Hormann and Derflinger (1993) give algorithms for effecting this portability for M =
231 - 1 and A < 230 . Implementations of these algorithms are demonstratably faster than
that in Fishman (1996, Section 7.5).
For M = 231 - 1, a third alternative for ensuring portability across platforms restricts
the choice of the multiplier A to a subset of the primitive roots of M that satisfy specified
conditions (Schrage 1979, Park and Miller 1988, and Fishman 1996). Since a major objective
is to choose an A that makes VI, V z , ... resemble an i.i.d. uniform sequence as closely as
possible, these additional restrictions can in no way improve the resemblance and are likely
to reduce it. Therefore, we do not encourage this approach to portability.

9.4 HOW RANDOM ARE PSEUDORANDOM NUMBERS?

How well does VI, V z, ... resemble a sequence of i.i.d. random variables from U(O, l)? We
have already indicated that prime M and A a primitive root of M establish one-dimensional
uniformity over the integers {l, ... , M -1}, whereasM:= 2f3 (f3 > 3) and A = 5 (mod 8)
do likewise over the integers {3 + 8j; j = 0, 1, ... , 2f3 - 3 } for Zo = 3 and {5 + 8j; j =
0, 1, ... , 2f3 - 3 } for Zo = 5. But what about the independence property? Do large values of
Vi+ 1 follow large (small) values of Vi, indicating dependence; or is Vi+1 independent of the
value of Vi?
Example 8.1 illustrates why independence within 2-tuples is necessary. If Vi and Vi+ 1
are independent and each from U(O, 1) and Vi .::: e-(Ui + 1-i)2/ z , then -In V i+1 has the half
normal p.d.f. (8.12a). If Vi and Vi+ 1 are not independent then -In Vi+ 1 does not have the
desired p.d.f.
Because of the deterministic way in which the generator (9.1) produces Zi' some
dependence between Zi and Zi+1 is inevitable. Presumably, we want to choose A for a given
M in a way that keeps this dependence to an acceptable minimum. But what about the joint
dependence of Zi, Zi+l, Zi+2 or, more generally, of Zi, ... , Zi+k-I? Regrettably, analysis
428 C HAP TE R 9. P5 E U D 0 RAN DO M N U M B ERG ENE RAT ION

(a) A = 2
(b) A=8

6 7 9 10 11
2 3 4 5 6 7 8 9 10 11
Zi

Figure 9.1 2-tuples for Zi = AZi - 1 (mod 11)

shows that no A performs equally well in all dimensions 1 through k for arbitrary k with
respect to given error norms. However, for a given M we can identify those A whose worst
errors across the k dimensions are smallest.
If Vi and V i + 1 are independent from U(O, 1), then the sequence {(Vi, Vi+d; i =
1,2, ... } would have its elements (Vi, Vi+d uniformly distributed in the unit square [0, IF.
Since the generator (9.1) induces discrete (Vi, Vi+d, the closest that we can come to this
idealization is to have 2-tuples uniformly distributed on the grid {x j := it;
j = 1, ... , M -1
and Yj = it; j = 1, ... , M - I}.
Figure 9.1 gives us a preview of what we can expect. It shows overlapping 2-tuples for
the generator in Table 9.1 with A = 2 and 8. Observe that ten 2-tuples appear in each figure,
whereas one expects one hundred 2-tuples, one at each vertex of the 2-dimensionallO x 10
grid. Also, the points for A =2 appear less random than those for A = 8. A second illustration
makes the issue of nonuniformity more salient.
Recall that the most commonly employed multipliers for M = 231 -1 in discrete-event
simulation are 16807 (e.g., SIMAN), 630360016 (e.g., SIMSCRIPT n.5), and 742938285 (e.g.,
GPSS/H). To reveal the discrete patterns they induce, Figure 9.2· shows their overlapping 2-
tuples that fall in the square [0, .001F. Since any square of width and length .001 in the unit
square exhibits approximately the same appearance, [0, .001 F contains about (.001)2 X M ~
2147 2-tuples. Figure 9.2 reveals that for A = 16807 these 2-tuples are packed together tightly
on 17 parallel hyperplanes. By contrast, Figure 9.2b shows thatthe 2-tuples for A = 630360016
are considerably more uniformly distributed on 29 parallel lines, whereas Figure 9.2c shows
50 parallel lines for A = 742938285.
These numbers arise from counting the number of parallel lines within the set of
parallel lines that have maximal distance between successive lines. For (0, I?, these numbers
 9.4. How RANDOM ARE PSEUDORANDOM NUMBERS? 429

(a)'" - 16807 (b) II = 630360016

0.001 - , -----

0.0009

0.0008

0.0001

0.0006

U,., O.ooos
0.0004

0.0003

0.0002

0.0001

O ~~~-L~~-L~-L~~~

o 0,0002 0.0004 0.0006 0.0008 0.001 o 0,0002 0.0004 0.0006 0.0008 0.001

U. U,

(e) II = 742938285

u,.,

0.0002 0]1104 0.0006 0.0008 0.1101

U,

Figure 9.2 2-tuples for three pseudorandom number generators:

Zj = AZ j _ 1 (mod 231 - 1)

suggest that 2-tuples for A = 16807 are distributed on about 17,000 parallel lines in the unit
square, 2-tuples for A = 630360016 are distributed on about 29,000 parallellines, and 2-tuples
for A = 74938285 are distributed on about 50,000 parallel lines.
Since the number of these lines increases with the distance between successive parallel
lines and since for a given multiplier the distribution of 2-tuples becomes less uniform as
the distance between parallel lines increases because of the increasing area void of 2-tuples,
it follows that A = 16807 offers less-uniform 2-tuples than A = 630360016, which offers
less-uniform 2-tuples than A = 742938285.
These observations for 2-tuples extend more generally to k-tuples. In particular, all
prime modulus generators oftheform induce M -1 k-tuples {Zj = (Zj, ... , Zi +k-I); 1 ::::
i :::: M - I} on the vertices of the k-dimensional grid 9 = {z = (ZI, .. . , zd : Zj = j, 1 ::::
430 CHAPTER 9. PSEUDORANDOM NUMBER GENERATION

j ::::: M - 1, 1 ::::: i ::::: k}, thereby leaving (M - 2)[(M - 1)k - 1] vertices unoccupied.
Hence, the density of points diminishes with increasing dimension k. Since all vertices are
not occupied, the second observation regarding uniformity becomes increasingly important
as k increases. That is, some arrangments of the M - 1 k-tuples appear closer to uniformity
than others. Since uniformity is a principal consideration, one prefers the primitive root A,
for a specified prime modulus M, whose k-tuples come closest to uniformity as measured
by an objective criterion for each dimension k = 1,2, .... Theorem 9.1 provides a basis for
making this choice.

Theorem 9.1.
For each k ::: 1, letq := (qo, ql> ... ,qk-d denote points in the set

Qk(M) := {-M, -M + 1, ... , -1,0,1, ... , M - l}k \ {O, ... , O}

'-..-'
k times

and let

k-l
Zi(q, k) := L qsZi+s, (9.14)
s=o

where

Zi = AZi-l (mod M),

M is prime, and A is a primitive root of M. Consider the subset ofq vectors

(9.15)

i. For each q E Qk(A, M) the k-tuples {Uik := (Ui> Ui+l> ... , Ui+k-d;
i = 1, ... , M - I}, where Ui := ZdM, lie on parallel hyperplanes in (0, 1)k.

ii. The total number of these parallel hyperplanes does not exceed L!:~ I qs I -1.
-[/2

iii. The distance between two parallel hyperplanes is proportional to (L!:~ q; )

PROOF OF PART I. Since Zi+s = (A' Zi) (mod M),

k-l )
Zi(q, k) (mod M) = ( Zi ~qsA' (modM)
 9.4. How RANDOM ARE PSEUDORANDOM NUMBERS? 431

for each q E Qk(M). Therefore, for each q E Qk(A, M)

Z;(q, k) (mod M) = 0,
or, equivalently,

Z;(q, k) == 0 (mod M).

This implies that for i = 1, ... , M - 1 there exist integers c;(q, k) E {-M, -M +
1, ... , -1,0,1, ... , M - I} such that

Z;(q, k) = c;(q, k)M,

or, equivalently,

k-I
LqsU;+s =c;(q,k). (9.16)
s=o

Therefore,allk-tuples{U;; i = 1,···, M-l}lieonhyperplanes{qouo+ .. . +qk-IUk-1 = c:

o< U j < 1; j = 0, ... , k -I} indexed by integers c E CkCq) := U~~I{C; (q, k)}. Moreover,
since all hyperplanes have the same coefficients qo, ql, ... , qk-l> they are parallel, and this
parallelism is preserved when equation (9.15) is reduced modulo 1 onto the k-dimensional
unit hypercube. 0

PROOF OF PART II. Since 1 ::s Zj < M,

k-I
Z;(q,k) < M Lmax(O,qs),
s=o

so that

k-I
c;(q,k)::s: Lmax(O,qs).
s=o

Conversely,

k-I
Z;(q, k) > M L mineO, qs),
s=o

so that

k-I
c;(q, k)::: L mineO, qs) + 1.
s=o
432 CHAPTER 9. PSEUDORANDOM NUMBER GENERATION

Let Cmin and Cmax denote the largest and smallest integers in Ck(q). Since I Ck(q) I denotes
the number of parallel hyperplanes,

k-l k-l k-l

I Ck(q) I::: Cmax-Cmin+ l ::: Lmax(O,qs)-I- L min(O,qs)-I+1 = L I qs 1-1.
s=o s=O s=O 0

Let x := (xo, ... , Xk-l) and y := (Yo, ... , Yk-d denote points on the
°
PROOF OF PART III..
parallel hyperplanes {qouo + ... + qk-lUk-l = C : < Uj < I; j = 0"", k - I} and
{qouo + ... + qk-lUk-l = c' : ° < Uj < I; j = 0, ... , k - I}, respectively, for C and
c' E Ck (q). To find the distance between these hyperplanes, we fix x and find the solution to

y:= (Yo, Yl,"" Yk-d,

subject to

and

k-l
LqsYs = c'.
s=O

Let Zs = Xs - Ys' Then the problem is equivalent to

Z := (zo, Zl> ... , zk-d,

where A is a Lagrange multiplier. The solution is

2zs = Aqs, s = 0, ... , k -I,

so that

2(c - c')
A - "k-l 2'
L."s=O qs
 9 .5 . C H 00 SIN GAM 0 N G M U L TIP LIE RS 433

and the resulting distance between hyperplanes is

k-1 )1/2 I e-e I

2::>;
I
(
= k 2 1/2' o
s=o (Ls=o qs)

Corollary 9.2.
For{Zi = AZi- 1 (mod2 P);{3 > 3,i::: I} with A == 5 (mod B), Theorem 9.2 continues
toholdwithL!:~ qsA S == 0 (mod2 P- 2 ).

This implies that all k-tuples for generators with M = 2P and A == 5 (mod 8) lie
on parallel hyperplanes, and thus any methods for discerning departures from uniformity
apply to these as well as to prime modulus generators.

9.5 CHOOSING AMONG MULTIPLIERS

To predict the seriousness of error and thereby to identify perferred multipliers, we need
to understand more about the k-tuples in expression (9.15). Reducing both sides of the
expression modulo 1 leads to

k-1
LqsUi +S == 0 (mod I) (9.17)
s=o

forall2 kM k- 1 -1vectorsqE Qk(A, M).Therefore,foreachq,allk-tuples{Ujk.1 S j S P}

lie on the intersection of k-dimensional parallel hyperplanes and [O,l)k. We now use this
representation to study the randomness properties of k-tuples. As one basis for assessment,
consider the Euclidean distance between adjacent parallel hyperplanes,

(9.18)

where

~k(q):= min fe - e'l. (9.19)

c#c'ECdq)

In particular,

dk(A, M) = qEQk(A,M)
max dk(q, A, M) (9.20)

denotes the maximal distance between two adjacent hyperplanes over all families of parallel
hyperplanes that the multiplier A and modulus M induce. Since one can show that for every
434 C HAP TE R 9. PS E U D 0 RAN D 0 M N U M B ERG ENE RAT ION

q E QdA, M) with ~k(q) > 1 there exists a q E Qk(A, M) with ~k(q) = 1 such that

we hereafter restrict attention to the q with ~k (q) = 1.

If two multipliers Al and A2 have dk(A I , M) > dk (A 2, M), then Al indicates a more
sparse and therefore less uniform arrangement for k-tuples in k-dimensional space than A2
implies. More generally, for multipliers A I, ... , Ar one in principle prefers the Ai for which
dk(A i , M) is minimal. This method of analyzing multipliers is called the spectral test.
As examples, Figures 9.1a and 9.1b give d 2(2, 11) = 1 1(2 2 + 1)1/2 = .4472 and
d 2(8, 11) = 1 1 (3 2 + 1) 1/2 = .3162, respectively, which supports the earlier assertion based
on visual evidence that multiplier A = 8 gives more uniformly distributed 2-tuples than
multiplier A = 2.
A limit exists on how small the maximal distance (9.19) can be; in particular, it is
known that (Cassels 1959, p. 332)

k =2,

r l/6 k = 3,
rl/4 k=4,
[t(M)]llkdk(A, M) :::: Yk:= r 3/10 k = 5, (9.21)

( -643)1 /12 k=6 '

r 3/7 k = 7,
r l/2 k = 8,

where

t(M) .-
.- 12fJM - 2
if M is prime,
if M = 2fJ.

For M = 11, d2 (A, 11) :::: .2806, revealing that d2 (8, 11) is within 13 percent of the best
achievable bound for 2-tuples. For M = 231 - 1,

.2008 X 10-4 k = 2,
.6905 X 10-3 k = 3,
ddA,2 31 - 1):::: .3906 x 10- 2 k = 4, (9.22)
.1105 X 10- 1 k = 5,
.2157 X 10- 1 k = 6,
 9.5. CHOOSING AMONG MULTIPLIERS 435

and for M = 248 ,

.1109 X 10-6 k = 2,
.2158 X 10-4 k = 3,
dk(A, 248 ) 2: .2903 X 10-3 k = 4, (9.23)
.1381 X 10-2 k = 5,
.3814 X 10-2 k = 6,

indicating the relative coarseness of the grids in as few as four dimensions. This coarseness
enhances the importance of choosing an A that makes dk(A, M) as small as possible across
dimensions.
Table 9.3 gives results for high-performing multipliers, with respect to dk(A, M) for
M = 231 - 1, 232 , and 248. For modulus M, the quantity Sk(A, M) denotes the ratio of
the minimal achievable distance Yk/t1/k(M) to the worst-case distance dk(A, M) for the
multiplier A. For M = 231 - 1, the first five entries are the best among the 534,600,000
primitive roots of M. For M = 232 , the entries are best among the 2{3-3 = 229 =536,870,912
possible candidates, and for M = 248 the five entries are the best among 60 million multipliers
tested. For M = 232 , the second entry under each multiplier is its multiplicative inverse
modulo M. For M = 231 - 1, the table also shows results for the two multipliers A =
630360016 in SIMSCRIPT 11.5 and A = 16807 in SIMAN and SLAM. As anticipated from
Figure 9.2, d2 (742938285, 231 - 1) < d2 (630360016, 231 - 1) < d2 (16807, 231 - 1). More
generally, each of the best five dominates 630360016 and 16807 for 2 :::: k :::: 6. Recall
that GPSS/H uses 742938285. Clearly, there is room for improvement in the pseudorandom
number generators in SIMAN and SLAM.

9.5.1 NUMBER OF PARALLEL HYPERPLANES

Other mathematical norms exist for choosing among multipliers, and they generally
lead to the same, or close to the same, ordering among multipliers in each dimension k. See
Fishman (1996, Chapter 7). These alternatives include a bound on the number ofhyperplanes
(Theorem 9.2 ii),

k-l
Nk(A, M):= L I qs 1-1,
s=o

for each q E Qk(A, M).

Although Nk(A, M) tends to decrease as the distance between successive parallel hy-
perplanes dk(A, M) decreases, Nk(A, M) is not as discerning a measure as dk(A, M). First,
it is an upper bound, whereas dk(A, M) is the actual minimal distance. Second, the number
of hyperplanes depends on the orientation of the hyperplanes in [O,I]k, whereas dk(A, M)
is rotation invariant.
436 CHAPTER 9. PSEUDORANDOM NUMBER GENERATION

Table 9.3 Distance between parallel hyperplanes for {Zj = AZ j - 1 (mod M), i 2: I}

Sk(A, M) := Yk I dk(A, M)t 1/ k (M)

Modulus Multiplier Dimension (k)
M A 2 3 4 5 6

231 - 1 742938285 b .8673 .8607 .8627 .8320 .8342

950706376 b .8574 .8985 .8692 .8337 .8274
1226874159b .8411 .8787 .8255 .8378 .8441
62089911 b .8930 .8903 .8575 .8630 .8249
1343714438b .8237 .8324 .8245 .8262 .8255
630360016 .8212 .4317 .7832 .8021 .5700
16807 .3375 .4412 .5752 .7361 .6454a

232 1099087573 c .8920 .8563 .8604 .8420 .8325

4028795517
2396548189 c .8571 .9238 .8316 .8248 .8248
3203713013
2824527309 c .9220 .8235 .8501 .8451 .8332
1732073221
3934873077c .8675 .8287 .8278 .8361 .8212
1749966429
392314069c .9095 .8292 .8536 .8489 .8198
2304580733

248 68909602460261 c .8253 .8579 .8222 .8492 .8230

33952834046453 c .9282 .8476 .8575 .8353 .8215
43272750451645 c .8368 .8262 .8230 .8400 .8213
127107890972165c .8531 .8193 .8216 .8495 .8224
55151000561141 c .9246 .8170 .9240 .8278 .8394

a Entries are Sk(A, M) x 1000.

b Best among all primitive roots of2 31 - 1. Source: Fishman and Moore (1986).
c Best among all multipliers A == 5j (mod 2.8), j 2: 1. Source: Fishman (1990).

To illustrate the problem, Figure 9.3a contains seven parallel lines with incremental
distance d when the lines are oriented parallel to the 45-degree line; but Figure 9.3b shows
only five lines with incremental distance d when the lines are oriented parallel to the y-axis.
While this example places more value on dkCA, M) than on NkCA, M) as an indicator of
uniformity, NkCA, M) nevertheless can provide useful information, as Section 9.5.3 shows.

9.5.2 SHOULD THE SIMULATION 1ST BE CONCERNED ABOUT A THEORETICALLY

POOR MULTIPLIER?

Since Figure 9.2 reveals that A = 16807 induces larger null regions in [0, 1)2 than
A = 360360016 and A = 742938285, its use in practice is more suspect than the others.
However, some say that even with these large null regions A = 16807 continues to be an
acceptable generator "to the level of approximation" one aims for when using simulation
 9 .5 . C H 00 SIN GAM 0 N G M U LTIP LIE RS 437

(b)
(a)

o
o

Figure 9.3 Number of parallel hyperplanes

output statistics to approximate unknown system quantities. Regrettably, both the criticism
and defense of A = 16807 are far too abstract for a reader to assess the strength of each
argument. To move the debate closer to reality, we return to sampling from the half-normal
p.d.f. (8.12a) and corresponding d.f. {2cJ>(x) - 1, 0 ::: x < oo}. In particular, we compare
the discretization errors that each ofthe three multipliers induces for half-normal generation
via the acceptance-rejection method (Section 8.5). To benchmark these errors, we first study
discretization error for the inverse transform method (Section 8.1.3).
Applying the inverse transform method leads to the sequence

Xl' = "'_1(1 +2 U;) ,

'¥ i=1,2, .... (9.24)

Since a prime-modulus generator produces all fractions {j / M; j 1, ... , M - I} per

period,

x(j) := cJ>
_l(l+ 2j /M) ' j = 1, ... , M -1,

defines, for all prime-modulus generators, the ordered sequence of approximating half-
normal values that expression (9.23) produces. Moreover, the discretization errors between
438 C HAP TE R 9. P5 E U DO RAN DO M N U M B ERG ENE RAT ION

successively ordered values have bounds

9.622 X 10-10 if 0 < X(j) ::: 1,

x( ·+1) -
J
x(·)
J
<
-
14.312 x 10- 9 if 1 <
5.254 X 10- 8 if 2 <
x(j) :::

x(j) :::
2,
3,
1.740 x 10-6 if 3 < x(j) ::: 4.

Since 2<1>(4) - 1 = 1 - .6334 X 10-4 ::::: .99993, we conclude that discretization error is at
most 1.740 x 10-6 for all numbers except those greater than 4, which are rarely encountered.
These favorable results apply for any full-period generator. However, no closed form
exists for the inverse <1>-1, so that this method of half-normal generation must rely on time-
consuming methods ofnumerical approximation. Nevertheless, the bounds on discretization
error provide a basis for judging the discretization errors induced by the acceptance-rejection
method based on Algorithm AR for A = 16807,630360016, and 742938285. In the present
notation:

ALGORITHM H-N

Given: Zo (seed).
Method: i +-- -1.
Repeat
i +--i+2
Zi +-- (AZi-d ( mod M)
Zi+! +-- (AZ i ) ( mod M)
Ui +-- Zi/ M
Ui+! +-- Zi+d M
Y +-- -In(l- Ui)
Until Ui+1 ::: e-(Yi-1)2/2.
Return Y.

Clearly, discretization error depends on the properties of the 2- tuples {( Ui , Ui +1) } that
lie in

C:= { (Uh U2): -U2 + AU1 == 0 (mod 1), U1 E {~, ... , M;; 1 } } .

Observe that the points fall on A parallel lines with slope A, whereas the accepted points are
ill

U2 ::: e-[ln(l-Ul)-1]2/2} •
 9 .5 . C H 00 SIN GAM 0 NG M U L TIP LIE RS 439

Let j 1, ... , A index the parallel lines (from left to right). Then UI = k(j)IM,
where

k(j) := iM(j -1)IAl,

denotes the smallest U I on line j and I (j) 1M, where

l(j) := min {I EM: All M - (j - 1) > e-[ln(I-I/M-Ij'/2}

denotes the smallest U I on line j that is rejected.

Let

Y(l) := -In(l -II M), 1=1, ... ,M-1,

where Y(l) is strictly increasing in I. Then Algorithm H -M produces no values on the positive
axis in the intervals

A-I
U
j=1
(Y(l(j)-I), Y(k(j+I)))'

Since Y(l(j)) is rejected, no value occurs in the intervals

(Y(rnax[k(J)'/(J)-I]), y(l(j))] and

Let ~Y denote the length of an excluded interval, which we hereafter call a gap. Table 9.4
displays bounds on gaps for the three multipliers of interest. For A = 16807, the gaps are
considerably larger than for the other multipliers. They are also substantially larger than
those that the inverse transform distribution induces. Although Algorithm H-N is not the
commonly encountered method for generating half-normal samples, the gaps for A = 16807
represent what can happen when a multiplier that performs relatively poorly on the spectral
test is used. At a minimum, Table 9.4 should encourage potential users of this multiplier to
switch to any of the first five multipliers in Table 9.3 that perform best on the spectral test in
dimensions 2 through 6.

9.5.3 DA.... GEROUS MULTIPLIERS

Recall that A == ± 3 (mod 8) and M = 2f3 (f3 :=:: 3) yield a generator with maximal
period 2f3 - 2 • Consider a multiplier A E {2" - 3; v :=:: f3 12} U {2" + 3; v :=:: f3 12} that ensures
440 CHAPTER 9. PSEUDORANDOM NUMBER GENERATION

Table 9.4 Bounds on lengths of excluded intervals, 1 :::; Y < 4

A y f),.y

16807 1:::; <2 1.27 x 10- 9 :::; :::; 1.73 X 10-4

2:::; <3 1.73 x 10-4 :::; :::; 1.03 X 10- 3

3:::; <4 1.03 x 10-3 :::; :::; 3.22 X 10- 3

6300360016 1:::; <2 1.27 x 10-9 :::; :::; 4.61 X 10-9

2:::; <3 4.61 x 10-9 :::; :::; 2.76 X 10-8

3:::; <4 2.76 x 10-8 :::; :::; 8.57 X 10-8

742938285 1:::; <2 1.27 x 10-9 :::; :::; 3.91 X 10-9

2:::; <3 3.91 x 10- 9 :::; :::; 2.34 X 10- 8

3:::; <4 2.34 x 10- 8 :::; :::; 7.27 X 10- 8

this maximal period and leads to

Zi+l = (2 V ± 3)Zi (mod 2fJ ),

Zi+2 = (± 3 x 2 v+I + 9)Zi (mod 2 fJ ),

(mod 2fJ). (9.25)

In the early days of simulation, multipliers of this form offered the appeal of allowing
shift and add operations to replace the more costly direct multiplication AZi • For example,
A = 2v + 3 allows:

• Shift Zi v bits to the left and call this number B.

• Shift Zi one bit to the left and add it to B.

• AddZi to B.

• Zi+l = B.
 9.6. MULTIPLE REPLICATIONS 441

Normalizing both sides of equation (9.25) and rearranging terms leads to

Ui + 2 =f 6 Ui + 1 ± 9 Ui == 0 (mod 1),

revealing that the triplets {(Ui , Ui + l , Ui + 2 ); i ~ I} all lie on no more than N3(A, M) = 15
parallel hyperplanes.
Regrettably, a multiplier A = 216 + 3 in this class was used for many years in the pseu-
dorandom number generator RANDU in IBM's Scientific Subroutine package. Interestingly,
the bad performance in three dimensions was discovered empirically.
This last example demonstrates that although dk(A, M) is a better measure of k-
dimensional nonuniformity than Nk(A, M), a bound on the number of hyperplanes may
be more easily computed and thus be of more immediate use.

9.6 MULTIPLE REPLICATIONS

As discussed in Section 4.20, using separate pseudorandom number streams, one for each
source of stochastic variation, creates an opportunity for variance reduction. Suppose there
are r sources of variation and the jth requires k j pseudorandom numbers to generate a
sample from its corresponding probability distribution. This focuses our interest on the
(k l + ... + krHuples

Z/+(r-l)l, Zi+(r-l)l+l, ... , Zi+(r-I)l+k,-I, i ~ 1, (9.26)

where I > max j kj denotes the offset in the pseudorandom number sequence between
successive streams.
It is easily seen that these points lie on (k l + ... + kr )-dimensional hyperplanes

r kj-I
LL q(j-I)I+sZi+(j-I)I+s == 0 (mod M), (9.27)
j=1 s=O

where q := (qO' ql, ... , q(r-I)l+k,-I) is an element of

Q (r-I)l+k, (A, M):= I q E Q ~(M) : t; ~%_I)I+sA(j-I)I+S ==

r kz-I
0 (mod t(M))
I ,
442 CHAPTER 9, PSEU DORAN DOM NUMBER G EN ERATION

t(M) is defined in Section 9.5, and the maximal distance between successive parallel
hyperplanes is

r kj-l ) -1/2
d'
k1+,+k,
(A, M) := max
qeQ '(r-l)I+k,,(A,M)
(
~ L
J=l s=O
2,
q(J-l)l+s
(9.28)

This motivates our interest in identifying the multiplier A that minimizes this maximal
distance.
Since inequality (9.21) has already revealed the grid coarseness for the generator (9.1)
for dimensions as small as 4, identifying the best A in kl + ... + kr > 6 dimensions would
serve little purpose. This motivates us to defer discussion of multiple streams until Section
9.7.1 and to turn to an alternative form of pseudorandom number generation that offers
an expeditious way of reducing this coarseness up to significantly higher dimensions. While
not currently implemented in any simulation language, its availability in C code makes it a
prime candidate for future implementation.

9.7 COMBINED GENERATORS

Consider J multiplicative pseudorandom number generators (L'Ecuyer and Tezuka 1991)

j = 1, ... , J, (9.29)

where M l , •.. , MJ are distinct primes and Aj is a primitive root of M j . For example, for
J =4,

Ml = 231 - 1 = 2147483647, Al = 45991,

M2 = 231 - 105 = 2147483543, A2 = 207707, (9.30)
M3 = 231 - 225 = 2147483423, A3 = 138556,
M4 = 231 - 325 = 2147483323, A4 = 49689.
Let 81> ... , 8J denote arbitrary integers such that for each j,8 j and M j have no
common factors. Consider the combined generator

(9.31)

and the normalized sequence {Ui := Zil M 1 , i :::: I}. What can we say about its properties?
 9.7. COMBINED GENERATORS 443

0.001
, ....
.... .. ;...;- ..........-:. .
~-~;.-'~~;"\ 'il"'~':-~'

.:
"" . "--:,------.--.---------------
..). ~ ~
.- ••

.... . . ~. -: :.:",..
. ). '"'. .~ -;.-",.:........
......._.::...: ......
r·: .... ' :~
• •••••• .. .".... ~ ...: ... ,.. • •••• I :: •

•: • _. ,~ •• ' . • '. ~....

" .. .
••••• ......... • ...... " ........1

0.0008 : • •1' .; ',' I. :: ~ •• ' .' '.,~ ':.: , " . : • • • •, ' • - ••"._

•
...
: :• • ~::.:.:~ •• I ••• , .... :".!~ '... : ~" ••: ,',
1'\" • •••••• ','.. • ......1 \
::. II: ::...... :. ':.: '~" .. :.:..;".::.!'~:;.:. 'II",:: ••:,' .,~
:: :;.::' .... : ,.1 I':, ':' .,~ ::." : :'.
.1-: ... \.
':'.1 ...•. ' .. ~. I.,"
.. :: .-:.... :., ' .:'. I. '.' , '.

0.0006 ~
,

..: .1:...~ ..." .1. .,

• __ . " • • ...... • .1 I. •••• '"

.
. ..' ",'.' ....~.. .. .:' I.:. '::-,,\"! \. '. ':~. I ' ' ..
.' •••.
..: .... I.,. ".:' .,...... ........ . ... :: ..,
U i+l ~,~~

..:.. ..:". ...

~_.
,.. ... -. ... :. .....
• ......., •• , •••• ::..
. . ::.: -.. ),.I··
~ •••• \. ::'f :. • :••• :J.:~:

0.0004 .. ...:. .:.:.:. :.1:·.:·· ..,:

~ . .. ~.... I· .:

.'-.. .;' . .... .,..., ...\ .•....:1::'·.. ::.: ... ..

.. , .....................
, ....: .. .'.' :....
• .,. ••• : .lIj :'.. •••• • .: .•••• ., : •• ,..... ••• • .... .

........
:. ,: :...
•. ...... :.:. ....
...
' .., ......
._'" .. -:.:.-.!
~: :,.........., ....'
... ..............:. ~.,

. .......
. ...., :..,:,......... .........
....' ...
.
.\ .:.~
:~ <4
:'
•• .. •• ., • •It • "'.. ..,.. • • • • • I. • ':
0.0002 !. .1.:.. .." ..
Jt :0......t,• ~
••". ....;:" •• ' •• ::. _.. \ •• •••
. :.......:. ' .....:... .......'.....
r:'... •.:......' : •• ••• ! •.. .
.., .... ... ...--... ::.._........
... .....'.::. ....
:. : .. .. : ..:..... . .'. -. ~
:~
~
•... : : .. 'y,:.:: ... .., ..
:
ie, !I . . .J'. ~'·.:.r .~.

o ~: .~.",. :>~::,:'::: . ~~·:L. ::.::r:~:~"· :'.:::: .. :.;:,

o 0.0002 0.0004 0.0006 0.0008 0.001

Ui

Figure 9.4 2-tuples for combined generator

To answer this question, we need to consider an ancillary generator. Let

Zji/Mi ,

(t1=1
OjUji) (mod 1), > 1, (9.32)

and

- J
M:= Ilj=IMj.

Then

• {Vi, i > I} obeys the recurrence

Yi = A Yi - I (mod !VI), (9.33)

where Yi = !VI Vi and A is computable (L'Ecuyer and Tezuka 1991) and does not
depend on {OJ}.
444 CHAPTER 9. PSEUDORANDOM NUMBER GENERATION

• {~} and therefore {Yi } have period

P = lcm(Ml - 1, ... , M, - 1),

the least common multiple of MI - 1, ... , M, - 1, where

• Ui = ~ + Ei, where tight lower and upper bounds can be computed on Ei.
These are important properties. They imply that the period is at least min l:::;j:9 (Mj -
1) and that judiciously chosen M I, ... , M, can induce a period as large as (M I - 1) x ... X
(M, - 1)/2'-1. In particular, the maximum is achieved if (M I - 1)/2, ... , (M, - 1)/2
have no factors in common, as is the case for M I , ..• , M4 in expression (9.29), for which
P ~ 2.13 X 1037 , in contrast to a period of 231 - 2 ~ 2.14 X 109 for the generator (9.1)
with M = 2 31 - 1. Indeed, the density of points in [0, 1) increases by a factor of 1027 .
The first property tells us that the normalized {~, i :::: I} satisfy a one-step recursion
~ = A ~_I (mod 1). The third reveals that Ui, based on the generator (9.30), differs from
~ by a negligible amount. In particular, the assignment (9.29) leads to

£1 = 21267641435849934371830464348413044909 ~ 2 x 1037 ,
A=5494569482908719143153333426731027229,

and induces the bounds -1.04308128 x 10-7 ~ Ei ~ 1.99303031 X 10-7 (L'Ecuyer 1999).
Therefore, it is not unreasonable to characterize the properties of IUd by those of {~},
which, fortunately, we can determine via the spectral test.
Table 9.5 lists the distances {dk(A, £1), k = 2, ... , 1O} between successive hyperplanes
for the generator based on the assignment (9.29). For comparison, it also lists the distances
for A = 16807,630360016, and 742938285 and M = 231 -1. The unequivocal superiority of
the generator (9.30) is clear. It induces a considerably denser grid than the other generators
do.
Recall that Figures 9.2a, 9.2b, and 9.2c each show regular patterns of about 2147
2-tuples in [0, .001) x [0, .001) over the period M - 1 = 231 - 2 of the three prime-
modulus multiplicative congruential generators of the basic form (9.1). For the combined
generator defined in expressions (9.28), (9.29), and (9.30), run with arbitrarily selected
seeds for each of its component generators, Figure 9.5 displays the subset of2147 2-tuples of
{(Ui, Ui+I); i = 1, ... , 231 -2}thatfallinto [0, .001) x [0, .001). The choice of run length
was made to allow comparison between Figures 9.2 and 9.5 with density held relatively
fixed. Most notably, the absence of any regularity in Figure 9.5 allows us to conclude that for
sequences of length 231 - 2 the combined generator succeeds in eliminating the apparent
lattice structure inherent in generators of the form (9.1) and depicted in Figures 9.2a, 9.2b,
and 9.2c.
 9.7. COMBI N ED G EN ERATORS 445

Table 9.5 Distance between parallel hyperplanes

dk(A,2 31 - 1) dk(A, if)

k A = 16807 A = 630360016 A = 742938285
2 5.95 x 10- 5 2.42 X 10- 5 2.32 X 10-5 2.38 X 10- 19
3 1.57 X 10- 3 1.60 X 10- 3 8.02 X 10- 4 4.18 X 10- 13
4 6.79 x' 10- 3 4.99 X 10- 3 4.53 X 10- 3 5.24 X 10- 10
5 1.50 X 10-2 1.38 X 10- 2 1.33 X 10-2 3.41 X 10- 8
6 3.34 X 10- 2 3.34 X 10- 2 2.59 X 10- 2 6.31 X 10- 7
7 6.04 X 10- 2 5.53 X 10- 2 4.73 X 10-6
8 7.91 X 10- 2 6.82 X 10- 2 2.09 X 10- 5
9 .113 .106 7.78 x 10- 5
10 .125 .108 2.30 x 10-4

9.7.1 MULTIPLE PSEUDORANDOM NUMBER STREAMS

Recall from Section 9.6 that it is the properties of the (k1 + ... + kr )-tuples that
ultimately interest us. Moreover, we cannot expect these to be particularly good for M =
231 -1 and 232 • To simplify the exposition in the current context, we take k1 = ... = kr = k
and focus on the kr-tuples

Zi := (2i, 2i+1> ... , 2i +k-1> •.• , 2i+l, 2i+ I+1> •.• , 2i + I+k-1>""
2i +(r-1)1, 2i +(r-1)1+1> .•• , 2i +(r-1)1+k-d,
which, because of the third property of generator (9.31), we assume are close to those for
the kr-tuples

Yi := (y;, Yi + 1,.··, Yi+k-1> Yi+l, Yi +l+1, ... , Yi+l+k-1> .•. ,

Yi +(r-1)1, Yi +(r-1)1+1, ... , Yi +(r-1)I+k-d.

For assignment (9.29), it is of interest to determine d~lr (,4, M), the maximal distance
between parallel hyperplanes, for r pseudorandom number streams with displacement I
between successive streams for k dimensions. An empirical analysis in L'Ecuyer and Andres
(I997a) reveals that

2.02 X 10- 19 r = 2,
3.22 X 10- 13 r = 3,
3.92 X 10- 10 r = 4,
d~lr(A, M)::::: 2.78 x 10-8 r = 5,
4.66 X 10-7 r = 6,
3.46 X 10-6 r = 7,
1.53 X 10-5 r = 8,
446 CHAPTER 9. PSEUDORANDOM NUMBER GENERATION

for k = 1, ... , 20, and I = 241. This choice of displacement I induces the least upper bound
on dle1rCA, M) for all 2 :::: r :::: 8 and 1 :::: k :::: 20. Since these are considerably smaller than
one could expect to achieve for generators of the form (9.1) for k-tuples taken on r streams
for these ranges of k and r and any displacement I, we recommend the L'Ecuyer and Andres
combined generator (9.31) with the assignment (9.30) for replacing currently implemented
pseudorandom number generators. Although the L'Ecuyer-Andres implementation takes
r = 231 streams each oflength I = 241 by default, it allows the user to change these values.

9.7.2 COST OF IMPROVED QUALITY

These improved properties come at the expense of increased computing time per pseu-
dorandom number generation. The increase is between three and four times the computing
time for an efficient implementation of the generator (9.1). The upper bound offour arises
because the function call to the generator occurs once regardless of whether generator (9.1)
or (9.29) is used.
As already mentioned in Section 9.2, careful attention to multiplication modulo M
can save time. The C programs clcg4.c and clcg4.h and the Modula-2 code clcg4.mod
and clcg4.def, available at http://www.iro.umontreal.ca/...lecuyer/papers.html. take full
advantage of the option to conserve time. A test program, prg 2.c, is also available there.
For the practitioner, the issue now becomes one of quality of results versus computing
time. If a simulation using generator (9.1) devotes 100 x a percent of its time to generating
pseudorandom numbers, then a switch to generator (9.30) induces a relative increase in
computing time of at most 4a + (1- a) = 3a + 1. For example, a = .05, .10, and .20 induce
3a + 1 = 1.15, 1.30, and 1.60, respectively.
In Monte Carlo experiments that focus narrowly on numerical integration, pseudo-
random number generation can be the principal time consumer. If, for example, a = .80,
then the switch from generator (9.1) to generator (9.30) implies as much as a 3a + 1 = 3.4
time increase in computing time. No doubt, this substantial increment in cost is what has
retarded the implementation of better pseudorandom number generators in this type of
experiment.
By contrast, pseudorandom number generation in discrete-event simulation rarely
consumes more than a marginal amount of computing time, most of which is devoted to
memory management and, in particular, list processing. It is not unusual to have a < .20. In
this case, the penalty of increased computing time seems worth the benefit ofhigher-quality
pseudorandom numbers.

9.8 LESSONS LEARNED

• Every simulation language provides a routine for generating pseudorandom integers
Zi, Zi+1, ... , where Zi is less than some positive integer M Vi. The integers are
 9.8. LESSONS LEARNED 447

normalized by mapping them (Ui = Zi 1M) into points in (0, 1), which are then used
as input for generating samples from diverse distributions.

• Multiplicative congruential pseudorandom generators are the most common mecha-

nism for generating these integers. Each has modulus M and integer multiplier A. The
generators execute rapidly and need little memory. Each require a seed Zo to prime
the generator. The simultaneous choice of M and A determines the density of inte-
gers generated from {I, ... , M - I}. For given M, the choice of A determines how
well k-tuples generated on (0, 1)k resemble random k-tuples uniformly distributed
on (0, 1)k.

• If M is prime, then choosing A to be a primitive root of M ensures that the sequence

runs through a complete permutation of the integers {I, ... , M - I} before repeating
itself.

• If M = 2fJ for f3 ::: 3, then choosing A = ± 3 (mod 8) ensures a maximal period MI4
on a subset of the MI2 odd integers in {l, 3, ... , MI2 - I}. The period denotes the
number of iterations of the generator before it repeats itself.

• k-tuples (Ui , U i + l , ... , Ui+k-d for i ::: 1 lie on parallel hyperplanes in (0, 1)k. The
choice of M and A dictates the number of parallel hyperplanes and the maximal
distance between successive parallel hyperplanes. A small number of widely spaced
hyperplanes signals a poor approximation to randomness for the k-tuples. Among
all multipliers A that maximize the period for a given M, one prefers the one that
minimizes the maximal distance between successive hyperplanes in k dimensions.
This optimal A usually varies with k.

• Table.9.3 gives good choices of multipliers for M = 231 - 1,232 , and 248.

• Since Ui , Ui +l , ... are the input to a procedure for generating samples X j, XHI, ...
from a particular distribution F, departures from uniformity in the Ui induce a
distribution F for the X j that only approximates F. Most importantly, a given sequence
Ui , Ui + l , ... used as input to different procedures for generating samples from F
induces different F's and therefore different approximations to the desired distribution
F. The acceptance-rejection and ratio-of-uniforms methods can induce substantial
errors in this way.

• By combining several multiplicative pseudorandom number generators according to

specified rules, generators can be created with exceptionally large periods. For example,
combining four generators with individual periods of about 2.14 x 109 leads to a
combined generator with period of about 2 x 1037 •

• In dimensions k = 2, ... , 8, the maximal distances between successive hyperplanes

in (0, 1)k for the combined generator are substantially smaller than for the more
conventional multiplicative congruential generators.
448 CHAPTER 9. PSEUDORANDOM NUMBER GENERATION

• Implementations of the combined generator are available via the Internet. See Section
9.7 for the address.

9.9 EXERCISES
<ff? 9.1 Show that A = 742938295 is a primitive root of2 31 - 1.

<ff? 9.2 Show that A = 630360016 is a primitive root of 231 - 1.

<ff? 9.3 Show that A=746061395 is the multiplicative inverse of630360016 (mod2 31 -1).

<ff? 9.4 Determine whether A = 48271 and M = 231 - 1 gives a full period M - 1.

<ff? 9.5 Determine whether A = 343 and M = 231 - 1 gives a full period M - 1.

<ff? 9.6 For the multiplicative congruential generator

Zi = AZi - 1 (mod 2731),

a. Determine the number of values (mod 2731) for A that give a full period 2730.

b. Enumerate all these values.

c. Enumerate all pairs of values of Al and A2 for which the generator

gives the same sequence

Zi = AzZi - 1 (mod 2731),

but in reverse order, and for which both generators have full period. Show your work in
detail.

<ff? 9.7 Consider the full-period pseudorandom number generator

Zi = AZi - 1 (mod M),

where M = 2fJ - y, y > 0, {J > log y I log 2, and M is prime. Construct a portable
pseudorandom number generator that will produce the identical sequence of numbers on
all computers whose floating-point representations have at least a f3{J 12l-bit mantissa.
 9.9. Ex ERe I S E S 449

Gf!? 9.8 Asimulatorwantstogeneratealltheintegers{O, 1, ... , 4997} in a pseudorandom

order. He plans to use the sequence {Yj = Zj - 1, i = 1, ... , 4998}, where

with M = 4999 prime, Al = 2187, and Zo E {l, 2, ... , 4998}. A second simulator suggests
an alternative multiplier A2 = 2182. Determine which, if either, of the multipliers generates
all the desired integers. Give complete details.
Gf!? 9.9 Consider the pseudorandom number generator

Zj = 7340035Zj_1 (mod(65536 x 524288)).

a. Determine the period of the generator.

b. Determine an upper bound on the maximal number of parallel hyperplanes on which
the 2-tuples {(Zj, Zj+d, i 2: I} lie.
c. Show that the maximal number of parallel hyperplanes on which the 3-tuples
{(Zj, Zj+J, Zi+2), i 2: I} lie cannot exceed 15.

Gf!? 9.10 The programming language Prime Sheffield Pascal provides a library function
RANDOM that uses a multiplicative congruential generator Zj = AZj _ 1 (mod M) with
A = 16807 and M = 231 to produce random numbers. Determine whether this generator
has the maximal achievable period for this modulus. Display all steps in your proof.
Gf!? 9.11 Because of the small word size available on her computer, a simulator decides to
employ the random number generator

Her experiment calls for generating exponential samples with p.d.f.

f(x) = e- x , x 2: O.

She decides to apply the inverse transform method to the normalized Zj to generate these
exponential samples. Let X be a resulting sample. Find the smallest and largest values that
X can assume before repeating itself (a) if Zo = 1 and (b) if Zo = 7. Give their numerical
values to six significant digits.
Gf!? 9.12 Consider the multiplicative congruential generator

Zj = 1048579Zj_ 1 (mod 536870912).

a. Determine the period.

450 C HAP TE R 9. P5 E U DO RAN DO M N U M B ERG ENE RAT ION

b. State the conditions under which the numbers 2875, 4876321, and 578766 would
each appear in a sampling experiment that uses P pseudorandom numbers, where P
denotes the maximal period.

~ 9.13 For A = 65541 and M = 232 , show that the period of generator (9.1) is 230 •

~ 9.14 For A = 16777221 and M = 248 , show that the period of generator (9.1) is 246 •

9.1 0 REFERENCES

Cassels, J.W.S. (1959). An Introduction to the Geometry ofNumbers, Springer-Verlag, Berlin.

Fishman, G.S., and L. Moore (1986). An exhaustive analysis of multiplicative congruential

random number generators with modulus 231 - 1, SIAM J. Sci. and Statist. Comput.,
7,24-45.

Fishman, G.S. (1990). Multplicative congruential random number generators with modulus
2fJ: an exhaustive analysis for fJ = 32 and a partial analysis for fJ = 48, Math. Comp.,
54, 331-334.

Fishman, G.S. (1996). Monte Carlo: Concepts, Algorithms, and Applications, Springer-Verlag,
New York.
Hormann, W., and G. Derflinger (1993). A portable random number generator well suited
for the rejection method, ACM Trans. Math. Software, 19, 489-495.

Jannson, B. (1966). Random Number Generators, Almquist and Wiksell, Stockholm.

L'Ecuyer, P., and S. Tezuka (1991). Structural properties for two classes of combined random
number generators, Mathematics of Computation, 57, 735-746.
L'Ecuyer, P., and T.H. Andres (1997a). A random number generator based on the combina-
tion of four LCGs, version: November 25, 1997, University of Montreal, available at
http://www.iro.umontreal.ca/..-. lecuyer.
L'Ecuyer, P., and T.H. Andres (1997b). A random number generator based on the
combination offour LCGs, Mathematics and Computers in Simulation, 44, 99-107.
L'Ecuyer, P. (1999). Personal communication.

Overstreet, Jr., c.L. (1974). The relationship between the multiplicative and mixed generators
modulo 2b, Computer Science Department, Virginia Polytechnic Institute and State
University, Blacksburg.
Park, S.K., and K.W. Miller (1988). Random number generators: good ones are hard to find.
Comm. ACM, 31, 1192-120l.
 9.10. REFERENCES 451

Payne, W.H., J.R. Rabung, and T.P. Bogyo (1969). Coding the Lehmer pseudorandom number
generator, Comm. ACM, 12, 85-86.
Schrage, L. (1979). A more portable FORTRAN random number generator, ACM Trans.
Math. Software, 5,132-138.
 CHAPTER 1

G. S. Fishman, Discrete-Event Simulation

© Springer Science+Business Media New York 2001
Preparing the Input

The modeling representations of Chapter 2 and the programming concepts of Chapter 3

provide us with the ability to create relatively sophisticated simulation models and executable
programs, but only after we have identified all sources of stochastic variation, specified
sampling distributions that characterize each source, and assign numerical values to the
parameters of the distributions. This chapter addresses these issues.
In practice, specifying distributions and assigning numerical value go hand in hand.
Collectively, we refer to them as input preparation. If no historical numerical data are
available, then a simulationist customarily relies on plausibility arguments. These arise
from at least three sources:

• Theoretical deductions that follow from an input process specified properties

• Expert opinion based on knowledge of how the particular input process behaves in
the real world

• Expert opinion about the behavior of input processes in closely related systems.

For example, if an arrival process can be viewed as the superposition of a very large number
of independent renewal processes each with relatively small arrival rate, then theoretical
454 C HAP TE R 1 0 . PRE PAR I N G THE I N PUT

deduction can show that inter arrival times in the arrival process of interest have a distribution
well approximated by the exponential (Section 1004.1).
If, in addition, expert opinion based on historical recollection indicates that the arrival
rate A lies in [A [, Azl for 0 < AI < Az < 00, then the simulationist has a candidate
distribution and an interval for the value of its parameter. Using the lower bound A[ in the
simulation would induce lower congestion than expected in the real system, whereas using
Az would induce the opposite behavior. This may encourage the simulationist to compare
the output of, say, two runs, one with A = A[ and the other with A = Az, to see whether
the uncertainty about A induces variation in the output that is within tolerable bounds. If
it does, then the actual value of A is not a major issue. If variation is not within tolerable
bounds, then attention must be devoted to finding a better way to estimate A.
Naturally, the availability of empirical data on the input process of interest or on one
closely resembling it is of great value. It creates an opportunity for input preparation that
enables the executing simulation program to generate stochastic sample paths that mimic
those observed in either the real or related system. How well these match depends on the
amount of historical data available and on the techniques employed in analyzing data.
Some input preparation depends exclusively on either expert opinion or on empirical
data analysis. However, a blend of both can endow a simulation with a considerably more
representative input than either alone. To effect this blend often calls for hypothesis testing
as well as estimation. Several factors militate against embracing this joint approach. First,
it requires a deeper knowledge of statistical theory. Second, it inevitably takes more of the
simulationist's time and effort. Third, many theoretical results in the study of delay systems
have their greatest value in facilitating analytical solution. Since a simulation model provides
a numerical solution, why burden the analysis with specification aimed at analytical solution?
While the first two factors need to be evaluated in light of the simulationist's skills and
time constraint, the third needs to be tempered by recognizing the limitations of relying on
a data analysis devoid of plausibility arguments. Using a sample record of finite length to fit
an input distribution inevitably invites errors both in selecting the distribution and estimat-
ing its parameters. By combining this analysis with plausibility arguments, a simulationist
prevents the sampling errors in the distribution fitting process from having undue influence
in the final form of the input.

10.1 ARRIVAL PROCESSESS

Interarrival times, service times, and batch sizes (number of items demanding service in
an arrival) are principal sources of stochastic variation in a delay system. Our discussion
focuses on these with the admonition that other sources of stochastic variation peculiar to
particular systems can often be analyzed using techniques applicable to these three.
From the perspective of modeling and programming it is convenient to regard inter-
arrival times as a sequence of i.i.d. random variables. This applies equally well to service
 10.1 . A RRI V ALP ROC E SSE 5 5 455

(a) Batch arrivals

Batch
size

OL-~________- i_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _~_ _ ~L ___~ __________~·

o 10 12 14 16 18 20

time

Figure 10.1 Arrival processes

times and batch sizes. Then one need only specify a single distribution for each source from
which to generate samples when executing a simulation run. In reality, the i.i.d. assumption
does not always apply, and input preparation needs to acknowledge that and accommodate
dependence. Section 10.11 does this. For now, we retain the assumption merely as a con-
venient backdrop to addressing issues relevant to the objectives of input preparation. The
issues raised here apply equally well when the i.i.d. assumption does not hold.
Discrete-event simulation methodology often characterizes the arrival process in one
of two ways. One views it as a sequence of2-tuples (T], Dd, (Tz , Dz), ... , whereT], Tz , .. . is
a sequence ofi.i.d.interarrival times and D], D 2 , ••• is a sequence of batch sizes. If D; = 1 Vi,
then we are back to the classical single-item arrival queueing model. If D], D z, ... are i.i.d.
random variables, then we have a random batch-size arrival process. Figure 1O.1a illustrates
the concept.
The second characterization consists of a sequence of i.i.d. 3-tuples (T], T{, D]), (T2 ,
T;, D2)' ... where T], T2, ... and T{, T;, ... are independent sequences ofi.i.d. nonnegative
random variables, and D; items in theith (i > 1) batch begin arriving at time T] + :L;;;;;;
(T/+
1[') and end arrival at time :L;=](If + 1[').
The transmission of records in a communication network often fits this second char-
acterization, where T;' denotes the time required to transmit the i th record and T; + T;'
denotes the time that elapses between the end of transmission ofthe (i ~ 1)st record and the
end of transmission of the i th record. Occasionally, it is convenient to think of this as a fluid
456 C HAP TE R 1 0 . PRE PAR I N G THE I N PUT

(b) Fluid-flowarrivais
5 ----------------------------------------,

Arriving
volume

0~~_~ _ _ _L __ _ ~ _____ -L~_L_ _______ L__~

o 10 12 14 16 18 20

time

Figure 10.1 (continued)

flow into a storage tank rather than as a discrete count on items (Section 2.6). Figure 1O.lb
gives one possible conceptualization of this process.

10.2 EMPIRICAL AND FinED DISTRIBUTIONS

When empirical data are available on input processes, a simulationist can use them in at
least two ways to prepare input. One constructs empirical distribution functions (e.d.f.s),
based exclusively on the data, and generates samples from them during the simulation. The
other fits the data to theoretical distributions and then generates samples from the fitted
distributions.
The e.dJ. approach induces stochastic variation in a simulated sequence that exactly
mimics that in the empirical data. Some of these fluctuations arise from the intrinsic proper-
ties of the underlying stochastic process. These are desirable. However, others arise from the
less-than-complete stochastic representation on which the e.dJ. is based. These are undesir-
able. Therefore, simulated sample sequences generated by an e.dJ. show desirable variation
due to the intrinsic character of the processes and undesirable variation due to finite sample
size.
As the length n of the empirical record increases, the unwanted fluctuations diminish
in relative importance when compared to the wanted fluctuations, so that the e.dJ. approach
generates samples that are increasingly (as n increases) representative of sample paths on
the underlying stochastic process. When very long empirical records are available, the e.dJ.
 10.3. EM P I RIC A L D. F. A P PRO A C H 457

approach has much to commend it as a means of generating sample sequences. Section 10.3
describes its essential features.
For small and moderate n, the fitted d.f. approach has appeal. Fitting a theoretical
distribution to data smoothes out the variations in the empirical distribution due to finite
sample size. It also enlarges the domain of values for sampling. For example, given a sample
record of service times X(l) ::: X(l) < ... ::: X(n), the e.d.f. approach generates values on
[X(l), X(n)] at best. However, if expert opinion tells us that in reality service times take values

in the support [Xl, Xl], where Xl ::: X(l) < ... ::: X(n) < Xl, then fitting a distribution
with this support allows this expert opinion to be merged with the sample record to generate
samples on [Xl, Xl].
Smoothing and increasing the domain for sampling are the desirable features of the
fitted d.f. approach. As n grows, these benefits diminish in importance when compared to the
e.d.f. approach. First, the e.d.f. becomes smooth; second, its domain for sampling increases,
and third, because the fitted d.f. is inevitably an approximation, tests of its adequacy begin
to detect statistically significant discrepancies between fitted and empirical distributions.
Section 10.12 discusses distribution fitting in several different contexts.
Although our discussion reveals a tendency to favor the e.d.f. approach for large sample
sizes and the fitted d.f. approach for small and moderate sizes, following this inclination
would be too simplistic in practice. Circumstances, as in Section 10.13, exist where the
e.d.f. approach offers more convenience and more accuracy for relatively small sample sizes.
Moreover, the characterization of arrival processes in Section 1004.1 encourages the reader
to focus on the (theoretical) exponential distribution as a candidate for fitting, even in large
samples. Therefore, we encourage the reader to recognize that circumstances for her/his
particular study must be the determinant of which approach to follow. More often than not,
both play roles in a single simulation study.

10.3 EMPIRICAL D.F. APPROACH

Let X denote a random variable with d.f. {F (x), XE X} and p.d.f. or p.m.f. {f (x), X E X}.
If X denotes interarrival or service time, then F is usually continuous with support X E
[0,00), [a, 00) for some fixed a > 0, or [a, b) for fixed 0 ::: a < b < 00. For the moment,
assume that X is continuous on [0, 00).
Let Xl, ... , Xn denote n i.i.d. observations from F. Then the e.d.f.

1
L
n
Fn(x) := - I[xj,oo) (x), X::: 0, (10.1)
n i=l

provides an estimator of {F (x)} and the empirical histogram

(10.2)
458 C HAP TE R 1 0 . PRE PAR I N G THE I N PUT

(a)

0.14

6. =.2758
1=20
0.12

0.1

0.08
0'"
.~

e
0.

P- 0.06

0.04

0.02

0 I I I I
0.3 0.6 0.9 1.1 1.4 1.7 2.0 2.2 2.5 2.8 3.1 3.3 3.6 3.9 4.2 4.4 4.7 5.0 5.3 5.

Figure 10.2 Empirical histograms for service-time data, n = 100

where for some fixed 1 < I < n,

~ .
Yl := - + mm Xi,
2
Yj := Yj-l + ~ j = 2, ... , I,
max X. - min X.
~:= I I

approximates f(Y)~ evaluated at Y E {Yl, ... , yd. Figure 1O.2a shows an empirical
histogram for n = 10 I i.i.d. service times based on an I = 20 cell partition of the data.
The graph of a histogram can provide insight into which theoretical distributions are
good candidates for fitting. For example, the Gamma and the Wishart distributions with
their skewed right tails are good prospects for Figure 10.2. However, the ability to discern
shape in a histogram depends crucially on 1. Customarily, I is chosen to balance the need
to discern the underlying shape of this frequency chart against the need to smooth out
fluctuations that obscure shape. Large I improves resolution but admits large fluctuations.
Small I smooths out fluctuations but reduces resolution. Inevitably, the choice of I is a
compromise. A comparision between Figures IO.2a and 1O.2b shows the tradeoff.
 10.3. EM P I RIC A L D. F. A P PRO A C H 459

(b)
0.2

0.18

0.1 6

0.14

0.12
c
0
'f 0.1
0
0.
0
~
0.08

0.06

0.04

0.02

0
0.4 1.0 1.5 2.1 2.6 3.2 3.8 4.3 4.9 5.4

Figure 10.2 (continued)

10.3.1 SAMPLING PROPERTIES

To assess how well {Fn(x)} approximates {F(x)}, we turn to its sampling properties.
For each x E X they are:

• Fn(x) -+ F(x) w.p.l asn -+ 00.

• s.e.(Fn(x)):= JvarFn(x) = JF(x)[l- F(x)l!n (standard error).

• r.e.(Fn(x)):= VarFn(X) = J[1 - F(x)l! F(x)n (relative error).

EFn(x)

• r.e.(l- Fn(x)):= VarFn(X) = IF(x)/[1 - F(x)]n.

E[I-Fn(x)] Y

• corr[Fn(x), Fn(Y)]:= COrr[Fn(x),Fn(y)] = F(y)[l-F(x)] Y :::: x.

JvarFn(x)VarFn(y) F(x)[l-F(y)]'

• SUPXEX I Fn(x) - F(x) I has the Kolmogorov-Smirnov (K-S) distribution for each
fixed positive integer n. Therefore, ([Fn (x) - dn(8), Fn (x) + dn(8)], '<Ix EX} covers
the true dJ. {F(x), x EX} w.p. 1 - 8, where dn(8) denotes the 1 - 8 quan-
tile of the Kolmogorov-Smirnov distribution. As n -+ 00, n 1j2 dn (.05) -+ 1.3581
460 CHAPTER 10. PREPARING THE INPUT

and n l / 2 dn (.01) -+ 1.6276. For n ::: 100, 1.3581/n l / 2 and 1.6276/n l / 2 approximate
dn (.05) and dn (.01), respectively, with negligible error.
d
• .jiiFn(x) -+ N(F(x), F(x)[I-F(x)])asn -+ 00. Therefore, for x EX, {nFn(x)+
4>2/2 =f 4>{4>2 /4 + nFn (x)[1 - Fn (x )]}1/2}/(n + 4>2), where 4> := <1>-1(1- 8/2), covers
F (x) with probability that converges to 1 - 8 as n -+ 00.

These properties also hold if X is continuous over an interval of finite length X := [XI, X2).
If X is discrete, the properties hold except that the confidence region based on the K-S
distribution corresponds to a confidence level greater than or equal to 1 - 8.

10.3.2 ASSESSING THE APPROXIMATION

The standard error, relative error, K-S distribution, and normal limit all provide ways
of assessing how well the e.dJ. approximates the true, but unknown, dJ. Replacing F(x)
by Fn(x) in s.e. (Fn(x)) and r.e. (Fn(x)) provides estimates of these errors. The K-S and
limiting normal properties provide more incisive measures.
The K-S distribution leads to a confidence region for the entire dJ. {F(x)} at the
100 x (1 - 8) percent confidence level for each positive integer n. By contrast, the limiting
normal result provides a 100 x (1 - 8) percent confidence interval for a selected x as
n becomes large. Both results have benefits and limitations. For example, the K-S result
gives a half-length of 1.6276/n l / 2 for 8 = .01 andn ~ 100 for all x, leading to {F(x) E
[max[O, Fn(x) - .1628), min[l, Fn(x) + .1628)), Vx E X} for the e.d.f. with n = 100 on
which the empirical histogram in Figure 10.2 is based.
Figure 10.3 shows {Fn(x)} and its 99 percent K-S confidence region where {Fn(x)},
as defined in expression (10.3 ), differs from {Fn(x)} in a relatively minor way. The width
of the region is far too large to regard the e.d.f. as a relatively accurate approximation of
{F (x)}. However, the limiting normal result reveals a better pointwise accuracy. Table 10.1
shows individual and simultaneous confidence intervals (Section 7.1) for five values of x .
These reveal shorter inteval lengths than the K-S approach, suggesting in general,
that for moderate and large n, computing confidence intevals for selected x provides a more
informative picture of statistical accuracy than the K-S approach. Nevertheless, in the present

Table 10.1 99% individual and simultaneous confi-

dence intervals, n = 100
Individual Simultaneous
x Fn(x) Lower Upper Lower Upper

.17 .09461 .2865 .08419 .3133

2 .47 .3474 .5964 .3252 .6200
3 .78 .6578 .8674 .6309 .8803
4 .93 .8342 .9723 .8084 .9767
5 .98 .9042 .9961 .8793 1.000
 10.3. EM P IRIC A L D. F. A P PRO A C H 461

case, the lengths of the normal intervals also imply that {Fn (x)} is not a particularly accurate
approximation to {F(x)}.

10.3.3 SAMPLlt-4G FROM {Fn(x)}

To generate a random sample X' from {Fn(x)}, we proceed as follows. Let {XI < X2 <
... < Xk(n)} denote the ordered collection of the k(n) distinct values in {X I, ... , X n },

a:= 1,
b := k(n),
PI := Fn(XI),
ql := PI,
Pi := Fn(Xi) - Fn(Xi-d,

and

qi := qi-I + Pi, i = 2, ... ,k(n).

0.9

0.8

0.7

0 .6

0.5

0.4

0.3

0 .2
- e.d.f.

0.1 - 99% lower and upper

con6:!ence limits

o
o 2 3 4 5 6

Figure 10.3 Empiral distribution function {l\(x), X E X} and .99 K-S limits for
service-time data, n = 100
462 C HAP TE R 1 0 . PRE PAR I N G THE I N PUT

Then the following steps produce a sample X' from {Fn(x)}:

• Sample Z using the cutpoint method (Section 8.2) with {qa, ... , qb} as input
or using the alias method (Section 8.4) with {Pa, ... , Pb} as input.

• X' =Xz.

Repeated application of these steps generates a sequence of i.i.d. random samples from
the restricted support {Xl < ... < Xk(n)} ratherthan from the infinite support X. To increase
the set of values that can be sampled from {Xl < ... < Xk(n)} to [0, Xk(n)j, let

Xa-l := 0,
Fn(O) := 0,
lex) := min(i ::: a: Xi > X),

and

(10.3)

Then repeated execution of the steps

• Sample Z using the cutpoint method with {qa, ... ,qb} as input and U from U(O, 1);

• X +-
I
XZ-l
+ (Xz - XZ-l
) Fn(xz)-Fn(xz-Il'
U-Fn(XZ_I) •

produces a sequence of i.i.d. samples from {Fn (x), 0 < X :s Xk(n)}' For an alternative method
of interpolation that relies on the alias method, see Ahrens and Kohrt (1981).

10.3.4 THE QUANTILE METHOD

As n increases, so does k(n) and, therefore, the memory space requirements for both
the cutpoint and alias methods. Indeed, each grows proportionately to k(n). If space is an
issue, one can resort to a space-conserving algorithm that, in effect, relies on an estimator
of the inverse d.f.

Q(f3) = XEX
inf [F(x) ::: f31 , 0:sf3:s1.

Let X(l) :s X(2) :s ... :s X(n) denote the order statistics of the sample Xl, ... , X n • For some
positive integer k :s n,

Qn(O) := 0,
Qn (j / k) := X(rjnfkl), j = 1, ... ,k,
is a k-point estimate of {Q(f3), 0 :s f3 :s I}. Then repeated execution of the step
 10.3. EMPIRICAL D.F. ApPROACH 463

• Sample U from U(O, 1).

• J +- LkuJ.
• Z +- Qn(Jlk) +k(U - Jlk) [Qn((J + l)lk) - Qn(Jlk)].

• X" +- Z;

produces a sequence of Li.d. samples from

.. -~~[ Qn((j x-Qn(jlk)

Fn(x) - k ka + l)lk) - Qn(jlk) +] 'J!.. x
[Qn{J/k),Qn({J+l)/k)) ( ),

O:::;x:::;Qn(l),

which is an approximation to {F (x), x E X} based on linear interpolation between sample

quantiles. If X is discrete on the integers {O, 1,2, ... }, then LZJ replaces Z in the assignment.
As in the cutpoint and alias methods, the time to generate a sample via this quantile
approach is constant and independent of nand k. However, its unique appeal is that it
partitions the unit interval into k subintervals, where k remains fixed as n --+ 00. Unlike the
cutpoint and alias methods, whose memory requirements grow linearly in n, the quantile
approach has a memory requirement proportional to fixed k but independent of n.
How statistically accurate is this quantile approach to estimation? Unlike the e.d.f. the
sample quantiles {Qn (j I k), j = 1, k - I} are not algebraic functions of the observations,
and thus its sampling properties are not available in algebraic form for finite sample size n.
However, limiting results exist. Suppose F has a continuous and positive p.d.f. {f(x), x E
X}. Then for each j = 1, ... , k - 1 with jnl k integer:

• Qn(jlk) --+ Q(jlk) w.p.l as n --+ 00

d ~r (Q( 'Ik)
• n 1/2Q n ('Ik)
] --+ JV ] ,
(j/k)(l-j/k))
f(Q(jfk)) as n --+ 00.

With minor adjustments, these properties continue to hold for nonintegral j n I k as n

becomes large. As a consequence of the consistency property, we have

w.p.l n --+ 00,

ensuring that Fn (x) converges to F(x), for each x E Qk := {Q(11 k), ... , Q( (k - 1)1 k)},
as desired.
However, no such convergence occurs for x E X \ Qk. How serious is this absence of
convergence? The answer depends on k and n. If n = 100 and k = 10, the quantile method
is an imprecise method at best. For n = 106 , as may occur when large data sets are available,
choosing k = 1000 makes {Qn(j I k)} a sequence of.OOlj quantiles for j = 1, ... , 1000, so
that linear interpolation in the quantile method is over relatively small probability intervals
464 CHAPTER 10. PREPARING THE INPUT

where the error of approximation is far less than in the k = 10 and n = 100 case. Since
n = 106 would require space of order four (single precision) or eight (double precision)
megabytes for the cutpoint and alias methods, a simulationist inevitability has to make a
choice between a large space commitment and an error of interpolation at the .00 I-increment
level.

10.4 CHARACTERIZATIONS FROM THEORY

A considerable body of theory exists about delay systems. Although much ofit relates to those
with special properties, some theory applies under relatively weak assumptions about the
system under study. For the simulationist, these properties can guide the choice of sampling
distributions and the estimation of their parameters. We discuss two widely applicable results:
the characterizations of interarrival times (Section 10.2.1) and of delay times (Section 10.2.2).

10.4.1 AN EXPONENTIAL LIMIT

Definition 10.1.
Let Xl, Xz, ... denote a sequence of i.i.d. nonnegative random variables and let

So:= 0,
Sn := Xl + ... + X n, n = 1,2, ....

Then {N(t) := sup(n : Sn :s t), t 2: O} is a called a renewal process. In particular, for

t > 0, N(t) denotes the number of events that occur in the time interval (0, t].

Definition 10.2.
Let {N 1 (t), t 2: O}, ... , {Nm(t), t 2: O} denote m independent renewal processes and let
{Nom(t) := Nl (t) + ... + Nm(t), t 2: O} denote their superposition. That is, Ni(t) denotes
the number of events oftypei that occur in (0, t], whereas Nom(t) denotes the number of events
of all types that occur. In general, {Nom (t)} is not a renewal process. However, in at least one
special case, it exhibits that property in the limit.

Foreachi = 1, ... , m, let Xii, X iZ , ••• denote a sequence ofi.i.d. nonnegative random
variables with d.f. {Fim(t), t 2: O}, and let

Ni(t) := sup(n : Sin :s t),

Aim := IjEXi1 ,
 10.4 . CHARACTERIZATIONS FROM THEORY 465

and

If A --+ constant as m --+ 00 and if for each m there exist E > 0 and t' > 0 such that

then (e.g., Heyman and Sobel 1982, p. 156)

as m --+ 00.

Since by definition

pr [Nom(t) = 0] = pr(Sl > t),

= probability that the next event occurs at time greater than t,

the limiting distribution (as m --+ 00) of the time to the next event is the exponential £ (1 / A).
Gnedenko and Kovalenko (1989) attribute this result to Grigelionis (1962).
For many open -loop delay systems, it is not untenable to characterize the arrival process
as the limit of a large number of independent renewal processes each with relatively small
arrival rate. As a consequence, the exponential distribution becomes a candidate for data
analysis aimed at fitting an interarrival time distribution. This does not mean that we should
consider only this distribution. It does mean that within the framework of data analysis a test
for the suitability of the exponential distribution offers us some protection from allowing
the empirical data, based on a finite sample size, to dictate exclusively the estimated values
of the parameters. Estimates based on theory together with data analysis inevitably result in
more thoughtfully fitted distributions.

10.4.2 HEAVY TRAFFIC

It is of interest to know how sensitive simulated output is to the choice of input

distributions as a function of the level of congestion (traffic) in a simulated system. A result
in K611erstrom (1974) provides a partial answer in the case of heavy traffic. A system with
traffic intensity p close to unity is said to be in heavy traffic. Let Tl , T2 , ••• denote Li.d.
interarrival times with mean t-tT and variance af. to an m-server queue using a first-come-
first-served discipline. Let Sl, S2, ... denote i.i.d. service times with mean t-ts and variance
a], p := t-ts/m t-tT,

._ t-tT (p2y] + yj.) + (1 _ p)2

(J) . - ,
2(1- p)
466 CHAPTER 10. PREPARING THE INPUT

aT
Ys and YT : = - .
J-Ls J-LT

If W denotes waiting time, then under relatively mild conditions on Si, T;, and Si - kT;, W/ ())
converges to a random variable from £(1) as p t 1 (K611erstrom 1974).
For p close to unity, this implies that the distribution of W is approximately £ (1/ ())),
where W depends only on the traffic intensity p, the coefficients of variation Ys and YT, and
is independent of the parent distributions of the interarrival times T\, T2 , ••• and the service
times S\, S2, ....
For this commonly encountered delay system, this result implies that as the system
approaches saturation, the dependence of at least one of its output distributions becomes less
sensitive to the form of its input distributions and more sensitive to their first two moments.
For input data analysis for heavy-traffic systems, the result lessens our concern about input
distributional shapes but encourages us to obtain estimates for their means and variances
that are as accurate as possible.

10.5 EXPERT OPINION

Just as the characterization of a systems's logic flow benefits from the expert opinion of those
most familiar with the real system, so can the specification of input characteristics benefit
from their knowledge. Combining that knowledge with statistical inferences drawn from an
analysis of historical data is the most likely recipe for successful input modeling. When no
data are available, we need to rely exclusively on expert opinion. An example shows how this
knowledge can be converted. into an input distribution.
Recall the example of Section 1.5 where drums were delivered daily to the incinerator
complex. No historical data were available for delivery times. However, incinerator operators
revealed that drums never arrived before 11 a.m. nor after 4 p.m. When pressed by the
simulationist as to the most likely time of arrival, they indicated 2 p.m.
With only these pieces of information, distribution fitting had limited possibilities. The
simulationist chose to fit the triangular distribution (Exercise 8.7). In particular, m\ = 11,
m2 = 14, and m3 = 16 after the times were converted to a 24-hour basis. As a consequence,

2(t - 11)/15, 11 ~ t ~ 14,

f(t) =\
(16 - t)/5, 14 ~ t ~ 16,
(t - 11)2/ 15, 11 ~ t ~ 14,
F(t) =\
1 - (16 - t)2/1O, 14 < t ~ 16.
 10.6. CHOOSING A DISTRIBUTION 467

In particular,

2
30'
t = 12,
6
30'
t = 13,
10
F(t) - F(t - 1) = t = 14,
30'
9
30'
t = 15,
3
30'
t = 16.
Sampling delivery times from this {F (t)} would result in almost two-thirds of delivery
occurring between 1 and 3 p.m. Since the incinerator operators were more certain about the
endpoints mI and m3 than the modal value mz, the simulationist chose to run at least two
additional simulations, one with mz = 13 and the other with mz = 15, and to compare the
output with that for mz = 14. Little variation in output would indicate that system response
was relatively insensitive to the choice of most likely delivery time.

10.6 CHOOSING A DISTRIBUTION

By fitting a distribution, we mean selecting a theoretical p.m.f. or p.d.f., estimating the values
of its parameters from data or expert judgment, and, in the data case, assessing how well the
resulting fitted d.f. approximates the corresponding e.d.f. Graphical comparison provides
one means of assessment; applying one or more statistical tests provides a second.
Commercial software is available for fitting. For example, BestFit (www.lithec.
skJukJdts_bestfit.htm), ExpertFit (www.does.orglmasterli/s3.htm), and Stat::Fit (www.
geerms.com/default.htm) each use empirical data to estimate the values of parameters for
a host of candidate distributions and provide graphical displays and test statistics to guide
the choice among them. Also, some simulation systems such as Arena provide a capability
for fitting distributions. Using this software saves the simulationist's time; but he/she must
ultimately make the choice among the candidates.
Unfortunately, goodness-of-fit tests for distributions tend to be considerably less
discriminating than desired. In particular, the standard approach tests Ho: a particular dis-
tribution fits the data, against the omnibus alternative HI: one of the many nonspecified
alternatives fits the data. This setup is known to have low power, meaning that the test
accepts Ho more often than it should, thus being less discerning than an analyst would like.
Moreover, some of these software applications rank the desirability of the host of candidate
distributions by the values of their test statistics on a particular test. This practice encourages
users to choose the best-performing distribution according to the ranking. However, little
basis exists in the statistical literature to support these ranking schemes. In the remainder
468 CHAPTER 10. PREPARING THE INPUT

of this chapter we concentrate more on graphical analysis to assess the adequacy of fitted
distributions. These are usually available from the software fitting applications or are easily
computed by more general-purpose applications such as Microsoft Excel.
For nonnegative data, the most difficult part of a distribution to fit is its right tail.
Graphical analysis may reveal discrepancies between the tails of empirical and fitted d.f.s,
but except in the case of an exceptionally large sample size, the ability of statistical tests to
reject based on these discrepancies is relatively low.
If a simulationist knows something about alternative tail behaviors, then her/his ca-
pacity to choose among alternative fits can be greatly enhanced. This section provides an
overview of these alternatives.

10.6.1 EXPONENTIAL TAILS

The Gamma distribution (Section 8.13 and Table 10.3) is a commonly fitted distri-
bution that has an exponentially decreasing right tail. It often provides a basis for fitting
interarrival and service time data. It has mean IL = af3, variance a 2 = af3 2 , mode at
x = max[O, f3(a - 1)]. It also facilitates hypothesis testing for exponentiality (a = 1)
within the context of standard statistical theory. Section 10.12.2 illustrates this feature. For
a = 1 its (finite) mode is at x = 0. For a < 1, its (infinite) mode is also at zero. For a > 1,
its (finite) mode is at some positive x. Table 10.2 lists the maximum likelihood equations
that need to be solved iteratively for this distribution.

10.6.2 SUB- AND HVPEREXPONENTIAL TAILS

The Weibull distribution (§8.9.1 and Table 10.3) provides an alternative basis for
fitting with mean f3r(1 + Ija), variance f32[r(1 + 2ja) - r2(1 + Ija)], mode at x =
f3 max[O, (1 - ja)l/a], and right tail that decreases as exp[ -(xj f3)a]. This last property
implies that the data, more than model selection, dictate tail behavior, being more rapid
a a
than exponential if > 1 and less if < 1. The Weibull also facilitates hypothesis testing
for exponentiality (a = 1) within the context of the likelihood ratio test (Section 10.12.2).
The location of its mode follows analogously to that for the Gamma. Table 10.3 lists the
maximum likelihood estimates that need to be solved iteratively for the Weibull.

10.6.3 LOGNORMAL DISTRIBUTION

Fitting data to an exponential, Gamma, or Weibull p.d.f. implicitly restricts its mode
and right tail. For exponential fitting, the mode is at zero and the tail exponential. A Gamma fit
with a < 1 puts the (infinite) mode at zero and retains the exponential tail. A Weibull fit with
a < 1 again puts the (infinite) mode at zero but makes the tail diminish subexponentially. A
Gamma fit with a > 1 puts the mode at a positive value and continues to give an exponential
tail. A Weibull fit with a > 1 puts the mode at a positive value but makes the right tail decrease
hyperexponentially.
 10.6. C H 00 SIN GAD I S TR I BUT ION 469

Occasionally, the empirical histogram reveals tail behavior that does not fit one of these
profiles. This is especially true when the empirical histogram shows the mode at a positive
value and a right tail that decreases at less than an exponential rate. If a simulationist were to
ignore this discrepancy between the e.dJ. and a fitted exponential, Gamma, or Weibull d.f. the
resulting sampling sequence during a simulation experiment would show proportionately
fewer large right tail values than the empirical evidence suggests. Accordingly, we need to
look at other options. This section discusses the lognormal distribution and §10.6A the
inverted Gamma and inverse Weibull distributions.
From Section 8.13, a random variable from the lognormal distribution eN (f-L, a 2) has
mean exp [-(f-L+a2 /2)], varianceexp(2f-L+a 2) x [exp(a 2)-I],modeatx = exp(f-L+a2/2),
and right tail whose dominant terms decreases as exp[-(lnx)2/2a 2] as x increases. Most
notably, tail behavior is slower than exponential in x, thus providing a candidate for data
with a "heavy" tail.

10.6.4 I~VERTED GAMMA A~D I~VERSE WEIBULL DISTRIBUTIO~S

While an alternative to subexponential, exponential, and hyperexponential decrease,

the lognormal distribution can place so much probability mass in the right tail that the
fitted distribution compares poorly with the e.dJ. in other regions. The inverted Gamma
distribution (Section 8.13.1) and the inverse Weibull distribution (Section 8.9.2) provide
alternatives with nonexponential tails that may allow for a better fit.
For ct > 2, the inverted Gamma has finite mean 1/(ct - 1),8 and variance 1/(ct -
2)(ct - 1)2,82. It also has positive mode at 1/(ct + 1),8, right tail that decreases as x-(a+l) as
x increases, and a flatter left tail than the Gamma.
For ct > 2, the inverse Weibull distribution has finite mean r (1 - 1/ct) / ,8 and variance
[r(l- 2/ct) - r2(l- 1/ct)],82. It also has positive mode at I/O + l/ct)l/a,8, right tail that
decreases as x-(a+O as x increases, and left tail that is flatter than that for the Weibull.
The inverted Gamma and inverse Weibull distributions can provide better fits than the
Gamma and Weibull distributions for data whose empirical histogram tends to decrease at a
polynomial rate. Figure lOA shows the difference among these distributions for a common
mean f-L = 2.6 and variance a 2 = 3.38. In particular, the p.dJ.s in Figure 10Aa reveal the
differences in left tail and modal behavior. Figure lOAb shows the exceedance probabilities.
It reveals that the y tend to be larger for the inverted Gamma and inverse Weibull for both
small and large x but smaller (in this example) for x E [2,3].
Figure WAc shows the left-tail exceedance probabilities

pr[X > x I X ::: x(.99)],

where

x(1]) := inf[x : F(x) ::: 1]], 0< 1] < 1,

470 C HAP TE R 1 0 . PRE PAR I N G THE I N PUT

and F is the d.f. In the present case with 1] = .99,

for 9 (2, 1.300),

r
9.5 for Ig (2.4, .1282),
x(.99) =
9.0 for W (2.953, 2.914),
4.9 for IW (2.938, .5257).

In addition to revealing the wide disparity among the .99 quantiles, Figure 1O.4c shows
that although samples are generated from this part of each distribution no more than once
per hundred trials on average, when such an outcome X does occur the values of X tend to
be considerably larger for the inverted Gamma and inverse Weibull distributions than for
the Gamma and Weibull. We can easily recognize the significance of this property for service
times. In some repair processes, the vast majority of items require moderate repair times,
but occasionally an item requires an excessively large repair time.
For completeness, recall that the Beta-prime distribution of Section 8.14.1 also has a
right tail that decreases as a polynomial in x. We leave to the reader the exercise of showing
that this distribution has J.L = fJ/(a _1),0'2 + J.L2 = fJ(l + fJ)/(a - 1)(a - 2), and, for the
particularJ.L = 2.6 and 0'2 = 3.38,a = 4.7692andfJ = 9.800. A graph of the corresponding
p.d.f. is of interest when compared to those in Figure 10.4.

10.7 SUPPORT

Let X denote a continuous random variable with p.d.f. or p.m.f. Ix. The support of X,
supp(X), denotes all points in mfor which fx(x) is positive. For example, Z '" 9 (a, fJ)
has supp(Z) = [0, (0), whereas X := a + bZ, for b > 0, has supp(X) = [a, (0). If
Z '" Be(a, fJ), then supp(Z) = [0,1], whereas X := a + (b - a)Z, for b > a, has
supp(X) = [a, b].
Given a sample sequence Xl, ... , Xn for which supp(Xi ) = [a, (0) for known a i= 0,
one customarily works with the transformed data, X; := Xi - a, i = 1, ... , n, when
fitting a distribution. Ifsupp(X i ) = [a,b] forknownb > a,thenX;:= (Xi -a)/(b-
a), i = 1, ... , n, has support [0, 1]. If unknown, the support must be estimated prior to
estimating, or along with, the values of the parameters of the selected distribution. Clearly,
a ::::: minl:9::on Xi and b 2: maxl::oi::on Xi. When expert opinion indicates finite support
[a, b] and gives estimatesG and b satisfyingG ::::: minl::oi::on Xi andb > maxl::oi::on Xi, it is not
unusual to fit X; := (Xi - G)/(b - G), i = 1, ... , n, to the beta distribution Be(a, fJ). An
approach, more firmly grounded in theory, simultaneously fits a, b, a, fJ using the maximum
likelihood method (e.g., Johnson et al. 1994a and 1994b).
When b is assumed infinite, similar approaches are used with X; = Xi - G, i =
1, ... , n, and the exponential, Gamma, Weibull, and other distributions. As an example,
 1 0.7. SUPPORT 471

(a) p.d.f..
0.8 ' ' ' ' . - - - - - - , . . - - - - - - -....''.... ,.."'''''....., , - - - - - - - - - - - - ;

0(2. 1.3)
-1O(4_12J2)
0-5
-W(1.9n.2-914)
- IW(2.9JlI,>U...'1

0.4

0.3

0.2

0.1

o 2 4 8 8 10 12

(b){pr(X >x)}

0.9

0.8

0.7

0.8

0-5

0.4

0.3

0.2

0.1

8 10 12
x
(0) {pr[X >xlX >x(.99)]J . x(.99):=.99quantile
1 "--------,-~.,-----------

0.8

0.8

Figure 10.4
Selected distributions
with {l = 2.6 and
(72 = 3.38
5 8 7 8 9 10 11 12 13 14 15
x
472 C HAP TE R 1 0 . PRE PAR I N G THE I N PUT

suppose that the Xi have the p.d.f.

fx(x) = Ae-A(x-a), A> 0, x::::: a::::: O.

Then it is easily seen that the m.l.e. of a and A are, respectively,

fj = min Xi
l~i~n

and

A 1
A= -
n
L (Xi - fj).
n

i=1

For discrete data, comparable techniques can be used with the binomial distribution if the
support is finite or with the Poisson, negative binomial, or other distributions, if the support
is infinite.

10.8 MAXIMUM LIKELIHOOD ESTIMATION

Many alternative methods exist for estimating the values of the parameters of a distribution,
each having its own virtues and limitations. The present account focuses on the method of
maximum likelihood principally because of the optimal large-sample statistical properties
with which it endows its estimators, its well-developed theory for testing hypotheses about
parameter values, and in to day's computing environment the relative ease of computing by
exact or iterative methods the estimated values of parameters of distributions of interest
from sample data.
Let {f(x; 81 , ••• , 8q ), x EX} denote a p.d.f. or p.m.f. with x taking values in the
support X and with unknown parameter vector 8 := (81 , ••• , 8q ) taking a value in some
set e. Let X := (X I, ... , Xn) denote a vector of n i.i.d. observations from f. Then

denotes the corresponding likelihood function, and (j, an element of e satisfying

L(O, X) = sup L(8, X).

OEE!

By way of interpretation, 0 is the value that maximizes the likelihood of observing the given
sampleX.
For the distributions we consider here, all of which are described in Chapter 8, the
maximum likelihood estimator (m.l.e.) 0 of 8, has the folloing important properties:
 10.8. MAXIMUM LIKELIHOOD ESTIMATION 473

• Uniqueness. 8 is unique
• Strong Consistency. 8 --+ 0 as n --+ 00 w.p.1

• n 1/2 8 has covariance matrix

-I

Lql ... Lqq

where

i, j = 1, ... , q

• Effeciency. If 8 := (OJ, ... , Oq) is an alternative estimate of 0, then

lim ::: I, i=I, ... ,q,

n~oo

n var Oi being the ith diagonal element of ~(O)/n, where

~(O) := lim ~n(O)

n~oo

• Asymptotic Normality. n l / 2 (8 - 0) ~ N (0, ~(O» as n --+ 00

• Invariance. For any function, {h (O)}, {h ( 8)} is its m.l.e., thus inheriting the properties
of strong consistency, efficiency, and asymptotic normality.

In large samples, efficiency implies that no alternative estimator of (}i has smaller
variance than the m.l.e. Asymptotic normality provides a distribution theory for assessing
how well 8 approximates O. In particular, for each i [Oi ± <1>-1 (1 - 8/2)s.e.(Oi)] with

1/2
s.e.«(}d := (var (}d
A A

provides a 100 x (1 - 8) percent confidence inteval for (}i, ifvar Oi were known. Since it is
not, we replace it with the i th diagonal element of ~ (8) / n, which by the invariance property
is the m.l.e. of var Oi and therefore inherits the property of efficiency. That is, for large n no
alternative estimator of var Oi has smaller variance.
474 CHAPTER 10. PREPARING THE INPUT

The relative error

1/2
r.e.(ei ) (var ei ) lei
A._ A

.-

e;
provides an additional measure of how well approximates ei •
These m.l.e. properties hold for any distribution representable in the form

f(x, 0) = exp [A(x)B(O) + C(x) + D(O)] ,

where A, B, C, and D are arbitrary functions. Collectively, they define the class ofDarmois-
Koopman distributions (e.g., Johnson et al. 1994a). Virtually all distributions discussed in
this chapter admit representations of this form.

o EXAMPLE 10.1

Let XI, ... , Xn denote i.i.d. observations from £(f3) with p.d.f.

f3 > 0, x::: o.

Then q = 1 and e l = f3, so that

n
InL(O,X) = -nlnf3 + r l LXi,
i=1

1~
el = f3 = -
A A

~ Xi,
n i=1

varf3
A

= f3 2 In,
1/2
s.e.(f3) = f3ln ,
A

and

A s.e.(~) 1
r.e. (f3) := -f3- - nl/2'

By the invariance property, ~ Inl/2 is the m.I.e. of s.e.(~).

o
 10.8. MAXIMUM LIKELIHOOD ESTIMATION 475

Table 10.2 lists selected continuous and discrete distributions whose m.l.e.s are avail-
able in closed form. Table 10.3 lists selected continuous and discrete distributions whose
m.l.e.s are computed by iterative methods. In particular, Algorithm N -R in Section 8.1.1 de-
scribes the Newton-Raphson method for finding the m.l.e. for one parameter using {k(z)}
and {k'(z)} in Table 10.3 for the Gamma, Weibull, and negative binomial distributions.
Then the rightmost column gives the form of the second parameter estimator. For example,
Algorithm N-R determines & for the Gamma distribution and ~ = A/&. An error bound
v .::: 10-6 is recommended for error tolerance in the algorithm.
e.
Tables 10.3 and 10.4 also list the large-sample form for var In the remainder of this
chapter these quantities are evaluated with vare replaced by its m.l.e.
All iterative computations rely on the availability of procedures for computing the
digammafunction {1/I(z) := dlnd~(z)} and the polygamma function {1/1(1) (z) := d~~z)}. These
are available in Mathematica and Mathcad. Since these are slowly converging infinite series,
we encourage a user of the iterative procedures in Table 10.3 to employ one of these packages
rather than to write his or her own procedures for these functions from first principles.

o EXAMPLE 10.2

In fitting the Gamma distribution to the service-time data with empirical histogram
in Figure 10.2, the m.l.e. are

& = 3.028, ~ = .7138,

s.e.(&) = .4069, s.e.(~) = .1043,
r.e.(&) = .1344, r.e.(~) = .1461,
corr(&,~) = -.9194.

Figure 10.5a compares the exceedance probabilities for the e.dJ. in Figure 10.2 with
those for the fitted dJ. Figure 1O.Sb shows their difference, and Figure lO.5c compares the
conditional right tail exceedance probabilities. The first two graphs reveal a relatively good fit
for just n = 100 observations. In particular, Figure 1O.5a shows that the fitted dJ. introduces
smoothness that is notably absent in the e.dJ.
Because X(i) and X(n) are finite, sampling from an e.dJ. limits its sample values to
[X(i), X(n)l. This can be a serious limitation when X is believed to have infinite support on
the nonnegative line or at least an upper bound much larger than X(n). Figure 1O.Sc shows
how the fitted d.f. overcomes this limitation of the e.d.f.
 ~
~
(")
I

distribution mean variance mode m.Le. properties ):~ »

"0
0" -t
"Ii ;; m
A u2 ::I
....
Q
normalN (/L, 0"2) 0"2 (L=A E(L = /L var /L = -;;
'"
/L /L !.N
II _
Mt/) o
--;:3' ("D
-00 < /L < 00 a2 = B _ A2 M2=n~I(J'2 var 8- 2 = 20"4 corr«(L, a2 ) = 0
- n>
><~ ""0
-. n>
0"2> 0 - p.. '"m
b:lp.. "0
•• S;"
II ;::t »
-00 < x < 00 ::I 6' '"Z
!.g
Gl
a+b (b_a)2 var a= n(b-a)2 Mg·
uniformU (a, b) 2 I2
a= minX, Eli = a + ~~~ (n+1)(.+2) corr(a, b) = ~ -t
[~~ I
><::r m
A n(b-a)2 -'IV 0
-00 <a < b< 00 b= max X, Eb=b- b-a
n+1 var b = (n+1)(.+2)
- '"
~n> Z
- S "0
a::;:x::;:b C
"Ii i;;" -t
::I •

f32 ~~
exponential £ (f3) f3 o P=A EP = f3 varp = ~ MI»
n· ~ g,
f3>O -5" g:
x:=:O
:>< s·
b:lc.
- 0
2 ·Ii 8.-
lognormal CN (/L, 0"2) e"+u 2 /2 e2JL+u2 (eu2 - 1) e"+u 2 /2 (L = A' E(L = /L var(L=~
::II 8'
-..,
-00 < /L < 00 8- 2 = B' - (A')2 E.a 2 =R:IU 2 var u1 = 20"4 corr«(L, a2) = 0 MS
n -. ~

!!.
0"2> 0 S
~
->< N
x:=:O ~
 distribution mean variance mode m.l.e. properties Si
g:
p(I;:P)
Bernoulli J3er(p) p p(l- p) Oifp < t p=A Ep=p var p =
...?n
N

O::;p::;1 lifp2:t 8'

=
c.
x = 0,1 ~
=
n
E::
Poisson P (>..) >.. >.. >.. - 1 and>.. i=A Ei=>.. vari=~ -"
0
OJ
>">0 if L>..J = >..,

L>..J otherwise
~
»
x = 0,1, ... x
-
3:
c
geometric ge(p)
3:
r-
A A
O::;p::;1 ....L 0 Ep--+p nvar p --+ p(1 _ p)'
I-p (I_~)2 p = I+A /<;
m
as n --+ 00 as n --+ 00 I
0
X = 0,1, ... 0
0

m
(J)
-I

3:
»
-I

0
Z

~
....
....
 ~
""
.... G'l~~ (")
distribution mean variance mode I
~~
N' N' ',', no
Gamma Q(a, f3) af3 af32 max[O, (a - 1)f3] •.•. ;:s
e:.... >
."
a, f3 > 0 II II I Q --I
;:sI ;:sI M- t.. m
x~O
'"
.......
Weibull W(a, f3) max[O, f3(1 - l/a)l/a]
MMr~ o
f3r(1 + l/a) f3 2[r(2 + l/a) - r2(1 + l/a)] [~[ ~ ->s a
a, f3 > 0 ~><:- 8.-
-;::::-'N C"':l Q..
x~O -0
i: S" H la' '"m
..!L rp + :.><:;:s;a, ."
negative binomial NB(r, p) (l_p)2 wand w + 1 if LwJ = w !.. ~
I-p >
r > 0, 0 ~ p ~ 1 Lw J otherwise N -. ~
~~Mg·::s '"z
-II (I)
x = 0,1, ... w := (rp - 1)/(1- p) G)
',i -S" ~
--I
;:s ><: ~ I
!..:. '" m
asymptotic properties M tl ?
.... ;:1 •• ~
Z
distribution variances correlation !!. II!' ."
~ C
A -'N (1)
p2
><: ;:sI
A a 1 --I
Gamma vara ~ [a1fr l l)(a)-I[n varf3 ~ 1fr(1)(a)[a1fr(1)(a)-I)n ~ - a1fr(1) (a) '0 -~
::s M-.
><:-.~ti1
Weibull vara ~ 1.109(f3/a)2/n varfi ~ .608a 2 /n ~ .3812 ::::....!!. ::;."
.. N 5'" ~
.... ~I»
A ~ 1___ -p - ....... t:!".
negative binomiala varr ~ M(r,p)n varfi ~ ! [P(I+P) + ~] ~ [ 0
n r r 2M(r,p) ~ M(r,p)r[ var, varpf/2 •• ::s
II.?<~
;:s - ••
!..~ II
iterate second MN' ;:Sl
distribution onz = k(z) k'(z) estimator ~'~~M­
A A - II
Gamma a 1/f(l)(z) - ~ ~;:s TI'~
1/f (z) - In z + In A - C f3 = - ~ I-
z a N _ ><:
! _ G(z) +c 1 H(z)
+M:'
Weibull a E(z) E(z)
fi = [E(a)j1/a _.
><: -. ~ II
z E(z) - zr - + [ G(z) r
A A 7'=~
negative binomial r In( z~A) + /(z) -1/f(z) ~ - Z~A + J(z) -1/f(l)(z) P=-A-
r+A
j+2
aM(x, Y):= L~o Bj~r> (Jiy) ,where B denotes the beta function.
 10.8. M A X I MUM LIKE L I H 00 DEs TIM A T ION 479

(a) Estimateaf (pr(X >x»)

0.9 - l-e..d.f
0.8 - I · f'i l~cd d.r.

0.7

0.6
0.5
0.4
0.3
0.2
0.1

o o~------------------~~==~==~
3 4 5 6
X

(b) Empirical- fined

0.01

- 0.01

- 0.03

(e) Estimate of (pr{X > x I X > x(.99»)). x(.99):= .99 quantile

0.9 --- c.d.f.

- - filtedd.f.
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
O L---------__________________________ ____
3.5 4 4.5 5 5.5 6
x

Figure 10.5 Service-time e.dJ. and fitted dJ.

10.8.1 BETA DISTRIBUTION

For n observations Xl •... , Xn from the Beta distribution Be(a, (3) (Section 8.14),
the logarithm of the likelihood function is

In L(a. f3. X) = n [In rea + (3) -In rca) -In r(f3) + (a - l)C + (f3 - l)D] ,
480 CHAPTER 10. PREPARING THE INPUT

where Table 10.3 defines C and D. The challenge is to solve

1fr(&; + ~) - 1fr(&;) + &; C = 0,

1fr(&; + ~) -1fr(~) + ~ D = 0,

simultaneously for the m.l.e. &; and~, where 1fr denotes the digamma function. This presents
more formidable problems than solving for the other m.l.e.s in Table 10.3.
Bivariate search provides one approach, starting with the initial values

Ot := (A - B)AI(B - A2)

and

S := (A - B)(1 - AI(B - A 2),

the moment estimators ofa and fJ (Fielitz and Myers 1975). Familiarity with bivariate search
techniques is necessary.
A second approach reduces the problem to a univariate search for &; by combining the
likelihood equations to obtain (Beckman and Tietjen 1978)

1fr(&;) -1fr(1fr-1(D - C + 1/I(a» + &;) - C = 0,

where 1/1-1 denotes the inverse of 1/1, and then computes

Since mathematical-applications software rarely provides for the direct computation of

1fr -1 ( . ), this approach also has a limitation.
A third approach approximates &; and ~ by relatively gross interpolation in a table of
selected values of C and D (Gnanadesikan et al. 1967). The table is reproduced in Fishman
(1973, p. 359). Law and Kelton (1991, pp. 411-413) contains a more extensive table.
A fourth approach again resorts to bivariate search using entries in the aforementioned
tables to localize the region to be searched.
A fifth alternative uses the moment estimators Ot and Sin place of the m.l.e. Since var
Ot ::: var&; and var S ::: var ~ for large n, this approach produces less accurate estimates.
However, it does offer the convenience of being easy to compute.
For large n, &;, and ~ have

A 1
vara ~ 1fr(l)(a + fJ)[IIQ(fJ, a) - Q(a, fJ)]n'
A 1
var fJ ~ 1/1 (I) (a + fJ)[IIQ(a, fJ) - Q(fJ, a)]n'
 10.8. MAXIMUM LIKELIHOOD ESTIMATION 481

A -1
corr(a, (3) ~ [Q(a, (3)Q(f3, a)]l/2'

where

10.8.2 GEI'olERALIZED BETA DISTRIBUTIOI'ol

The Beta distribution of Section 8.14 is a special case of the generalized Beta
distribution

f(z) = r(a + (3) . (z - zd a - I (Z2 - Z),B-I ,

Zl ~ Z ~ Z2'
r(a)r(f3) (Z2 - Zda+,B-I

Ifzl andZ 2 are known exactly, then one transforms Xl, ... , Xn to {(Xi -Zd/(Z2 -Zl); i =
1, ... , n} and proceeds as above. If Zl and Z2 are unknown, estimation becomes more prob-
lematic. In particular, the m.l.e.s for a and f3 are sensitive to values assumed for Zl and
Z2' AbouRizk et al. (1994) provide software called BetaFit for estimating a, f3, Zl> and Z2
by several different methods. The reference shows that least-squares estimation generally
gives as good and often a better fit than the m.I.e. VIBES, accessible on the Internet at
http://www.ie.ncsu.edu/jwi1son/page3.html. provides the software for BetaFit.

10.8.3 BETA-PRIME DISTRIBUTIOI'ol

To estimate the m.I.e. of the Beta-prime distribution, we make use of its relationship
to the Beta distribution. Recall from Section 8.14.1 that if Z is from Be(a, (3), then X =
Z-l -1 is from the Beta-prime distribution with support [0,00) and right tail that decreases
polynomially. To estimate a and f3 for this distribution for sample data Xl, ... , Xm one
replaces Xi by 1/(1 + Xi) for i = 1, ... ,n everywhere in the adopted m.I.e. procedure for
Be(a, (3).

10.8.4 II'olVERTED GAMMA AI'olD II'olVERSE WEI BULL M.L.E.S

Maximum likelihood estimators are easily computed for the inverted Gamma distri-
bution (Section 8.13.1) and the inverse Weibull distribution (Section 8.9.2) by exploiting
their relationships with the Gamma and Weibull distributions respectively. Recall that if X
is from rQ(a, (3), then 1/ X is from Q(a, (3). Accordingly, the m.l.e.s a and fi for rQ(a, (3)
are obtainable using {k(z)} and {k'(z)} for the Gamma distribution in Table 10.3 with the
revised definitions
n

Li=l X-I
I
482 CHAPTER 10. PREPARING THE INPUT

and
n

Li=l X:-I 2 •

In an analogous manner, the m.l.e.s a and fi for IW(a, fJ) are obtainable using
{k(z)} and {k'(z)} for the Weibull distribution with

n
E(z) := n- l Li=l X- z
i '
n
G(z) := n- l Li=l Xi z In(Xi l ),

and
n
H(z):= n- l Li=l XiZ[ln(Xil)]2.

10.9 ADEQUACY OF FIT

Previous sections encourage the reader to rely on graphical comparison to assess the adequacy
of fit of a particular distribution. The approach allows the simulationist to spot discrepancies
between fitted and empirical curves, be they dJ.s or densities. The magnitude of these
discrepancies, their locations, and the use to be made of a curve provide a basis for deciding
adequacy. For example, a particular fitted distribution may show substantial deviations from
the e.dJ. in a region. If, when used to generate samples, this part of the fitted curve comes
into playa relatively small percentage of the time, the error may be tolerable.
While certainly an informal approach, graphical comparisons are easy to effect via
commonly available spreadsheet software such as Microsoft Excel. Ease of use gives the
approach its appeal. A more formal assessment uses goodness-oJ-fit tests. For a sample i.i.d.
sequence Xl, ... , Xn and selected distribution F, one tests the hypothesis H': The data
Xl, ... , Xn come from F. If T(X l , ... , Xn), the test statistic for a particular goodness-
of-fit test, has d.f. G n under H', then H' is accepted at the user-specified significance level
8 E (0, 1) if 1 - Gn(T(Xb ... , Xn» ::::: 8. Otherwise, it is rejected.
Conover (1999) describes goodness-of-fit tests, including chi-squared, Kolmogorov-
Smirnov, and Anderson-Darling. Spreadsheet and distribution-fitting software packages
commonly offer one or several of these tests.
A goodness-of-fit test provides a single number on the basis of which one accepts
or rejects H'. While this yes-no approach has a natural appeal, it can at best be used as a
supplement but not as a substitute for graphical comparison.
 10.10. ATTRIBUTES AND FITTING DISTRIBUTIONS 483

Several dJ. and density-fitting software packages recommend or at least encourage

the use of a fitted distribution whose test-statistic value on a particular test implies a higher
rank than the values of the corresponding statistic for, say, N - 1 other fitted distributions,
where N varies with the package. Experience indicates that reliance on this comparison of
test-statistic values in no way replaces the benefit to be gained from graphical comparison
of a fitted dJ. with the corresponding empirical dJ.

10.10 ATTRIBUTES AND FITTING DISTRIBUTIONS

Recall that Chapter 2 focuses on entities of delay systems and on their attributes. The values
that the attributes assume frequently influence the dynamics of the simulation. For example,
they can affect decision-flow logic for:

• Routing that temporary entities follow as they flow through the system

• Sequencing of jobs (entities) for service

• Matching resources with workload

• Scheduling of resources.
They can also affect sample averages collected during a simulation. For example, management
decision-making may require estimates oflong-run averages and exceedance probabilities
for waiting time and queue length for jobs partitioned by attribute values.
Whenever any of these issues arise in a simulation study and attibute values affect
the values that interarrival and service times assume, relying on distributional fitting as in
Section 10.8 runs the risk of seriously misrepresenting system dynamics within the corre-
sponding simulation model. Section 10.10.1 addresses this problem. When none of these
issues arise but attribute values nevertheless affect inter arrival and service times, the fitting
of unimodal distributions as in Section 10.8 may again be inappropriate. Section 10.10.2
addresses this topic.

10.10.1 AnRIBUTES AFFECT DECISION FLOW

If an attribute affects decision flow and also interarrival and service times, then, to
ensure the fidelity of the system's logic, the corresponding simulation program must incor-
porate procedures for explicitly sampling these quantities from distributions conditioned on
attribute values. For example, if customers in the airline reservation problem of Chapter 2
were selected for service based on type, then this mandate would apply. We use the service
time distributions for that problem to describe the issues that arise in estimating parameter
values.
In the airline reservation problem, recall that lOOp percent of callers want multides-
tination reservations and are assumed to have service times from the Erlang distribution
484 CHAPTER 10. PREPARING THE INPUT

Q(2, 1j w). Also, 100(1- p) percent of callers want a single-destination reservation and have
service times from £(Ijw). Suppose that the service-time distributions are unknown and
need to be estimated.
Three data-analysis scenarios are possible:

1. The proportion P is known and service-time data are available by type of customer.

2. The proportion P is known and service-time data are not identified by type.

3. The proportion P is unknown and service-time data are available by type.

4. The proportion p is unknown and service-time data are not identified by type.

When Scenario 1 obtains, a service-time distribution is fitted separately for each type
of data. Given the value of p and the two fitted distributions, a simulationist now has the
essentials for generating service times by type in the simulation.
If the historical data in Scenario 3 contain n I service times for single-destination
callers and n2 service times for multiple-destination callers, then one estimates p by P =
nz/(nI + n2) and fits the two service-time distributions as with Scenario 1.
If p is known but the historical data do not reveal type (Scenario 2), estimation becomes
much harder. Suppose we want to fit Q(aI, fh) to service times for single-destination callers
and Q(a2, f32) to service times for multiple-destination callers. That is, we want to estimate
aI, f31> a2, f32 in the mixtureofp.d.f.s

xa2-Iex/fJ2
+ p f3~2f'(a2)'
x ::: 0,

which is the p.d.f. of a randomly selected customer. Since we cannot identify service times
by type, the likelihood function for the service-time data X I, ... , Xn becomes

Finding aI, f31> a2, and f32 that maximize L is a considerably more challenging nonlinear
optimization problem than what Section 10.8 describes. The severity ofthe problem increases
if p is unknown and also has to be estimated as in Scenario 4.
The airline reservation problem with unknown service-time distribution merely il-
lustrates a more general data-fitting problem that arises whenever the data are not available
in a partition that reflects how a critical attribute or attributes influence the data values.
Inevitably, the challenge is to estimate the values of parameters in a mixture of distributions

f(x) = PI!I(X) + P2fz(x) + ... + prfr(x),

Pi> 0, i = I, ... , r,
 10.11 . DEPENDENT HISTORICAL DATA 485

P + ... + Pr = 1,
where Pi denotes the proportion of type i and It, ... , Ir are p.d.f.s or p.m.f.s. Whenever
Scenarios 2 and 4 prevail, convenient estimation via the maximum likelihood method is
usually not possible.

10.10.2 ATTRIBUTES DO NOT AFFECT DECISION FLOW

When attributes that affect service times do not influence decision flows, one can, in
principle, generate these times either by attribute type from the conditional distributions
11, ... , Ir or directly from the unconditional distribution I. Both options induce sample
paths during simulation execution that lead to sample averages that converge to correct
limits. However, data availability again limits the choice of options.
If Scenario 1 or 3 prevails, the simulationist is well advised to fit It, ... , Ir separately.
Since they are more likely to be unimodal than the unconditional distribution I, the method
of Section 10.8 for entries in Tables 10.2 and 10.3 applies to directly fitting It, ... , Ir more
often than to I. When Scenario 2 or 4 prevails, fitting 11, ... , Ir in this way is not possible.
Moreover, using the maximum likelihood method to fit I presents a formidable problem.
An alternative fits the data to a possibly multimodal distribution from which samples
can be drawn relatively directly during simulation execution. In principle, this approach
puts the mixture issue aside and concentrates on curve fitting. Wagner and Wilson (1996)
offer Windows-based software, PRIME (Probabilistic Input Modeling Environment), to
fit the data to a continuously differentiable I formulated by exploiting the properties of
Bezier curves. These are a special class of spline curves whose mathematical and numerical
properties are well suited to simulation input modeling.
PRIME offers a user the choice optimizing criteria for fitting the resulting Bezier
distribution. These include the methods ofleast squares, minimum L1 norm, minimum
Loo norm, moment matching, percentile matching, and maximum likelihood. A limita-
tion of PRIME is that it assumes bounded support. PRIME is available on the Internet at
http://www.ie.ncsu/jwilson by clicking on "software."

10.11 DEPENDENT HISTORICAL DATA

When XI, ... , X n , the historial data to be used to fit a distribution, are not i.i.d. the method
of Section 10.8 does not apply. A departure from the i.i.d. assumption occurs if either

XI, ... , Xn are not strictly stationary

or

Xi> ... , Xnare strictly stationary but dependent (Section 6.5).

486 CHAPTER 10. PREPARING THE INPUT

Since nonstrictly stationary input phenomena are inconsistent with a simulation method-
ology aimed at deriving estimates oflong-run averages, we ignore this case and concentrate
on stationary but dependent input phenomena. In what follows, we describe dependence as
it relates to an arrival process. However, the applicability of the concepts to a more general
setting should be apparent to the reader.
If dependence exists among the observations in a historical record of interarrival
times that one plans to use to create an interarrival-time generator in a simulation model,
neither does the method of Section 10.8 apply for fitting the data to a distribution nor
does the sample-generating procedure for i.i.d. interarrival times apply. To overcome these
impediments, we partition our discussion into two parts. Sections 10.11.1 and 10.14 address
problems in which the data are all functions of some exogenous phenomenon such as time.
Section 10.11.2 addresses the problem in which dependence in the data is tied to considerably
more localized considerations.

10.11.1 TIME DEPENDENCE

Problem setting often prompts a simulationist to suspect dependence in historical

data. For example, the simulation of the photocopying problem in Section 1.4 focuses on
customer arrivals between 10 a.m. and 5 p.m. on weekdays. Elementary considerations
about library traffic encourage the suspicion that the arrival rate may vary by time of day
and possibly by day of week. Therefore, data on arrivals need to be tested to confirm or reject
this suspicion. Section 10.12.1 does this by resorting to the standard statistical tools of the
analysis of variance for testing.
Dependence related to time of day, day of week, week of month, and month of year
frequently exhibits a periodic pattern. Section 10.14 describes how the concept of a non-
homogeneous Poisson process can capture the essential features of this time dependence in
fitting data and then generating interarrival times. When studying the testing procedures
in Section 10.12.1 and modeling concepts in Section 10.14, the reader is encouraged to
recognize that these ideas apply to a considerably wider range of problems in which time
dependence is suspected.

10.11.2 LOCAL DEPENDENCE

The commonly encountered concept of arrivals to a delay system is that they arise
from a large number of independent sources each of which contributes at a relatively small
rate. However, other arrival patterns do occur. For example, it may be that short interarrival
times occur in clusters, merely reflecting an institutional feature of the external world that
provides input to the system being simulated.
How does dependence among interarrival times effect queueing behavior? If depen-
dence takes the form of positive correlation and a procedure ignores this property but
nevertheless generates interarrival times from the correct marginal distribution, the result-
ing sample paths for queue length and waiting time tend to understate the level of congestion
 10.11 . D E PEN DEN T HIS TOR I CAL DA TA 487

in the system. When suspected, the first task is to test for dependence. Testing the data for
a flat spectrum using the periodogram and Kolmogorov-Smirnov test provides a compre-
hensive approach (e.g., Brockwell and Davis 1991, p. 339). Rejection implies the presence
of dependence. Whenever applications software is available to perform this test, the sim-
ulationist is well advised to take advantage of it. When not available, other less powerful
procedures are occasionally used.
One approach relies on the von Neumann ratio (6.65)

with a two-sided test. That is, one rejects the hypothesis of independence H at the 100 x 8
percent significance level if

Cn < -h(8, n) or Cn > h(8, n),

where

h(8, n) := <1>-1(1 - 8/2)J(n - 2)/(n 2 - 1), n > 2.

As Section 6.6.8 indicates, this procedure tests the hypothesis PI = 0, where PI denotes
the autocorrelation oflag 1 in the sequence (Section 6.5). If, in reality, the data are dependent
with a non-monotone-decreasing autocorrelation function (6.6), then it is possible that PI is
close to zero but subsequent auto correlations, Pj, I j I> 1, are substantially different from
zero. This consequence limits the value of the procedure.
To overcome this limitation, a third procedure uses the sample autocorrelations

_1_ ",n-j - )( - )
n-j L...d=1 (Xi - Xn X i + j - Xn
Pj = -"-----;:-::=::------=----- j = 1, ... , j*,
1 "'~ (X., - X n )2
n L..,,=I

for some user-selected lag j* « n. If the data are independent, then under relatively general
conditions

as n --+ 00.

We describe one ofseveral ways ofexploiting this limiting result. For some a E (0, 1), suppose
a simulationist regards a lagged correlation Pj E [-a, a 1as ofnegligible importance. If A j (a)
denotes this event, then for large n
488 CHAPTER 10. PREPARING THE INPUT

As an example, a = .05 and n = 1000 gives

pr[Aj(a) I H] ~ 2<1>(.OS.J1000) -1 = .8862,

implying that under H we expect to encounter I Pj l:s a about 88.62 percent of the time.
If j* = 100 then the mean number of exceedances for PI, ... , PIOO is 100 x (1 - .8862) =
11.38. If a = .05, n = 2000, and j* = 100, the mean number of exceedances under H is
100(1 - .9746) = 2.54. Observing no more than the average number of exceedances gives
credibility to H.
Sometimes a significance level a is specified so that a = <I>-I(1-a/2). Then the mean
number of exceedances under H is j*a.

10.12 LIBRARY PHOTOCOPYING (SECTION 1.4 EXAMPLE)

The photocopying problem described in Section 1.4 required the simulationist to specify
many sampling distributions and to assign numerical values to the parameters of each to
facilitate sampling. Principal among these distributions are those for customer interarrival
time T, number of pages to be copied N, and the corresponding photocopying time S.
To estimate the parameters, data were collected and analyzed for each floor of the
library. The present account describes the analysis for the third floor. For each customer i,
the data consisted of

T; := interarrival time,
Ni := number of pages to be copied,
Si := photocopying time.

Times came to the simulationist rounded to the nearest minute. That is, if Tj' and S; were
customer i's true interarrival and photocopying times, respectively, in minutes, then the data
were the integers

T; = LTj' + .SJ

and

Si = LS; + .SJ.

Since both interarrival and photocopying times are continuous, the implications of the
recording method need to be understood. In particular, T; = 0 for 0 < Tj' < .5 and Si = 0
for 0 < S; < .5, which substantially misrepresented the true behavior of small interarrival
 10.12. LIBRARY PHOTOCOPYING (SECTION 1.4 EXAMPLE) 489

and photocopying times. To reduce the severity of this error, Ti and Si were redefined as

{"25
if If' < .5,
Ti = LT{ +.5J if T{ :::: 1,

and

Si =
ILS;
.25
+.5J
if
if
S; < .5,
S; : : .5,
respectively. This enables us to regard {Ti} and {Sd as sequences of midpoints of the
discretization intervals [0, .5), [.5, 1.5), [1.5,2.5), ....

10.12.1 INTERARRIVAL TIMES

Table lOA shows the number of arrivals by time of day and day of week. If arrivals
were independent of these two phenomena, then a single sampling distribution would suffice
for all interarrival times, thereby simplifying the estimation procedure and the subsequent
sampling procedure. Accordingly, a two-way analysis of variance (ANOVA) was performed
on these data to test the hypotheses

HI : Arrivals are independent of day of week

and

H2 : Arrivals are independent of hour of day.

Table 10.5 lists the results.

Table 10.4 Number of hourly arrivals

Hour Mon. Tues. Wed. Thurs. Fri. Hourly

total

10 a.m. 17 14 18 18 13 80
11 23 32 20 11 17 103
12 p.m. 15 19 19 17 17 87
1 27 21 22 19 25 114
2 23 24 6 14 27 94
3 15 18 13 32 20 98
4 13 18 15 11 8 65
Daily
total 133 146 113 122 127 641
490 C HAP TE R 1 0 . PRE PAR I N G THE I N PUT

Table 10.5 ANOVA for HI and H2

Test dJ. F p-value:= pr(exceeding F)

H IUH2 (10,24) 1.18 .3520
HI (4,30) .65 .6327
H2 (6,28) 1.53 .2110

For each test, the p-value estimates the probability of observing a test statistic in excess
of the corresponding computed F. Hence, the p-value decreases with increasing F. Common
practice rejects a hypothesis if its p-value is less than .05.
In the present case, the ANaVA provides little support for rejecting HI> H 2, or HI U
H 2. However, we must be cognizant of two factors. First the statistical power of a test to
detect differences is often low when the alternative hypothesis consists of all possible types
of differences, as is the case here. Second, the pattern of library usage was known to have
fewer arrivals in the early morning (10-11) and in the late afternoon (4-5) than during the
remainder of the day (11-4). These factors encouraged testing H2 against the hypothesis

H3 : Arrival rates in hours 10-11 and 4-5 differ from those in hours 11-4.

The data in Table lOA also arouse suspicion that the large number of arrivals on Tuesday
indicate that this day differs from the remaining four. This observation encouraged testing
of HI against the hypothesis

H4 : Arrival rate on Tuesday differs from that on the remaining days.

Testing one hypothesis against a specific alternative, as in the case ofH 2 versus H3 and HI
versus H 4, increases the statistical power of ANaVA to detect differences, when they exist.
Table 10.6 displays results of these ANavAs. They reject H2 in favor of H3 at the
.05 level but provide no support for H4 over HI. Accordingly, interarrival time data were
partitioned into two groups. The analysis that follows focuses on the arrivals in the peak
interval 11-4.
The aggregated sample consisted of n = 493 interarrival times with sample mean,
variance, and standard errors

(LT = 3.030, a"f, = 11.12, (1004)

s.e.((LT) = .1502, r.e. ((LT) = .04956, (10.5)

the last of which attests to the relative stability of (LT as an estimate of the true mean J-tT.
Figure 1O.6a shows the empirical histogram. Together with the discretization, described
earlier, the absence of mass at 11, 17, 18, 20, 23, 25, 27, and 28 minutes would severely
limit the values of interarrival times, were the histogram used as the basis for sampling
in a simulation. Accordingly, we fit a continuous distribution. The shape of the histogram
 10.12. LIB RA RY PH 0 Toe 0 P YIN G (S E C T ION 1.4 Ex AMP L E ) 491

Table 10.6 ANOVA for H2 vs. H3 and HI vs. H4

Test d.f. F p-value

H2 vs. H3 (1,33) 6.74 .0140
HI vs. H4 (1,33) 1.63 .2105

encouraged the fitting of a Gamma distribution 9 (a, (3) (Section 8.13). However, the sample
coefficient of variation fir / fiT = 1.100 is relatively close to unity, which is the coefficient of
variation of the exponential distribution, a special case, 9 (1, (3), of the Gamma. Both options
were considered. First, we describe the effect of discretization on the parameter estimation
procedure.
Because of discretization, the likelihood function is

L(a, f3, no, nl, .. ·, nk) = [F(.5Wo Il~=l [F(j + .5) - F(j - .5Wj, (10.6)

where

no := # of observations with T; = .25,

n j := # of observations with T; = j,
k = # of distinct integer categories,

and F(t) = J~ f(x)dx, where f is the Gamma p.d.f. in Section 8.13.

Because of the exceptional demands in computing the m.l.e. for f3 in the exponential
case and for a and f3 in the Gamma case for the likelihood function (10.3), they were
computed for the approximate likelihood function

In effect, the approximations .5f(.25) ~ F(.5) and f(j) ~ F(j + .5) - F(j - .5) for
j = 1, ... , k were used.
Maximizing with respect to a and f3 led to the m.l.e.

& = 1.047, ~ = 2.893,

s.e.(&) = .05897, s.e.(~) = .2068,
r.e.(&) = .05631, r.e.(~) = .07l47,
corr(&,~) = -.7819,

where the relative errors attest to the stability of &and ~ as estimates of the unknown a and
f3. The corresponding fitted distribution, which would serve as the basis for sampling in the
492 CHAPTER 10. PREPARING THE INPUT

(a) Empirical histogram

0.2S

n ~ 493. !1 ~ 3.030.<1 ' ~ 11 .12

0.20

I
0,115

0.10

0.06

0.00
10 12 14 16 18 20 22 24 2S 28 30

time (minutes)

(b) Gamma fi t
1.00

0 .90 - F ~ 1CiI u C"ccdlulu. 4.(.

OC I.en.l .U].)
0 .30 -E mp ltlt~ 1 4 . t· rrU tO d,',

0 .10

0 .60

:E 0 .500
2
£ 0 .40

0 .30

0 .20

0,10

0.00

'0 ,10 - 10
" 20 25
••
time (minutes)

(c) Comparison of difference curves

0.04

- ~ri::al d,f, G(1 .o.t7.l.A9J)

0.03 - Er11I'ri:-.lld.f. · lDed d,! E(3.DJO)

0.02
~

~ 0 .01
""6
0

.(l.Ot

-0.02
0 5 to 15 20 25 30

lime (minutes)

Figure 10.6 Interarrival times

 10.12. LIB RA RY P HOT 0 COP YIN G (5 E C T ION 1.4 ExAMP L E) 493

simulation, is

Note that & < 1 implies a monotone decreasing p.d.f. unbounded at t = O. Statistical tests
of how well {F(t)} approximates {F(t) = [,8ar(a)]-1 f;
za-l e-z/Pdz, t :::: O} customarily
rely on a comparison of F and the empirical d.f.

1
L I[Ti,oo)(t),
n
F(t) = - (10.8)
n i=l

Figure 1O.6b displays the fitted exceedance d.f. {l- F(t), t :::: O} in black and the dif-
ferences {F(t) - F(t), t :::: O} in gray. As discussed in Section 10.3.2, visual assessment of the
difference graph provides one statistical evaluation, albeit a subjective one. The appearance
of the difference curve suggests a relatively good fit.
Now to the special case of the exponential distribution g(1,,8) with p.d.f.

,8 > 0, t :::: O.

Clearly, & = 1.047 adds considerable plausibility to the hypothesis Q: a = 1. Moreover, the
aggregate arrival process to the library's photocopiers fits our description in Section 1004.1
of a superposition of many independent renewal processes each with comparatively small
arrival rate. Thus the distribution of the interarrival times for the aggregate process are well
approximated by the exponential where the error of approximation tends to decrease as the
number of renewal processes increases.
Maximizing the likelihood function (10.7) with respect to ,8 yields the m.l.e. =Pe
ILT = 3.030, which has the same standard errors as ILT in expression (10.1). The
corresponding fitted d.f. is

t :::: O. (10.9)

Figure 1O.6c shows a marginal superiority for the Gamma fit, which is to be expected.
How are we to choose between the two fits? Let us summarize the facts:

• The interarrival time data estimate the mean as ILT = 3.030 and variance as &/ =
11.12, implying a coefficient of variation 1.10.

• The Gamma fit estimates the mean as &P = 3.030 and variance as &p 2 = 8.766,
implying a coefficient of variation .9771.

• The exponential fit estimates the mean as Pe = 3.030 and variance as P; = 9.181,
implying coefficient of variation unity.
494 C HAP TE R 1 0 . PRE PAR I N G THE I N PUT

• Sampling from [(Se) is less time consuming than sampling from Q(a, S).
Interestingly, the Gamma fit has the lowest coefficient of variation. As discussed earlier,
system congestion tends to grow with increasing coefficient of variation. Therefore, sampling
from either Q(a, S) or [(Se) induces less congestion than sampling from the empirical
dJ. {F(t)} would. However, this reduction is less for [(Se)' This, together with less costly
sampling, favors the exponential choice.
a
In the present case, the proximity of to unity encourages us to choose the exponential
a
fit. However, what would we have done if were, say, 1.1 or 1.2? A perusal of difference curves,
as in Figure 10.6c, would provide some help. However, a more objective statistical criterion of
acceptability is desirable. The likelihood-ratio test provides a basis for increased objectivity.

10.12.2 LIKELIHOOD RATIO TEST

Recall that L*(1, fJ) is merely the likelihood function L*(a, fJ) in expression (10.7)
under the hypothesis Q : a = 1. Because of this constrained optimization in the exponential
case, the likelihood ratio

R ._ _L_*_(1_,-,-S.".e,_n_o_,_n_l,_._._._'n_k...:...)
.- L*(a, S, no, nl, ... , nk)

lies in (0, 1). Moreover, it is known that

- 2lnR -! X~-l asn ~ 00, (10.10)

where X~-l denotes a chi-squared random variable with n - 1 degrees of freedom. In the
present case

-21n R = 17.15.
For the distributional limit (10.10),

p-value = 1- 8.681 X 10- 258 ,

which is an estimate of the probability of observing a Xi92 random variable greater than
-21n R = 17.15. In the present case the p-value gives no support to rejecting Q.

10.12.3 OFF-PEAK HOURS

An analysis of the data for the off-peaks hours (10, 11] U (4,5] gave sample mean
(IT = 3.528 and variance 8-j = 12.80 for inter arrival time. Moreover, maximum likelihood
estimation and likelihood ratio testing led to the conclusion that off-peak interarrival times
 10.12. LI B RARY PHOTOCOPYI NG (S ECTI 0 N 1.4 EXAMP LE) 495

were from £(3.528). Section 10.14.1 addresses the issue of how to merge the peak and
off-peak arrival processes for sampling.

10.12.4 NUMBER OF PAGES

The number-of-pages or page-quantity sequence N J , ••• , Nn is a sequence of positive

integers, and it is reasonable to assume that they are i.i.d. The Poisson and negative binomial
distributions are the most commonly employed for fitting nonnegative integer data with
infinite support. If X has the Poisson distribution peA), then

e-AA i
pr(X = i) = -.-'r-'
I.
A> 0, i = 0,1, ....
If Y has the negative binomial distribution N B(r, p), then

( .) r(r+i) ( )r i = 0,1, ....

pr Y = l = r(r)r(i + 1) 1 - P p, r >0, 0< p < 1, i

Chapter 8 describes bounded-computing-time algorithms for sampling from each.

A distinguishing feature ofP(A) is that EX = A, and varX = A, so that

varX
variance-to-mean ratio = EX = 1.
By contrast, N B(r, p) has EY = rp/(l- p) and var Y = rp/(l- p)2, so that
. . varY 1
vanance-to-mean ratio = - - = - - > 1.
EY 1- P

Since the data yielded the sample mean {LN = 22.94 and sample variance a~ = 1661
for a sample variance-to-mean ratio 72.40, and since the empirical histogram in Figure 10.7a
revealed substantial right skewness uncharacteristic of the Poisson, the negative binomial
distribution was chosen for fitting. The resulting m.l.e.s are

P = .9740, ft = .9594,
s.e.(p) = .0520, s.e.(ft) = .002626,
r.e.(P) = .05360, r.e.(ft) = .00274,
corr(P,ft) = .7947.
Figure 10.7b shows the fitted exceedance dJ. in black and the difference between the empirical
and fitted d.f.s (in gray). The difference curve reveals no absolute difference greater than .06
between these dJ.s. It also shows that each difference is relatively small when compared with
its corresponding ordinate on the fitted dJ.
496 C HAP TE R 1 0 . PRE PAR I N G THE I N PUT

(a) Empirical histogram

o 25 50 75 100 125 150 175 200 225 250

(b) egalive binomial fil

0.90
• F'ned ~~ 6,( N8(.970lw9S94)
. clT1'iicald.(-IRdd.L

0 .70

I
~ 0 .50

0 30

0.1 0 mUlIUlIlIIllIlIlIIlIllIIlIlllllllllllllllllllllIlIIlIIllIIlIllilllIlllllllllllilII', ""''' ,,••

.(l.10 .
o 10

(c) Poisson iiI

0.6
• c-!rpri.:aI d.r. - ~ d.E P(2:VM)
0.5

0.4

0.3
>.

'2i 0.2
~ 0.1

0.0

.(l.1
1'111~1111I1111111111I1111I1I11111I1U11111I1I1II11I11I',"""''''.',"'
.(l.2
o 10 20 30 40 50 60 70 80 90 100
N

Figure 10.7 Page quantity N

 10.12. LIB RA RY P HOT 0 COP YIN G (S E C T ION 1. 4 Ex AMP L E ) 497

While the sample variance-to-mean ratio supported the decision to fit the negative
binomial rather than the Poisson distribution, it is nevertheless instructive to show the
results of fitting the latter with 5. = MN = 22.94. Figure 1O.7c shows the differences with
the empirical d.f. The superior fit of the negative binomial is unequivocal.

10.12.5 PHOTOCOPYING TIME

Since photocopying time is clearly a function of number of pages to be copied, it is

necessary to take this dependence into account when sampling this time. To do this, we need
to quantify this relationship and then turn it into a mechanism for sampling.
The plot of {(N;, S;); i = 1, ... , I} in Figure 1O.8a reveals a linear relationship but
with increased variability in S; for large N;. To model this behavior, let N denote number
of copies, S the corresponding setup plus photocopying time, and let

Vj := copying time for page j

with

and j = 1, ... ,N.

Multiple copies of the same page or automatic page feeding customarily induces a
copying time S directly proportional to the number of pages N and with virtually no time
variation per page. However, copying in a library tends to be from journals and books,
one page at a time, and thus copying times VI, ... , VN are reasonably assumed to be i.i.d.
Collectively, the graph and the i.i.d. assumption imply the representation

N
S=a+ LVj =a+bN+N I / 2 E, (10.11)
j=1

where

a := setup time,

L (Vj - b),
N
E := N I / 2
j=1

EE = 0,
var E = (f2.

In terms of the data, we have

S; = a + bN; + N;1/2 E;, i = 1, ... , I,

498 C HAP TE R 1 0 . PREPARING THE INPUf

<a) Photocopying time S vs. page quantity N

120

100 I...:..=.- 1.297+.111411,1

80
]'
~
60
~
">

40

20

0
0 50 100 150 200 250 300

(b) Standardized residuals vs. page quantity N

: ::::::=:=:::~:~:::~:~:=~:=:~
3 .. '

.2 ---------------------.------______ . ___________ ...__ ..._... __ ..._...__.. __: ......._..._..._..._... _..._ . .....1

o 25 50 75 100 125 150 175 200 225 250 275 300
N

(c) Unconditional Gamma fit

"""""""""""""""""""'" """'-,,-------_.. _,,_.._ - - - - - - ,

0.90 - fnod .««danc. d.f
iri<:ald.f. · &led d.f

0.70

0.50

0.30

0.10

.(l.10 --------------,-""""""'''''''''''''''''''''-

o 5 10 15 20 25 30 35 40

S (minutes)

Figure 10.8 Photocopying

 10.12. LIBRARY PHOTOCOPYING (SECTION 1.4 EXAMPLE) 499

with three parameters a, b, and 0'2 to be estimated. This was done by choosing a and b to
minimize the sum of squared errors

1 1
L El = L (Si -a -bNY,
i=l i=l

giving

a= [I:{=1 Si - I:{=1 Ni I:~=1 (jI N = 1.297 minutes j)

[ 2 _ "I N. "I N- 1
L-i=l I L-j=l j

and

b = [I:{=1 (S;j Ni ) - I:{=1 Si I:~=1 Nj1 = .2114 minutes = 12.68 seconds.

[2 _ " I N. "I N- 1
L-i=l I L-j=l j

With regard to the residual variance 0'2, we have

~L
1
&2 = N i- 1 (Si - a- b N )2 i
[ i=l

~L
1
= N i- 1 {(Si - a - b Ni) - [a - a + (b - b)N;]}2
I ;=1
= .7685,

so that for NI := (N1 , ••• , NI ),

Therefore,

&2 = &2 [IU - 2) = .7709

is an unbiased estimate of 0'2.

Since E(a I N 1 ) = a and E(b I N1 ) = b, it is easily seen that
500 CHAPTER 10. PREPARING THE INPUT

and

a
Substituting &2 for &2, we estimate the standard errors of and b as

s.e(a) = .1217 and s.e.(b) = .008724,

r.e.(a) = .09383 and r.e.(b) = .04128.

In principle, (S - a - b N)/.jNa 2 -!N (0, 1) as N --+ 00. This property

suggests that a plot of the standardized residuals Ili := (Si - a-
b Ni )/.jNi &2 versus
Ni for i = 1, ... , I should reveal a tendency toward symmetry around the zero axis as Ni
increases. Although Figure 1O.8b confirms this tendency, it also reveals a marked asymmetry
and thus nonnormality of the residuals around zero for small Ni.
Since our principal objective is to devise a means of sampling S given N, we need to
choose a sampling distribution that accounts for asymmetry for small number of copies N
and for the tendency toward normality as N increases. One solution to this problem relies
on the Gamma distribution (Section 8.13).
A random variable X from 9(a, f3) has mean af3 and variance af32. If we regard S
a
given N as a Gamma distributed random variable with mean + b N and variance N &2,
then it is from 9(aN, f3N), where

and

so that al = 2.951, f31 = .5111, and

aN
- --+ .05797 and f3N --+ 3.647 as N --+ 00.
N

One limitation of this approach is that aN decreases from aN = 2.951 at N to a

minimum of aN = alb = 1.423 at N = 6.135 and increases thereafter. This behavior for
small N suggests that convergence to normality for S is not monotone. This fact encourages
us to check the adequacy of the fitted distributions.
The Gamma dJ. is

s > 0,
 10.12. LIB RA RY PHOT 0 COP YIN G (S E C T ION 1. 4 ExAMP L E) 501

so that S given N has dJ. {F(s; aN, fiN), s ::: OJ. Then

1
L
A n
FG(s) := - F(s; aNi' fiN,), s > 0,
n ;=1

provides an estimator of the unconditional dJ. of copying time S independent of the number
of copies N. For the data SI, ... , Sf> the empirical d.f. is

1
L
I
Pes) =I I(Si. oo) (s), s ::: o.
;=1

Figure 1O.8c displays the fitted unconditional exceedance dJ. {l - FG (s)} and the
difference {Pes) - FG(s)}. The relatively small absolute differences encourage us to accept
this method of approximating the conditional dJ.s of photocopying time.

10.12.6 RELATIONSHIP WITH STANDARD LINEAR REGRESSION

It is of interest to cast this estimation procedure in the context of general linear re-
gression and to contrast the model (10.11) with the more commonly encountered form of
linear regression. Let

and

Then G, fj, and (12 have the representations

and

If the data in Figure 1O.8a had revealed deviations {S; - a - bN;; i = 1, ... , I}
that fluctuated independently of the N;, then we would have been tempted to use the more
502 CHAPTER 10. PREPARING THE INPUT

conventional regression model with

and
y'=c}
10.13 WASTE GENERATION (SECTION 1.5 EXAMPLE)
An analysis of daily waste-generation data for the incinerator problem in Section 1.5 illus-
trates one way in which empirical distributions can be the basis for sampling workload. Recall
that each of the four sites or campuses produces liquid waste, which it stores in 55-gallon
drums until they can be transferred to the incinerator for disposal. Let

Ni := number of days of data collected at campus i

and

Nij : = number of days on which campus i generated

j 55 - gallon drums ofliquid waste.

Table 10.7 shows these frequencies. Since no waste generation occurs on weekends and
holidays, these days have been omitted. The generating processes at the four campuses
proceed independently of day of week, week of month, and month of year. These properties
suggest that daily waste generation at each campus be sampled from its own empirical
distribution.

Table 10.7 Daily waste generation, j := number of drums

Nij N;

Campusi 0 1 2 3 4 Total
1 193 32 0 227

2 145 60 28 13 4 250

3 204 25 13 3 246

4 176 6 0 0 183
 10.13. WASTE GENERATION (SECTION 1.5 EXAMPLE) 503

For campus i, let

Pij := probability that on an arbitrarily selected day the campus generatesj drums,

L j Pij = mean daily waste generation,

4
ILi := (10.12)
j=o

a/ := LCi - IL;)2 Pij = L i 2Pij - IL7 =

4 4
variance of daily waste generation. (10.13)
j=O j=O

Given the marginal sums N 1 , ••. , N 4 , the maximum likelihood estimates of the Pij
are

Nij
(10.14)
A

Pij=-
Ni

with standard errors

and correlations

j =j:. k.

Table 10.8 displays the Pij.

Let Ai and 8/denote the maximum likelihood estimates of ILi and a/ obtained by
substitutingpij for Pij in expressions (10.12) and (10.13). For each ILi and8 = .01, Table 10.9
displays the point estimate Ai and the approximating 100 x (1-8) percent confidence interval
[Ai ± <1>-1 (1 - 8/2)&;/ N//2] for ILi and the relative half-length <1>-1 (1 - 8/2)&;/ Ai Nil/z .
For 8 = .05, the relative half-lengths would be .7609 as long. The relatively large half-lengths
imply that we would be shortsighted in accepting Ai as more than a rough approximation
to ILi' especially for Campus 4.
Since waste generation is the driving force in this system, attention needs to be paid to
how sensitive system congestion is to the sampling variation in the Ai. We address this topic

Table 10.8 Maximum likelihood estimates of the Pij for daily waste generation
Campus Pij
j=O j=1 j=2 j=3 j=4
1 .8502 .1410 .004405 .004105 O•
2 .5800 .2400 .1120 .05200 .0160
3 .8293 .1016 .05285 .01220 .004065
4 .9617 .03279 .005464 O. O.
504 CHAPTER 10. PREPARING THE INPUT

through 1'; := the number of days that elapse between successive positive waste generation
at campus i. Since 1 - PiO is the probability of waste generation on a given day,

t = 1,2, ... ,

with mean

1
E1'; = ri:= - - - .
1 - PiO

Since

we have the approximating 100 x (1 - 8) percent confidence interval [WI (Nij, N i , <I> -1 (1 -
8/2», w2(Nij' N i , <1>-1(1 - 8/2»] for Pij, where
z + fJ2/2 + 8( _I)' [fJ2 /4 + zen - z)/nj1/2
w,(z, n, fJ) := (n + fJ)2
This follows from solving the inequality

for Pij'
For j = 0, these results yield the point estimate 1/(1 - PiO) and interval
estimate [1/[1 - WI (NiO , N i , <1>-1(1- 8/2»], 1/[1 - W2(NiO , Ni, <1>-1(1 - 8/2»1] for rio
Table 10.10 displays these for 8 = .01. The variation inherent in the mean estimate is con-
sistent with that seen in the fli of Table 10.9. However, in this form we can more easily offer
a sensitivity analysis that exploits the method of sampling.
The most direct way to generate waste daily in the simulation of Section 1.5 is to sample
the number of drums Di for Campus i from the empirical distribution tpiO, ... , Pi4} via

Table 10.9 Mean daily waste generation

99% Confidence interval
Campus Relative
(Li Lower Upper half-length s.e.({L;)
1 .1630 .09216 .2338 .4346 .02750
2 .6840 .5257 .8423 .2314 .0615
3 .2602 .1537 .3666 .4092 .04132
4 .04372 O. .0874 1.000 .01698
 10.13. WAS TE G ENE RAT ION (S E C T ION 1. 5 Ex AMP L E ) 505

Table 10.10 Mean number of days between suc-

cessive waste generations

Sample 99% Confidence interval

Campus mean Lower Upper
1 6.676 4.531 10.12
2 2.381 1.994 2.918
3 5.857 4.150 8.490
4 26.14 10.67 66.40

Algorithm ITR in Section 8.1.3. For five ordinates, this algorithm compares favorably in
computing time with the cutpoint method in Algorithm CM in Section 8.2 and the alias
method in Algorithm ALIAS in Section 8.4, whose superiority materializes for larger numbers
of ordinates. However, this direct generating approach has a limitation. For example, if one
were to generate waste daily in this way for Campus 1, then the mean number of days between
successive positive waste generations in the simulation would be I/O - PlO) = 227/34 =
6.676. That is, this approach generates zero drums on 5.676 successive days on average before
generating a positive number. This result follows from the observation that the number of
days between successive positive waste generations is 1 + Z, where Z has the geometric
distribution Qe(PlO), as in Section 8.19.
To improve computational efficiency, we resort to a procedure that skips the zeros. For
j = 1, ... , 4 and i = 1, ... , 4, let

I Pij
Pij = 1- PiO
= probability of generating j drums
given that at least one drum is generated

with maximum likelihood estimate

Let

fl Ti := number of days between positive waste generations on Campus i.

Then on the day of the most recent positive waste generation on Campus i, Algorithm
WASTE samples fl Ti' schedules the next positive waste generation, and samples the number
of drums Di that are to be generated on that day.

ALGORITHM WASTE

Purpose: To schedule the next positive waste generation for Campus i and to
determine its number of drums.
506 CHAPTER 10. PREPARING THE INPUT

Given: AI
Pio an d {AI
qij:= AI
qi,j-l
A I
; ] = 1, ... ,4}.
+ Pij; qiO:= O'
Method:
Sample Z from ge(PiO).
I!!.:r:i +-- 1 + Z.
Schedule the next positive waste generation on day t' + Do t'i •
Sample U for U(O, 1).
j +-- 1
While U > %, j +-- j + 1.
Di +-- j.

Table 10.11 displays the relevant distributions along with Pio, PiO/O - pio), the mean
number of zero waste generations between successive positive generations, and the mean
number of comparisions in sampling Di := Pil + 2Pi2 + 3Pi3 + 4Pi4' Note the substan-
tiai mean numbers of days skipped with regard to zero-waste generation. Also note that
the relatively small mean numbers of comparisons are consequences of P;j ::: P;,j+! for
j = 1, ... , 3. When these orderings do not arise naturally in a problem, reordering the
probabilities so that comparisons proceed from largest to smallest always reduces the mean
number of comparisons.

10.14 NONHOMOGENEOUS POISSON PROCESSES

Recall from Section 10.12.1 that peak period interarrival times were from £(3.030), whereas
during off-peak (10, 11] U 06, 17] hours interarrival times were from £(3.528). If one
intends to simulate a complete seven-hour day, how does one merge these two processes to
reflect the distinct values of their parameters? We begin with the well-known relationship
between the exponential and Poisson distributions.
1fT!, Tz. ... are i.i.d. from £0/)...), where)... > 0, and

N(t) := number of arrivals in (0, t], (10.15)

Table 10.11 Waste generation probabilities (p;j := pij / (I - PiO); j = 1, ... ,4)

Campus Mean no. of

A, A, A, A, ...ilL
PiO Pi! Pi2 Pi3 Pi4 1-', comparisons
.8502 .9412 .0294 .0294 0 5.676 1.088
2 .5800 .5714 .2667 .1238 .0381 1.381 1.627
3 .8293 .5952 .3096 .0714 .0238 4.858 1.524
4 .9617 .8571 .1429 0 0 25.11 l.l43
 10.14. NON HOM 0 G ENE 0 U S POI S SON PRO C E SSE S 507

then N(t) has the Poisson distribution peAt). Suppose we simulate the times of arrivals for
a day starting at hour 10 (a.m.) and ending at hour 17 (5 p.m.). Let

to := 10,
tl := 11,
t2 := 16,
t3 := 17,
Al := 60/3.050 = hourly arrival rate in peak interval(tl' t2],
A2 := 60/3.528 = hourly arrival rate in off-peak intervals (to , tIl U (t2, t31.

Then

N(tl - to) '" P(Al(tl - to)),

N(t2 - tl) '" P(A2(t2 - td),
N(t3 - t2) '" peAl (t3 - t2)),

and by the reproductive property of the Poisson distribution

where

A(t):= itto
A(s)ds, to < t ~ t3, (10.16)

r
and

to < t ~ tlo
A(t):= A2, tl < t ~ t2, (10.17)
AI, t2 < t ~ t3.

Here interarrival times form a nonhomogeneous Poisson process with instantaneous intensity
function {A(t)} that varies with time of day.
A special case of this nonhomogeneous Poisson process is the homogeneous Poisson
process, wherein {A(t)} is invariant with respect to t so that A(t) ex: (t - to). More generally,
{N (t ), t 2: O} is said to be a nonhomogeneous Poisson process with instantaneous intensity
function {A(t), t 2: O} iffor each h 2: 0 (e.g., Ross, 1980 p. 187)

i. N(O) = 0,
508 CHAPTER 10. PREPARING THE INPUT

ii. {N(t), t ~ O} has independent increments,

iii. pr[N(t + h) - N(t) ~ = o(h),

2)

iv. pr[N(t + h) - N(t) = 1) = A(t)h + o(h).

Since {A(t)} in (10.17) is right continuous, property iv can be satisfied in the present setting.
The relationship between homogeneous and nonhomogeneous Poisson processes pro-
vides a means for sampling interarrival times for the latter, provided that {A(t)} is finite.
Algorithm NHPP generates samples from the homogeneous Poisson process P(A max t),
where Amax ~ A(t) Vt, and thins them by an acceptance-rejection test. The acceptance
probability in an arbitrarily selected trial is no less than AdAJ = .9354. More generally, if
Amax is considerably larger than most values of {A(t)}, then thinning is not a particularly
efficient sampling procedure.

ALGORITHM NHPP

Purpose: To sample an interarrival time from a nonhomogeneous Poisson

process with instantaneous intensity function (10.17).
Source: Lewis and Shedler (1979).
Input: {A(t), to < t ::: t3}, Amax := maxtO<t:::::t3 A(t), and r := current
simulated time in hours (to < r ::: t3).
Method:
Y ~ r.
Repeat:
sample Z from £(1jAmax) and U from U(O, 1)
Y~Y+Z
if Y > t3, return indicating no additional arrival in (r, t3)
Until U ::: A(Y)jAmax .
Return Y as the next arrival time.

10.14.1 CLEARING BOOKS

In addition to the photocopying problem in Section 1.4, the library requested a simu-
lation study to evaluate the benefits of alternative reshelving policies for books and journals
that patrons leave on study tables and reshelving carts. Several times daily a student employee
makes tours of these areas and collects the materials for eventual reshelving. Hereafter, we
use the generic term "book" to mean journal as well as book.
The simulation called for a means of sampling the number of books that are cleared
from tables and reshelving carts each time a clearing tour occurs. For this study, a day began
at hour 7.75 (7:45 a.m.) and ended at hour 22 (10 p.m.). Table 10.12 shows data collected
on 31 tours. It reveals that tours did not occur on a fixed time schedule. This variation in
clearing times created a challenge for the simulationist to provide a generating procedure.
 10.14. NONHOMOGENEOUS POISSON PROCESSES 509

Table 10.12 Library book dearing (k = 31 observations)

Clearing time No. of Books! Clearing time No. of Books!
Obs. interval books hour Obs. interval books hour
Si < t ~ ti Bi Bi/(ti - Si) Si < t ~ ti Bi B;/(ti - Si)

7.75 15 160 22.07 17 18 22 63 15.75

2 15 21 146 24.33 18 7.75 11 60 18.46
3 21 22 10 10 19 11 17 285 47.5
4 7.75 15 133 18.34 20 17 19 63 31.5
5 15 22 92 13.14 21 7.75 10 75 33.33
6 7.75 11 67 20.62 22 10 13 49 16.67
7 11 15 116 29 23 13 16 66 22
8 15 20 66 13.20 24 16 20 15 3.75
9 7.75 11 79 24.31 25 7.75 9 106 84.8
10 11 14 75 25 26 9 13 60 15
11 14 18 208 52 27 13 15.25 55 24.44
12 7.75 14 150 24 28 15.25 17 166 94.86
13 14 16 107 53.50 29 7.75 9 76 60.80
14 7.75 11 51 15.69 30 9 11 25 12.5
15 11 15 216 54 31 11 19 159 19.88
16 15 18 88 29.33

One approach to this problem again relies on the concept of a nonhomogeneous

Poisson process with instantaneous intensity function estimator

~(t) = ~~=1 [B;/(ti - Si)] [(Si,t;] (t), 7.75 < t ~ 22, (10.18)
Li=1 [(Si,t'] (t)

where Bi , Si, and ti are as defined in Table 10.12 for i = 1, ... , k. Figure 10.9 shows
{A(t), 7.75 < t ~ 22}, which shows lowerintensityin the earlymorningpriorto hour 9 and
in the evening after hour 17.
Let ro := 7.75 and r1 := 22. To devise a procedure for sampling B, the number of
books cleared on a tour, from P (A (r )) efficiently for

A(r) := l'
'0
A(t)dt,

where {A(t)} is defined in expression (I0.18), we first partition the hours (7.75, 22] into 57
quarter-hour intervals of which the jth corresponds to (ro + .25{j - I), ro + .25j]. Then

Aj := intensity in interval(ro + .25{j - 1), ro + .25j], (10.19)

Ao :=0,
j = 1, ... ,57.
510 CHAPTER 10. PREPARING THE INPUT

40

11
35

30
r--- niL-
..---J
25

A.(t) 20 - L...--
~
L, Il
15 '---' L-
10

o
7 8 9 10 11 12 13 14 15 18 17 18 19 20 21 22

t (hours)

Figure 10.9 Sample instantaneous intensity function for book clearings

In terms of definitions (10.16) and (10.17),

Aj = A(ro + .25 j) (10.20)

(ro+.25 j j
= lro A(t)dt = ~Aj, j = 1, ... ,57.

By contrast with the need to generate arrival times in the photocopying problem in Sec-
tion 10.14, the clearing problem provides a daily sequence of clearing times and requires the
simulationist to sample B, the number of books cleared, at time r from P(A(r) - A(t)),
where t < r denote the times of two successive clearings on a single day. Algorithm CLEAR
does this by retaining Aold := A (t) from the previous sampling.

ALGORITHM CLEAR

Purpose: To sample B, the number of cleared books.

Input: Clearing time r E (ro, rd, P'j, j = 1, ... , 57}, and {A j ; j = 1, ... , 57}.
Comment: Aold is retained on successive calls within a day.
Method:
If first tour of day, Aold ~ o.
I ~ 1 + f4(r - ron.
Anew ~ AZ-l +Az[r -4(1-1)].
Sample B from P(A new - Aold).
Aold ~ Anew.
Return B.
 10.14. NONHOMOGENEOUS POISSON PROCESSES 511

Definition (10.18) provides a relatively convenient basis for computing a piecewise

constant approximation to the instantaneous intensity function {A(t)} when the system
under study is stationary. In the present case, a day consists of 14.25 hours starting at hour
7.75 and ending at hour 22. If simulated time consists of a sequence of days of 14.25 hours,
then for all t :::: 7.75 (in hours) stationarity implies that

A(t + 14.25j) = A(t), j = 0, ±1, ±2, .... (10.21)

Likewise, for the example in Section 10.12.1, if simulated time consists of a sequence of seven-
hour days beginning at hour 10 and ending at hour 17, then for all t :::: 10 its instantaneous
intensity function satisfies

A(t + 7 j) = A(t), j = 0, ±1, ±2, .... (10.22)

Note thatthe assignment (10.17) satisfies expression (10.22), whereas the assignment (10.18)
satisfies expression (10.21).
The forms of these intensity functions suggest another more formal method of esti-
mating {A(t)} that exploits their inherent periodicities. In this alternative, {A(t)} assumes
the form

A(t) = exp[h(t, p, 0)], (10.23)

where TO denotes the simulation's starting time,

p
h(t, p, 0) := ao + Lak sin(bkt + Ck), (10.24)
k=l

and

Here {A(t)} is an everywhere continuous and differentiable function, allowing us to regard

the phenomenon being modeled as one whose intensity changes relatively smoothly rather
than abruptly with time, as is the case in assignments (10.17) and (1 0.l8). Then the challenge
is to derive estimates of p and 0, which, when substituted into {h (t, p, O)}, make operational
the sampling from a nonhomogeneous Poisson process.
Some of the parameters in 0 may be known in advance of estimation. This occurs in
the book-clearing problem and in the arrival process for photocopying in Section 10.12.1.
For the clearing problem, we have

TO = 7.75,
bj = 2rrj/14.25, j = 1, . .. ,p,
512 CHAPTER 10. PREPARING THE INPUT

where b I is the daily fundamental frequency of book clearing intensity and b2 , •.• , bp are
its first p - 1 harmonics. For the photocopying problem,

'fo = 10,
b j = 2rrj/7, j = 1, ... ,p.
In both cases the relevant p is unknown and needs to be estimated along with aI, ... , a p
and CI, ... , cpo
The form (10.23) guarantees thatA(t) is nonnegative for allt ::: 'fo. Moreover, it allows
us to cast parameter estimation in terms of the likelihood function. Recall that N (t) denotes
the number of arrivals or more generically, the number of events in (0, tl. If N(t) is from
P(A(t», then

j = 0,1, ... ,

where

Therefore,

(10.25)
= probability that exactly one event occurs in(tI' t2)'
Suppose {A(t)} has theform (10.23) and that Zn = {Zo := to < ZI < ... < Zn} denotes a
sample sequence of times at which n successive events occur. Then expression (10.25) implies
the likelihood function

and therefore, the loglikelihood function

Then the m.l.e. are the solution to

aIn L(p, fJ, Zn) =0, (10.26)

aaj
aIn L(p, fJ, Zn) =0, j = 1, ... , p,
 10.15. LESSONS LEARNED 513

when b l , ... , b p are known. If they are not, then the additional equations

8 In L(p, 8, Zn)
- - - ' - - - - = 0, j = 1, ... ,p,
8b j

apply as well.
The form (10.26) severely impedes a direct approach to computing the m.l.e. Kuhl
et al. (1997) describe an iterative technique that overcomes the impediment by exploiting
approximations that lead to useful initial values for iterative estimation. Public-domain
software that implements their procedures is available at http://www.ie.ncsu.edu/jwilson/
page3.html. It also allows for estimation of{). (t ) } when it contains an exponential polynomial
trend. For an alternative approach to estimating {).(t)}, see Kao and Chang (1988).

10.15 LESSONS LEARNED

• Every discrete-event simulation requires a distribution function (d.f.) to generate
samples for each source of stochastic variation it models.
• If a historical data record is available and is sufficiently large, then its empirical distri-
bution function (e.d.f.) may provide an adequate basis for sampling. When this is not
so, fitting a theoretical distribution is the next best alternative.
• Every discrete-event simulation requires the assignment of numerical values to all
parameters of the theoretical distributions from which stochastic behavior is to be
generated. When available, historical data provide the basis for estimating these values.
• The method of maximum likelihood provides a means for estimating the parameters
of many distributions. These estimates have optimal properties. Iterative numeri-
cal methods need to be used in most cases. In some special cases other methods of
estimation may prove more convenient without sacrificing much of this optimality.
• For a specified historical data record, a graphical comparison of its e.dJ. with the fitted
d.f. for a particular distribution provides an assessment of how good the fit is. It is
especially important for detecting discrepancies in the tails.
• Some data suggest distributions with tails that diminish exponentially. Other data sug-
gest tails that diminish polynomially. The ability to distinguish between tail behaviors
increases as the size of the historical data record increases.
• Expert opinion often needs to be used in conjunction with historical data in assigning
values to parameters. In the absence of historical data, expert opinion provides the
most useful guidance for setting parameter values.
• Regression methods provide a convenient way to formulate generators for conditional
sampling.
514 CHAPTER 10. PREPARING THE INPUT

• Interarrival times are frequently modeled as i.i.d. random variables. When time of
day, week, month, or year affects the arrival rate, this assumption is inappropriate.
The concept of a nonhomogeneous Poisson process provides a means of accounting
for this dependence in generating samples.

10.16 REFERENCES

AbouRizk, S.M., D.W. Halpin, and J.R Wilson (1994). Fitting Beta distributions based on
sample data, Journal of Construction Engineering and Management, 120, 288-305.

Ahrens, J.H., and K.D. Kohrt (1981). Computer methods for efficient sampling from largely
arbitrary statistical distributions, Computing, 26,19-31.

Beckman, RJ., and G.L. Tietjen (1978). Maximum likelihood estimation for the Beta
distribution, J. Statist. Comput. Simul., 7, 253-258.

Brockwell, P.J., and RA. Davis (1991). Time Series: Theory and Methods, second edition,
Springer-Verlag, New York.

Conover, w.J. (1999). Practical Nonparametric Statistics, third edition, Wiley, New York.

Fielitz, B.D., and B.L. Myers (1975). Estimation of parameters in the Beta distribution,
Decision Sciences, 6, 1-13.
Fishman, G.S. (1973). Concepts and Methods in Discrete Event Digital Simulation, Wiley, New
York.
Gnanadesikan, R, RS. Pinkham, and L.P. Hughes (1967). Maximum likelihood estimation
of the parameters of the Beta distribution from smallest order Statistics, Technometrics,
9,607-620.
Gnedenko, B.Y., and LN. Kovalenko (1989). Introduction to Queueing Theory, Birkhiiuser,
Boston.
Grigelionis, B.L (1962). Accuracy of approximation of a superposition of renewal processes
by a Poisson process, Litovskit Mathemalicheskit Sbornik, II, (2), 135-143.

Johnson, N.L., S. Kotz, and N. Balakrishnan (1994a). Continuous Univariate Distributions,

Volume I, second edition, Wiley, New York.
Johnson, N.L., S. Kotz, and N. Balakrishnan (1994b). Continuous Univariate Distributions,
Volume II, second edition, Wiley, New York.
Kao, E.P.C., and S.-L. Chang (1988). Modeling time-dependent arrivals to service systems:
a case in using a piecewise-polynomial rate function in a nonhomogeneous Poisson
process, Man. Sci., 34,1367-1379.
 10.16. REF ERE NeE 5 515

K611erstrom, J. (1974). Heavy traffic theory for queues with several servers. I, J. Appl. Prob.,
11, 544-552.

Kuhl, M.E., J.R. Wilson, and M.A. Johnson (1997). Estimating and simulating Poisson
processes having trends or multiple periodicities, lIE Transactions, 29, 201-211.
Kulkarni, V.G. (1995). Modeling and Analysis of Stochastic Systems, Chapman and Hall,
London.
Law, A.M., and w.D. Kelton (1991). Simulation Modeling and Analysis, McGraw-Hill, New
York.
Lewis, P.A.W., and J. Shedler (1979). Simulation of nonhomogeneous Poisson process by
thinning, Nav. Res. Logist. Quart., 26, 403-413.

Nelson, B.L., and M. Yamnitsky (1998). Input modeling tools for complex problems, 1998
Winter Simulation Conference Proceedings, D.J. Medeiros, E.F. Watson, J.S. Carson,
and M.S. Manivannan, eds., Association for Computing Machinery, New York.AU :see
editors note
Ross, S.M. (1980). Introduction to Probability Models, Academic Press, New York.
Wagner, M.A., and J.R. Wilson (1996). Using univariate Bezier distributions to model
simulation input processes, lIE Transactions, 28, 699-711.
Zacks, S. (1971). The Theory of Statistical Inference, Wiley, New York.
ApPENDIX
More on the
Batch-Means
Method

A.1 FNB RULE (SECTION 6.6.4)

Procedure FNB describes the principal steps for implementing the FNB rule in Section 6.6.4.

Procedure A.1 (FNB (LABATCH.2)).

Given: II, bl> and t (> Ilbd
I +-- II (fix # of batches)
J(t) +-- 1 + Llog(t/llbd/log2J (# of reviews)
tl(t) +-- 2J (t)-l/lb l (# of observations used on review J(t»
j +-- 1, to +-- 0, tl +--/lbl ,
While tj .:::: tl(t):
Collect Xtj=l+l, ... , X tj additional observations
Compute X tj
Form batches of size bj based on X I, ••. , Xtj
Compute WZb j
bj+1 +-- 2b j
j +--j+1
518 ApPENDIX. MORE ON THE BATCH-MEANS METHOD

tj +- 2tj_l (= 11bj )
Collect observations X t /(tHl, ••• , Xt
ComputeXt
Output:
Xt is the point estimate of /1-.
C[(8) +- [X t ± t'1_1 (1 - 81 2 )Jb J (I) Wlbl(tJt 1is the approximating
100 x (1 - 8) percent confidence interval for /1-.
{b j Wlb j; j = 1, ... , J (t)} is the sequence of successive estimates of a~.

As an example, consider again the MIMII simulation queueing model with v .90 =
interarrival rate and unit service rate. The simulation began in the steady state, used II = 7
and b 1 = 5, and terminated when customer t = 107 entered service. Figure A.I shows
the LABATCH.2-generated sequences {A =
Jb j Wllb j ; j =
1, ... , J(t)}. The graphs
reveal a tendency for Vlj initially to increase with j and eventually to fluctuate around their
respective true aoo's.
Figures A.la and A.2b reveal an additional property. After dissipating systematic vari-
ance error, the size of fluctuations remains relatively constant in both graphs, in contrast to
the behavior in Figures 6.7 a and 6.7b. The origins of this constancy are not hard to find.
For {l(tj) = II, j :::: I}, EVII(t) -+ a~ as t -+ 00, but

2a~(l1 + 1) 2
var VrJ(t) -+ (11 _ 2)2 + 0(11/ 1 ) as t -+ 00, (A.I)

implying that Vr does not converge in mean square and thus is not a consistent estimator
of a~. However, under relatively weak. conditions on {X;}, the limit (6.53) holds, implying
that CII (8) remains an asymptotically valid confidence interval for /1-.
Although this settles the issue of validity, it does not address the issue of statistical
efficiency, which, in the present case, can be measured by2t'1,_, (1- 81 2 )JVII (t) It, the width
of the confidence interval for /1-. Expanding V//2 in Taylor series about a oo leads to

(A.2)

which, together with expressions (6.50) and (6.51), leads to

as t -+ 00. (A.3)

However, if VI were a strongly consistent estimator of a~, then necessarily IJ(I) -+ 00 as

t -+ 00, implying that
 A.1. FNB RULE (SECTION 6.6.4) 519

350

300 -

250 -

!if
o:!
t .
200-

~
IT 150 -
til .

100-

50-

0
0 5 10 15 20

Interim review J

1.75 -

1.5 -

!if 1.25 :
::!
~t: 1~

IT
til
0.75 -

0.5 -

0.25 -

0
0 5 10 15 20

Interim review J

Figure A.I LABATCH.2 ABATCH estimates ,JBW(L, B) of a oo , (a) Series 1, a oo =

189.5 (b) Series 2, a oo = 1.308
520 A P PEN D I X. M 0 REO NTH E BAT C H - MEA N S MET HOD

Therefore, for fixed 8, the ratio of expected widths would be

(AA)

revealing that the expected interval width based on a strongly consistent estimator of a~
never exceeds the mean interval width based on the FNB rule. In this regard, statistical
efficiency favors a strongly consistent estimator.

A.2 SQRT RULE (SECTION 6.6.5)

Let

Z ._ X t - f..i (A.S)
Ib .- -y'-=W=lb=I=Z

and let Fib denote the distribution function (d.f.) of Zib with Edgeworth expansion

where cI> denotes the standard normal d.f. and Ki (Zlb), the i th cumulant of Zlb. If K; denotes
the ith cumulant of a random variable Y, say for i = 1, ... ,4, then KI = EY, K2 =
E(Y - Kd 2, K3 = E(Y - KI)3, and K4 = E(Y - Kl)4 - 3Ki-
IfEIX I - f..i1 20 < 00 and {X;} is ¢-mixing with ¢i = 0(i-l3), then (Chien 1989, p.
46)

Kl (Zlb) = 001 ZI/2)00Ib 1/ 2),

K2(Zlb) = 1 + 001 l) + OOlb),
K3(Zlb) = 001 ZI/2)00Ib 1/ 2),
K4(Zlb) = 001 l).

Actually, Chien gives slightly weaker conditions for the first three cumulants.
As an immediate consequence,

F1b(Z) - cI>(z) = 001 ZI/2)00Ib l /2) + 001 l) + OOlb), (A.6)

 A. 2 . S Q RT RULE (S E C T ION 6. 6 . 5 ) 521

revealing thatl(t) ()( t i / 2 and bet) ()( t i / 2 induce the fastest convergence of the true coverage
rate to the specified theoretical coverage rate 1 - 8. In this case, Vt has 0(1/t i / 2 ) systematic
error and 0(1/ti/2) random error. To exploit these convergence properties, the SQRT rule
defines {(l j, b j); j :::: I} as in expressions (6.55) and (6.56).
Procedure SQRT describes the principal steps in employing the SQRT rule.

Procedure A.2 (SQRT (LABATCH.2».

Given: II, b!> and t(> IIb I )
II +- LJ2I1 +.5J
bi +-3;ifb i > 1,bI +- LJ2b I +.5J
J(t) +- 1 + Llog(t/llbd/log2J (# of reviews)
t'(t) +- 2J (t)-IIIb i (# of observations used on review J(t»
j +- 1, to +- 0, tl +- lIb!. t2 +- II, bi
While tj ::::: t'(t):
Collect Xlj _ 1+I , ... , X lj additional observations
Compute Xlj
Compute W/jb j
Ifj=l:
Ij+I +- II and bj+I +- b i
Otherwise:
lj+I +-Ij-I and bj+I +- bj - I
j +-j+1
tj +-Ijbj
Collect observations XI'(/)+I, •.• , X t
ComputeXt
Output:
XI is the point estimate of IJ..
C1J(t)(8) +- [Xt ± r1J(t)-1 (1 - 8/2)Jb J (/) W/J(t)bJ(t) /t 1is the approximating
100 x (1 - 8) percent confidence interval for IJ..
{b j W/jbj; j = 1, ... , J(t)} is the sequence of successive estimates of a~.

Choosing (II, bd from B in Table 6.3 ensures that 2I Ib i =

lIb!> so that tj =
Ijbj = 2j - 1l l bl> and therefore tj+dtj =2, as desired. This constraint proves valuable
in Section 6.6.6, which describes batch-size rules that combine the FNB and the SQRT rules.
.As illustration, we again use the MIMI 1 simulation example with a steady-state sample
record Xl,"" X t of waiting times in queue with t = 107 , II = 7, and b i = 5, so that
II = 10 and b i = 7. Thus 2I Ib i = IIbI , LABATCH.2 generates J(t) = 19 interim reviews,
=
and the last review uses t'(t) 9, 175,040 observations.
Figure A.2 compares {Jb j W/jb j ; j =1, ... , 19} for the FNB and SQRT rules. After
review 12, the FNB sequence fluctuates around the true aoo, whereas the SQRT sequence
522 A P PEN 0 I X, M 0 REO NTH E BA TC H - MEA N S MET HOD

350

,- ~
.
\
\
I \
300 - I
I

•
I

.
2.5 0 -

iif \
d 200- I • ~ ~ \ ,.

--------------------------------~-----------~---~-----~---~----
t
E' '
" ../ \ I

•
.,. 150-
<IJ

100 -
,, ~
•
. . . _.,,
' ..~
.. ' /

SO- I
I
I
~ .

0
0 5 10 15 20

Interim review J

Figure A.2 LABATCH.2 estimation of a oo for waiting time in queue (FNB rule: light
dashed line, SQRT rule: solid line, a oo = 189.3: heavy dashed line)

comes close to a oo after review 15. This contrast in behavior reflects the O{ l/tj) systematic
variance error dissipation rate for the FNB rule as compared to the 0{I/tJ I2) rate for the
SQRT rule. However, sampling fluctuations for the FNB sequence are 0(1), in contrast to
0{I/tJ I2) for the SQRT rule. We conclude that the FNB rule offers a benefit initially, but
the SQRT rule ultimately offers an advantage. The next section describes batching rules that
exploit the benefits of both rules.

A.3 CHOOSING [, h, AND hI (SECTION 6.6.6)

For given (II, bd E l3, choosing t such that log(t / II bd / log 2 is an integer results in t ' (t) =
t. As a consequence, the LBATCH and ABATCH rules use all t observations to estimate
WL](t)B]lt) and Xt'(t) = X t • Since choosing II> bl> and t subject to the constraint may be

too burdensome for some users, LABATCH.2 merely requires that t be specified and then
chooses II and b l to maximize t'(t).
Let l3(t) denote the subset of l3 that maximizes t'(t). LABATCH.2 chooses (II, b l )
to be the element of l3(t) that maximizes J(t), the number of interim reviews. As the
following example shows, this algorithm reduces the number of elements in l3 that need to
be considered in maximizing t'(t). Suppose that (ai, a2) and (a3, a4) E l3 maximize t'(t)
and that for some integer ex :::: 1, alaz = 2aa3a4. Then clearly, the choice (ai, az) induces
 A.3. C H 005 I N G t I II I AND b1 (S E C T ION 6.6.6) 523

TableA.l Data utilization for final estimate of a~ min, UmaxUl,bIlEB t'(t)]

10 < t < 24 25 < t < 49 50~t~99 100 ~ t ~ 499 500 ~ t ::::: 107
l\ ~ 10 .522 .706 .696 .688 .686
l\ ~ 20 .522 .706 .696 .798 .795
l\ ~ 30 .522 .706 .696 .805 .889
l\ ::::: 100 .522 .706 .696 .805 .898

more reviews than (a3, a4). As a consequence of this example, LABATCH.2 considers only
the underlined 2-tuples in B in Table 6.3 in maximizing t'(t). As illustration, t'(107)
9,175,040 is maximal for each 2-tuple in

B(107) = {(7, 5), (14, 5), (25,5), (10, 7), (20, 7), (14, 10), (28, 10), (20,14), (28, 20)}.

but (7, 5) allows for the maximal number of reviews, J(107) = 19.
For 10 :s t :s 107 , Table A.l shows the smallest proportion of observations used
for WLJII)BJ(I) when t'(t) is maximized subject to alternative upper bounds on II. Recall that
11 :s 100 implies optimization over all entries in B. In this case, choosingt ::: 500 implies that
no less than 89.8 percent of the data is used for computing WLlltl BJII)' Alternatively, imposing
the constraint 11 :s 30 in choosing (11, bd to maximize t'(t) reduces this percentage to 88.9.
We recommend either of these options for t ::: 500.
We now reconcile

Xt - fL (A.7)
t
A := JBJ(t) WLJ(t)BJIt)/t'

whose properties establish the basis for the confidence interval (6.48), with l(t)
LJ(t), b(t) = BJ(t) and Wl(t)b(t) = Vt'(t)/ BJ(t) and

Xt'(I) - fL
(A.8)
G I := JBJ(t) WLJ(I)BJltJt'(t) '

whose properties are described in Theorem 6.6 for j = J(t). Recall that t'(t) = tJ(t) =
LJ(t)BJ(t). Whereas At uses all t observations to estimate fL, G t uses only the firstt' (t) obser-
vations.However, both use BJ(t) WLJlt)BJ(tl asanestimateof(J'~. Since (Xt'(t) -fL)/ J(J'~/t'(t)
and (X t - fL)/J(J'~/t are both asymptotically N (0,1), by ASA, and since BLJ(t) WLJ(t)BJ(t)
is strongly consistent, it appears that a modification of the proof of Theorem 1 in Yarberry
(1993) can yield asymptotic normality for At as well. With regard to statistical efficiency, At
reduces interval width bya factor of 1 - Jt'(t)/t :s 1- J.889 = .0571 when (llbl) E B(t)
for t ::: 500.
524 A P PEN D I X. M 0 REO NTH E BAT C H - MEA N 5 MET HOD

A.4 REFERENCES

Chien, C.H. (1989). Small-sample theory for steady state confidence intervals, Technical
Report -37, Department of Operations Research, Stanford University.
Yarberry, L.S. (1993). Incorporating a dynamic batch size selection mechanism in a fixed-
sample-size batch means procedure, unpublished Ph.D. dissertation, Department of
Operations Research, University of North Carolina, Chapel Hill.
Author Index

AbouRizk, S.M., 481, 514 Best, D.J., 369, 411

Abramowitz, M., 410 Birtwistle, G., 33, 35
Afflerbach, L., 328, 410 Blomqvist, N., 245, 248, 292
Agrawal, A., 410 Bogyo, T.P., 451
Aho,A.V.,41O Box, G.E.P., 364, 411
Ahrens, J.H., 337, 341, 352, 361, 362, 364, Bray, T.A., 413, 414
365,368,369,375,381,385,400, Brockwell, P.J., 487, 514
410,411,427,462,514 Broder, A.Z., 412
Alexopoulos, c., 267, 292
Ananthanarayanan, K., 414 CACl, 100, 104, 126, 174,292
Anderson, T.W., 411 Carson, J.S., III, 324
Andres, T.H., 445, 446, 450 Cassels, J.W.S., 434, 450
Asau, Y., 334,412 Chang, S.-L., 513, 515
Atkinson, A.C., 375, 381, 411 Chen, H.-C, 334, 412
Cheng, R.C.H., 67, 69,369,371,372,373,
Babu, A.J.G., 377, 414 374,377,378,399,412
Balakrishnan, N., 413,514 Chien, C.H., 251, 292, 520, 524
Banks, J., 163, 173,307,324 Chow, Y.S., 268, 292
Bechhofer, R.E., 296, 324 Chung, K.J.S., 198, 199,210,217
Beckman, R.J., 480,514 Conover, W.J., 482, 514
526 IN D EX

Cornish, E.A., 345, 412 Kachitvichyanukul, Y., 381, 384, 388, 413,
414
Dahl, O.J., 35 Kao, E.P.C., 513, 515
Daley, D., 38, 69 Karlin, A.R., 412
Damerdji, H., 252, 292 Karr, H.W., 104, 174
Davis, R.A., 487,514 Kelton, W.D., 100, 148, 149, 174,254,268,
Derflinger, G., 328,413,427,450 293,296,307,324,480,515
Devroye, L., 329, 330, 367, 381, 385, 412 Kendall, M.G., 324
Dieter, u., 341, 343, 352, 361, 362, 364, Kinderman, A.J., 352, 355, 402, 413
365,368,369,375,381,385,400, Kingston, J.H., 209, 217
411,412 Kiviat, P.J., 104, 174
Knuth, D., 186,217
Easton, M.C., 415 Kohrt, K.D., 462, 514
Evans, J.B., 203, 217 Kollerstrom, J., 182,217,465,466,515
Kotz, S., 413, 514
Feast, G.M., 369, 371, 372, 374, 412 Kovalenko, LN., 465, 514
Feller, W., 412 Kronmal, R.A., 343, 413
Fielitz, B.D., 480 Kuhl, M.E., 513, 515
Fieller, E.C., 282, 292 Kulkarni, Y.G., 413,515
Fisher, R.A., 345, 412 Kumaran, K., 60
Fishman, G.S., 40, 69,148,159,174,259,
265,268,292,296,324,334,336, Lal, R., 369, 414
343,378,385,399,412,413,450 Lamb, J.D., 67, 69
LaVal, D.K., 69
Gerontidis, 1., 413 Law, A.M., 148, 174,293,296,307,324,
Gibson, R., 163, 173 480,515
Gnanadesikan, R., 480, 514 L'Ecuyer, P., 442, 443, 444, 445, 446, 450
Gnedenko, B.V., 465,514 Lewis, P.A.w., 515
Goldsman, D., 250, 292, 293, 324 Lurie, D., 413
Gordon, G., 58, 69,149,174
Grigelionis, B.I., 465, 514 Ma, X., 20, 35
MacLaren, M.D., 339, 361,413,414
Halpin, D.W., 514 Markowitz, H.M., 104, 174
Hartley, H.O., 413 Marsaglia, G., 358, 364, 369, 413, 414
Hastings, c., Jr., 382, 413 Marsaglia, J., 414
Hausner, B., 104, 174 Matejcik, EJ., 296, 324, 325, 404
Henriksen, J.O., 100, 174,207,209,214, McCormack, W.M., 198, 199,209,217
217 Meketon, M.S., 251, 293
Hoeffding, W., 413 Melamed, B., 292
Hopcroft, J.E., 410 Miller, K.w., 427, 450
Hormann, W., 328, 410, 413, 427, 450 Minh, D.L., 369, 414
Huff, B., 35 Mitra, D., 60
Hughes, L.P., 514 Monahan, J.E, 352, 355, 402, 413
Moore, L.R., 334, 336, 412, 413, 450
Jannson, B., 424, 450 Moro, B., 331, 414
Johnson, M.A., 515 Muller, M.E., 364, 411
Johnson, N.L., 378, 413, 470, 474, 514 Myers, B.L., 480
Jones, D.W., 200, 210, 217 Myhraug, B., 35
 IN D EX 527

Naderisamani, A., 385 Shannon, R.E., 174

Neely, P.M., 238, 293 Shedler, G., 515
Nelson, B.L., 296, 324, 325, 515 Smith, RL., 413
Nygaard, K., 35 Song, W.T., 251, 293
Stadlober, E., 352, 377, 382, 384, 385, 386,
O'Reilly, J.J., 69,100,174 389,414,415
Overstreet, c.L., Jr., 425, 450 Stegun, I., 410
Stidham, S., Jr., 93, 97
Palaniswamy, S., 35 Stout, w., 252, 293
Park, S.K., 427, 450 Stuart, A., 324
Paul, N.J., 413 Swain, J.J., 30, 35, 99, 174
Payne, W.H., 451 Systems Modeling Corporation, 149, 156,
Pegden, C.D., 150, 174 174
Peterson, A.Y., Jr., 343, 413
Philipp, w., 252, 293 Tezuka, S., 442, 443, 450
Pinkham, R.S., 514 Tietjen, G.L., 480, 514
Pritsker, A.A. B., 40, 69, 100, 174 Tzenova, E., 15, 35

Rabung, J.R., 451 Ullman, J.D., 410

Rego, V., 217 Villanueva, R., 104, 174
Robbins, H., 268, 292 von Neumann, J., 260, 293, 345,414
Ross, S.M., 414, 515
Wagner, M.A., 485, 515
Sadowski, D.A., 174,293 Walker, A.J., 342, 414, 415
Sadowski, RP., 174, 293 Whitman, L., 34, 35
Sang, J., 217 Whittaker, J., 375, 411
Santner, T.J., 324 Wilks, S.S., 415
Sargent, RG., 198, 199,209,217 Williams, J.w.J., 203, 217
Satyanarayana, A., 410 Wilson, J.R, 485, 514, 515
Schechner, Z., 414 Wong, C.K., 415
Scheffe, H., 325
Schmeiser, B.W., 251, 293, 369, 377, 381, Yamnitsky, M., 515
384,385,388,413,414 Yarberry, L.S., 259, 265, 267, 293, 413,523,
Schrage, L., 427, 451 524
Schreider, Y.A., 414 Young, L.c., 260, 293
Schriber, T.J., 33, 35,100,174
Schucany, W.R., 414 Zacks, S., 515
Seila, A.T., 292 Zaman, A., 414
Servi, L., 68, 69 Zechner, H., 377,415
Subject Index

ABATCH, 250, 519 Algorithm

rule, 252,256,259, 260, 262 ALIAS, 342, 505
Acceptance-rejection method, 329, 337, ALSET,343
345,348,356,362,395,398 AR, 346, 347,438
Access, 30 ARS,350
Accumulated round-off error, 334 AW,376
Accuracy of the sample average, 232 BB,377
Active processes, 57 BB*,378
Activity, 39 BER,385
Adequacy of fit, 482 BISECTION, 330
Advanced process, 149 BRUA, 380, 393
Advanced transfer, 149 BRUA*, 386, 387
AGV, 31,101,103 CA, 360, 365, 366
AIR-EV.sim, 105, 229 CLEAR, 510
program, 105,229 C~,336,337,397,408,505
sample output, 129, 130 C~SET, 336, 408
Airline example, 300 CO~P~,338
Airline reservation problem, 271 DI, 333, 334
Airline reservation simulation, 282 EA, 361, 362
530 INDEX

Algorithm (cont.) Arrival rate, 85

EAM,359 ASA, 252, 259
GEO,391 ASRS, 31,102,103
GKMl,373 ASSEMBLE statement, 33
GKM2,373 Assumption of strong approximation, 252
GKM3, 313, 374 Assumption of weak approximation, 239
GS,368 Asymptotic normality, 473
GS*,369 Asymptotic unbiasedness, 89
H-L,208 Attribute, 38, 43, 104, 485
H-N,438 Autocorrelated sequences, 250
HRUA,389 Autocorrelated function, 242, 245, 260,
HRUA*, 389, 390 261,263,487
HRUN,389 Autocovariance function, 241
HYP, 388, 390 Automated storage and retrieval systems,
ITR, 333, 334, 381, 387,388,391,505 31
MULTN, 393, 394 Automatic guided vechicles, 31, 101
NA, 364, 365 Automatic storage and retrieval systems,
NEGBIN,392 102
NHPP, 508 AutoMod, 31, 32
N-R, 330,475 Average computing time, 213
PD, 381, 384 Average queue length, 75
Average throughput, 11
PRUA,382
AWA,239
PRUA*, 382, 383, 386
PRUN,382
BAAT panels, 149, 150, 162
ROU,354
Basic process, 149
RT,364
Batch arrivals, 455
RWTE, 340
Batch averages, 260
SA, 361
BATCH~EANS, 270
SIFTDOWN(x), 204
Batch-means method, 241, 246, 255, 517
SIFTUP(x),204
Arena, 254
SKIP, 394
Bernoulli
SPACE_FINDER, 191
distribution, 477
SPACEJlINDER*, 194, 196
sampling, 342
SPACEJlINDER*', 196, 197
trials, 344
SPACE-RELEASER,191 Best among alternative systems, 296
SPACE-RELEASER*, 195 BestFit, 467
WASTE,505 Beta distribution, 331, 375, 479
Alias method, 341, 395, 397, 408, 461 Beta-prime distribution, 378, 481
Analysis of variance, 486, 489 Binary heap, 203, 206, 211
ANOVA, 489, 490 Binomial distribution, 335, 384, 406
Antithetic variates, 296 Binomial generation, 406
Arena, 99,220, 268 Block, 44, 101, 149, 154
batch means, 273 Block arrays, 192
Arena/SIMAN, 231, 248 Blocks panels, 162
airline reservation example, 156 Bonferroni's inequality, 298, 321
Arithmetic relationships, 40 Bounded mean computing time,S, 328
Arrival process, 453, 454, 455 Bridge cranes, 31, 103
 INDEX 531

Brownian motion, 238, 239 Confidence interval, 159,219,232,240,

Buffer occupancy, 6, 7l 246,263,275,282,295,505
Buffer, 7, 58, 59, 61 asymptotically valid, 219
Byte-addressable memory, 207 difference of means, 321
Congestion, 96
C,30, 33, 100, 151, 165,246,265,267,270, Congruence, 417
284 Congruential pseudorandom number
C++, 30, 33,100,151 generators
CAD,30 linear, 418
CALL event, 48, 120 multiplicative, 418, 447
Capacitated travel, 211 Continuous distributions, 330
Capacity, 59 Control variates, 296
Cauchy distribution, 365, 366, 405 Convergence, 84, 85, 275
Central Limit Theorem, 236 in distribution, 89
Correlation, 312
CES, 56, 57, 64
Covariance, 241
Choosing a distribution, 467
cpn, 296, 316,317, 318, 320,321
Class attribute, 49
Credibility analysis, 296, 304, 311
Clearing books, 508
Critical value, 301
Clock attribute, 49, 207
CST panels, 150
Closed-loop systems, 8
Customer, 43
Coefficient of variation, 182
CUSTOMER process, 55
COLLECT evert, 99
Cutpoint method, 334, 337, 395, 397, 407,
Collecting data, 131
461
Combined generator, 442, 443
Cutpoints, 334
Common panel, 150, 151, 153
Common pseudorandom number streams,
Dangerous multipliers, 439
146, 165 Data collection, 71, 171
Common pseudorandom numbers, 142, Dbase, 30
145,296,300,311,321 Deactivation point, 54
Communications networks, 6 Debugging, 162
Comparing two means, 300 Dec workstations, 100
Comparison of means Decision flow, 483, 485
on two sample paths, 306 Degrees of freedom, 303
Comparison of results, 296 Delay, 6, 7l, 96
COMPLETION event, 49, 122 Delay models, 211
Composition method, 329, 338, 341, 395 Dependent historical data, 485
Compound events, 44, 62 Destination node, 58
Computational complexity,S Detail,24
Computational properties, 232 Deterministic, 12
Computer simulation, 26 Discrete distributions, 332, 334, 335
Computer-assisted design, 30 Discrete-event simulation, 5, 455, 513
Computing time, 178 teaching, 34
Conditional distribution, 81 web-based,33
Conditional event, 44 Discrete-event systems,S
Conditional mean function, 241 Discrete-valued variables, 154
Conditional probability, 345 Distance between parallel hyperplanes,
Conditional-event set, 56, 64 436
532 IN D EX

Distribution, 97 Feel Good Incorporated, 19

Distribution systems, 6 FES,47,50,56, 79,181,183,204,205
Dynamic space management, 190 manager, 209
Final Tableau, 264
Efficiency, 201,473 Finite sample average, 220
Elementary events, 43, 62 Finite sample-path length, 88
Elements, 149, 155 Fitting distributions, 483
Elements panels, 162 Fixed-time advance, 50
Empirical and fitted distributions, 456 Flow logic, 52
Empirical data,S, 454 Flowcharts, 43
Empirical D.E, 457 Fluid,58
Empirical distribution function, 92 Fluid-flow, 59
Empirical histogram, 457 arrivals, 456
EMPTY event, 61 compound events, 61
END-MESSAGE event, 49,121 FNB rule, 253, 260, 517
Entities, 104, III FORTRAN, 33,165,246,265,267,270,
Entity, 38, 43 284,361
Entity record, 48 Fox Pro, 30
Equilibrium distribution, 81, 83 FULL event, 61
Erlang distribution, 483 Full period, 420
Errors in optimality, 319 Future-event set, 39,46,64,79, 119, 197,
Errors of measurement, 27 206,211
.Euler totient function, 420
Event, 39,62 Gamma
management, 119 distribution, 367, 478, 491,500
notice, 39, 49, 111 fit, 494
Events generation, 405, 408
in SIMAN and SLAM, 162 GASP, 150
Event-scheduling approach, 41, 47, 48, 49, Generalized Beta distribution, 481
50,62,104 Geometric distribution, 335, 390, 477
Exact-approximation method, 329, 358, Getting an executable program, 125
395,404 Global variables, 105
Exceedance probabilities, l3, 96, 97, GPSS, 33, 149, 150
299 GPSS/H, 30, 34, 41, 52,100,103,117,165,
Excel,30 166,210,421,428,435
Execution, 95
EXP file, 154 Half-normal
Experiment source file, 154 distribution, 351
Expert opinion, 466, 513 generation, 348, 349, 357
ExpertFit, 467 Header, 39, 187
Exponential Health Sciences Library, 14
distribution, 331, 361, 476 Health-care delivery systems, 6
generation, 404 Heap, 203, 205
limit, 464 Heavy traffic, 465
tails, 468 Henriksen's algorithm, 207, 210, 211
Histogram, 161
cp-mixing, 250 HSL,14
FACTOR/AIM,31 Hyperexponential tails, 468
 INDEX 533

Hypergometric K-S confidence region, 460

distribution, 335, 387 K-S distribution, 459
generation, 407 k-tuples, 433
Hyperplanes, 435
LABATCH.2, 4, 220, 241, 246, 250, 253,
I.i.d. replications, 232 255,260,262,265,284,308,517,
IBM-compatible PCs, 100 519,521
Icons, 101 tableaus, 248
Implementation, 426 Lagrange multiplier, 432
LABATCH, 2, 270 LBATCH rule, 256, 257
Incinerator, 19 Length of the warm-up interval, 228
Increasing efficiency, 295 Library photocopying, 307,488
Independent Likelihood, function, 491, 493
Bernoulli trials, 394 Limiting properties, 90
replication, 118 Linear event list
replications, 225, 235 multiple, 201
simulation run, 232 single, 199
Infinite-server system, 213 Linear linked list, 211
Inflow rate, 61 Linear regression, 501
Inheritability, 33 Linkage editing step, 126
Initial conditions, 78, 82, 96, 118,236,283 Linked list, 39
Initializing distribution, 83 List processing, 186
Input, 85, 453 List, 187
prepartation, 453 Little's law, 93, 97, 282
process, 454 Load module, 100, 126
Insertion and deletion, 189 Loading tankers, 65, 286, 322
Instantaneous intensity function, 507 Local dependence, 486
Interacting with the analysis, 267 Local variables, 105
Interarrival times, 182,489 Logical relationships, 40
Interim review, 253 Logical system-operating rules, 296
Invariance, 473 Logical-operating rule, 40
Inventory systems, 6 Logistic distribution, 331
Inverse distribution function, 329 Lognormal distribution, 366, 468, 476
Inverse transform method, 329, 332, 334, 338 Long-run average, 96, 234
Inverse Weibull queuelength,85
distribution, 363, 469 waiting-time analysis, 310
m.l.e.s., 481 Long-term averages, 78
Inverse-transform approach, 395 Long-term system average, 220
Inverted Gamma Loss rate, 290
distribution, 363, 469
m.l.e.s., 482 M.l.e., 470, 476
M/MIl
JAVA, 33 model, 278
Job-selection rule, 40 queueing model, 245
simulation, 264
Kolmogorov-Smirnov Macromodules, 101, 163
distribution, 459 Manufacturing and materials handling
test, 487 systems, 102
534 INDEX

Manufacturing plants, 6 initial states, 227

Matching resources, 483 linear linked list, 211
Mathcad, 475 pseudorandum number streams, 445
Mathematica, 475 replications, 284, 320, 441
Maximal Multiplicative
achievable period, 423 congruential generator, 420, 422
distance, 433, 434 inverse, 420, 421
period generators, 423 pseudorandum number generators, 442
period, 424, 447 Multiplier, 418, 434
Maximum likelihood
estimates, 503 Natural ordering, 308
estimation, 472 Negative binomial distribution, 335, 391,
method,485 472,478,495
Mean,476 Negative binomial generation, 407, 409
intercall time, 115 Negatively correlated, 313
service time, 115 Newton-Raphson method, 330, 345, 475
Mean-square Next-event
convergence, 251 approach, 50
error, 89, 251 selection, 44
Memory space, 96, 178 time advance, 39
Memory space, 96 No programming, 34
Microsoft Non-central Cauchy distribution, 331
PowerPoint, 30 Nonhomogeneous Poisson process, 506,
Windows 29, 98 507,511,513
Word,30 Nonnormality, 500
Minimal model, 25 Nonoverlapping batches, 246
Mixed congruential generators, 425 Nonstationary, 115
Mobile-telephone service, 68,173,289,323 Normal distribution, 235, 331, 364, 476
MOD file, 154 Normal generation, 405
Modalities for programming, 100 Number of pages, 495
Mode, 476 Number of parallel hyperplanes, 435
Model,38 Number of reservationists, 115
Model file, 154 Number waiting
Modeling concepts, 37 average, 11
Modeling dangers, 26
MODSIM III, 32 Object program, 100
Module opaqueness, 165 Object-oriented programming, 31
Modulo, 417 Off-peak hours, 494
Modulo reduction, 425 Open-loop
Modulus, 418 queueing system, 200
Modulus 2fJ , 422 system, 8, 85
Monte Carlo, 327 Opt, 296,315,317,318,320,322
experiment, 385, 446 Optimal sample-path lengths, 296, 315
Multinomial distribution, 392 Origin node, 58
Multiple Outflow rate, 61
comparisons, 306, 315 Output analysis, 219
independent replications, 223 Output, 160,295
 INDEX 535

Packet, 58 Pseudorandom number generation,S, 116,

Pairwise comparisons, 315 395,417-451
Pairwise differences, 307 Pseudorandom number generator, 117,
Parallel hyperplanes, 430, 433 128,159,417
Parameters, 219 Pseudorandom number streams, 142
Pareto distribution, 331 Pseudorandom numbers, 328, 441
Passive resources, 57 p-value, 262, 263, 490
Path-length parameters, 118
P-code,100 Quality control, 8
Performance statistics, 72 Quantile method, 462, 463
Performance, 7 Quarry-stone crushing, 64, 167,285,322
Period, 419 QUEUE, 57, 103
Periodic data collection, 94 Queue length average, 13
Periodogram, 487 Queue length, 7, 11,71,74,85
Permanent entities, 111 Queueing discipline, 8
Photocopying, 14 Queueing model, 8
problem, 280, 310
Random error, 225
time, 497
Random sampling fluctuations, 97
Plausibility arguments, 453
RANDOM, 449
Png, 117,423
RANDU,441
Point, click, drag, drop, and connect, 100
RATE event, 62
Point-click-drag-drop-connect approach,
Ratio analysis, 305
149
Ratio estimates, 97, 284
Pointers, 334
Ratio estimation, 278
Poisson
Ratio-of-uniforms method, 329, 352, 356,
distribution, 335, 380, 472, 477, 495, 507
357,370,379,395,402
generation, 406, 408
Reactivation point, 39, 54, 64
Portability, 99, 426, 427
Rectangle-wedge-tail method, 339
Power, 467
Regression methods, 513
Prebatching, 278 Relative errors, 295
Predecessor address, 39 RELEASE, 57
Predecessor-successor relationship, 187, Renewal process, 453, 464
188 Replicate, 27
PRIME, 485 Reproducibility, 418
Prime modulus, 420 REQUEST, 57
Primitive root, 420, 430 Reservationist, 43
Priorities, 121 Resizing, 196
Priority order, 52, 112, 170 Resource in SIMSCRIPT II, 5, 132
Probability of waiting, 16 Resource utilization, 6, 71, 76, 85
Process, 39, 53 Resources, 7, 13 7
Processes, 13 7 Restricted sampling, 332, 337
in SIMSCRIPT 11.5, 132 Robots, 31,103
Process-Interaction approach, 52, 55, 62,131 Round-off error, 238
Programming, 95 Routing, 7, 483
ProMod,32
ProModel, 31 Sample averages, 96
Pseudorandom integers, 446 Sample path,S, 73, 233
536 INDEX

Sample path oflength, 240 Spectrum, 487

Sample record, 73, 78, 242 SPLIT statement, 33
Sample-path SQRT rule, 256, 258, 520
data, 220, 283 Squeeze method, 349, 351, 352, 395, 400
length, 79 Stability, 12
Sampling Standard errors, 295
error, 219, 220 Standard normal distribution, 89
from diverse distributions, 5 Starting execution, 119
from probability distributions, 327-415 Stat::Fit, 467
from tables, 337 State, 5, 79
properties, 459 Statistical accuracy, 78
Scenarios, 312 Statistical properties, 232
Schedule of executions, 46 Steady-state distribution, 83
Scheduling, 7, 8, 201, 483 Stopping execution, 119
SEIZE, 57, 103 Storage capacity, 19
Selecting the next event, 201 Stratified sampling, 296
Sequencing, 7, 8, 483 Strict stationarity, 241
Sequential sampling experiment, 346 Strictly stationary stochastic processes, 241
Strong consistency, 473
Series-parallel queueing model, 179
Strongly consistent, 240
Server utilization average, 12
Strongly consistent estimator, 86, 267
Service routine, 50
Student's t distribution, 232, 301, 309
Service-time e.dJ., 479
Subhyperexponential tails, 468
SIMAN, 30, 34, 41, 52, 99,100,117,150,
Successor address, 39
154,165,166,421,428,235
Summary statistics, 112
blocks and elements, 157
Sun workstations, 100
SIMDEBUG, 126
Superposition, 464
SIMFACTORY II.s, 31
Support, 470
SIMSCRIPT II.s, 26, 30, 34, 41, 52, 99, Support panel, 150
100,103,104,117,165,210,221, System
229,246,265,267,270,275,284, loss rate, 7, 71
421,428,435 performance,S
SIMULA,33 state, 79
Simulation programming time, 71
minimal approach, 163 Systematic error, 88, 219, 225
programming language, 45, 205 Systematic variance error, 248, 251
Simultaneity, 296, 307
Simultaneous Tables, 343
confidence intervals, 460 Tanker loading, 170
confidence statements, 296, 297 Taylor II, 31
Single-server queueing problems, 225 Telecommunications
SLAM, 30,34,41, 52, 100, 117, 165, 166, network modeling, 58
210,435 Templates, 100
SLX, 34,100 Temporary entities, III
Source program, 100, 126 Terminal packet assembly/disassembly
Source of variation, 26 controller model, 67
Space, 206 Terminal pad controller, 323
Spectral test, 434, 439, 444 model, 172,216,287
 INDEX 537

Terminating execution, 90 Variance inflation, 244, 245

Testing Ho, 260 Variance reduction, 145
Throughput, 6,19,71 Variance-reducing
Time methods, 148
averages, 10 techniqie, 296, 315
dependence, 486 Virtual machine, 100
dilation, 26 Virtual Basic, 30, 151
Time-persistent variables, 74, 154 von Neumann ratio, 260, 487
Timing routine, 39, 47, 56, 64, 119
Traffic intensity, 115, 181,200,245 Waiting time, 7, 13, 14,71,77,85,92
Trailer, 39, 189 average, 12
Transfer process panel, 150 long-term average, 92
Transportation networks, 6 mean, 16
Truncated sample paths, 88 Warm-up
Truncated time averages, 221 interval, 11,79,84,97,220,225,229,
240,264,283
Unbiased estimator, 242 parameters, 118
Uncapacitated travel, 212 Waste generation, 502
Unconditional event, 44 Weakly consistent estimator, 252
Unequal sampling costs, 316 Weibull distribution, 331, 362, 478
Uniform distrubution, 331, 335, 476 Welch's solution, 303
Uniqueness, 473 Weiner process, 239
University of North Carolina at Chapel Windows NT, 29
Hill, 14 Windows-based operating systems,
Unix envoronments, 29 100
WITNESS, 31, 32
Variable transit times, 212 Word,207
Variable-time advance, 50 Work-in-process, 7