Modeling and analysis of stochastic systems 2nd ed Edition Kulkarni 2024 scribd download
Modeling and analysis of stochastic systems 2nd ed Edition Kulkarni 2024 scribd download
com
https://ebookname.com/product/modeling-and-analysis-of-
stochastic-systems-2nd-ed-edition-kulkarni/
OR CLICK BUTTON
DOWLOAD EBOOK
https://ebookname.com/product/modeling-and-analysis-of-dynamic-
systems-second-edition-esfandiari/
ebookname.com
https://ebookname.com/product/micromechatronics-modeling-analysis-and-
design-with-matlab-2nd-ed-edition-giurgiutiu/
ebookname.com
https://ebookname.com/product/analysis-and-design-of-heating-
ventilating-and-air-conditioning-systems-2nd-ed-edition-spach/
ebookname.com
https://ebookname.com/product/multiple-sclerosis-and-related-
disorders-1st-edition-douglas-s-goodin-eds/
ebookname.com
Netter s Concise Radiologic Anatomy Netter Basic Science
1st Edition Edward Weber Do
https://ebookname.com/product/netter-s-concise-radiologic-anatomy-
netter-basic-science-1st-edition-edward-weber-do/
ebookname.com
https://ebookname.com/product/advances-in-strength-and-conditioning-
research-1st-edition-michael-duncan/
ebookname.com
https://ebookname.com/product/endometriosis-an-enigma-1st-edition-
seema-chopra-editor/
ebookname.com
https://ebookname.com/product/contemporary-psychoanalytic-field-
theory-stories-dreams-and-metaphor-s-montana-katz/
ebookname.com
https://ebookname.com/product/pragmatic-version-control-using-cvs-2nd-
edition-edition-david-thomas/
ebookname.com
Handbook of Neurocritical Care 1st Edition Wendy C. Ziai
(Auth.)
https://ebookname.com/product/handbook-of-neurocritical-care-1st-
edition-wendy-c-ziai-auth/
ebookname.com
Modeling and
Analysis of
Stochastic
Systems
Second Edition
CHAPMAN & HALL/CRC
Texts in Statistical Science Series
Series Editors
Bradley P. Carlin, University of Minnesota, USA
Julian J. Faraway, University of Bath, UK
Martin Tanner, Northwestern University, USA
Jim Zidek, University of British Columbia, Canada
Modeling and
Analysis of
Stochastic
Systems
Second Edition
Vidyadhar G. Kulkarni
Department of Statistics and Operations Research
University of North Carolina at Chapel Hill
U.S.A.
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2009 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid-
ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright
holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may
rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti-
lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy-
ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
To
my wife
Radhika
Jack and Harry were lost over a vast farmland while on their balloon ride.
When they spotted a bicyclist on trail going through the farmland below, they
lowered their balloon and yelled, “Good day, sir! Could you tell us where we
are?”
The bicyclist looked up and said, “Sure! You are in a balloon!”
Jack turned to Harry and said, “This guy must be a mathematician!”
“What makes you think so?” asked Harry.
“Well, his answer is correct, but totally useless!”
The author sincerely hopes that a student mastering this book will be able to
use stochastic models to obtain correct as well as useful answers.
Contents
Preface xix
1 Introduction 1
xi
xii CONTENTS
3.1 Definitions 55
3.2 Cumulative Distribution Function of T 56
3.3 Absorption Probabilities 60
3.4 Expectation of T 69
3.5 Generating Function and Higher Moments of T 74
3.6 Computational Exercises 76
3.7 Conceptual Exercises 81
Epilogue 489
References 533
Index 539
Preface
Of course, if the results of Step 2 show that the model does not “fit” the real-life situa-
tion, then one needs to modify the model and repeat Steps 1 and 2 until a satisfactory
solution emerges. Then one proceeds to Step 3. As the title of the book suggests, we
emphasize the first two steps. The selection, the organization, and the treatment of
topics in this book are dictated by the emphasis on modeling and analysis.
Based on my teaching experience of over 25 years, I have come to the conclusion
that it is better (from the students’ points of view) to introduce Markov chains be-
fore renewal theory. This enables the students to start building interesting stochastic
models right away in diverse areas such as manufacturing, supply chains, genet-
ics, communications, biology, queueing, and inventory systems, etc. This gives them
a feel for the modeling aspect of the subject early in the course. Furthermore, the
analysis of Markov chain models uses tools from matrix algebra. The students feel
comfortable with these tools since they can use the matrix-oriented packages, such
as Matlab, to do numerical experimentation. Nothing gives them better confidence
in the subject than seeing the analysis produce actual numbers that quantify their in-
tuition. We have also developed a collection of Matlab-based programs that can be
downloaded from:
1. www.unc.edu/∼vkulkarn/Maxim/maxim.zip
2. www.unc.edu/∼vkulkarn/Maxim/maximgui.zip
The instructions for using them are included in the readme files in these two zip files.
After students have developed familiarity with Markov chains, they are ready for
renewal theory. They can now appreciate it because they now have a lot of renewal,
renewal-reward, or regenerative processes models. Also, they are more ready to use
the tools of Laplace transforms.
xix
xx PREFACE
I am aware that this sequence is contrary to the more prevalent approach that starts
with renewal theory. Although it is intellectually appealing to start with renewal the-
ory, I found that it confuses and frustrates students, and it does not give them a feel for
the modeling aspect of the subject early on. In this new edition, I have also changed
the sequence of topics within Markov chains; I now cover the first passage times be-
fore the limiting behavior. This seems more natural since the concepts of transience
and recurrence depend upon the first passage times.
The emphasis on the analysis of the stochastic models requires careful develop-
ment of the major useful classes of stochastic processes: discrete and continuous time
Markov chains, renewal processes, regenerative processes, and Markov regenerative
processes. In the new edition, I have included a chapter on diffusion processes. In or-
der to keep the length of the book under control, some topics from the earlier edition
have been deleted: discussion of numerical methods, stochastic ordering, and some
details from the Markov renewal theory. We follow a common plan of study for each
class: characterization, transient analysis, first passage times, limiting behavior, and
cost/reward models. The main aim of the theory is to enable the students to “solve”
or “analyze” the stochastic models, to give them general tools to do this, rather than
show special tricks that work in specific problems.
The third aspect, the implementation, involves actually using the results of Steps
1 and 2 to manage the “real-life” situation that we are interested in managing. This
requires the knowledge of statistics (for estimating the parameters of the model) and
organizational science (how to persuade the members of an organization to follow
the new solution, and how to set up an organizational structure to facilitate it), and
hence is beyond the scope of this book, although, admittedly, it is a very important
part of the process.
The book is designed for a two-course sequence in stochastic models. The first
six chapters can form the first course, and the last four chapters, the second course.
The book assumes that the students have had a course in probability theory (measure
theoretic probability is not needed), advanced calculus (familiarity with differential
and difference equations, transforms, etc.), and matrix algebra, and a general level
of mathematical maturity. The appendix contains a brief review of relevant topics. In
the second edition, I have removed the appendix devoted to stochastic ordering, since
the corresponding material is deleted from the chapters on discrete and continuous
time Markov chains. I have added two appendices: one collects relevant results from
analysis, and the other from differential and difference equations. I find that these
results are used often in the text, and hence it is useful to have them readily accessible.
The book uses a large number of examples to illustrate the concepts as well as
computational tools and typical applications. Each chapter also has a large number
of exercises collected at the end. The best way to learn the material of this course
is by doing the exercises. Where applicable, the exercises have been separated into
three classes: modeling, computational, and conceptual. Modeling exercises do not
involve analysis, but may involve computations to derive the parameters of the prob-
lem. A computational exercise may ask for a numerical or algebraic answer. Some
PREFACE xxi
computational exercises may involve model building as well as analysis. A concep-
tual exercise generally involves proving some theorem, or fine tuning the understand-
ing of some concepts introduced in the chapter, or it may introduce new concepts.
Computational exercises are not necessarily easy, and conceptual exercises are not
necessarily hard. I have deleted many exercises from the earlier edition, especially
those that I found I never assigned in my classes. Many new exercises have been
added. I found it useful to assign a model building exercise and then the correspond-
ing analysis exercise. The students should be encouraged to use computers to obtain
the solutions numerically.
It is my belief that a student, after mastering the material in this book, will be well
equipped to build and analyze useful stochastic models of situations that he or she
will face in his or her area of interest. It is my fond hope that the students will see
a stochastic model lurking in every corner of their world as a result of studying this
book.
Vidyadhar Kulkarni
Department of Statistics and Operations Research
University of North Carolina
Chapel Hill, NC
CHAPTER 1
Introduction
The discipline of operations research was borne out of the need to solve military
problems during World War II. In one story, the air force was using the bullet holes
on the airplanes used in combat duty to decide where to put extra armor plating. They
thought they were approaching the problem in a scientific way until someone pointed
out that they were collecting the bullet hole data from the planes that returned safely
from their sorties.
Consider a system that evolves randomly in time, for example, the stock market
index, the inventory in a warehouse, the queue of customers at a service station,
water-level in a reservoir, the state of the machines in a factory, etc.
Suppose we observe this system at discrete time points n = 0, 1, 2, · · ·, say, every
hour, every day, every week, etc. Let Xn be the state of the system at time n. For
example, Xn can be the Dow-Jones index at the end of the n-th working day; the
number of unsold cars on a dealer’s lot at the beginning of day n; the intensity of the
n-th earthquake (measured on the Richter scale) to hit the Continental United states
in this century; or the number of robberies in a city on day n, to name a few. We say
that {Xn , n ≥ 0} is a discrete-time stochastic process describing the system.
If the system is observed continuously in time, with X(t) being its state at time
t, then it is described by a continuous-time stochastic process {X(t), t ≥ 0}. For
example, X(t) may represent the number of failed machines in a machine-shop at
time t, the position of a hurricane at time t, or the amount of money in a bank account
at time t, etc.
More formally, a stochastic process is a collection of random variables
{X(τ ), τ ∈ T }, indexed by the parameter τ taking values in the parameter set T .
The random variable takes values in the set S, called the state-space of the stochastic
process. In many applications the parameter t represents time. Throughout this book
we shall encounter two cases:
1
2 INTRODUCTION
1. T = {0, 1, 2, · · ·}. In this case we write {Xn , n ≥ 0} instead of {X(τ ), τ ∈ T }.
2. T = [0, ∞). In this case we write {X(t), t ≥ 0} instead of {X(τ ), τ ∈ T }.
t
(a) Continuous-time, discrete state space
X(t)
t
(b) Continuous-time, continuous state space
Xn
× × × ×
× × ×
× × ×
× × ×
n
(c) Discrete-time, discrete state space
Sir: I have the honor to inform you that I held a council to-
day with the Arapahoes and Apache Indians, at which I
explained to them why their arms and ammunition had been
withheld; that the white settlers were now well armed and
determined that no more raids should be made through their
country by large bodies of Indians; and that while the whites
were friendly and well disposed toward the Indians, yet if the
Indians attempted another raid such as they recently made on
the Kaw reservation, I feared themselves and the whites would
have a fight, and that it would bring on war.
The head chief of the Arapahoes, Little Raven, replied “that
no more trips would be made by his people into the settlements:
that their hearts were good toward the whites, and they wished
to remain at peace with them.” I told him I would now give
them their arms and ammunition; that I hoped they would use
them for the sole purpose of securing food for themselves and
families, and that in no case would I ever hear of their using
these arms against their white brethren. Little Raven and the
other chiefs then promised that these arms should never be
used against the whites, and Agent Wynkoop then delivered to
the Arapahoes one hundred pistols, eighty Lancaster rifles,
twelve kegs of powder, one and one-half kegs of lead, and
fifteen thousand caps; and to the Apaches he gave forty pistols,
twenty Lancaster rifles, three kegs of powder, one-half keg of
lead, and five thousand caps, for which they seemed much
pleased.... I would have remained here to see the Cheyennes
did I deem it important to do so. From what I can learn there
will be no trouble whatever with them. They will come here, get
their ammunition and leave immediately to hunt buffalo. They
are well and peacefully disposed toward the whites, and, unless
some unlooked-for event should transpire to change their
present feelings, they will keep their treaty pledges.
This was before the peace policy had become supreme, or the
appointment of agents from the Society of Friends had been
discovered as a supposed panacea for all our Indian difficulties.
General Sherman, as stated in his letter, forwarded all the papers
relating to the arms question to the headquarters of the army.
General Grant, then in command of the army, forwarded them to the
Secretary of War, accompanied by the following letter, which clearly
expresses the views he then held:
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
ebookname.com