FUNDAMENTALS OF APPLIED PROBABILITY THEORYMcGraw-Hill Classic Textbook Reissue Series AMYX, BASS and WHITING: Petroleum Reservoir Engineering: Physical Properties CHOW: Open-Channel Hydraulics DAVENPORT: Protability Random Process: An Introduction for Applied Scientists and Engineers DRAKE: Fundamentals of Applied Probability Theory GOODMAN: Introduction to Fourier Optics HARRINGTON: Time-Harmonic Electromagnetic Fields HINZE: Turbulence KAYS and CRAWFORD: Convective Heat and Mass Transfer KRYNINE and JUDD: Principles of Engineering Geology and Geotechnics MEIROVITCH: Methods of Analytical Dynamics MELSA and SCHULTZ: Linear Control Systems MICKLEY, SHERWOOD ard REED: Applied Mathematics in Chemical Engineering PAPOULIS: The Fourier Integral and Its Applications PHELAN: Fundamentals of Mechanical Design SCHLICHTING: Boundary Layer Theory SCHWARTZ and SHAW: Signal Processing: Discrete Spectral Analysis, Detection, and Estimation ‘TIMOSHENKO: Theory of Plates and Shells TIMOSHENKO and GOODIER: Theory of Elasticity TIMOSHENKO and GERE: Theory of Elastic Stability 'TREYBAL: Mass-Transfer Operations TRUXAL: Introductory Systems Engineering WARNER and McNEARY: Applied Descriptive Geometry WELLMAN: Technical Descriptive GeometryALVIN W. Operations Research Center and Department of Electrical Engineering Massachusetts Institute of Technology -ABARZsW McGRAW-HILL CLASSIC TEXTB®K REISSU E SRS fundamentals of applied probability theory AAACN McGraw-Hill, Inc, New York St. Louis San Francisco Auckland Bogoté Caracas Lisbon Losdon Madrid Mexico City Milan Montreal New Delhi San Juan Singapore Sydney Tokyo Torontofundamentals of applied probability theory Copyright © 1967 by the McGraw-Hill Book Company, Ine. Reissued 1988 by the McGraw-Hill Book Company, Ine. Printed in the United States of America. Except as permitted under the United States Copyright Act of 1976, no part ofthis publication may be reproduced or distributed in any form or by any means, or stored in a data base or retrieval system, without the prior written permission of the publisher. Library of Congress Catalog Card Number 67-11208 ISBN O-O?7-017815-1 22 23 2h 25 26 27 BKMBKH 9876S 4To this Great Nation, ‘which haz allowed millions of refugees, including my parents, to enter a land of freedom and opportunityPREFACE This is a first textbook in applied probability theory, assuming a background of one year of calculus. The material represents a one- semester subject taught at M.I-T. to about 250 students per year, most of whom are in the Schools of Engineering or Management. About two-thirds of these students are undergraduates. The subject, Probabilistic Systems Analysis, serves both as a terminal course and as a prerequisite for more advanced work in areas such as communica- tion theory, control systems, decision theory, operations research, quantitative management, statistics, and stochastic processes.viii PREFACE My intention is to present a physically based introduction to applied probability theory, with emphasis on the continuity of funda- mentals. A prime objective is to develop in the new student an under- standing of the nature, formulation, and analysis of probabilistic situations. This text stresses the sample space of representation of probabilistic processes and (especially in the problems) the need for explicit modeling of nondeterministic processes, In the attempt to achieve these goals, several traditional details have either been omitted or relegated to an appendix. Appreciable effort has been made to avoid the segmentation and listing of detailed applications which must appear in a truly comprehensive work in this area. Intended primarily as a student text, this book is not suitable for use as a general reference. Scope and Organization ‘The fundamentals of probability theory, beginning with a dis- cussion of the algebra of events and concluding with Bayes’ theorem, are presented in Chapter 1. An axiomatic development of probability theory is used and, wherever possible, concepts are interpreted in the sample space representation of the model of an experiment (any non- deterministic process). ‘The assignment of probability measure in the modeling of physical situations is not necessarily tied to a relative frequency interpretation. In the last section of this chapter, the use of sample and event spaces in problems of enumeration is demonstrated. Chapter 2 is concerned with the extension of earlier results to deal with random variables. This introductory text emphasizes the local assignment of probability in sample space. For this reason, we work primarily with probability density functions rather than cumula- tive distribution functions. My experience is that this approach is much more intuitive for the beginning student. Random-variable con- cepts are first introduced for the discrete case, where things are particularly simple, and then extended to the continuous case. Chap- ter 2 concludes with the topic of derived probability distributions as obtained directly in sample space. Discrete and continuous transform techniques are introduced in Chapter 3. Several applications to sums of independent random varia- bles are inchided. Contour integration methods for obtaining inverse transforms are not discussed. Chapters 4 and 5 investigate basic random processes involving, respectively, independent and dependent trials. Chapter 4 studies in some detail the Bernoulli and Poisson processes and the resulting families of probability mass and density functions. Because of its significance in experimentation with physi-PREFACE ix cal systems, the phenomenon of random incidence is introduced in the last section of Chapter 4. Discrete-state Markov models, including both discrete-transi- tion and continuous-transition processes, are presented in Chapter 5. ‘The describing equations and limiting state probabilities are treated, but closed form solutions for transient behavior in the general case are not discussed. Common applications are indicated in the text and in the problems, with most examples based on relatively simple birth- and-death processes. Chapter 6 is concerned with some of the basic limit theorems, both for the manner in which they relate probabilities to physically observable phenomena and for their use as practical approximations. Only weak statistical convergence is considered in detail, A transform development of the central limit theorem is presented. ‘The final chapter introduces some common issues and techniques of statistics, both classical and Bayesian. My objectives in this obvi- ously incomplete chapter are to indicate the nature of the transition from probability theory to statistical reasoning and to assist the student in developing a critical attitude towards matters of statistical inference. ‘Although many other arrangements are possible, the text is most effectively employed when the chapters are studied in the given order. Examples and Home Problems Many of the sections which present new material to the student contain very simple illustrative examples. More structured examples, usually integrating larger amounts of material, are solved and discussed in separate sections. For the student, the home problems constitute a vital part of the subject matter. It is important that he develop the skill to formu- late and solve problems with confidence. Passive agreement. with other people’s solutions offers little future return. Most of the home problems following the chapters are original, written by the author and other members of the teaching staff. These problems were written with definite objectives. In par- ticular, wherever possible, we have left for the student a considerable share in the formulation of physical situations. Occasionally, the probability assignments directly relevant to the problems must be derived from other given information. It did not seem feasible to sample the very many possible fields of application with other than superficial problems. The interesting aspects of each such field often involve appreciable specialized struc- ture and nomenclature. Most of our advanced problems are based onx PREFACE relatively simple operational situations. From these common, easily communicated situations, it seemed possible to develop compact repre- sentative problems which are challenging and instructive. The order of the problems at the end of each chapter, by and large, follows the order of the presentation in the chapter. Although entries below are often not the most elementary problems, relatively comprehensive coverage of the material in this text is offered by the following skeleton set of home problems: 032.04 3.05 4.05 5.05 6.02 7.04 08 2.07 3.08 4.09 5.06 6.03 7.06 09 2.41 3,094.12 5.10 6.04 7.08 A2 247 3.40 4.13 5.12 6.07 7.15 AB 2.26 3.12 4.17 5.13 6.08 7.16 21 2.97 8.18 4.18 5.14 6.13 7.19 2 2.98 3.2 4.22 516 6.17 7.20 30 2.30 ‘Some Clerical Notes I have taken some liberties with the usual details of presentation. Figures are not numbered but they do appear directly in context. Since there are few involved mathematical developments, equations are not numbered. Whenever it appeared advantageous, equations were repeated rather than cross-referenced. Recommended further reading, including a few detailed refer- ences and referrals for topies such as the historical development of probability theory are given in Appendix 1. Appendix 2 consists of a listing of common probability mass and density functions and their expected values, variances, and transforms. Several of these proba- bility functions do not appear in the body of the text. A brief table of the cumulative distribution for the unit normal probability density function appears in context in Chapter 6. The general form of the notation used in this text seems to be gaining favor at the present time. To my taste, it is one of the sim- plest notations which allows for relatively explicit communication. My detailed notation is most similar to one introduced by Ronald A. Howard. Acknowledgments My interest in applied probability theory was originally sparked by the enthusiasm and ability of two of my teachers, Professors George P. Wadsworth and Ronald A. Howard. For better or worse,PREFACE = xi itt is the interest they stimulated which led me to this book, rather than one on field theory, bicycle repair, or the larger African beetles. Like all authors, I am indebted to a large number of earlier authors. In this case, my gratitude is especially due to Professors liam Feller, Marek Fisz, and Emanuel Parzen for their excellent works. ‘Teaching this and related material during the past six years has been an exciting and rewarding experience, due to the intensity of our students and the interchange of ideas with my colleagues, especially Dr. Murray B. Sachs and Professor George Murray. The many teaching assistants associated with this subject contributed a great deal to its clarification. Some of the problems in this book represent their best educational (and Machiavellian) efforts During the preparation of this book, I had many productive discussions with Professor William Black. In return for his kindness, and also because he is a particularly close friend, I never asked him to look at the manuscript. Professor Alan V. Oppenheim and Dr. Ralph L. Miller were less fortunate friends; both read the manuscript with great care and offered many helpful suggestions. ‘The publisher's review by Dr. John G. Truxal was most valuable. Some award is certainly due Mrs. Richard Spargo who had the grim pleasure of typing and illustrating the entire manuscript—three times! My devoted wife, Elisabeth, checked all examples, proofread each revision of the manuscript, and provided unbounded patience and encouragement. Finally, I express my gratitude to any kind readers who may forward to me corrections and suggestions for improvements in this text. Alvin W. DrakePreface CONTENTS Events, Sample Space, and Probability Le Le 13a M4 15 16 “7 Ls 19 A Brief Introduction to the Algebra of Events Sample Spaces for Models of Experiments Probability Measure and the Relative Likelihood of Events Conditional Probability and Its Interpretation in Sample Space Probability Trees for Sequential Experiments ‘The Independence of Events Examples Bayes’ Theorem Enumeration in Event Space: Permutations and Combinations eR ARE 10 3B qT 19 25 27xiv CONTENTS 2. Random Variables a 22 28 24 25 26 27 28 29 210 un 212 243, 214 245 Random Variables and Their Event Spaces ‘The Probability Mass Function Compound Probability Mass Functions Conditional Probability Mass Functions Independence and Conditional Independence of Random Variables Functions of Random Variables Expectation and Conditional Expectation Examples Involving Discrete Random Variables A Brief Introduction to the Unit-impulse Function ‘The Probability Density Function for » Continuous Random Variable Compound Probability Density Functions Conditional Probability Density Functions Independence and Expectation for Continuous Random Variables Derived Probability Dens Examples Involving Continuous Random Variables Functions 3. Transforms and Some Applications to Sums of Independent Random Vi a1 a2 33 34 35 36 37 38 39 s ‘The Transform ‘The z Transform ‘Moment-generating Properties of the ‘Transforms Sums of Independent Random Variables; Convolution ‘The Transform of the PDF for the Sum of Independent Random Variables A Further Note on Sums of Independent Random Variables Sum of a Random Number of Independent Identically Distributed Random Variables An Example, with Some Notes on Inverse Transforms Where Do We Go from Here? 4, Some Basic Probabilistic Processes 41 42 43 44 45 46 47 48 49 ‘The Bernoulli Process Interarrival Times for a Bernoulli Process ‘Summary of the Bernoulli Process An Example ‘The Poisson Process Interarrival Times for the Poisson Process Some Additional Properties of Poisson Processes and Poisson Random Variables Summary of the Poisson Process Examples 42 “4 46 5h 52 53 87 69 2 ™ “4 78 7 98 99, 100 103 106 107 109 12 6 123 1 128 129 130 133 137 139 140 142CONTENTS XV £10 Renewal Processes 4g. 4-11 Random Incidence M9 Discrete-state Markov Processes 163 5-1 Series of Dependent ‘Trials for Diserete-state Processes 164 5-2 Disorete-state Discrete-transition Markov Processes 164 5-3 State Classification and the Concept of Limiting-state Probabilities 167 54 ‘The Ergodic Theorem 170 5-5 The Analysis of Discrete-state Discrete-transition Markov Processes 171 5-6 Examples Involving Discrete-transition Markoy Processes 175 5&7 Diserete-state Continuous-transition Markov Processes 180 5-8 Examples Involving Continuous-transition Processes 185 6, Some Fundamental Limit Theorems 3 6-1 The Chebyshev Inequality 204 62 Stochastic Convergence 205 63 ‘The Weak Law of Large Numbers 206 6-4 The Gaussian PDF 207 6-5 Central Limit Theorems 22 6-6 Approximations Based on the Central Limit Theorem 215 67 Using the Central Limit Theorem for the Binomial PMF 216 68 The Poisson Approximation to the Binomial PMF 219 69 A Note on Other ‘Types of Convergence 220 7. An Introduction to Statistics 29 TA Statistics Is Different 230 7-2 Statistical Models and Some Related Issues 230 7-3. Statistics: Sample Values and Experimental Values, Bi 7-4 Significance Testing 234 75 Parametric and Nonparametric Hypotheses 239 7-6 Hypothesis Testing 240 7-7 Estimation 243 7-8 Some Properties of Desirable Estimators 245 7-9 Maximum-likelihood Estimation 7 7-10 Bayesian Analysis 250 711 Complementary PDF's for Bayesian Analysis 253 7-12 Some Comments on Bayesian Analysis and Classical Statistics 255 Appendix 1. Further Reading 385 Appendix 2. Common PDF's, PMF’s, and Their Means, Variances, and Transforms 271 Index a7CHAPTER ONE events, == sample space, | and probability Our study of probability theory begins with three closely related topics: 1 The algebra of events (or sets or areas or spaces) will provide us with a common language which is never ambiguous. This algebra will be defined by a set of seven axioms. 2 Sample space is the vital picture of a model of an “experiment” (any nondeterministic process) and all of its possible outcomes. 3 Probability measure is the relevant measure in sample space. ‘This measure is established by three more axioms and it provides for the measurement of events in sample space.2 EVENTS, SAMPLE SPACE, AND PROBABILITY 1-1 A Brief Introduction to the Algebra of Events We first introduce an explicit language for our discussion of probability theory. Such a Innguage is provided by the algebra of events. We wish to develop an understanding of those introductory aspects of this algebra which are essential to our study of probability theory and its applications. Let’s begin with a brief tour of some of the definitions, concepts, and operations of the algebra of events. Events (or sets) are collections of points or areas ina space. The physical interpretation of this space will be provided in the following section. The collection of all points in the entire space is called U, the universal set or the universal event. Event A’, the complement of event A, is the collection of all points in the universal set which are not included in event A. The null set ¢ contains no points and is the complement of the universal set. ‘The intersection of two events A and B is the collection of all points which are contained both in A and in B. For the intersection of events A and B we shall use the simple notation AB. ‘The union of two events A and B is the collec- tion of all points which are either in A or in B or in both. For the union of events A and B we shall use the notation A + B.A BRIEF INTRODUCTION TO THE ALGEBRA OF EVENTS. 3 If all points of U which are in B are also in A, then event B is said to be included in event A. ‘Two events A and B are said to be equal if every point of U whieh is in A is also in B and every point of U which is in A’ is also in B’. Another way to state the condition for the equality of two events would be to say that two events are equal if and only if each event is included in the other event. We have sampled several of the notions of the algebra of events. Pictures of events in the universal set, such as those used above, are known as Venn diagrams, Formally, the following is a set of laws (axioms) which fully defines the algebra of events: 1A+B=B+A Commutative law 2A+(B+C)=(A+B)4+C Associative law 3 A(B + C) = AB+ AC Distributive law aA aA 6 AA = 6 5 (AB) = A’ +B" TAU =A The Seven Axioms of the Algebra of Events Technically, these seven axioms define everything there is to know about the algebra of events, ‘The reader may consider visualiz- ing each of these axioms in a Venn diagram. Our selection of a list of axioms is not a unique one. Alternative sets of axioms could be stated which would lead to the same results. Any relation which is valid in the algebra of events is subject to proof by use of the seven axioms and with no additional information. Some representative relations, each of which may be interpreted easily ona Venn diagram, are AtA=A A+U A+AB=A A+ BC =(A+BY(A +0) A+A'B=A+B Ab=% A+A'=U A(BC) = (AB)C If it were our intention to prove (or test) relations in the algebra of events, we would find it surprisingly taxing to do so using only the4 EVENTS, SAMPLE SPACE, AND PROBABILITY seven axioms in their given form. One could not take an “obvious” step such as using CD = DC without proving the validity of this rela- tion from the axioms. For instance, to show that CD = DC, we may proceed C+D =D+C Axiom (1) with A = C’ and B = D! (C+D! = (D'+C’)’ Take complement of both sides “CD = DC By use of axioms (5) and (4) The task of proving or testing relations becomes easier if we have relations such as the above available for direct use. Once stich rela tions are proved via the seven axioms, we call them theorems and use them with the same validity as the original axioms. We are already prepared to note those definitions and properties of the algebra of events which will be of value to us in our study of probability. A list of events Ay, As, . . . , Aw is said to be composed of mutually exclusive events if and only if Ac itty AA; fs iting 1,2,...,N; f=1,2, iN A list of events is composed of mutually exclusive events if there is no point in the universal set which is included in more than one event in the list. A Venn diagram for three mutually exclusive events A, B, C could be but in no case may there be any overlap of the events, A list of events Ar, As... , Ay is said to be collectively exhaustive if = and only if ZS Art Arts ++ + Ay =U =SAMPLE SPACES FOR MODELS OF EXPERIMENTS 5 A list of events is collectively exhaustive if each point in the universal set is included in at least one event in the list. A Venn diagram for three collectively exhaustive events 4, B, C could be but in no ease may there be any point in the universal set which is included in none of the events. A list of events may be mutually exclusive, collectively exhaustive, both, or neither. After the discus- sion of sample space in the next section, we shall have many oppor- tunities to consider lists of events and familiarize ourselves with the use of these definitions. We note two additional matters with regard to the algebra of events. One is that we do not make errors in the algebra of events if we happen to include the same term several times ina union. If we are trying to collect all the points in a rather structured event, we need only be sure to include every appropriate ‘point at least once—multiple inclusion will do no harm. Another consideration is of particular importance in dealing with actual problems. The algebra of events offers a language advantage in. describing complex events, if we are careful in defining all relevant simple events. Since we shall be making conscious and subconscious use of the algebra of events in all our work, one necessary warning should be sounded. “He who would live with the algebra of events had better know exactly-with which events he is living.” The original defined events should be simple and clear. If this is the case, it isthen an easy matter to assemble expressions for complex events from the original definitions. For instanc even Event 4: Neither Tom nor Mary goes without Harry unless they see Fred accompanied by Phil or it is raining and... BETTER Event A: Tom goes. _ Event B: Mary goes. Event C: Harry goes. Ete. 1-2 Sample Spaces for Models of Experiments In this book, we use the word “experiment” to refer to any process which is, to some particular observer, nondeterministic. It makes no6 EVENTS, SAMPLE SPACE, AND PROBABILITY difference whether the observer's uncertainty is due to the nature of the process, the state of knowledge of the observer, or both. ‘The first six chapters of this book are concerned with the analysis, of abstractions, or models of actual physical experiments. Our last, chapter is concerned with the relation of the model to the actual experiment. It is probably safe to state that there are more variables asso- ciated with the outcome of any physical experiment than anybody could ever care about. For instance, to describe the outcome of an actual coin toss, we could be concerned with the height of the toss, the number of bounces, and the heating due to impact, as well as the more usual consideration of which face is up after the coin settles. For most purposes, however, a reasonable model for this experiment would involve a simple nondeterministic choice between a head and a tail. ‘Many of the “trick” probability problems which plague students are based on some ambiguity in the problem statement or on an inexact formulation of the model of a physical situation, The precise state- ment of an appropriate sample space, resulting from a detailed deserip- tion of the model of an experiment, will do much to resolve common difficulties. In this text, we shall literally live in sample space. Sample space: Thefinest-grain, mutually exclusive, collectively exhaus- tive listing of all possible outcomes of a model of an experiment, The “finest-grain” property requires that all possible distinguishable outeomes allowed by the model be listed separately. If our model for the flip of a coin is simply a nondeterministie selection between possible outeomes of a head and a tail, the sample space for this model of the experiment would include only two items, one corresponding to each possible outcome. ‘To avoid unnecessarily cumbersome statements, we shall often use the word experiment in place of the phrase “model of an experi- ment.” Until the final chapter, the reader is reminded that all our work is concerned with abstractions of actual physical situations, ‘When we wish to refer to the real world, we sball speak of the “physical experiment.” Soon we shall consider several experiments and their sample spaces. But first we take note of two matters which account for our interest in these spaces. First, the universal set with which we deal in the study of probability theory will always be the sample space for an experiment, The second matter is that any event described in termsSAMPLE SPACES FOR MODELS OF EXPERIMENTS 7 of the outcome of a performance of some experiment can be identified ‘as a collection of events in sample space, In sample space one may collect the members of any event by taking a union of mutually exclu- sive points or areas from a collectively exhaustive space. The advan- tage of being able to collect any event as a union of mutually exclusive members will become clear as we learn the properties of probability measure in the next section. ‘A sample space may Jook like almost anything, from a simple listing to a multidimensional display of all possible distinguishable out- comes of an experiment. However, two types of sample spaces seem to be the most useful. ‘One common type of sample space is obtained from a sequential picture of an experiment in terms of its most convenient parameters. Normally, this type of sample space is not influenced by our particular interests in the experimental outcome, Consider the sequential sample space for a reasonable model of two flips of a coin. We use the notation | A (ia a { 7,3 * | Tails | on the nth toss of the coin This leads to the sequential sample space WH, OHH, a “The union ofthese two sample points corresponds tothe event "net outcome {S exactly one head! or "both fips did rot produce the same result” or CH, 1 +7, H,) oF (HH, +7, 73)" a — ont Above we picture the experiment proceeding rightward from the left origin, Each sample point, located at the end of a erminal tree branch, represents the event corresponding to the intersection of all events encountered in tracing a path from the left origin to that sample point. On the diagram, we have noted one example of how an event may be collected as a union of points in this sample space. Formally, the four sample points and their labels constitute the sample space for the experiment, However, when one speaks of a sequential sample space, he normally pictures the entire generating tree as well as the resulting sample space. For an experiment whose outcomes may be expressed numeri- cally, another useful type of sample space is a coordinate system on which is displayed a finest-grain mutually exclusive collectively exhaus- tive set of points corresponding to every possible outcome. For4, EVENTS, SAMPLE SPACE, AND PROBABILITY ance, if we throw a six-sided die (with faces labeled 1, 2, 3, 4, 5, 6) twice and call the value of the up face on the nth toss 24, we have a sample space with 36 points, one for each possible experimental outcome, ——— The union of these sample ‘© points corresponds to the event (x—x,)>20 - This sample point corresponds to ‘¢ the experimental outcome "a four on the first toss and a three on © the second toss" Often we shall have reason to abbreviate a sample space by working with a mutually exclusive collectively exhaustive listing of all possible outcomes which is not finest-grain. Such spaces, possessing all attributes of sample spaces other than that they may not list separately all possible distinguishable outcomes, are known as event spaces. If alist of events Ay, As,.. . , Ay forms an event space, each possible finest-grain experimental outcome is included in exactly one event in this list. However, more than one distinguishable outcome may be included in any event in the list. We now present some sample and event spaces for several experiments. Experiment: Flip a coin twice. heads| . A Notation: Let {r) be the event { cn | ea the th on We already have seen the sequential sample space for this experiment, but let’s consider some event spaces and some other displays of the sample space. ‘The sample space in a different form isSAMPLE SPACES FOR MODELS OF EXPERIMENTS 9 ‘The sample space displayed as a simple listing is OHH, HT, OTH, OTT ‘An example of an event space but not a sample space for this experi- ment is oH, oT; Another example of an event space but not a sample space is eHiT, — (HiT2)" ‘The following is neither an event space nor a sample space: oH oT, HOT Experiment: Flip a coin until we get our first head. heads| | on the nth toss. tails Notation: Let {a} be the event { ‘A sample space generated by a sequential picture of the experiment is eH, THe OTH, OLE MHe A " Lo 1% ‘A sample space which is a simple listing for this experiment, with N representing the number of the toss on which the first head occurred, is H, Is Hy 4 : i ———T.- N=1 oN 2 W=3 N=4 Note that this is still a finest-grain description and that one can specify the exact sequence of flips which corresponds to each of these sample points. ‘An event space but not a sample space for this experiment is N>4 NS4 Neither a sample space nor an event space for this experiment is N>1 N>2 NEB WDE --- Experiment: Spin a wheel of fortune twice. ‘The wheel is continuously calibrated, not necessarily uniformly, from zero to unity. Notation: Let z, be the exact reading of the wheel on the nth spin.10 EVENTS, SAMPLE SPACE, AND PROBABILITY A sample space for this experiment is » y Every possible distinguishable outcome (x, x2) is represented 2a different point inthis space e 1 A Let's collect several typical events in this sample space. Any point in a sample space corresponds to one possible outeome of a performance of the experiment. For brevity, we shall often refer to the sample point as though it were the event to which it corresponds, 1-3. Probability Measure and the Relative Likelihood of Events ‘To complete our specification of the model for a physical experiment, we wish to assign probabilities to the events in the sample space of the experiment. The probability of an event is to be a number, represent- ing the chance, or “relative likelihood,” that a performance of the experiment will result in the occurrence of that event. ‘This measure of events in sample space is known as probability ‘measure. By combining three new axioms with the seven axioms of theRELATIVE LIKELIHOOD OF EVENTS eB algebra of events, one obtains a consistent system for associating real nonnegative numbers (probabilities) with events in sample space and for computing the probabilities of more complex events. If we use the notation P(A) for the probability measure associated with event A, then the three additional axioms required to establish probability measure are: = 1 For any event A, P(A) > 0 2 PU) =1 = 3 If AB = ¢, then P(A + B) = P(A) + P(B) The Three Axioms of Probability Measure = = The first axiom states that all probabilities are to be nonnega- tive. By attributing a probability of unity to the universal event, the second axiom provides a normalization for probability measure. ‘The third axiom states a property that most people would hold to be appro- priate to a reasonable measure of the relative likelihood of events. If we accept these three axioms, all of conventional probability theory follows from them. ‘The original assignment of probability measure to a sample space for an experiment is a matter of how one chooses to model the physical experiment, Our axioms do not tell us how to compute the probability of any experimental outcome “from scratch.” Before one can begin operating under the rules of probability theory, he must first, provide an assignment of probability measure to the events in the sample space. Given a physical experiment to be modeled and analyzed, there is no reason why we would expect two different people to agree as to what constitutes a reasonable probability assignment in the sample space. One would expect a person’s modeling of a physical experiment to be influenced by his previous experiences in similar situations and by any other information which might be available to him. Proba- bility theory will operate “correctly” on any assignment of probability to the sample space of an experiment which is consistent with the three axioms. Whether any such assignment and the results obtained from itt are of any physical significance is another matter. Tf we consider the definition of sample space and the third axiom of probability measure, we are led to the important conclusion, as Given an assignment of probability measure to the finest-grain events === in a sample space, the probability of any event A may be computed by |= summing the probabilities of all finest-grain events included in event A. ===12 EVENTS, SAMPLE SPACE, AND PROBABILITY One virtue of working in sample space is that, for any point in the sample space there must be a “yes” or “no” answer as to whether the point is included in event A. Were there some event in our space for which this was not the case, then either the space would not be a sample space (because it did not list separately certain distinguishable outcomes) or our model is inadequate for resolution between. events Aand A’. We shall return to this matter when we discuss the sample- space interpretation of conditional probability. One can use the seven axioms of the algebra of events and the three axioms of probability measure to prove various relations such as P(A’) =1— P(A) P(A +B) = P(A) + PB) — P(AB) P($) =0 PA+B+C) — P(A'B'C’) Because we shall live in sample space, few, if any, such relations will be required formally for our work. We shall always be able to employ directly the third axiom of probability measure to compute the proba- bility of complex events, since we shall be expressing such events as unions of mutually exclusive membersina sample space. In computing the probability of any event in sample space, that axiom states that. we must include the probability of every sample point in that event exactly once. Multiple counting caused no error in taking a union in the algebra of events to describe another event, but one must carefully confine himself to the axioms of probability measure when determining the probability of a complex event. There are many ways to write out a “formula” for the proba- bility of an event which is a union such as P(A +B +C). Using the third axiom of probability measure and looking at a Venn diagram, (By we may, for instance, write any of the following: P(A + B+ C) = P(A) + P(A'B) + P(A’B'C) P(A + B+ C) = P(A) + P(B) + P(C) ~ P(AB) ~ P(AC) — P(BC) + P(ABC) P(A + B+ C) = P(AB‘C’) + P(A’BC’) + P(A'B'C) + P(AB) + P(AB'C) + P(A'BC) P(A + B+ C) =1~ P(A'B'CONDITIONAL PROBABILITY IN SAMPLE SPACE 13 Obviously, one could continue listing such relations for a very long time. Now that we have considered both the algebra of events and probability measure, it is important to recall that: 1 Events are combined and operated upon only in accordance with the seven axioms of the algebra of events. 2 The probabilities of events are numbers and can be computed only in accordance with the three axioms of probability measure. 3 Arithmetic is something else. PIA + CD + BA + C'D)) ‘The algebra of events applies eee inside the brackets. It oper- ates upon events and has noth- peo Doe ing to do with their measure. ory are used in obtaining the numerical value of this quan- tity. In practice, we shall usually evaluate such probabilities by collecting the event as a union of mutually exclusive points in sample space and summing the probabilities of all points included in the union. 1-4 Conditional Probability and Its interpretation in Sample Space Assumé that we have a fully defined experiment, its sample space, and an initial assignment of probability to each finest-grain event in the sample space.’ Let two events A and B be defined on the sample space of this experiment, with P(B) ¥ 0. We wish to consider the situation which results if the experiment is performed once and we are told only that the experimental outcome has attribute B. Thus, if B contains more than one sample point, we are considering the effect of “partial information” about the experi- mental outcome. Let’s look at a sample-space picture of this situation. Consider a sample space made up of sample points S;, Sz,. . . , Sy.14 EVENTS, SAMPLE SPACE, AND PROBABILITY Given that event B has occurred, we know that the sample point repre- senting the experimental outcome must be in B and cannot be in B’, We have no information which would lead us to alter the relative proba- bilities of the sample points in B. Since we know one of the sample points in B must represent the experimental outcome, we scale up their original probabilities by a constant, 1/P(B), such that they now add to unity, to obtain conditional probabilities which reflect the influence of our partial information. We formalize these ideas by defining P(S; | B), the “conditional probability of S, given B,” to be (P(S;) ‘ + P(s)|B) = {Py Fin BY _ PLS) 0 if S; in B’ The conditional probability of any other event, such as A, is to be the sum of the conditional probabilities of the sample points included in A, leading to the common definition of the conditional probability of event A given B. P(AB) P(A| B) = He) defined only for P(B) # 0 which may be obtained from our previous statements via P(S;B) _ P(AB) P(A|B)= Y P(S)| B) = Piby ~ Pi) afina a Fina We may conclude that one way to interpret conditional proba- bility is to realize that a conditioning event (some partial information about the experimental outcome) allows one to move his analysis from the original sample space into a new conditional space. Only those finest-grain events from the original sample space which are included in the conditioning event appear in the new conditional space with a non- zero assignment of conditional probability measure. The original (“a priori”) probability assigned to each of these finest-grain events is multiplied by the same constant such that the sum of the conditional (‘‘a posteriori”) probabilities in the conditional space is unity. In the resulting conditional sample space, one uses and interprets these a posteriori probabilities exactly the same way he uses the a priori probabilities in the original sample space. The conditional probabilities obtained by the use of some partial information wil, in fact, serve as initial probabilities the new sample space for any further work. We present one simple example. A fair coin is flipped twice, and Joe, who saw the experimental outcome, reports that “at least one toss { } {CONDITIONAL PROBABILITY IN SAMPLE SPACE 15 resulted in a head.” Given this partial information, we wish to deter- mine the conditional probability that both tosses resulted in heads. T. sample space for the experiment and indicate, in the P(e) column, our a priori probability for each sample point. Using the notation (#1 and the problem statement, we may draw a Pe) 4 B AB oH, 0.25 v v v omr, 0.25 v THs 0.25 v TT, 0.25 PA) = 0.75 P(B) = 0.25 P(AB) = 0.25 For the above display, we defined the events A and B to be “at least one head” and “two heads,” respectively. In the A, B, and AB columns we check those sample points included in each of these events. The proba- bility of each of these events is simply the sum of the probabilities of the sample points included in the event. (The only use of the B column in this example was to make it easier to identify the sample points associated with the complex event AB.) The desired answer P(B | A) may be found either directly from the definition of conditional probability P(AB) _ 0.25 1 “PAY © 075 ~ 3 P(B| A) = or by noting that, in the conditional sample space given our event A, there happen to be three equally likely sample points, only one of which has attribute B. ‘The solution to this problem could have been obtained in far less space. We have taken this opportunity to introduce and discuss a type of sample-space display which will be valuable in the investigation of more complex problems. It is essentisi that the reader realize that certain operations (such as collecting events and conditioning the space) which are always simple in a sample space may not be directly applicable in an arbitrary event space. Because of the finest-grain property of a sample space, any sample point must be either wholly excluded from or wholly included in any arbitrary event defined within our model. However, in an event, space, an event point A might be partially included in B, some other event of interest. Were this the case, the lack of detail in the event, space would make it impossible to collect event B in this event space or to condition the event space by event B.16 EVENTS, SAMPLE SPACE, AND PROBABILITY For instance, suppose that for the coin example above we were given only the event space Hy, TH, OTT: its probability assignment, and no other details of the experiment. Because this event space lacks adequate detail, it would be impossible to calculate the probability of event Hz or to condition this space by event Hy. When we are given a sample space for an experiment and the probability assigned to each sample point, we can answer all ques- tions with regard to any event defined on the possible’ experimental outcomes. 1-5 Probability Trees for Sequential Experiments Sequential sample and event spaces were introduced in Sec. 1-2. Such spaces, with all branches labeled to indicate the probability structure of an experiment, are often referred to as probability trees. Consider the following example: 5 ee Ag Rola — 0, "Faas, GTA By This would be a sample space for an experiment in which, for instance, the “C; or C” trial occurs only if A,B: has resulted from the earlier stages of the sequential experiment. As before, one sample point appears for each terminal branch of the tree, representing the intersec- tion of all events encountered in tracing a path from the left origin to a terminal nod: Each branch is labeled such that the product of all branch proba- bilities from the left origin to any node equals the probability that the event represented by that node will be the outcome on a particular performance of the experiment. Only the first set of branches leaving the origin is labeled with a priori probabilities; all other branches must be labeled with the appropriate conditional probabilities. ‘TheTHE INDEPENDENCE OF EVENTS akg sum of the probabilities on the branches leaving any nonterminal node must sum to unity; otherwise the terminal nodes could not represent. a collectively exhaustive listing of all possible outcomes of the experiment. Of course, any tree sample space which contains the complete set of finest-grain events for an experiment. is a “correct” sample space. In any physical situation, however, the model of the experiment will usually specify the sequential order of the tree if we wish to label all branches with the appropriate conditional probabilities without any calculations. Sometimes, a complete picture of the sample space would be too large to be useful. But it might still be of value to use an “outline” of the actual sample space. These outlines may be “trimmed” proba- bility trees for which we terminate uninteresting branches as soon as possible. But once we have substituted such an event space for the sample space, we must again realize that we may be unable to perform certain calculations in this event. space. 1-6 The Independence of Events ‘Thus far, our structure for probability theory includes seven axioms for the algebra of events, three more for probability measure, and the definition and physical interpretation of the concept of conditional probability. We shall now formalize an intuitive notion of the inde- pendence of events. This definition and its later extensions will be of considerable utility in our work. Inan intuitive sense, if events A and B are defined on the sample space of a particular experiment, we might think them to be “inde- pendent” if knowledge as to whether or not-the experimental outcome had attribute B would not affect our measure of the likelihood that the experimental outcome also had attribute A. We take a formal state- ment of this intuitive concept to be our definition of the independence of two events. ‘Two events A and B are defined to be independent if and only if Eee P(A|B) = P(A) a From the definition of conditional probability, as long as P(A) # 0 and P(B) # 0, we may write P(AB) = P(A)P(B| A) = P(B)P(A | B) When we substitute the condition for the independence of 4 and B into this equation, we learn both that P(A |B) = P(A) requires that18 only if EVENTS, SAMPLE SPACE, AND PROBABILITY P(B| A) = P(B) and that an alternative statement of the condition for the (mutual) independence of two events is P(AB) = P(A)P(B). If A and B are other than trivial events, it will rarely be obvious whether or not they are independent. To test two events for inde- pendence, we collect the appropriate probabilities from a sample space to see whether or not the definition of independence is satisfied. Clearly, the result depends on the original assignment of probability measure to the sample space in the modeling of the physical experiment. We have defined conditional probability such that (as long as none of the conditioning events is of probability zero) the following relations always hold: P(AB) = P(A | B)P(B) = P(B| A)P(A) P(ABC) = P(A)P(BC | A) = P(B)P(C | B)P(A | BC) = P(AC)P(B| AC) = but only when two events are independent may we write P(AB) = P(A)P(B). We extend our notion of independence by defining the mutual independence of N events A1, dy... , Av. = N events Ar, An... , Aw are defined to be mutually independent if and === = P(As| AyAn > Ay) = P(A) — for all xe 3, = This is equivalent to requiring that the probabilities of all possible intersections of these (different) events taken any number at a time {such as P(A1A3444y)] be given by the products of the individual event probabilities [such as P(A,)P(A)P(A»)P(A,)]. Pairwise independ- ence of the events on a list, as defined at the beginning of this section, does not necessarily result in the mutual independence defined above. One should note that there is no reason why the independence or dependence of events need be preserved in going from an a priori sample space to a conditional sample space. Similarly, events which are mutually independent in a particular conditional space may or may not. be mutually independent in the original universal set or in another conditional sample space. Two events A and B are said to be condi- tionally independent, given C, if it is true that P(AB|C) = P(A| C)P(B| C)EXAMPLES 19 This relation does not require, and is not required by, the separate condition for the unconditional independence of events A and B, P(AB) = P(A)P(B) We close this section with a consideration of the definition of the independence of two events from a sample-space point of view. One statement of the condition for the independence of events A and B, with P(B) + 0, is P(A|B) = P(A) or, equivalently, P(A) = ee \) Thus, the independence requirement is that event AB is assigned a fraction of the probsbility measure of event B which is numerically equal to P(A). As one would expect, we see (from the above diagram and the last equation) that, as long as P(A) > 0 and P(B) % 0, events A and B cannot be independent if they are mutually exclusive. 1-7. Examples example 1 Our first example is simply an exercise to review the properties of probability measure, conditional probability, and some definitions from the algebra of events, The reader is encouraged to work out these examples for himself before reading through our discussion of the solutions. Suppose that we are given three lists of events, called lists 1, 2, and 3. All the events in the three lists are defined on the same experi- ment, and none of the events is of probability zero. List 1 contains events Ay, 42, . . . , Aeand is mutually exclusive and collectively exhaustive. List 2 contains events By, Bs, . . . , Byand is mutually exclusive and collectively exhaustive. List 3 contains events C1, C2, . . . ,Cn and is mutually exclusive but not collectively exhaustive.20 EVENTS, SAMPLE SPACE, AND PROBABILITY Evaluate each of the following quantities numerically. If you cannot evaluate them numerically, specify the tightest upper and lower numerical bounds you can find for each quantity. * & @ ¥ Pe () Plds| A) a aN ho i @ YY Pay @ Y Pes aco But & i Aa @ ¥ Pay © YY Paor@l ao A faim * @ J Pwlay Part (a) requires that we sum the probability measure associated with each member of a mutually exclusive but not collectively exhaus- tive list of events defined on a particular experiment. Since the events are mutually exclusive, we are calculating the probability of their union. According to the conditions of the problem, this union represents an event of nonzero probability which is not the universal set; so we obtain 00 < ¥ PC) <10 i For part (b), the A's are mutually exclusive so we note that P(A;| A,) is zero unless j = 2. When j = 2, we have P(As| Az), which is equal to unity. Therefore, we may conclude that 2 Jy PAs A) = 10 fa Again in part (c), the mutually exclusive property of the A's will require P(A;A;) = 0 unless j = i. If j = 4, we have P(4sA,), which is equal to P(A,); so upon recalling that the A; list is also col- lectively exhaustive, there follows ho ats A YY Pad) = YY PAA) + Y P(A) = O41 =1, Bi fafa mM tei In part (d), we know that Cz and C; are mutually exclusive. ‘Therefore, Cz and Cy can never describe the outcome of the same per- formance of the experiment on which they are defined. So, with no attention to the properties of the Aj’s (assuming that we can neglect any pathological cases where the conditioning event A,C; would be of probability zero and the conditional probability would be undefined), we haveEXAMPLES = 21. & > PCr AC) = 0.0 i Part (e) is most easily done by direct substitution, f Pia) = Yu - Puy =e F Pay mn mn nh and since we are told that the A/’s are mutually oxelusive and collec- tively exhaustive, we have } P(Aj) =k — 10 i For part (f), we can use thedefinition of conditional probability and the given properties of lists 1 and 2 to write ¥ SY Peaarwylao = FY Paw) = Y Pca) = 10 mm Ai om ‘The quantity to be evaluated in part (9) involves the summation of conditional probabilities each of which applies to a different condi- tioning event. ‘The value of this sum need not be a probability. By decomposing the universal set into A, 42, . . . , Ax and considering a few special cases (such as B; = U) consistent with the problem state- ment, we see o< 3 PIA) sk A example 2 To the best of our knowledge, with probability 0.8 Al is guilty of the crime for which he is about to be tried. Bo and Ci, each of whom knows whether or not Al is guilty, have been called to testify. Bo is a friend of Al’s and will tell the truth if Al is innocent but will lie with probability 0.2 if Al is guilty. Ci hates everybody but the judge and will tell the truth if Al is guilty but will lie with probability 0.3 if Al is innocent. Given this model of the physical situation: (a) Determine the probability that the witnesses give conflicting testimony. (b) Which witness is more likely to commit perjury? (©) What is the conditional probability that Al is innocent, given that Bo and Ci gave conflicting testimony? (a) Are the events “Bo tells a lie” and “Ci tells a lie” independent? Are these events conditionally independent to an observer who knows whether or not Al is guilty?22 EVENTS, SAMPLE SPACE, AND PROBABILITY We begin by establishing our notation for the events of interest: Event A: Alis innocent. Event B: Bo testifies that Al is innocent. Event C: Ci testifies that Al is innocent. Event X: The witnesses give conflicting testimony. Event ¥: Bo commits perjury. Event Z: Ci commits perjury. Now we'll draw a sample space in the form of a probability tree and collect the sample points corresponding to the events of interest. __ as y ec fan one A —— & “We cane amy voy > (ew tape a ee oe POO PY) P(Z) PLAX) 022 =016 =005 =0.06 To find the probability of any event in sample space, we simply sum the probabilities of all sample points included in that event. (It is because of the mutually exclusive property of the sample space that we may follow this procedure.) Now, to answer our questions, a P(X) = P(BC’ + B'C) = P(BC’) + P(B'C) = 0.22 b PCY) = P(AB’ + A’B) = P(AB’) + P(A’B) = 0.00 + 0.16 = 0.16 PZ) = P(AC’ + A'C) = P(AC)) + P(A'C) = 0.06 + 0.00 = 0.06 Therefore Bo is the more likely of the witnesses to commit perjury. P(AX) 0.06 3 £ PAIX) = “poy = O0e-+ OG ~ IT Conflicting testimony is more likely to occur if Al is innocent than if he is guilty; so given X occurred, it should increase our proba- bility that Al is innocent. ‘This is the ease, since 3/11 > 1/5. © (One other method of solution.) Given X occurred, we go to a condi- tional space containing only those sample points with attribute X. The conditional probabilities for these points arefound byscaling up theEXAMPLES 23 original a priori probabilities by the same constant (1/P(X)] so that they add to unity. e ABC 014 © ABC’ 006 © arse’ 0.16 | *X occurred® 2aBce Conditional sample space given X occurred, including conditional proba- bilities Original sample space including a ror probabilities Now, in the conditional space we simply sum the conditional prob- abilities of all sample points included in any event to determine the conditional probability of that event. P(A| X) = P(ABC'| X) = 3/11 d To determine whether “Bo tells a lie” and “Ci tells a lie” are independ- ent in the original sample space, we need only test P(YZ) £ P(Y)P(Z). Since P(YZ) = 0 but P(¥) > 0 and P(Z) > 0, we see that events ¥ and Z are not independent, in fact they are mutually exclusive, To determine whether Y and Z are conditionally independent given A or A’, we must test P(YZ| A) 2 P(Y|A)P(Z| A) and P(YZ|A') 2 P(Y| A)P(Z A’). From the sample space, we find that the left-hand side and one term on the right-hand side of each of these tests is equal to sero, so events Y and Z are conditionally inde- pendent to one who knows whether or not Al is innocent. Since the testimony of the two witnesses depends only on whether or not Al is innocent, this is a reasonable result. If we don’t know whether or not Al is innocent, then Y and Z are dependent because the occurrence of one of these events would give us partial information about Al's innocence or guilt, which would, in turn, change our prob- ability of the occurrence of the other event. example 3 When we find it necessary to resort to games of chance for simple examples, we shall often use four-sided (tetrahedral) dice to keep our problems short. With a tetrahedral die, one reads the “down” face either by noticing which face isn’t up or by looking up at the bottom of the die through a glass cocktail table. Suppose that a fair four-sided die (with faces labeled 1, 2, 3, 4) is tossed twice and we are told only that the product of the resulting two down-face values is less than 7. (a) What is the probability that at least one of the face values is a two? (b) What is the probability that the sum of the face valuesis less than 7? (c) If we are told that both the product and the sum of the face values is less than 7 and that at least one of the face values is a two, determine the probability of each possible value of the other face.24 EVENTS, SAMPLE SPACE, AND PROBABILITY We use the notation Event F.: Value of down face on first throw is equal to i Event S,: Value of down face on second throw is equal to j to construct the sample space nr, a % Sy ss in which, from the statement of the problem (“fair die”), all 16 sample Points are equally likely. Given that the product of the down faces is less than 7, we produce the appropriate conditional space by elimi- nating those sample pointe not inciuded in the conditioning event and seating up the probabilities of the remaining sample points. We obtain A i A A % nee % | - | in which each of the remaining i0 sample points is still an equally likely outcome in the conditional space. a There are five sample points in the conditional space which are included in the event “At least one down face is a two”; so the conditional probability of this event is 5 x 0.1 = 0.5. b All the outcomes in the conditional space represent experimental out- comes for which the sum of the down-face values is less than 7; s0 the conditional probability of this event is unity. ¢ Given all the conditioning events in the problem statement, the result- ing conditional sample space is simplyBAYES’ THEOREM 25 A i % % oe ona a si Ss and the conditional probability that the other face value is unity is 2/5; the same applies for the event that the other face value is 3, and there is a conditional probability of 1/5 that the other face value is also a two. For Example 3 we have considered one approach to a simple problem which may be solved by several equivalent methods. As we become familiar with random variables in the next chapter, we shall employ extensions of the above methods to develop effective techniques for the analysis of experiments whose possible outcomes may be expressed numerically. 1-8 Bayes’ Theorem ‘The relation known as Bayes’ ¢heorem results from a particular applica- tion of the definition of conditional probability. As long as the con- ditioning events are not of probability zero, we have P(AB) = P(A)P(B| A) = P(B)P(A| B) We wish to apply this relation to the case where the events Ay As... Ay form a mutually exclusive and collectively exhaustive list of events. For this case where the events 4, Ay, . . - , Aw form an event space, let’s consider the universal set and include some other event B. For the case of interest, assume that we know P(A\) and P(B | Ad) for all L *pcENUMERATION IN EVENT SPACE 29 ‘There were N choices for the first member, any such choice results in N — 1 poseible choices for the second member, and this continues until there are N — (K + 1) possible choices for the Kth (and last) member of the permutation. ‘Thus we have - Number of K-permutations of N distinguishable items = = NW DW -2) +++ — K+ = GE, N Vv Lal A selection of K out of NV distinguishable items, without regard to ordering, is known as a K-combination. Two K-combinations are identical if they both select exactly the same set of K items out of the original list and no attention is paid to the order of the items. For the example considered earlier, AB and BA are different, permutations, but both are included in the same combination. By setting N = K in the above formula, we note that any com- bination containing K distinguishable items includes K! permutations of the members of the combination. To determine the number of K-combinations which may be formed from N distinguishable items, we need only divide the number of K-permutations by K! to obtain Number of K-combinations of N distinguishable items NI ‘N’ = RWW —K) =(%) N2K ‘Many enumeration and probability problems require the orderly collection of complex combinatorial events. When one attacks such problems in an appropriate sequential event space, the problem is reduced to several simple counting problems, each of which requires nothing more than minor (but careful) bookkeeping. ‘The event-space approach provides the opportunity to deal only with the collection of mutually exclusive events in a collectively exhaustive space. ‘This is a powerful technique and, to conelude this section, we note one simple example of its application. Suppose that an-urranked committee of four members is to be formed from a group of four males R, S, T, U and five females V, W, X, Y, Z. It is also specified that and S cannot be on the same com- mittee unless the committee contains at least one female. We first wish to determine the umber of different such committees which may be formed.30 EVENTS, SAMPLE SPACE, AND PROBABILITY We draw an event space for the enumeration of these commit- tees. Our only methodology is to decompose the counting problem into smaller pieces, always branching out into a mutually exclusive decomposition which includes all members of interest. (We may omit items which are not relevant to our interests.) Notation Event X: X is on the committee. Event fa: Exactly n females are on the committee. ws 4(3) [the ove fomele member ot) | paepenwee | £ sttermate member (Por 0) & o(3)*(7) Rs ~~. +(3) 2 The quantity following each terminal node is the number of acceptable committees associated with the indicated event. For instance, for the RS’ committees, note that there are as many acceptable committees containing R but not S as there are ways of selecting three additional members from 7, U, V, W, X, ¥, Z without regard to the order of selection. We have obtained, for the number of acceptable committees, 2 (;) + (') + (}) () + (3) = 125 possible aeceptable committees Finally we determine, were the committees selected by lot, the probability that the first four names drawn would form an acceptable committee. Since any committee is as likely to be drawn as any other, we note that there are 125 acceptable committees out of (%) = 126 possible results of the draw. Thus there is a probability of 125/126 that a randomly drawn committee will meet the given constraint. Had we decided to solve by counting unacceptable committees and sub- tracting from 126, we would have noticed immediately that only one draw, RSTU, would be unacceptable. ‘The extension of our event-space approach to problems involving permutations, such as ranked committees, merely requires that we work in a more fine-grained event space.PROBLEMS: 31 PROBLEMS a 1.01 Use the axioms of the algebra of events to prove the relations aU'=¢ bA+B=A+4A'B+ABC CA+U=U 1.02 Use Venn diagrams, the axioms of the algebra of events, or anything else to determine which of the following are valid relations in the algebra of events for arbitrary events A, B, and C: a(A+B+Cy =A +B +0 bA+B+C=A+A'B+ (A+ A'BYO ¢ (A + B)(A’ + BY) = AB’ + A’B + A’BC’ d AB + AB’ + A’B = (A’B’! 1.03 Fully explain your answers to the following questions: a Ifevents A and B are mutually exclusive and collectively exhaustive, are A’ and B’ mutually exclusive? b If events A and B are mutually exclusive but not collectively exhaustive, are A’ and B’ collectively exhaustive? ¢ If events A and B are collectively exhaustive but not mutually exclusive, are A’ and B’ collectively exhaustive? 1.04 We desire to investigate the validity of each of four proposed rela tions (called relations 1, 2, 3, and 4) in the algebra of events. Discuss what evidence about the validity of the appropriate relation may be obtained from each of the following observations. We observe that we ean: a Obtain a valid relation by taking the intersection of each side of relation 1 with some event E1 b Obtain an invalid relation by taking the intersection of each side of relation 2 with some event Es ¢ Obtain a valid relation by taking the union of each side of relation 3 with some event Es 4 Obtain an invalid relation by taking the complement of each side of relation 4 2 1.05 In a group of exactly 2,000 people, it has been reported that there are exactly’ 612 people who smoke 670 people over 25 years of age 960 people who imbibe 86 smokers who imbibe 290 imbibers who are over 25 years of age32 EVENTS, SAMPLE SPACE, AND PROBABILITY 158 smokers who are over 25 years of age 44 people over 25 years of age each of whom both smokes and imbibes 250 people under 25 years of age who neither smoke nor imbibe Determine whether or not this report is consistent. 1.06 Consider the experiment in which a four-sided die (with faces labeled 1, 2, 8, 4) is thrown twice. We use the notation Event (E}: Down face value on {JTS | throw equals n. a cond For each of the following lists of events, determine whether or not the list is (i) mutually exclusive, (ji) collectively exhaustive, (ii) a sample space, (iv) an event space: a Fy Ps (Fi + Fs)! b FiS:, PS, Fa, Fs, Fe © FiSz, FiSs, (Fi)', (Si + 82)” d (Fi + 8), Pe + 82), (Ps + 83), (Fa + 8s) © (Pit F2)(Si + So + 83), (Fi + Fa + Pa)So, (y+ P)(Si + 82 + 82), Sal's 1.07 For three tosses of a fair coin, determine the probability of: a The sequence HHH b The sequence HTH ¢ A total result of two heads and one tail d The outcome “More heads than tails” Determine also the conditional probabilities for: e “More heads than tails” given “At least one tai f “More heads than tails” giver “Less than two tails” LT) 1,08 Joe is a fool with probability of 0.6, a thief with probability 0.7, and neither with probability 0.25. a Determine the probability that he is a fool or a thief but not both, b Determine the conditional probability that he is a thief, given that he is not a fool. 1.09 Given P(A) # 0, P(B) # 0, P(A + B) = P(A) + P(B) — 0.1, and P(A|B) = 0.2. Either determine the exact values of each of the following quantities (if possible), or determine the tightest numerical bounds on the value of each: 2 P(ABC) + P(ABC’) bb P(A’|B) & P(B) od P(A’) e P(A+B+A’B’) | 1.10 If P(A) = 0.4, P(B’) = 0.7, and P(A + B) = 0.7, determine: a P(B) b P(AB) ¢ P(A’|B) Eee ‘ae PROBLEMS 33, L11 A game begins by choosing between dice A and B in some manner such that the probability that A is selected isp. ‘The die thus selected is then tossed until a white face appears, at which time the game is concluded. Die A has 4 red and 2 white faces. Die B has 2 red and 4 white faces. After playing this game a great many times, it is observed that the probability that a game is concluded in exactly 3 tosses of the selected die is sr. Determine the value of p. 1.22 Oscar has lost his dog in either forest A (with « priori probability 0.4) or in forest B (with a priori probability 0.6). If the dog is alive and not found by the Nth day of the search, it will die that evening with probability N/(N + 2). If the dog is in A (either dead or alive) and Oscar spends a day searching for it in A, the conditional probability that he will find thedog that day is 0.25. Similarly, if the dog is in B and Oscar spends a day looking for it there, he will find the dog that day with probability 0.15. ‘The dog cannot go from one forest to the other. Oscar can search only in the daytime, and he can travel from one forest to the other only at night. All parts of this problem are to be worked separately. a In which forest should Oscar look to maximize the probability he finds his dog on the'first day of the search? b Given that Oscar looked in A on the first day but didn’t find his dog, what is the probability that the dog is in A? ¢ If Oscar flips a fair coin to determine where to look on the first day and finds the dog on the first day, what is the probability that he Jooked in A? d Oscar has decided to look in A for the first two days. What is the a priori probability that he will find a live dog for the first timeon the second day? @ Oscar has decided to look in A for the first two days. Given the fact that he was unsuccessful on the first day, determine the prob- ability that he does not find a dead dog on the second day. f Oscar finally found his dog on the fourth day of the search. He looked in A for the first 3 days and in B on the fourth day. What is the probability he found his dog alive? 9 Oscar finally found his dog late on the fourth day of the search. The only other thing we know is that he looked in A for 2 days and and in B for2days. What is the probability he found his dog alive? 1.13 Considering the statement of Prob. 1.12, suppose that Oscar has decided to search each day wherever he is most likely to find the dog on that day. He quits as soon as he finds the dog.GN EVENTS, SAMPLE SPACE, AND PROBABILITY a What is the probability that he will find the dog in forest..A? b What is the probability that he never gets to look in forest B? ¢ Given that he does get to look in forest B, what is the probability that, he finds the dog during his first day of searching there? d Given that he gets to look in forest B at least once, what is the probability that he eventually finds the dog in forest A? 1.14 Joe is an astronaut for project Pluto. Mission success or failure depends only on the behavior of thre major systems. Joe decides the following assumptions are valid and apply to the performance of an entire mission: (1) ‘The mission is a failure only if two or more of the major systems fail. (2) System I, the Gronk system, will fail with probability 0.1. (3) If at least one other system fails, no matter how this comes about, System II, the Frab system, will fail with conditional probability 0.5. If no other system fails, the Frab system will fail with probability 0.1. (4) System III, the Beer Cooler, fails with probability 0.5 if the Gronk system fails. Otherwise, the Beer Cooler cannot fail. a What is the probability that the mission succeeds but that the Beer Cooler fai b What is the probubility that all three systems fail? ¢ Given that more than one system failed, determine the conditional probabilities that: The Gronk did not fail. The Beer Cooler failed. iii Both the Gronk and the Frab failed. @ About the time when Joe was due buck on Earth, you overhear radio broadcast about Joe in a very noisy room. You are not positive what the announcer did say, but, based on all available information, you decide that it is twice as likely that he reported “Mission a success” as that he reported “Mission a failure.” What now is the conditional probability (to you) that the Gronk failed? 1.15 A box contains two fair coins and one biased coin. For the biased coin, the probability that any flip will result in a head is $. Al draws two coins from the box, flips each of them once, observes an outcome of one head and one tail, and returns the coins to the box. Bo then draws one coin from the box and flips it. The result is a tail. Deter- mine the probability that neither Al nor Bo removed the biased coin from the box. 116 Joe, the bookie, is attempting to establish the odds on an exhibition baseball game. From many years of experience, he has learned that his prime consideration should be the selection of the starting pitcher.PROBLEMS: 35 The Cardinals have only two pitehers, oe of whom must start: C1, their best pitcher; C2, a worse pitcher. The other team, the Yankees, has only three pitchers, one of whom must start: Y,, their best pitcher; Ys, a worse one; Ys, an even worse one. By carefully weighting information from various sources, Joe has decided, to make the following assumptions: The Yankees will not start Y, if C; does not start. If C1 does start, the Yankees will start ¥, with probability 4. The Cardinals are equally likely to start C; or C2, no matter what the Yankees do. ¥z will pitch for the Yankees with probability $ if Y; does not piteh, no matter what else occurs The probability the Cardinals will win given C; pitches is m/(m +1), where m is the number (subscript) of the Yankee pitcher. The probability the Yankees will win given C2 pitehes is 1/m, where m is the number of the Yankee pitcher. a What is the probability that C2 starts? b What is the probability that the Cardinals will win ¢ Given that V2 does not start, what is the probability that the Cardi- nals will win? 4 If Cz and Y; start, what is the probability that the Yankees win? a 1.17 Die A has five olive faces and one lavender face; die B has three faces of each of these colors. A fair coin is flipped once. If it falls heads, the game continues by throwing die A alone; if it falls tails, die B alone is used to continue the game. However awful their face colors may be, it is known that both dice are fair. a Determine the probability that the nth throw of the die results in olive. b Determine the probability that both the nth and (n + I)st throw of the die results in olive. € If olive readings result, from all the first n throws, determine the conditional probability of an olive outcome on the (n + I)st toss. Interpret your result for large values of n. 1.18 Consider events A, B, and C with P(A) > P(B) > P(C) > 0. Events A and B are mutually exclusive and collectively exhaustive. Events A and C are independent. Can C and B be mutually exclusive? 1.18 We are given that the following six relations hold: P(A) #0 P(B) #0 PC) #0 P(AB) = P(A)P(B) — P(AC) = P(A)P(C) (BC) = P(B)P(C)36 EVENTS, SAMPLE SPACE, AND PROBABILITY Subject to these six conditions, determine, for each of the following entries, whether it must be true, it might be true, or it cazinot be true: a P(ABC) = P(A)P(B)P(C) __b P(B| A) = P(B(C) ¢ P(AB|C) = P(A| C)P(B| C) d P(A +B+C) < P(A) + P(B) + PLC) e If W = ABand V = AC, WV = 6 1.20 Events £, F, and @ form a list of mutually exclusive and collectively exhaustive events with P(E) x 0, P(F) 0, and PG) #0. Deter mine, for each of the following statements, whether it must be true, it might be true, or it cannot be true: a Z’, F’, and @ are mutually exclusive, 6 BE’, F’, and @’ are collectively exhaustive. ¢ BY and F” are independent. d PUB) + POP!) > 1.0. e P(E! + EF’ + EPG!) = 10. 121 Bo and Ci are the only two people who will enter the Rover Dog Food jingle contest. Only one entry is allowed per contestant, and the judge (Rover) will declare the one winner as soon as he receives a suitably inane entry, which may be never. Bo writes inane jingles rapidly but poorly. He has probability 0.7 of submitting his entry first. If Ci has not already won the con- test, Bo’s entry will be declared the winner with probability 0.3. Ci writes slowly, but he has a gift for this sort of thing. If Bo has not already won the contest by the time of Ci’s entry, Ci will be declared the winner with probability 0.6. a What is the probability that the prize never will be awarded? b What is the probability that Bo will win? ¢ Given that Bo wins, what is the probability that Ci’s entry arrived first? d What is the probability that the first entry wins the contest? e Suppose that the probability that Bo’s entry arrived first were P instead of 0.7. Can you find a value of P for which “First entry wins” and “Second entry does not win” are independent events? 122 t a i { 2 I m tt Each +} represents one communication link. Link failures arePROBLEMS 37 independent, and each link has a probability of 0.5 of being out of service. Towns A and B can communicateas long as they are connected in the communication network by at least one path which contains only in-service links. Determine, in an efficient manner, the probability that A and B can communicate. @ @ 1.23 Hi i -L,L, || 8 e H In the above communication network, link failures are inde- pendent, and each link has a probability of failure of p. Consider the physical situation before you write anything. A can communicate with B as long as they are connected by at least one path which contains only in-service links. a Given that exactly five links have failed, determine the probability that A can still communicate with B. b Given that exactly five links have failed, determine the probability that either g or h (but not both) is still operating properly. ¢ Given that a, d, and have failed (but no information about the condition of the other links), determine the probability that A can communicate with B. 124 The events X, Y, and Z are defined on the same experiment. For each of the conditions below, state whether or not each statement follow- ing it must necessarily be true if the condition is true, and give a reason for your answer. (Remember thet a simple way to demonstrate that a statement is not.necessarily true may be to find a counterexample.) a If P(X + Y) = P(X) + P(Y), then: i P(XY) = P(X)P(Y). ii P(Y) = 0 wX+V¥=X4+XY ww P(XY) =0 b If P(XY) = P(X)P(Y) and P(X) # 0, P(Y) #0, then: i P(X | ¥) = POY |X) ili P(XY) #0 ti P(X + ¥) = P(X) + P(Y) — P(XY) iv X.# YunlessX = U © If P(XY |Z) = 1, then: iP(XY)=1 ili P(XYZ) = PZ) iZ=U iv P(X'|Z) =0 d If P(X + ¥) |Z) and X # @, Y # ¢, Z = @, then: i P(XY|Z)=0 iii P(X’Y’|Z) WP(XX+Y¥) <1 Ww P(X|Z) <1 e Define the events R = X, 8 = X'Z, T = X'Y’ If the events R,38 = = EVENTS, SAMPLE SPACE, AND PROBABILITY S, and T are collectively exhaustive, then: i R, 8, and Tare mutually P(X'YZ’) =0 exclusive li P(RST) = P(S)P(RT|S) wv PU(R+S+7)| XYZ) =1 1.25 Only at midnight may a mouse go from either of two rooms to the other. Each midnight he stays in the same room with probability 0.9, or he goes to the other room with probability 0.1. We can’t observe the mouse directly, so we use the reports of an unreliable observer, who cach day reports correctly with probability 0.7 and incorreetly with probability 0.3. On day 0, we installed the mouse in room 1. We use the notation: Event Ri(n): Observer reports mouse in room k on day. Event Si(n): Mouse is in room k on day n, a What is the a priori probability that on day 2 we shall receive the report Ra(2)? b If we receive the reports (1) and (2), what is the conditional probability that the observer has told the truth on both days? ¢ If we receive the reports Rx(1) and Ra(2), what is the conditional probability of the event S2(2)? Would we then believe the observer on the second day? Explain 4 Determine the conditional probability Pli2q(2) | Ry(1)}, and compare it with the conditional probability P[S2(2) | S:(1)] 1.26 a The Pogo Thorhead rocket will function properly only if all five of its major systems operate simultaneously. ‘The systems are labeled A, B,C, D, and B. System failures are independent, and each system has a probability of failure of 1/3. Given that the Thorhead fails, determine the probability that system A is solely at, fault. b The Pogo Thorhead II is an improved configuration using the same five systems, each still with a probability of failure of 1/3. The ‘Thorhead II will fail if system A fails or if (at least) any two systems fail. Alas, the Thorhead II also fails. Determine the probability that system A is solely at fault. 1.27. The individual shoes from eight pairs, each pair a different style, have been put into a barrel. For $1, a customer may randomly draw and keep two shoes from the barrel. Two successive customers do this. What is the probability that at least one customer obtains two shoes from the same pair? How much would an individual customer improve his chanees of getting at leasi one complete pair by investing $2 to be allowed to draw and keep four shoes? (Some combinatorial problems are quite trying. Consider, if you dare, the solution to this problem for cight customers instead of two.) ene er Ree SHCA SSS A RNPPROBLEMS 39 1.28 Companies 4, B, C, D, and KE each send three delegates to a con- ference. A committee of four delegates, selected by lot, is formed. Determine the probability that: a Company A is not represented on the committee. b Company A has exactly one representative on the committee. ¢ Neither company A nor company E is represented on the committee. 1.29 A box contains N items, K of which are defective. A sample of AL items is drawn from the box at random. What is the probability the sample is a With replacement b Without replacement 1.30 ‘The Jones family household includes Mr. and Mrs. Jones, four chil- dren, two cats, and three dogs. Every six hours there is a Jones family stroll. ‘The rules for a Jones family stroll are: Exactly five things (people + dogs + cats) go on each stroll. Each stroll must include at least one parent and at least one pet. There can never be a dog and a cat on the same stroll unless both parents go. All acceptable stroll groupings are equally likely. Given that exactly one parent went on the 6 P.xt. stroll, what is the probability that Rover, the oldest dog, also went?CHAPTER TWO random variables A A RE Often we have reasons to associate one or more numbers (in addition to probabilities) with cach possible outcome of an experiment. Such numbers might correspond, for instance, to the cost to us of each experi- mental outcome, the amount of rainfall during a particular month, or the height and weight of the next football player we meet. ‘This chapter extends and specializes our earlier work to develop effective methods for the study of experiments whose outcomes may be described numerically.42 RANDOM VARIABLES 2-1 Random Variables and Their Event Spaces A random variabl random variable to each sample point i experiment. For the study of experiments whose outcomes may be specified numeri- cally, we find it useful to introduce the following definition: defined by a function which assigns a value of the the sample space of an Each performance of the experiment is said to generate an experimental value of the random variable. This experimental value of the random variable is equal to the value of the random variable assigned to the sample point which corresponds to the resulting experimental outcome. Consider the following example, which will be referred to in several sections of this chapter. Our experiment consists of three independent flips of a fair coin, We again use the notation even [i ent Tails | on the nth flip We may define any number of random variables on the sample space of this experiment. We choose the following definitions for two ran- dom variables, h and r: } = total number of heads resulting from the three flips r = length of longest run resulting from the three flips (a run is a set of suceessive flips all of which have the same outcome) We now prepare a fully labeled sequential sample space for this, experiment, Weinclude the branch traversal conditional probabilities, the probability of each experimental outcome, and the values of random variables h and r assigned to each sample point. ‘The resulting sample space is shown at the top of the following page. If this experiment were performed once and the experimental outcome were the event HiT:T, we would say that, for this per- formance of the experiment, the resulting experimental values of ran- dom variables h and r were 1 and 2, respectively. Although we may require the full sample space to describe the detailed probabilistic structure of an experiment, it may be that our only practical interest in each performance of the experiment, will relate to the resulting experimental values of one or more random variables |RANDOM VARIABLES AND THEIR EVENT SPACES 43. samole wns Pe) ee tlt os 3 WS enn 01s 22 Hy, eHTH, 012 20 1 a =~ AT Wn emmy ons 12 os emmy, 01s 22 1 eTHT, «012 1k 0125 1 2 a SNe 012 0 3 i fe When this is the case, we may prefer to work in an event space which distinguishes among outcomes only in terms of the possible experi- mental values of the random variables of interest. Let's consider this for the above example. Suppose that our only interest in a performance of the experi- ment has to do with the resulting experimental value of random varia- bleh. We might find it desirable to work with this variable in an event space of the form The four event points marked along the ho axis form a mutually exclu- sive collectively exhaustive listing of all possible experimental outcomes. ‘The event point at any ho corresponds to the event “The experimental value of random variable h generated on a performance of the experi- ment is equal to ho,” or, in other words, “On a performance of the experiment, random variable A takes on experimental value ho.” Similarly, if our concern with each performance of the experi- ment depended only upon the resulting experimental values of random variables h and r, a simple event space would be44 RANDOM VARIABLES ‘This event point represents the event . “exactly two heads resulted from the three flips and no pair of consecutive flips had the same outcome" An event point in this space with coordinates ho and rp corresponds to the event “On a performance of the experiment, random variables h and r take on, respectively, experimental values he and ry.” ‘The proba- bility assignment for each of these six event points may, of course, be obtained by collecting these events and their probabilities in the origi- nal sequential sample space. ‘The random variables discussed in our example could take on only experimental values selected from a set of diserete numbers. Such random variables are known as discrete random variables. Random variables of another type, known as continuous random variables, may take on experimental values anywhere within continuous ranges. Examples of continuous random variables are the exact instantaneous voltage of a noise signal and the precise reading after a spin of an infinitely finely calibrated wheel of fortune (as in the last example of Sec. 1-2). Formally, the distinction between discrete and continuous ran- dom variables can be avoided. But the development of our topics is casier to visualize if we first become familiar with matters with regard to discrete random variables and Inter extend our coverage to the con- tinuous case. Our discussions through Sec, 2-8 deal only with the diserete case, and Sec. 2-9 begins the extension to the continuous case. 2-2. The Probability Mass Function We have learned that a random variable is defined by a function which assigns a value of that random variable to each sample point. ‘These assigned values of the random variable are said to represent its possible experimental values. Each performance of the experiment generates tan experimental value of the random variable. For many purposes, we shall find the resulting experimental value of a random variable to be an adequate characterization of the experimental outcome. In the previous section, we indicated the form of a simple event space for dealing with asingle discrete random variable. To work with a random variable x, we mark, on an 2p axis, the points corresponding er net FN PSTHE PROBABILITY MASS FUNCTION 45 to all possible experimental values of the random variable. One such event space could be pe ty Each point in this event space corresponds to the event “On a per formance of the experiment, the resulting experimental value of random variable x is equal to the indicated value of zo.” We next define a function on this event space which assigns a probability to each event point. The function p,(20) is known as the probability mass function (PMF) for discrete random variable x, defined by p(t) = probability that the experimental value of random variable £ obtained on a performance of the experiment is equal to x» We often present the probability mass function as a bar graph drawn over an event space for the random variable. One possible PMF is sketched below: Pxl%) Since there must be some value of random variable x associated with every sample point, we must have ' Y p-(@o) = 1 and, of course, from the axioms of probability theory we also have 0S p(x) <1 for alll values of x» Note that the argument of a PMI is a dummy variable, and the PMF for random variable x could also be written as p.(y), p-(= ), or, as some people prefer, p_(:). We shall generally use the notation p, (te) for a PMP. However, when another notation offers advantages of clarity or brevity for the detailed study of « particular process, we shall adopt it for that purpose.46 BANDOM VARIABLES For an example of the PMF for a random variable, let's return, to the experiment of three flips of a fair coin, introduced in the previous section. We may go back to the original sample space of that experi- ment to collect py(ho), the PME for the total namber of heads resulting from the three flips. We obtain I Pgllig) E yllhg)= probability maes function fr random variable ote 2.3. Compound Probability Mass Functions We wish to consider situations in which values of more than one random Our discussion will be for two diserete random variables, but the exten- variable are assigned to each point in the sample space of an experiment. sion to more than two is apparent. For a performance of an experiment, the probability that random, variable z will take on experimental value x» and random variable y will take on experimental value yo may be determined in sample space by summing the probabilities of each sample point which has this com- pound attribate. To designate the probability assignment in an 0,40 event space, we extend our previous work to define the compound (or Joint) PMF for two random variables and y, Pew(20.¥o) = probability that the experimental values of random vari bles x and y obtained on @ performance of the experi- ment are equal to x9 and yo, respectively A picture of this funetion would have the possible event points marked on an 2,yo coordinate system with each value of pzy(z0,ye) indicated as a bar perpendicular to the zayo plane above each event point. (We use the word event point here since each possible (to,yo) point might represent the union of several sample points ia the finest-grain deserip- tion of the experiment. ] By considering an zo,yo event space and recalling that an event space is a mutually exclusive, collectively exhaustive listing of all pos-COMPOUND PROBABILITY MASS FUNCTIONS AT sible experimental outcomes, we see that the following relations hold: YT praltowe) = 1 Y peoltayes) =P) — Yi Pea(eoute w Px(X0) Insituations where we are concerned with more than one random varia ble, the PME’s for single random variables, such as p,(1.), are referred to as marginal PMF’s. No matter how many random variables may be defined on the sample space of the experiment, this function p,(0) always has the same physical interpretation. For instance, p.(2) is the probability that the experimental value of discrete random variable x resulting from a performance of the experiment will be equal to 2. Let's return to the example of three flips of a fair coin in See. 2-1 to obtain the compound PMF for random variables h (number of heads) and r (length of longest run). By collecting the events of interest and their probabilities from the sequential sample space in Sec. 2-1, we obtain pi,,(hors). We indieate the value of pi,r(he,7s) associated with each event by writing it beside the appropriate event point. The probability of any event described in terms of the experi- mental values of random variables h and r may be found easily in this event space once py,-(hor») has been determined. For instance, we may obtain the marginal PMI"s, p,(fte) and P2(70), simply by collecting the probabilities of the appropriate events in the hora sample space.48 RANDOM VARIABLES ‘The reader can check that the above procedure and a similar operation for random variable h lead to the marginal PMF’s Palo) Pelt) ony, _— ho r ° . 28 ‘o It is important to note that, in general, there is no way to go back from the marginal PMF’s to determine the compound PMI. 2-4 Conditional Probability Mass Functions Our interpretation of corditional probability in Chap. 1 noted that conditioning a sample space does one of two things to the probability of each finest-grain event. If an event does not have the attribute of the conditioning event, the conditional probability of that event is set to zero. For all finest-grain events having the attribute of the condi- tioning event, the conditional probability associated with each such event is equal to itsoriginal probability scaled up by a constant [1/P(A), where A is the conditioning event] such that the sum of the conditional probabilities in the conditional sample space is unity. We can use the same concepts in an event space, as long as we can answer either “yes” or “no” for each event point to the question “Does this event have the attribute of the conditioning event?” Difficulty would arise only when the conditioning event was of finer grain than the event space. This matter was discussed near the end of Sec. 1-4. When we consider a discrete random variable taking on a partic- ular experimental value as a result of a performance of an experiment, this is simply an event like those we denoted earlier as A, B, or anything else and all our notions of conditional probability are carried over to the discussion of discrete random variables. We define the conditional PMP by Peiy(t0 ye) = conditional probability that the experimental value of random variable z is zo, given that, on the same perfor- — aCONDITIONAL PROBABILITY MASS FUNCTIONS = 49. mance of the experiment, the experimental value of ran- dom variable y is yo TA From the definition of conditional probability, there follows = Pew(oito) Peww(Z0y¥Yo) Paiv(ee| yo) = SG) "paCe) ‘As was the case in the previous chapter, the conditional probabilities are not defined for the case where the conditioning event is of proba- bility zero. Writing these definitions another way, we have and, similarly, — pyie(yo | Zo) Pro(eye) = PeCto)Puic(Yo} 0) = Py(yo)Psiv(ee | yo) Notice that, in general, the marginal PMF’s p.(zo) and p,(ye) do not specify p.»(2oyo) just as, in general, p(A) and p(B) do not specify p(AB). Finally, we need a notation for a conditional joint probability mass funetion where the conditioning event is other than an observed experimental value of one of the random variables. We shall use Praja(zo;yo| A) to denote the conditional compound PMF for random variables x and y given event A. This is, by the definition of condi- tional probability, PeyF0U) if Cay) in A 0 if (#o,yo) in A’ We return to the ho,ro event space of the previous sections and its compound PMF in order to obtain some experience with conditional probability mass functions Pealalta,yo| A) = hg ro event space including value of joint PM for each ‘event point ste We begin by finding the conditional PMF for random variable r, the length of the longest run obtained in three flips, given that the experi- mental value of h, the number of heads, is equal to 2. Thus, we wish to find paa(r»| 2). Only two of the event points in the original heroour reasoning in sample space, there results Prin(tg!2) rnd Paaltgl2v“42 rom? for all other values of 7, ot 2S ° Had the conditioning event been that the experimental value of h were equal to 3, this would specify the experimental value of r for this experiment, because there is only one possible hero event point with hg = 3. The resulting conditional PMF would, of course, be Pagltol3) 50 RANDOM VARIABLES event space have the attribute of the conditioning event (hy = 2), and conditional event space. Either by direct use of the defini resulting from a performance of the experiment are not equal, In going to the appropriate conditional event space and allocation of con- ditional probability, we remove event points incompatible with the conditioning event and renormalize to obtain Conditional hor ‘event space gen "ex perimental values of hand r are not equal" Uf we let A cepresent tis con ditioning event, the numbers beside each event pont are e9Ual 10 P, nalltgr ol) ho Finally, we can note one resonable example of a phenomenon Suppose that we wish to condition the compound PMP in the faoyre event space by the event that the experimental values of h and r which was mentioned earlier. We have stated that we cannot alwaysRandom variables 2 and y are defined to be independent if and only if pye(yo | v0) = py(yo) for all possible values of x» and yo. INDEPENDENCE OF RANDOM VARIABLES 51 directly condition an event space by an arbitrary event defined on the experiment. For our example, if we were told that the second flip resulted in heads, our simple conditioning argument cannot. be applied in an ho,ro event space because we can’t answer uniquely “yes” or “no” as to whether or not each event point has this attribute. ‘The condi- tioning requires information which appeared in the sequential sample space of Sec, 2-1 but which is no longer available in an ho,rs event space. Independence and Conditional Independence of Random Variables In Sec. 1-6 we obtained our definition of the independence of two events by stating formally an intuitive notion of independence. For two random variables to be independent, we shall require that no possi- ble experimental value of one random variable be able to give us any new information about the probability of any experimental value of the other random variable. A formal statement of this notion of the inde- pendence of two random variables is From the definition of the conditional PMI"s, as long as the conditioning event is of nonzero probability, we may always write Pryloyo) = Pe(t0)Pus(yo| 20) = Py(yoPav(e | yo) and, substituting the above definition of independence into this equa- tion, we find that pyie(yel 0) = py(yo) for all xoyo requires that, Psiy(t0| Yo) = ps(to) for all zo,yo; thus, one equivalent definition of the independence condition would be to state that random variables x and y are independent if and only if p.y(xo,yo) = pz(to)Py(yo) for all ToYo We define any number of random variables to be mutually inde- pendent if the compound PMF for all the random variables factors into the product of all the marginal P\I"s for all arguments of the compound PMI, It is also convenient to define the notion of conditional indepen- dence. One of several equivalent definitions is i Random variables « and y are defined to be conditionally independent given event A (with P(A) # 0] if and only if Prxia(to,yo| A) = pata(eo | A)pyia(yo | A) for all (wa,yo) = = =52 RANDOM VARIABLES Of course, the previous unconditional definition may be obtained by setting A = U in the conditional definition. The function p.ia(t0| A) is referred to as the conditional marginal PMF for random variable z given that the experimental outcome on the performance of the experi- ment had attribute A. We shall learn later that the definition of independence has implications beyond the obvious one here. In studying situations involving several random variables, it will normally be the case that, if the random variables are mutually independent, the analysis will be greatly simplified and several powerful theorems will apply. ‘The type of independence we have defined in this section is often referred to as true, or statistical, independence of random variables. ‘These words are used to differentiate between this complete form of independence and a condition known as linear independence. The latter will be defined in Sec. 2-7. The reader may wish to use our three-flip experiment and its aro event space to verify that, although h and r are clearly not inde- pendent in their original event space, they are conditionally independent given that the longest run was shorter than three flips. 2-6 Functions of Random Variables A function of some random variables is just what the name indicates— it is a function whose experimental value is determined by the experi- mental values of the random variables. For instance, let A and r again be the number of heads and the length of the longest run for three flips of a fair coin, Some functions of these random variables are (hr) =A wlhyr) =|h—r]— xlhyr) Uhr) = vex ( and, of course, h and r. Functions of random variables are thus new random variables themselves. The experimental values of these new random variables may be displayed in the event space of the original random variables, for instance, by adding some additional markers beside the event points. Once this is done, it is a simple matter to assemble the PMF for a function of the original random variables. For example, let random variable w be defined by w = |i — We'll write the value of w assigned to each event point in a box beside the point in the hore event space, aEXPECTATION AND CONDITIONAL EXPECTATION 53 J 40 46) y ef) 30) oa and then working in the above event space, we can rapidly collect Pw(wa), to obtain Put) y ° 12a wo 2-7 Expectation and Conditional Expectation Let x be @ random variable, and let g(z) be any single-valued function of its argument. ‘Then g(z) is a function of a random variable and is itself a random variable. We define Elg(z)], the expectation, or expeited value, of this function of random variable x, to be Blg(e)) = S gtedpstes) = ge) and we also define E(g(x) | A}, the conditional expectation of g(x) given that the experimental outcome has attribute A, to be Elg(z) | A] = ¥ o(e0)paa(zo| A) = ge] A)RANDOM VARIABLES As usual, this definition for the conditional case includes the uncondi- tional case (obtained by setting A equal to the universal event) Consider the event-space interpretation of the definition of Elg(z)] in an zo event space. For each event point xs, we multiply (to) by the probability of the event point representing that experi- mental outcome, and then we sum all such products. Thus, the expected value of g(x) is simply the weighted sum of all possible experi- mental values of g(z), each weighted by its probability of occurrence on a performance of the experiment. We might anticipate a close relationship between E[g(z)] and the average of the experimental values of g(2) generated by many performances of the experiment. This type of relation will be studied in the last two chapters of this book. Certain eases of g(z) give rise to expectations of frequent interest and these expectations have their own names. Tf gw) = 2": Hlo(e)) = Y cerpalee) = Hi g(2) = fz — E(@)}: Elg(e)} = to — Ba) pelae) = (c — #)* ‘The quantity x" is known as the nth moment, and the quantity (t — 4)" is known as the nth central moment of random variable 2. Often we desire a few simple parameters to characterize the PMF fora particular random variable. ‘Two choices which have both intuitive appeal and physical significance are the expected value and the second central moment of the random variable. We shall discuss the intuitive interpretation of these quantities here. Their physical significance will become apparent in our later work on limit theorems and statistics, ‘The first moment (or expected value or mean value) of random variable x is given by E(a) = J: xpalae) and if we picture a PMF bar graph for random variable «to be com- posed of broomsticks on a weightless axis, we may say that E(x) specifies the location of the center of mass of the P:\ The second central moment H'{{r — H(z)|*} is a measure of the second power of the spread of the PMF for random variable x about its expected value. The second central moment of random variable x is known as its variance and is denoted by a,?. ‘The square root of the Sep seer a tra eer WEEE TREO TES RARE TE A EETEXPECTATION AND CONDITIONAL EXPECTATION 55 variance, ¢., is known as the standard deviation of random variable x and may be considered to be one characterization of the spread of a PMF about H(z). Here are a few PM's for random variable «, each with the same mean but different standard deviations. Palio) Pelt) E(2)=0 oa 1 ant 120 Bis) =00 Es) =00 of = 40 of = 250 = 20 7 158 A conditional central moment is a measure of the nth power of the spread of a conditional PMF for a random variable about its con- ditional mean. For instance, given that the experimental outeome had attribute A, the conditional variance of random variable 2, 03,4, is given by Ee a ata = S (ee— Ble] A)ltpaalvol A) te . a ga For functions of several random variables, we again define expec tation to be the weighted sum of all possible experimental values of the function, with each such value weighted by the probability of its occurrence on a performance of the experiment. Let g(zy) be a single-valued function of random variables x andy. By now we are familiar enough with the ideas involved to realize that a definition of Elg(z,y) | A], the conditional expectation of g(x,y), will include the definition for the unconditional case Flg(zy) | Al = YY g(eouyodpsaraeouyo| A) We should remember that in order to determine the expectation56 RANDOM VARIABLES of g(2) lor of g(z,y) or of any funetion of g(z,y)] it is not necessary that we first determine the PMF p,(go). The calculation of Elg(z,y)} can always be carried out directly in the xojyo event space (see Prob. 2.10). We wish to establish some definitions and results regarding the expected values of sums and products of random variables. From the definition of the expectation of a function of several random variables, we may write Ee + 9) = YY Got yo)pev(eoye) = LY cepew(t0.ye) + YY verew(eouo) Weil nim one Wet som tne ‘Seer yo Bist ‘over so feat = Y zepc(ze) + Y) yor (yo) = Ele) + Ely) = Ele + u) The expected value of the sum of two random variables is always equal to the sum of their expected values. This holds with no restrictions on the random variables, which may, of course, be functions of other random variables. ‘The reader should, for instance, be able to use this result directly in the definition of the variance of random variable x to show ost = Elle — E(@)|*} = E(t) — (B@))* Now, consider the expected value of the product ey, Eley) = YY cavepealoys) In general, we can carry this operation no further without some know! edge about p.(zoyo). Clearly, if x and y are independent, the above expression will factor to yield the result E(zy) = E(x)B(y). Even if and y are not independent, it is still possible that the numerical result would satisfy this condition. If E(zy) = B(z)E(y), random variables x and y are said to be linearly === independent. (Truly independent random variables will always satisfy == this condition.) BS An important expectation, the covariance of two random vari- ables, is introduced in Prob. 2.33. Chapter 3 will deal almost exclu: sively with the properties of some other useful expected values, the transforms of PMF’s. We shall also have much more to say about expected values when we consider limit theorems and statistics in Chaps. 6 and 7.EXAMPLES INVOLVING DISCRETE RANDOM VARIABLES 57 2.8 Examples Involving Discrete Random Variables exampl We have dealt with only one example related to our study of discrete random variables. We now work out some more detailed examples. 1 A biased four-sided die is rolled once. Random variable N is defined to be the down-face value and is described by the PMF, pu) = {RP for Ne = 1,2, 8,4 0 for all other values of No Based on the outcome of this roll, a coin is supplied for which, on any flip, the probability of an outcome of heads is (N + 1)/2N. The coin is flipped onee, and the outcome of this flip completes the experiment. Determine: (a) The expected value and variance of discrete random variable N. (b) The conditional PMF, conditional expected value, and conditional variance for random variable N given that the coin came up heads. (0) If we define the events Event A: Value of down face on die roll is either 1 or 4 Event H: Outeome of coin flip is heads are the events A and H independent? We'll begin by drawing a sequential sample space for the experi- ment, labeling all branches with the appropriate branch-traversal probabilities, and collecting some relevant information. PU) # A AH & v v v & Ble x x a Tre 8 y ay iewe ’ ns 2 pore 4 03 2 4 a ee y Nyn3 “mre 2 o 3 & : a ™~ 4 se MI = i58 RANDOM VARIABLES a Applying the definitions of mean and variance to random variable N, deseribed by the PMF py(No) given above, we have EN) = Y, Nopw(No) = boas + 2+ + 3a + 4 vy = 30 x ont = YY (Ne ~ EWN) F*px(No) m = (=2) fe (C1 ie + OF ae + IP ay = LO b Given that event H did occur on this performance of the experiment, ve may condition the above space by event H by removing all points with attribute 7 and scaling up the probabilities of all remaining events by multiplying by 1/PUZ) = 10/7. ‘This results in the four-point conditional event space shown below. To the right of this conditional event, space we present the resulting conditional PMI for random vari- able N, given that event H did occur. Pran(MolED P(+|H) eHMmnt & +HNo k eHm3 enNet & Applying the definitions of the mean and variance in this conditional event space, we have EWN | H) = ¥ Nopwix(No| H) = Pet Qtet Bitet b= & oxia? = (No — EWN | H)Itpwin(No| H) x = (Mt Ds +t Dt et OP te = ¢ We wish to test P(AH) 2 P(A)P(HH), and we have already collected each of these three quantities in the sample space for the experiment. PA)= HO PU) =a PAW) = So the events A and H are independent. SRS ERTS NS SE PARED SL MOORE SSSR IA RI OE SEE CNR CE TEE REEEXAMPLES INVOLVING DISCRETE RANDOM VARIABLES 59 example 2 Patrolman G. R. Aft of the local constabulary starts cach day by deciding how many parking tickets he will award that day. For any day, the probability that-he decides to give exactly K tickets is given by the PMP) pe(Ks) = [a0" for Ky = 1,2, 3, or4 0 for all other values of Ky But the more tickets he gives, the less time he has to assist old ladies at street crossings. Given that he has decided to have a K-ticket day, the conditional probability that he will also help exactly 1 old ladies cross the street that day is given by le ifl 5 — Ko His daily salary S is computed according to the formula S=2K+L (dollars) Before we answer some questions about Officer Aft, we should be sure that we understand the above statements. For instance, on a day when Officer Aft has decided to give two tickets, the conditional PMF states that he is equally likely to help one, two, or three old Indies. Similarly, on a day when he has decided to give exactly four tickets, it is certain that he will help exactly one old lady cross the street. (a) Determine the marginal PMF pr(1o). ‘This marginal PMF tells us the probability that Officer Aft, will assist exactly Ly old ladies on any day. Determine also the expected value of random vari- able L. (b) Random variable S, Officer Aft’s salary on any given day, is a fune- tion of random variables K and L. Determine the expected value of the quantity S(L,K). (c) Given that, on the day of interest, Officer Aft earned at least $6, determine the conditional marginal PMF for random variable K, the number of traffic tickets he awarded on that particular day. (d) We define Event A: Yesterday he gave a total of one or two parking tickets. Event B: Yesterday he assisted a total of one or two old ladies. Determine whether or not random variables K and L are condi- tionally independent given event AB. prx(Lo| Ko) = { From the statement of the example we can obtain a sample space and the assignment of a priori probability measure for the experiment. We could begin with a sequential picture of the experiment such as60 RANDOM VARIABLES probability assignment px.z(Ko,/.o) determined by Pxt(Ko,Lo) = px(Ko)prix(Lo| Ke) - (So Le if Ko = 1,2,3,4and 1 < Ip <5~—Ko 0 otherwise = {04 if Ko = 1,2, 3,4and1 < Ip <5 — Ky “10.0 otherwise We have established that each ‘of the ten possible event points has an a priori probability of 0.1 ‘of representing the outcome of ‘any particular day in the life of Officer GR. A or we might work directly in a Ko,» coordinate event space with the | eEXAMPLES INVOLVING DISCRETE RANDOM VARIABLES 61 a The calculation px(Lo) = J) pu.x(Lo,K,) is easily performed in our g event space to obtain Pill) oa 3 od onl 1 2 3 4 Ly To find the expectation of /., we could multiply the experimental value of 1. corresponding to each sample point: by the probability measure assigned to that point and sum these products. But since we already have pz(Z0), it is quicker to work in the event space for random variable L to obtain E(L) = ¥) Lopt(Io) = Livy + 2s + 3 fo t 4 ory = 20 a Although it has happened in this example, we should note that there is ‘no reason whatever why the expected value of a random variable need be equal to a possible experimental value of that random variable. b E(S) = PY (2Ko + Lo)px.x(Ko,l0). We can simply multiply the Bi experimental value of S corresponding to each event point by the probability assignment of that event point. Let’s write the corre- sponding experimental value of S beside each event point and then compute E(S).62 RANDOM VARIABLES ES) = WB +4+54+6+54+64+74+74+849) = 96 ¢ Given that Officer Aft earned at least $6, we can easily condition the above event space to get the conditional event space (still with the experimental values of $ written beside the event points representing the remaining possible experimental outcomes) Since the relative likelihood of all event points included in the conditioning event can't change, these remain. ing six event points are sti equally probable Conditional KaZo sample space given ‘experimental outcome had attribute S260 ‘Thus, by using the notation Event C: S > 6, we have PjeCKoIC) Recall that we can use this simple interpretation of conditional prob- y only if the event space to which we wish to apply it is of fine hough’ grain to allow us to classify each event point as being wholly in C or being wholly in C’. d There are only four (equally likely) event points in the conditional Keg event space given that the experimental outcome has attribute AB. Conditional Ky Lo event soe space given experimental outcome has attribute AB | ggA BRIEF INTRODUCTION TO THE UNIT-IMPULSE FUNCTION 63 We wish to check Pr.tiaa(Ko,lo| AB) © pxjao(Ko| AB)pujan(Le | AB) for all Ko,Lo Each of these three PMI”s is found from the conditional Ko,Ly event space presented above if Ko = 1, 2and Ly = 1,2 otherwise if Ky = 1,2 otherwise Px.yan(Kejbo | AB) { t Prian(Ko| AB) = { : Puan(Lo | AB) ; otherwise The defini \ wwe conclude that random variables K and L, which were not inde- i pendent in the original sample space, are conditionally independent i given that the experimental outcome has attribute AB. Thus, for | instance, given that AB has occurred, the conditional marginal PDF ; of variable Z, will be unaffected by the experimental value of random | variable K. | In this text, the single word independence applied to random j | | i n of conditional independence is found to be satisfied, and varinbles is always used to denote statistical independence 2.9 A Brief Introduction to the Unit-impulse Function To prepare for a gencral discussion of continuous random variables, it is desirable that we become familiar with some properties of the unit-impulse function. Our introduction to this function, though adequate for our purposes, will lack certain details required to deal with more advanced matters. The unit-impulse function wo(to ~ a) is a function of ry which is equal to infinity at x9 = a and which is equal to zero for all other values of x ‘The integral of yo(te — a) over any interval which includes the point where the unit impulse is nonzero is equal to unity. One way to obtain the unit impulse uo(ta — a) is to consider the Limit, as A goes to zero, of gl%o-a) ray | T t64 RANDOM VARIABLES which, after the limit is taken, is normally represented as g(a) where the scale height of the impulse is irrelevant and the total area under the impulse function is written beside the arrowhead. The integral of the unit-impulse function, [E walee ~ 0) do isa function known as the unit-step function and written as p_1(to ~ a). 29-4) ‘Thus an impulse may be used to represent the derivative of a funetion at a point where the function has a vertical discontinuity. As long as a function g(x0) does not have a discontinuity at 2» = a, the integral Jg(x0)uo(ro — a) do F over any interval whieh includes 7» = a is simply g(a). This results from the fact that, over the very small range where the impulse is non- zero, g(a) may be considered to be constant, equal to g(a), and factored out of the integral 2-10 The Probability Density Function for a Continuous Random Variable We wish to extend our previous work to include the analysis of situa- tions involving random variables whose experimental values may fall anywhere within continuous ranges. Some examples of such con- tinuous random variables have already occurred in Secs. 1-2 and 2-1. The assignment of probability measure to continuous sample and event spaces will be given by a probability density function (PDF). Let us begin by considering a single continuous random variable x whoseDENSITY FUNCTION FOR A CONTINUOUS RANDOM VARIABLE 65 event space is the real line from z= —% tom = ». We define the PDF for random variable x, fa(0), by . Prob(a a Prob(a < « $b) = peg(b) —Pr<(a@) forb>a reson) = flee ‘The CDF will be especially useful for some of our work with continuous random variables. For a diserete random variable, the CDF is dis- continuous and therefore, to some tastes, not differentiable. For our purposes, we have defined the derivative of the CDF at such a dis- continuity to be an impulse of infinite height, zero width, and area equal to the discontinuity. Let’s consider an experiment involving a random variable z defined in the following manner: A fair coin is flipped once. If the outcome is heads, the experimental value of z is to be 0.2. If the out- come is tails, the experimental value of z is obtained by one spin of a fair, infinitely finely calibrated wheel of fortune whose range is from zero to unity. This gives rise to the PDF |DENSITY FUNCTION FOR A CONTINUOUS RANDOM VARIABLE 67 fal) The scale height of the impulse sm is irrelevant, The ed under the impulse is written alongside it ‘002 ° and to the CDF Pasl¥) ¥o oa a6 10 Because of the integration convention discussed earlier in this section, we can note that p,<(z») has (in principle) its discontinuity immediately to the left of x» = 0.2. We consider one simple example which deals with a continuous random variable. Assume that the lifetime of a particular component is known to be a random variable described by the PDF Kay) Xp(months) Let’s begin by determining the a priori probability that the component fails during its second month of use. In an xp event space we can col- lect this event, The event: component fails during its / second month of use 068 RANDOM VARIABLES Thus, we require the quantity Prob(1 P(B). ‘One other approach to this question would he to condition the event space for random variable x by noting that experimental values of x between 0.0 and 1.0 are now impossible. ‘The relative probabilities of all events wholly contained within the conditioning event A’ are to remain the same in our conditional space as they were in the original event space. To do this, the a priori PDF for the remaining event points must be multiplied by constant so that the resulting conditional PDF for x will integrate to unity. This leads to the following condi- tional PDF faia-(zo| A’):COMPOUND PROBABILITY DENSITY FUNCTIONS 69 foal? /— vans xu a the conitanl pbabitiy of Ine te seared meth en tare ded etc ing te Wt mouth hes =f In this latter solution, we simply worked directly with the notion which was formalized in Sec, 1-4 to obtain the definition of conditional probability. Before closing this section, let us observe that, onee we are familiar with the unit-impulse function, » PMF may be considered to ropresent a special type of PDF. For instance, the following prob ability mass function and probability density function give identical descriptions of some random variable x: Belay) fel) Only when there are particular advantages in doing so shall we repre- sent the probability assignment to a purely discrete random variable by « PDF instead of a PME, ZL Compound Probability Density Functions We may now consider the case where several continnous random variables arc defined on the sample space of an experiment. ‘The assignment of probability mensure is then specified by a compound probability density function in an event space whose coordinates are the experimental values of the random variables, In the two-dimensional event space for the possible experimental values of random variables z and y, the compound PDF j,.,(0,¥s) may be pictured as » surface plotted above the x,y. plane. The volume enclosed between any area in the zo,ys plane and this fe (9,4) surface is equal to the probubility of the experimental outeome falling within that area, For any event A defined in the x,y plane we have70 RANDOM VARIABLES ee PUA) = ff Soalzoundeat ‘The probability that the experimental value of < will fall between zyand zp + dz, which we know must be equal to f(t0) dro, is obtained by integrating the compound PDF over the strip in the event space which represents this event, % Pob(s [2 olcoedfeataltone | A) de dyw All the definitions and results obtained in Sec. 2-7 carry over directly to the continuous case, with summations replaced by integrations. Density Functions We have learned that g(x,y), a function of random variables x and y, is itself a new random variable. From the definition. of expectation, we also know that the expected value of g(z,y), or any function of g(x,y),DERIVED PROBABILITY DENSITY FUNCTIONS 75 may be found directly in the zo, event space without ever determin- ing the PDF f,(go). However, if we have an interest only in the behavior of random variable g and we wish to answer several questions about it, we may desire to work in ago event space with the PDF f,(g.)._ A PDF obtained for a function of some random variables whose PDF is known is referred to as a derived PDF. : We shall introduce one simple method for obtaining a derived distribution by working in the event space of the random variables ; whose PDF is known. There may be more efficient techniques for particular classes of problems. Our method, however, will get us there and because we'll live in event space, we'll always know exactly what we are doing. ‘To derive the PDF for g, a function of some random variables, } we need to perform only tio simple steps in the event space of the original random variables: _ Finst STEP: Determine the probability of the event g < go for all values of go. = secon step: Differentiate this quantity with respect to gp to obtain Folge). ‘The first step requires that we calculate the cumulative probability distribution function p,<(go). To do so; we simply integrate the given PDF for the original random variables over the appropriate region of their event space. Consider the following example: A fair wheel of fortune, con- tinuously calibrated from 0,00 to 1.00, is to be spun twice. ‘The exper mental values of random variables x and y are defined to be the readings on the first and second spins, respectively. (By “fair” we mean, of course, that the wheel has no memory (different spins are independent events) and that any intervals of equal are are equally likely to include the experimental outcome.] We wish to determine the PDF f,(go) for the case where random var ble g is defined by g(ey) = 7 ‘The example states that the spins arc independent. Therefore we may obtain the joint PDF 10° if0 1. For our par- ticular example, because of the very special fact that feyCzo,ye) is everywhere equal to either zero or unity, we can replace the integration asia) = ff ferenon) aes dys by a simple calculation of arcas to obtain, for the first step of our two-step procedure, Poslgs) = 42 AL this point, of course, we may eheck that this CDF is a monotonically increasing function of rp and that it increases from zero at ge = —% tounity at go = +2, Now, for step 2, we differentiate Prs(ge) to find 0 Lolo) = 40. 0.5907 fe) 039} oo ag SaDERIVED PROBABILITY DENSITY FUNCTIONS 7 Hf wo wish to derive a joint PDF, say, for g(z,y) and M(z,y), then in step 1 we collect the probability Pasac(nhe) for all values of go and ho which represents the joint CDF for random variables g and 4, For step 2 we would perform the differentiation A Pozas(gesho) — go dhs As the number of derived random variables increases, our method beconies tnressonably cumbersome; but more efficient techniques exist for particular types of problems. One further detail is relevant to the mechanies of the work involved in carrying out our two-step method for derived distributions ‘Our method involves an integeation (step 1} followed by a differentia- tion (step 2). Although the integrations and differentintions are gen- erally with respect to different variables, we may wish to differentiate first before formally performing the integration. For this purpose, i useful to remember one very useful formula and the picture from which it is obtained. = Soanlgoho) Tn working with a relation of the form Ra) = 00" raya) dz wehavean ategral over x whose lower limit, upper limit, and integrand ave all functions of a If we desire to obtain the derivative it is more efficient to use the following formula directly than to first perform the integration with respect to x and then to differentiate with respect to a IO) ala) 2 + rable) BO 4. fr ones) This relation is easy to remember if we keep in mind the picture from which it may be obtained No) Riaym f rlaxdde ‘The dotted tines represent ‘ehanges in r( a,x) and the fits ofthe integra ‘due to small charges in cx atay Bay78 RANDOM VARIABLES ‘The reader will have many opportunities to benefit from the dR(a) above expression for “7? in obtai ing derived distributions. For instance, in Prob, 2.30 at the end of this chapter, this expression is used in order to obtain the desired PDF in a useful form. 2-15 Examples Involving Continuous Random Variables However simple the concepts may seem, no reader should assume that he has a working knowledge of the material in this chapter until he has successfully mastered many problems like those at the end of the chap- ter. Two examples with solutions follow, but we must realize that these examples are necessarily a narrow representation of a very large class of problems. Our first example gives a straightforward drill on notation and procedures, The second example is more physically motivated. example 1 Random variables x, y, and z are described by the compound prob- ability density function, Sewaltoyoj20) = co” + 3yo% if0 Sm xp hundreds of dollars) As a matter of convenience, we are assuming that the currency is infinitely divisible, (Rounding off to the nearest penny wouldn't mat- ter much.) Oscar has a lot of experience at this. All of it is bad. Deeades of experience have shown that, over the course of an evening, Oscar never wins, In fact, the amount with whieh he returns home on any particular night is uniformly distributed between zero and the amount with whieh he started out, Let random variable y represent the amount (in hundreds of dollars) Osear brings home on any particular night. {a) Determine /..,(z0y0), the joint PDF for bis original wealth 2 and his terminal wealth y on any evening. (b) Determine f,(ye), the marginal probability density function for the amount Oscar will bring home on a randomly sclected night. (c) Determine the expected value of Oscar's loss on any particular night. (d) On one particular night, we learn that Oscar returned home with less than $200. For that night, determine the conditional proba- bility of each of the following events: (i) He started out for the casino with less than $200. i) His loss was less than $100. His loss that night was exactly $75. a From the example statement we obtain Ayia ¥ol%? T E orysaget Ayjeltglagd= 4"? 4 ” 0 otherwiseEXAMPLES INVOLVING CONTINUOUS RANDOM VARIABLES 81 The definition of conditional probability is used with the given f(z») to determine ee Faun) = Hodis) = {3% g 0SuSos4 otherwise and this result may be displayed in an z0,yo event space, (smmgersa, (nse Joint POF is equal to 1/8 inside this triangle ‘and zero elsewhere. A quick check shows that the calculated POF does integrate to nity over the entire event space. fobal captain ondeeds of dttars of * x Event: <5 Ip + Ay 7 . For0 Sy <4, Ailuo) = [7 dre feleaive) = [dts b= HA = yo) a Leow And we can sketch this PDF as 400) The f, (yp) POF does integrate to unity and, as was obvious from the os. above sketch (since the joint POF ‘was constant inside the triangle), \tis linearly decreasing from a maximum at yo=0 19 7670 at yom4 ° 20 % © Eley) = ff. wifealeoved dzadyo We must always be careful of the limits on the integrals when we82 RANDOM VARIABLES substitute actual expressions for the compound PDF. We'll integrate over 2p first. yay o 2% Ezy) = fiw dyo fe daa #(z0 — yo) = $133.33 where again we are using the convention that the successive integrals are to be performed in order from right to left. By changing a sign in the proof in Sec. 2-7, we may prove the relation E(@ - y) = El@) ~ EW) Since we already have the marginal PDI”s, this relation allows us to obtain E(x — y) by another route, Ble) — Bly) = [7 aafezs) dea — [| vefilus) de = [tte aes — fh yD SM aye = = fro Baee — ff yA GM ye = $133.83 4 Given that Oscar returned home with less than $200, we work in the conditional sampte spree. At all points consistent. with the conditioning event, the conditional PDF is equal to the original joint PDF scaled up by the reciprocal of the a priori probability of the conditioning event. 7 Tne event: "he returnes home with ess than $200." The a priori probability ofthis event is quai tothe integral of the @ priori joint OF ver this event wich i equal to ao in area of interest conditioning event ‘Thus, the conditional joint PDF is equal to 2/# = in the region where it is nonzero, Now we may answer all questions in this conditional space,PROBLEMS 83 {in area of interest event of interest %r% aoyel J[he event "his toss was less than $100" PUmdag = (1 2)=} 12 ° / Conditionat joint POF! Area representing in area of interest ‘event of interest We realize that, in general, we would have to integrate the conditional PDF over the appropriate events to obtain their proba- bilities, Only because the conditional PDI is a constant have we been able to reduce the integrations to simple multiplications, iil As fong as we allow the currency to be infinitely divisible, the con- ditional probability measure associated with the event ~ y = 75 is equal to zero. The integral of the compound PDF f,(to,4o) over the line representing this event in the ze,yo event space is equal to zero. PROBLEMS 2,01 The geometric PMF for discrete random variable K is defined to be CU Pe! if Ke = 1,2,3,...and0

0) 0 for all other values of Ky a Show that this PMP sums to unity. b Discrete random variables @ and S are defined on the sample spaces of two different, unrelated experiments, and these random variables have the PMP's a pr(Ro) = px(S) = EP S$ =0,1,2,... Use an RoSo sample space to determine the PMF pr(T%), where discrete random variable 7’ is defined by T = R + 8. ¢ Random variable W is defined by W = cR, where c is a known nonzero constant. Determine the PMF pw(Wo) and the expected value of W. How will the nth central moment of W change if ¢ is doubled? 2.04 Discrete random variable x is described by the PMF. mee nteg = [RH ifzo=0,1,2 0 for all other values of xo Let di, ds, .. . , dy represent N successive independent experimental values of random variable 2. a Determine the numerical value of K.AAT PROBLEMS 85 b Determine the probability that di > de. ¢ Determine the probability that di + d2+ +++ + dy <1.0 d Definer = max(d.,dz) ands = min(d,,d:). Determine the following PMF’s for, all values of their arguments: i pals) ii pais(ro | 0) iii pea(rojss) iv pu(to), with = (1 -+d)/(L + 8) e Determine the expected value and variance of random variable s defined above. f Given d; + dy < 2.0, determine the conditional expected value and conditional variance of random variable s defined above. 2.05 Discrete random variable z is described by the PMF p,(x»). Before an experiment is performed, we are required to guess a valued. After an experimental value of x is obtained, we shall then be paid an amount A ~ Ble — d)? dollars a What value of d should we use to maximize the expected value of our financial gain? b Determine the value of A such that the expected value of the gain is zero dollars. 2.06 Consider an experiment in which a fair four-sided die (with faces labeled 0, 1, 2, 3) is thrown once to determine how many times a fair coin is to be flipped. In the sample space of this experiment, random variables n and & are defined by n = down-face value on the throw of the tetrahedral die k = total number of heads resulting from the coin flips Determine and sketch each of the following functions for all values of their arguments: a palo) —B prja(ko[2) © Payal] 2) alka) € Also determine the conditional PMF for random variable n, given that the experimental value of & is an odd number. 2.07 Joe and Helen each know that the a priori probability that her mother will be home on any given night is 0.6. However, Helen can determine her mother’s plans for the night at 6 p.xt, and then, at 6:15 p.m, she has only one chance each evening to shout one of two code words across the river to Joe. He will visit her with probability 1.0 if he thinks Helen’s message means “Ma will be away,” and he will stay home with probability 1.0 if he thinks the message means “Ma will be home.” But Helen has a meek voice, and the river is channeled for heavy barge traffic. ‘Thus she is faced with the problem of coding for a noisy channel. She has decided to use a code containing only the code words A and B.RANDOM VARIABLES ‘The channel is described by PalA)=% Pl@|B)=% PelA)=4 POIB)=-4 and these events are defined in the following sketch: 2 A 3 ga iS event "Joe thinks message is A’ ? 3 1 a Bs 5 bis event toe thinks veces [eae 7 tx ws ou wos a In order to minimize the probability of error between transmitted and received messages, should Helen and Joe agree to use code I or code II? Code 1 Code I A = Ma away A = Ma home B = Ma home B= Ma away b Helen and Joe put the following cash values (in dollars) on all possible outeomes of a day: Ma home and Joe comes ~30 Ma home and Joe doesn’t come 0 ‘Ma away and Joe comes +30 ‘Ma away and Joe doesn’t come —5 Joe and Helen make all their plans with the objective of maximizing the expected value of each day of their continuing romance. Which of the above codes will maximize the expected cash value per day of this romance? € Clara isn’t quite so attractive as Helen, but at least she lives on the same side of the river. What would be the lower limit of Clara's expected value per day which would make Joe decide to give up Helen? 4 What would be the maximum rate which Joe would pay the phone company for a noiseless wire to Helen's house which he could use ‘once per day at 6:15 p.st.? e How much is it worth to Joe and Helen to double her mother’s probability of being away from home? Would this be a better or worse investment than spending the same amount of money for aPROBLEMS 87 telephone line (to be used once a day at 6:15 p.m.) with the following properties: P(a| A). = P@|B)=0.9 P| A) = P(@|B) = 0.1 2.08 A frazale is equally likely to contain zero, one, two, or three defects. No frazzle has more than three defects. The cash price of each frazzle is set at $(10 — K*), where K is the number of defects in it. Gummed labels, each representing $1, are placed on each frazzle to indicate its cash value (one label for a $1 frazzle, two labels for a $2 frazzle, ete.). What is the probability that a randomly selected label (chosen from the pile of labels at the printing plant) will end up on a frazzle which has exactly two defects? 2.09 A pair of fair four-sided dice is thrown once. Each die has faces labeled 1, 2, 3, and 4. Discrete random variable z is defined to be the product of the down-face values. Determine the conditional vari- ance of x? given that the sum of the down-face values is greater than the product of the down-face values. 2.10 Discrete random variables and y are defined on the sample space of an experiment, and g(2,y) is a single valued function of its argument. Use an event-space argument to establish that Elo(ea) = YY 9(t0sve)Peal20,90) Bin and : E(a) = Y.pore(ao) cof are equivalent expressions for the expected value of g(x,y). 211 At a particular point on a busy one-way single-lane road, a study is made of, the distribution of the interarrival period T between suc- cessive car arrivals. A reasonable quantization of the data for a chain of 10,001 éars results in the following tabulation: Tr, seconds 2 4 6 8 12 Number of occurrences 1,000 2,000 4,000 2,000 1,000 (Consider the cars to be as wide as the road, but very short.) a A young wombat, who never turns back, requires five seconds to cross the street. Determine the probability that he survives if: i He starts immediately after a car has passed. li He starts at a random time, selected without any dependence on the state of the traffic.88 RANDOM VARIABLES b Is the safer method of the above problem always safer, no matter what data are given? 212 The life span of a particular mechanical part is a random variable described by the following PDF: ffl) os oat i ™ ~ 0 and yo > 0 Sevltay) = {i Meret ae a Are x and y independent random variables? b Are they conditionally independent given max(z,y) < 0.57 ¢ Determine the expected value of random variabler, defined byr = zy. d If we define events A and B by Event A:2(y—z)2y+x Event By >t obtain the numerical values of P(A), P(B), P(A’B), P{(A’B'Y|, and determine and plot the conditional probability density function Seyarne(a0 | A'B'). 2.15 One of two wheels of fortune, A and B, is selected by the flip of a fair coin, and the wheel chosen is spun once to determine an experimental value of random variable z. Random variable y, the reading obtained with wheel A, and random variable w, the reading obtained with wheel B, are described by the PDI’sPROBLEMS 89 1 if0 xq(hours) 4 ot > vgkhowrs) Stations A and B are connected by two parallel message chan- nels. “A message from A to B is sent over both channels at the same time. Random variables x and y represent the message delays over parallel channels I and IJ, respectively. ‘A message is considered “received” as soon as it arrives on any one channel, and it is considered “verified” as soon as it has arrived over both channels. a Determine the probability that a message is received within 15 minutes after it is sent. b Determine the probability that the message is received but not veri- fied within 15 minutes after it is sent. ¢ Let u represent the time (in hours) between transmission at A and verification at B. Determine the cumulative distribution function Pys(uo), and then differentiate it to obtain the PDF f,(uo)- 4 If the attendant at B goes home 15 minutes after the message is received, what is the probability that he is present when the message should be verified? @ If the attendant at B leaves for a 15-minute coffee break right after90 RANDOM VARIABLES the message is received, what is the probability that he is present at the proper time for verification? £ The management wishes to have the maximum probability of having the attendant present for both reception and verification. Would they do better to let him take his coffee break as described above or simply allow him to go home 45 minutes after transmission? 2.18 The data for an experiment that could be performed only once con- sist of experimental values of random variables x and y, which are described by the a priori probability density function: — [Ary if 0< Got+tm) <1, m20, w 20 Seulto¥e) = | 9 otherwise But the experimental data were lost. All the experimenter remembers is that the experimental value of z was either 0.4 or 0.6, and he decides that he observed one or the other of these values with equal probability. Based on the above information, determine the experimenter’s probability density function for his experimental value of random variable y. LET 2.19 The exponential PDF for random variable z is given by felts) = de for ao > 0. a Determine the probability that an experimental value of z will be greater than E(z). b Suppose the lifetime of a bulb is given by the above PDF and we are told that the bulb has already been on for T units of time; determine the PDF for the remaining lifetime y = x — T of the bulb, (This special result will be discussed in Chap. 4.) ¢ Assume that each bulb is replaced as soon as it burns out. Over a very long period; determine the fraction of this interval for which illumination is supplied by those bulbs whose lifetimes are longer than E(2). NGS 2.20 a For three spins of a fair wheel of fortune, what is the probability that none of the resulting experimental values is within +30° of any other experimental value? b What is the smallest number of spins for which the probability that at least one other reading is within +30° of the first reading is at least 0.97 2.21 Random variable z is described by a PDF which is constant between 0 and 2» = 1 and which is zero elsewhere. K independent suc- cessive experimental values of this random variable are labeledAaa A PROBLEMS: 91 dh, ds, ... dx. Define the random variables r = second largest of di, dy... , dx s = second smallest of di, ds... , de and determine the joint probability density function f,.«(r0,80) for all values of ro and 9. 2.22. The probability density function for continuous random variable " isa constant in the range a < x os. ¢ Ify = cx +d, determine E(y) and o, in terms of H(z) and oz. Do your results depend on the form of the PDF for random variable 2? 223 Random variable z is described by the PDF filo) = {01 €0S x < 100 2) = 19 otherwise Another random variable, y, is defined by y = — Inz. Determine the PDF f,(yo). 2.24 Random variables z and y are distributed according to the joint probability density function Am if1 SoS yo <2 Sea(eaiye) = {oe otherwise a Evaluate the constant A. b Determine the marginal probability density function f,(yo). ¢ Determine the expected valueof 1/z, given that y = #. d Random variable z is defined by 2 = y — 2, Determine the proba- bility density function f.(e). 2.25 Random variables x and y are described by the joint density function _{K f0 75.0. What is the probability that this will occur? ¢ Make a neat and fully Inbeled sketch of the probability density function f,(go). d Melvin, true to form, got a grade of exactly 75.0 on the exam. Determine the conditional probability that. he spent less than one hour working in preparation for tl exam. SA 2.27 Each day Wyatt Uycp shoots one “game” by firing at a target with the following dimensions and scores for each shot: = a a a =a E a B 2 — L 5 a E = = . zg = = 2 a The score on any shot depends only on its distance from the center of the target His pellet supply isn’t too predictable, and the number of shots for any day’s game is equally likely to be one, two, or three. Furthermore, ‘Wyatt tires rapidly with each shot. Given that it is the kth pellet in a particular game, the value of r (distance from target center to point of impact) for a pellet is a random variable with probability density function 1 Sele | ho) = | ko 0 otherwise if 0 S10 < ko a Determine and plot the probability mass function for random vari- able ss, where s) is Mr. Uyrp’s score on a three-shot game.re pe ne tpn PROBLEMS 93 b Given only that a particular pellet was used during a three-shot game, determine and sketeh the probability density function for r, the distance from the target center to where it hit. ¢ Given that Wyatt scored a total of exactly six points on a game, determine the probability that this was a two-shot game. d We learn that, in a randomly selected game, there was at least one shot which scored exactly two points. Determine the conditional expected value of Wyatt’s total score for that game. e A particular pellet, marked at the facfory, was used eventually by Wyatt. Determine the PMF for the number of points he scored on the shot which consumed this pellet. 2.28 Random variables x and y are described by the joint PDF 1 f0 nem day = sty (The unit step function »1(to — a) is defined in See. 2-9] For a second example, we consider the uniform PDF 1 if0 0 and c >o,>0. The E(z) > 0 condition will simplify our statements and expressions. Our reasoning need not hold for ¢.? = », and any PDF which has o, = 0 represents an uninteresting deterministic quantity. Tf we are willing to accept the standard deviation of r, or, a5 & type of linear measure of the spread of a PDF about its mean, some interesting speculations follow. ‘As n increases, the PDF for r gets “wider” (as V/n) and its expected value inereases (as n). ‘The mean and the standard deviation both grow, but the mean increases more rapidly.INDEPENDENT IDENTICALLY DISTRIBUTED RANDOM VARIABLES 109. This would lead us to expect that, for instance, as n goes to infinity, the probability that an experimental value of r falls within a certain absolute distance d of E(r) decreases to zero. That is, Er) 0 (we think) Ir - EO < & EG) grows as n, ‘Thus the area over this interval should, as n—+ c, come to include all the area of the PDF f(r). For the given conditions, we shall learn in Chap. 6 that these speculations happen to be correct. Although proofs of such theorems could be stated here, we would not have as satisfactory a physical inter- pretation of such Limit theorems as is possible after we become familiar with the properties of several important PMF’s and PDF's, 47 Sum of a Random Number of Independent Identically Distributed Random Variables Let x be a random variable with PDF J,(z0) and s transform f,"(s). If r is defined to be the sum of » independent experimental values of random variable x, we know from the results of Sec. 3-5 that the trans- form for the PDF f,(re) is = f(s) = [f.7(s)}" where r is the sum of n (statistically) independent = experimental values of x = We now wish to consider the situation when n is also a random variable0 TRANSFORMS AND SOME APPLICATIONS {with PME pa(n0)]. We are interested in the sum of a random (but integer) number of independent identically distributed random variables. For instance, if f,(2s) were the PDF for the weight of any indi- vidual in an elevator, if the weights of people in the clevator could be considered to be independent random variables, and if pe(na) were the PMF for the number of people in the clevator, random variable r would represent. the total weight of the people in the elevator. Our work will also require that x and n be independent. In our example, the PDF for the individual weights may not depend on the number of people in the elevator. If m can fake on the experimental values 0, 1, 2, . . . , N, the sample space for each performance of our experiment is of N +1 dimensions, since cach performance generates one experimental value of n and up to N experimental values of random variable x. It is usually difficult to obtain the desited PDF f,(re) directly, but its s transform is derived quite easily. Although it may be difficult to get back to f-(7o) in a useful form from J,7(s), it is a simple matter to evaluate the moments and variance of random variable r. We may determine the s transform for f,(re) by working in an event space for random variable n (the number of independent experi- mental values of z in the sum) and r (the value of thesum). ‘This event space, perhaps a strange choice at first sight, consists of a set of parallel lines in one quadrant and one point at the origin of the rojo plane. Every event point representing 2 possible outcome is either on one of these heavy lines or at the origin Along each heavy line, there applies a conditional PDF f,19(ro | no) which is the PDF for r given the experimental value of n. | We know that Jrja(ro | no) is that PDF which describes the sum of ny independent experimental values of random variable x. As we noted at the start of this section, PDF fjn(7s | no) has the ¢ transform [f"(s)}*. We use these observations to collect f,(r») as a summation in the er RSS ARE FT RIES EEE FSET EES Sy SSN SE SRINDEPENDENT IDENTICALLY DISTRIBUTED RANDOM VARIABLES 111. ‘roto event space, and then we take the s transform on both sides of the equation. Slr) = Y ald fata | ne) ICs) = [7 Y pal fplra| no) dry in("0 | m0) dro = Yreralset(orre We recognize the last equation for f,"(s) to be the z transform of PMF Pa(no), with the transform evaluated at z = /f,"(s). We now restate this problem and its solution. Tet n and x be independent random variables, where n is described by the PMF py(ns) and x by the PDF f,(»). Define r to be the sum of n independent experimental values of random variable x, The s transform for the PDF f.(re) is f-"(s} = paTlf."(8)1 We may use the chain rule for differentiation to obtain the expectation, second moment, and variance of r. [UO] _ _ fawwtisz00) arr) | [ ds ee “a.F@)) ds th To evaluate the first term in the right-hand brackets, we proceed, ldpa1f-"(s)]} _ [dpat(z) = “aie |. al |= FO) ‘The first step in the equation immediately above made use of the fact that [f.7()]u0 = 1. To solve for E(r), we use [32 = -E() | in the expression for E(r) to obtain = EG) = BlyE@) a A second chain-rule differentiation of f,7(s) and the use of the relation for o,* in terms of f,"(s) leads to the further result112 TRANSFORMS AND SOME APPLICATIONS of = Bln)o.® + (E(x)}¥on* We may note that this result checks out correctly for the ease where m is deterministic (¢,?= 0) and for the ease where x is deterministic (et = 0, (ZG) = 24). If we had required that 2, as well as n, be a discreterandom variable which takes on only nonnegative integer experimental values, we could have worked with the PMI p,(z9) to study a particular case of the above derivations. The resulting 2 transform of the PME for diserete random variable + is pet(e) = petlp.t(2)1 and the above expressions for //(r) and o,? still hold. An example of the sum of 2 random number of independent. identically distributed random variables is included in the following section, 3-8 An Example, with Some Notes on Inverse Transforms As we undertake the study of some common probabilistic processes in the next chapter, our work will include numerous cxamples of applications of transform techniques. One problem is solved here to review some of the things we learned in this chapter, to indicate one new application, and [in part (c)] to lead us into a diseussion of how we may attempt to go back from an s transform to its PDE, ‘There are, of course, more general methods, which we shall not discuss. Let discrete random variable & be described by the PMF ge cri ko = 0,1,2,... Palko) = (a) Determine the expected value and variance of random vari (b) Determine the probability that an experimental valuc of k is even. (©) Determine the probability that the sum of n independent experi- mental values of k is even. (d) Let random variable & represent. the number of light bulbs we happen to have on hand at time 7. Furthermore, fet 2, the life- time of each bulb, be an independent random variable with PDF Salta) = wate — O)Ae* We turn ox one bulb at time To, replacing it immediately with another bulb as soon as it fails. This continues until the lust of the i bulbs blows out. Determine the s transform, expectation, and variance for random variable 7, the time from Ty until the last bulb dies.AN EXAMPLE, WITH SOME NOTES ON INVERSE TRANSFORMS 113 (e) Determine the PDF f,(r0) from the s transform f,7(s) obtained in part (d). a Rather than attempt to carry out some troublesome summations directly, it seems appropriate to employ the z transform. pit(e) = Be) = Y Ap(he) = Making a quick check, we note that p,7(1) is equal to unity. =[2p,7 E(k) = [é Py ol, 8 a a d z ote [Sere + Em - (Loew) | = b We shall do this part two ways. First, for our particular PMF we can evaluate the answer directly. Let A represent the event that the experimental value of £ is even. =! Be Piay= J nity = a(t +o aah ) P(A) The monotonically decreasing PMF for random variable k makes it obvious that P(A) > 0.5, since pe(0) > pa(1), px(2) > pu(8), ete. Another approach, which is applicable to a more general problem where we may not be able to sum’ pa(ks) directly, follows: sbben P(A) = ato : [x Dale) (1) 2 mutta) P(A) = 31 + pe(—1)] For our example we have px(z) = (9 ~ 82)-', px™(—1) = vy, resulting in P(A) = 9/17. © Let r be the sum of n independent experimental values of random variable k. In See. 3-5, we learned that. P(e) = [r7(@)" which we simply substitute into the expression obtained in (6) above, to get Prob(exper. value of r is even) = 41 + p,7(—1)] = Hl + Gi") As we might expect on intuitive grounds, this probability rapidly approaches 0.5 as n grows. 4 This part is concerned with the sum of a random number of independent 2114 TRANSFORMS AND SOME APPLICATIONS identically distributed random variables. Continuous random variable is the sum of k independent experimental values of random variable z. From See. 3-7, we have F(a) = PuPlf.7(s)] = (PT e)emratco For f.(z0), the exponential PDF, we have s20) = [ Terme dy = ats which results in 570) = (0 Y= GX roe et We may substitute E(k), o+%, (2), and o,* into the formulas of See. 3-7 to obtain £(¢) and o,%, or we may use the relations E@ = — lz GAL end ot = [& FAI -EGAL We'll use the former method, with Bk) =8 ost = 72 from part (a) of this example and (from the example in Sec. 3-3), which, in the expressions of See, 3-7 for the expectation and variance of a sum of a random number of inde- pendent identically distributed random variables, yields E() = E(R)E(z) = £ ot = E(k)os + [B(2)Pox? The expected time until the last bulb dies is the same as it would be if we always had eight bulbs. But, because of the probabilistic behavior of k, the variance of this time is far greater than the value 8/A* which would describe o,? if we always started out with exactly eight bulbs. Let Ai, As,. . . , Ax be a list of mutually exclusive collectively exhaus- tive events. Assume that there is a continuous random variable y which is not independent of the A,’s, Then it is useful to write Sule) = Y PADInave| Ad I renee Sn RR RTAN EXAMPLE, WITH SOME NOTES ON INVERSE TRANSFORMS 115, And, from the definition of the s transform, we note that j,%s) would be the weighted sum of the transforms of the conditional PDF's Suave] Ad. If we define Sale) = f> have! Ad dys = Elem | Ad we have LE) = ¥ PANS a) When we wish to take inverse transforms (go back to a PDF from a transform), we shall try to express the # transform to be inverted, Jv"(s), in the above form such that we can recognize the inverse trans- form of each f7\4,(8). In our particular problem, where the PMF for k is of the form Palko) = (1—P)PS ig = 0,1 2,...; 1>P>O and the PDF for x is the exponential PDF Fo) = Nal — Oem it happens that we may obtain f,(rs) from f,"(s) by the procedure dis- cussed above. We begin by using long division to obtain _ str _1,8 19 WO = SEK Ot OTE which is of the form (8) = Bhials) + 8Ffials) From the examples carried out in See. 3-1, we note that Foals) = 1 has the inverse transform fuia(ro| A) = wo(ro — 0), and also Fase) =O e+ has the inverse transform fyas(ry| As) = u-s(ro ~ 0) hem? and, finally, we have the PDF for the duration of the interval during which the lights are on. Sire) = dlrs ~ 0) + Ba-s(ro ~ 0) ‘The impulse at r> = 0 in this PDF is due to the fact that, with proba- bility 4, we start out with zero bulbs. Thus our PDF f,(r0) is a mized PDF, having both a discrete and a continuous component. Wecon- clude with a sketch of the PDF f,(re)116 to) TRANSFORMS AND SOME APPLICATIONS ‘The reader should recall that the scale height of the impulse is arbitrary 3:9 Where Do We Go from Here? We are already familiar with most of the basic concepts, methods, and tools of applied probability theory, By always reasoning in an appropriate sample or event space, we have had little difficulty in going from concepts to applications. Three main areas of study are to follow: Probabilistic processes (Chaps. 4 and 5) Limit theorems (Chap. 6) Statistical reasoning (Chap. 7) Although we shall consider these topics in the above order, this does not necessarily reflect their relative importance in the world of applied probability theory. Further study of the consequences of the summation of a large number of random variables (limit. theorems) is indeed basic. Many probabilicists work solely at attempting to make reasonable inferences from actual physical data (statistical reasoning). Our choice of the order of these topies is based on the contention that, if we first develop an understanding of several processes and their properties, we may then begin a more meaningful discussion of limit theorems and statistics. The following two chapters are con- cerned with those probabilistic processes which form the most basic building blocks from which models of actual physical processes are constructed.PROBLEMS 117 PROBLEMS = 301 A sufficient condition for the existence of an integral is that the integral of the magnitude of the integrand be finite. Show that, at | __ least for purely imaginary values of s, 8 = jw, this condition is always satisfied by the s transform of a PDF. 3.02 If we allow s to be the complex quantity, © = a + jw, determine for which values of s in the ao plane the s transforms of the following PDF's exist: ew Me 1 > 0 8 false) = ( neo b files) i to >0 rome my SO © falas) = [OS me D0 stem [05d my <0 3.03 Express the PMF p.(zo) = (1 — P)P®, 2» = 0,1,2,. . . ,asaPDF. 3.04 If z can be the complex number z = a + 78, determine for which values of z in the a,8 plane the z transform of the PMF of Prob. 3.03 will exist. 3.05 All parts of this problem require numerical answers. a If f,7(s) = K/(2 + 8), evaluate K and E(y'). b If p."(2) = (1 + 2*)/2, evaluate p,[E(c)] and o.. © If f.(8) = 22 — e-W"? — €~*)/3s, evaluate E(e*). d If p."(2) = A(1 + 32), evaluate B(2*) and p,(2). 3.06 Determine whether or not the following are valid z transforms of a PMF for a discrete random variable which can take on only nonnega- tive integer experimental values: act+2—-2 b2—-2 © @-2) 3.07 Show that neither of the following is an s transform of a PDF: a(lL—e)/s — b 7(4 +39)! 3.08 Let J be 4 discrete random variable whose possible experimental values are all nonnegative integers. We are given 14 + 5: 2 pre) = K [ MaRS ¥] Determine the numerical values of E(l), p:(1) and of the conditional expected value of J given I ¥ 0. 3.09. Use the expected-value definition of the s transform to prove that,118 TRANSFORMS AND SOME APPLICATIONS if x and y are random variables with y = ax + b, f,7(s) = e-*f."(as). (This is a useful expression, and we shall use it in our proof of the central limit theorem in Chap. 6.) 3.10 For a particular batch of cookies, k, the number of nuts in any cookie is an independent random variable described by the probability mass function puke) = 48)" ko = 0,1, 2,3,. - - E Human tastes being what they are, assume that the cash value i of a cookie is proportional to the third power of the number of nuts in the cookie. The cookie packers (they are chimpanzees) eat all the cookies containing exactly 0, 1, or 2 nuts. All series must be summed. a What is the probability that a randomly selected cookie is eaten by the chimpanzees? b What is the probability that a particular nut, chosen at random from the population of all nuts, is eaten by the chimpanzees? © What is the fraction of the cash value which the chimpanzees consume? 4 What is the probability that a random nut will go into a cookie containing exactly & nuts? it probability 0.3) or in city B (with a priori probability 0.6), or he has left the country. If he is in city A and N« men are assigned to look for him there, he will be eaught with probability 1 — f%. If he is in city B and Ng men are assigned to look for him there, he will be caught with probability 1 —f", If he has left the country, he won’t be captured. Policemen’s lives being as hectic as they are, Na and Np are independent random variables described by the probability mass functions : a a i ‘The hitherto uncaught burglar is hiding in city A (with a priori Kent pul) = Ne Oa | PwaN) = (4)" otek a What is the probability that a total of three men will be assigned to search for the burglar? b What is the probability that the burglar will be caught? (All series are to be summed.) ¢ Given that he was captured in a city in which exactly K men had been assigned to look for him, what is the probability that he was found in city A?PROBLEMS 119 3.12 ‘The number of “leads” (contacts) available to a salesman on any given day is a Poisson random variable with probability mass function rile) = BF he 0,152, ‘The probability that any particular lead will result in a sale is 0.5. If your answers contain any series, the series must be summed. What is the probability that the salesman will make exactly one sale on any given day? b If we randomly select a sales receipt from his file, what is the proba- bility that it represents a sale made on a day when he had a total of B leads? ¢ What fraction of all his ieads comes on days when he has exactly one sale? 4 What is the probability that he has no sales on a given day? 3.13. The probability that a store will have exactly ke customers on any given day is Palko) = #4)" ko = 0,1,2,... On each day when the store has had at least one customer, one of the sales slips for that day is picked out of a hat, and a door prize is mailed to the corresponding customer. (No customer goes to this store sore than once or buys more or less than exactly one item.) a What is the probability that a customer selected randomly from the population of all customers will win a door prize? b Given a customer who has won a door prite, what is the probability that he was in the store on a day when it had a total of exactly ko customers? A 14 Independent random variables z and y have PDF's whose s trans- forms are j.*(s) and f,7(s). Random variable r is defined to be r=az+y. Usef,"(s) and the moment generating properties of trans- forms to show that E(r) = E(x) + B(y) and o,? = 04% + 4,2. 15 Let x and y be independent random variables with rem ay D0 felze) = | {o yo >0 mcd 2WO=j}rm 4 <0 Random variable r is defined by r = x+y. Determine: a f°(8), f7(), and f,7(8).EB az TRANSFORMS AND SOME APPLICATIONS b E(0) and 63, € filro). d Repeat the previous parts for the case r = ax + by. 3.16 Consider the PDF f.(2o) = w-1(to — 0) — n-1(zo — 1). Random variable y is defined to be the sum of two independent experimental values of x. Determine the PDF fy(yo): a In an appropriate two-dimensional event space b By performing the convolution graphically ¢ By taking the inverse transform of f,7(s) (if you can) 3.17 a If x and y are any independent discrete random variables with PMF’s pz() and p,(yo) and_we define r= x+y, show that Poli) = z Pe(o)Pylro — 0) = z Palyo)p=(ro ~ yo). ‘These summa- tions are said to represent the discrete convolution, Show how you would go about performing a discrete convolution graphically. b For the case where x and y are discrete, independent random varia- bles which can take on only nonnegative-integer experimental values, take the z transform of one of the above expressions for p,(ro) to show that p,"(z) = p."(z)p,"(2). 3.18 Random variable x has the PDF f.(z0), and we define the Mellin transform f."(s) to be $.4(8) = Ee) a Determine E(x) and ¢,? in terms of f.(s). b Let y be a random variable with Sule) = Kuycfe(yo) i Determine K. it Determine f,*(s) in terms of f-™(s). Wit Evaluate f,"(s) and f,"(e) for {3 #O

You might also like

• The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good Life

  

  From Everand

  The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good Life

  Mark Manson

  4/5 (6386)
• Principles: Life and Work

  

  From Everand

  Principles: Life and Work

  Ray Dalio

  4/5 (634)
• The Gifts of Imperfection: Let Go of Who You Think You're Supposed to Be and Embrace Who You Are

  

  From Everand

  The Gifts of Imperfection: Let Go of Who You Think You're Supposed to Be and Embrace Who You Are

  Brené Brown

  4/5 (1160)
• Never Split the Difference: Negotiating As If Your Life Depended On It

  

  From Everand

  Never Split the Difference: Negotiating As If Your Life Depended On It

  Chris Voss

  4.5/5 (983)
• The Glass Castle: A Memoir

  

  From Everand

  The Glass Castle: A Memoir

  Jeannette Walls

  4/5 (8302)
• Grit: The Power of Passion and Perseverance

  

  From Everand

  Grit: The Power of Passion and Perseverance

  Angela Duckworth

  4/5 (633)
• Sing, Unburied, Sing: A Novel

  

  From Everand

  Sing, Unburied, Sing: A Novel

  Jesmyn Ward

  4/5 (1254)
• Shoe Dog: A Memoir by the Creator of Nike

  

  From Everand

  Shoe Dog: A Memoir by the Creator of Nike

  Phil Knight

  4.5/5 (933)
• The Perks of Being a Wallflower

  

  From Everand

  The Perks of Being a Wallflower

  Stephen Chbosky

  4/5 (10337)
• Her Body and Other Parties: Stories

  

  From Everand

  Her Body and Other Parties: Stories

  Carmen Maria Machado

  4/5 (887)
• The Hard Thing About Hard Things: Building a Business When There Are No Easy Answers

  

  From Everand

  The Hard Thing About Hard Things: Building a Business When There Are No Easy Answers

  Ben Horowitz

  4.5/5 (361)
• Hidden Figures: The American Dream and the Untold Story of the Black Women Mathematicians Who Helped Win the Space Race

  

  From Everand

  Hidden Figures: The American Dream and the Untold Story of the Black Women Mathematicians Who Helped Win the Space Race

  Margot Lee Shetterly

  4/5 (1007)
• Steve Jobs

  

  From Everand

  Steve Jobs

  Walter Isaacson

  4/5 (3237)
• Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic Future

  

  From Everand

  Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic Future

  Ashlee Vance

  4.5/5 (581)
• The Emperor of All Maladies: A Biography of Cancer

  

  From Everand

  The Emperor of All Maladies: A Biography of Cancer

  Siddhartha Mukherjee

  4.5/5 (297)
• A Man Called Ove: A Novel

  

  From Everand

  A Man Called Ove: A Novel

  Fredrik Backman

  4.5/5 (5058)
• Angela's Ashes: A Memoir

  

  From Everand

  Angela's Ashes: A Memoir

  Frank McCourt

  4.5/5 (943)
• The Art of Racing in the Rain: A Novel

  

  From Everand

  The Art of Racing in the Rain: A Novel

  Garth Stein

  4/5 (4346)
• The Yellow House: A Memoir (2019 National Book Award Winner)

  

  From Everand

  The Yellow House: A Memoir (2019 National Book Award Winner)

  Sarah M. Broom

  4/5 (100)
• The Little Book of Hygge: Danish Secrets to Happy Living

  

  From Everand

  The Little Book of Hygge: Danish Secrets to Happy Living

  Meik Wiking

  3.5/5 (458)
• Brooklyn: A Novel

  

  From Everand

  Brooklyn: A Novel

  Colm Tóibín

  3.5/5 (2091)
• Yes Please

  

  From Everand

  Yes Please

  Amy Poehler

  4/5 (1993)
• Devil in the Grove: Thurgood Marshall, the Groveland Boys, and the Dawn of a New America

  

  From Everand

  Devil in the Grove: Thurgood Marshall, the Groveland Boys, and the Dawn of a New America

  Gilbert King

  4.5/5 (278)
• The World Is Flat 3.0: A Brief History of the Twenty-first Century

  

  From Everand

  The World Is Flat 3.0: A Brief History of the Twenty-first Century

  Thomas L. Friedman

  3.5/5 (2283)
• Bad Feminist: Essays

  

  From Everand

  Bad Feminist: Essays

  Roxane Gay

  4/5 (1077)
• The Woman in Cabin 10

  

  From Everand

  The Woman in Cabin 10

  Ruth Ware

  3.5/5 (2780)
• A Tree Grows in Brooklyn

  

  From Everand

  A Tree Grows in Brooklyn

  Betty Smith

  4.5/5 (2032)
• The Outsider: A Novel

  

  From Everand

  The Outsider: A Novel

  Stephen King

  4/5 (2838)
• The Sympathizer: A Novel (Pulitzer Prize for Fiction)

  

  From Everand

  The Sympathizer: A Novel (Pulitzer Prize for Fiction)

  Viet Thanh Nguyen

  4.5/5 (141)
• A Heartbreaking Work Of Staggering Genius: A Memoir Based on a True Story

  

  From Everand

  A Heartbreaking Work Of Staggering Genius: A Memoir Based on a True Story

  Dave Eggers

  3.5/5 (692)
• Team of Rivals: The Political Genius of Abraham Lincoln

  

  From Everand

  Team of Rivals: The Political Genius of Abraham Lincoln

  Doris Kearns Goodwin

  4.5/5 (1912)
• Wolf Hall: A Novel

  

  From Everand

  Wolf Hall: A Novel

  Hilary Mantel

  4/5 (4086)
• On Fire: The (Burning) Case for a Green New Deal

  

  From Everand

  On Fire: The (Burning) Case for a Green New Deal

  Naomi Klein

  4/5 (76)
• Fear: Trump in the White House

  

  From Everand

  Fear: Trump in the White House

  Bob Woodward

  3.5/5 (830)
• Manhattan Beach: A Novel

  

  From Everand

  Manhattan Beach: A Novel

  Jennifer Egan

  3.5/5 (906)
• Rise of ISIS: A Threat We Can't Ignore

  

  From Everand

  Rise of ISIS: A Threat We Can't Ignore

  Jay Sekulow

  3.5/5 (143)
• John Adams

  

  From Everand

  John Adams

  David McCullough

  4.5/5 (2544)
• The Light Between Oceans: A Novel

  

  From Everand

  The Light Between Oceans: A Novel

  M L Stedman

  4.5/5 (813)
• The Unwinding: An Inner History of the New America

  

  From Everand

  The Unwinding: An Inner History of the New America

  George Packer

  4/5 (45)
• Little Women

  

  From Everand

  Little Women

  Louisa May Alcott

  4.5/5 (2369)
• The Constant Gardener: A Novel

  

  From Everand

  The Constant Gardener: A Novel

  John le Carré

  4/5 (277)