Ch t 2

Chapter
Probability
 2-2

2 Probability
 Using Statistics
 Basic Definitions: Events, Sample Space, and Probabilities
 Basic Rules for Probability
 Conditional Probability
 Independence of Events
 Combinatorial Concepts
 The Law of Total Probabilityy and Bayes’
y Theorem
 The Joint Probability Table
 Using the Computer
 2-3

2 LEARNING OBJECTIVES

After studying this chapter

chapter, you should be able to:
 Define probability, sample space, and event.
 Distinguish between subjective and objective probability.
 Describe the complement of an event, the intersection, and the union of two
events.
 Compute probabilities of various types of events.
 E l i the
Explain h concept off conditional
di i l probability
b bili andd how
h to compute it. i
 Describe permutation and combination and their use in certain probability
computations.
 E plain Bayes’
Explain Ba es’ theorem and its applications.
applications
 2-4

2-1 Probability
y is:

 A quantitative
tit ti measure off uncertainty
t i t
 A measure of the strength of belief in the occurrence of an
uncertain event
 A measure of the degree of chance or likelihood of
occurrence of an uncertain event
 Measured by a number between 0 and 1 (or between 0% and
100%)
 2-5

Types
yp of Probability
y

 Objective or Classical Probability

 based on equally-likely events
 based on long-run relative frequency of events
 not based on personal beliefs
 is the same for all observers (objective)
 examples:
p toss a coin,, roll a die,, pick
p a card
 2-6

Types
yp of Probability
y ((Continued))

 Subjective Probability
 based on personal beliefs, experiences, prejudices, intuition - personal
judgment
 different for all obser
observers
ers (s
(subjective)
bjecti e)
 examples: Super Bowl, elections, new product introduction, snowfall
 2-7

2-2 Basic Definitions

 Set - a collection of elements or objects of interest

 Empty set (denoted by )
 a set containing no elements

 Universal set (denoted by S)

 a set containing all possible elements

 Complement
p (Not).
( ) The complement
p of A is  A
 a set containing all elements of S not in A
 2-8

Complement
p of a Set

V
Venn Di
Diagram ill
illustrating
t ti theth Complement
C l t off an eventt
 2-9

Basic Definitions ((Continued))

 Intersection (And)  A  B 
– a set
t containing
t i i all ll elements
l t in
i both
b th A andd B
 Union (Or)
 A  B
– a set containing
i i all ll elements
l in
i A or B or both
b h
 2-10

Sets: A Intersecting
g with B

A
B

A B
 2-11

Sets: A Union B

A
B

A B
 2-12

Basic Definitions ((Continued))

• Mutually exclusive or disjoint sets

–sets having no elements in common, having no
intersection, whose intersection is the empty set
• Partition
–a collection of mutually exclusive sets which
together include all possible elements, whose
union
i isi the
h universal
i l set
 2-13

Mutually
y Exclusive or Disjoint
j Sets

Sets have nothing in common

B
A
 2-14

Sets: Partition

S
A3
A1

A2 A4

A5
 2-15

Experiment
p
• Process that leads to one of several possible outcomes *, e.g.:
 Coin toss
• Heads, Tails
 Rolling a die
• 1 2,
1, 2 3,
3 4,
4 5,
5 6
 Pick a card
 AH, KH, QH, ...
 Introduce a new product
• Each trial of an experiment has a single observed outcome.
• The precise outcome of a random experiment is unknown before a trial.

* Also called a basic outcome, elementary event, or simple event

 2-16

Events : Definition
 Sample Space or Event Set
 Set of all possible outcomes (universal set) for a given experiment
 E.g.: Roll a regular six-sided die
 S = {1,2,3,4,5,6}
 Event
E ent
 Collection of outcomes having a common characteristic
 E.g.: Even number
 A = {2,4,6}
{2 4 6}
 Event A occurs if an outcome in the set A occurs
 Probability of an event
 Sum of the probabilities of the outcomes of which it consists
 P(A) = P(2) + P(4) + P(6)
 2-17

Equally-likely Probabilities
(Hypothetical or Ideal Experiments)
• For example:
p
 Roll a die
• Six possible outcomes {1,2,3,4,5,6}
• If each is equally-likely, the probability of each is 1/6 = 0.1667 =
16.67%
1
 P ( e) 
n( S )
• Probability of each equally-likely
equally likely outcome is 1 divided by the number of
possible outcomes
 Event A (even number)
• P(A) = P(2) + P(4) + P(6) = 1/6 + 1/6 + 1/6 = 1/2
• for e in A
P ( A )   P ( e)
n( A ) 3 1
  
n( S ) 6 2
 2-18

2-3 Basic Rules for Probability

y

 Range of Values for P(A): 0  P ( A)  1

 Complements - Probability of not A

P( A )  1  P( A)
 Intersection - Probability of both A and B

P ( A  B )  n( A  B )
n( S )
 Mutually exclusive events (A and C) :

P( A  C)  0
 2-19

Basic Rules for Probability

(Continued)
• U i - Probability
Union P b bilit off A or B or both
b th (rule
( l off unions)
i )

P ( A  B )  n( A  B )  P ( A)  P ( B )  P ( A  B )
n(S )
 Mutually exclusive events: If A and B are mutually exclusive, then

P ( A  B )  0 so P ( A  B )  P ( A )  P ( B )
 2-20

Sets: P(A
( Union B))

A
B

P( A  B)
 2-21

2-4 Conditional Probability

y

• Conditional Probability - Probability of A given B

P(A  B)
P(A B)  , where P ( B )  0
P(B)

 Independent events:

P( AB)  P( A)
P( B A)  P( B)
 2-22

Conditional Probability
y ((continued))

Rules of conditional probability:

P( A B)  P( A  B) so P( A  B)  P( A B) P( B)
P( B)
 P( B A) P( A)

If events A and D are statistically independent:

P ( A D )  P ( A)
so P( A  D)  P( A)P(D)
P ( D A)  P ( D )
 2-23

Contingency
g y Table

Counts

Probability that a cholera

is unattacked given it is
inoculated:
P ( un In ) 
Probabilities
Un-
attacked attacked Total
P ( un  In )
inoculated 0.055 0.445 0.50 P ( In )
uninoculated 0.105 0.395 0.50
0 . 445
  0 . 89
0 . 50
Total 0.160 0.840 1.00
 2-24

2-5 Independence
p of Events

Conditions for the statistical independence of events A and B:

P ( A B )  P ( A)
P ( B A)  P ( B )
and
P ( A  B )  P ( A) P ( B )
P ( Ace  H eart ) P ( H eart  Ace )
P ( Ace H eart )  P ( H eart Ace ) 
P ( H eart ) P ( Ace )
1 1
1 1
 52   P ( Ace )  52   P ( H eart )
13 13 4 4
52 52

4 13 1
P ( Ace  Heart )  *   P ( Ace ) P ( Heart )
52 52 52
 2-25

Independence of Events

Events tubercolosis(T) and breaking my

leg(B) are assumed to be independent.

a)P(T  B)  P(T ) P( B)
 0.04 * 0.06  0.0024
b)P(T  B)  P(T )  P( B)  P(T  B)
 0.04  0.06  0.0024  0.0976
 2-26

Product Rules for Independent

p Events

The probability of the intersection of several independent events

is the product of their separate individual probabilities:
P( A  A  A  An )  P( A ) P( A ) P( A ) P( An )
1 2 3 1 2 3
The probability of the union of several independent events
is 1 minus the product of probabilities of their complements:
P( A  A  A  An )  1  P( A ) P( A ) P( A ) P( An )
1 2 3 1 2 3
 2-27

2-6 Combinatorial Concepts

p

Consider picking 2 amino acids among {A, R, N, D, C, E}. There are

six possible outcomes from picking the first AA {A, R, N, D, C, E}
and six possible outcomes from for the second AA. Altogether, there
are 66*66 = 36 possible outcomes picking the two AAs
In general, if there are n events and the event i can happen in
Ni possible ways, then the number of ways in which the
sequence of n events may occur is N1N2...Nn.

 Pick 3 AAs from a list of {A, R, N,  Pick 3 AAs from a list of {A, R, N,
D, C, E} - with replacement D, C, E} - without replacement
 6*6*6=216 different possible  6 5 4 = 120 different possible
6*5*4
outcomes outcomes
 2-28

More on Combinatorial Concepts

(Tree Diagram)
Order the letters: A,, B,, and C
C
ABC
B
C B
ACB
A
A C
B BAC
C A
C BCA
A B
B CAB
A
CBA
 2-29

Factorial

How many ways are there to order the AAs:{ A, R, C, D, C, E} ?

Ans: (6*5*4*3*2*1 = 720)

Factorial: For any positive integer n, we define n factorial as:

n(n-1)(n-2)...(1). We denote n factorial as n!.

The number n! is the number of ways in which n objects can

be ordered. By definition 1! = 1 and 0! = 1.
 2-30

Permutations ((Order is important)

p )

What if we chose only 3 out of the 6 Aas: AA, R

R, N
N, DD, C
C, and E?
There are 6 ways to choose the first AA, 5 ways to choose the
second AA, and 4 ways to choose the third AA (leaving 3
AAss uunchosen).
c ose ). That makes
es 6*5*4=120 possible
poss b e oorderings
de gs oor
permutations.

Permutations are the possible ordered selections of r objects out

of a total of n objects. The number of permutations of n objects
taken r at a time is denoted by nPr, where
P  n!
n r ( n  r )!
For example :
6! 6! 6 * 5 * 4 * 3 * 2 * 1
6 P3     6 * 5 * 4  120
( 6  3 )! 3! 3 * 2 *1
 2-31

Combinations ((Order is not Important)

p )

Suppose that when we pick 3 patients out of the 6 available patients named A,
A B,
B C,
C
D, E, and F for a clinical trial? How many combinations of 6 different people,
taking 3 at a time, are there?

Combinations are the possible selections of r items from a group of n items  n
regardless of the order of selection. The number of combinations is denoted  r
and is read as n choose r. An alternative notation is nCr. We define the number
of combinations of r out of n elements as:
n n!
   n C r 
r r! (n  r)!
For example :
n 6! 6! 6 * 5 * 4 * 3 * 2 *1 6 * 5 * 4 120
   6 C 3       20
 
r 3! ( 6  3 )! 3!3 ! (3 * 2 * 1)(3 * 2 * 1) 3 * 2 * 1 6
 2-32

2-7 The Law of Total Probability and

Bayes’ Theorem
Bayes
The law of total probability:
P( A)  P( A  B)  P( A  B )

In terms of conditional probabilities:

P( A)  P( A  B)  P( A  B )
 P( A B) P( B)  P( A B ) P( B )

More ggenerallyy (where

( Bi make upp a partition):
p )
P( A)   P( A  B )
i
  P( AB ) P( B )
i i
 2-33

The Law of Total Probability-

Event U: E.coli population will grow in the next morning

Event W: The LB medium will not go bad by the next morning
P (U W )  . 75
P (U W )  30
P (W )  . 80  P (W )  1  . 8  . 2
P (U )  P (U  W )  P (U  W )
 P (U W ) P (W )  P (U W ) P (W )
 (. 75 )(. 80 )  (. 30 )(. 20 )
 . 60  . 06  . 66
 2-34

Bayes’
y Theorem
• Bayes’ theorem enables you, knowing just a little more than the
probability
b bili off A given
i B,
B to find
fi d the
h probability
b bili off B given
i A.
A
• Based on the definition of conditional probability and the law of total
probability.

P( A  B)
P ( B A) 
P ( A)
P( A  B) Applying the law of total
 probability to the denominator
P( A  B)  P( A  B )
P( A B) P(B) Applying the definition of

P( A B) P(B)  P( A B ) P(B ) conditional probability throughout
 2-35

Bayes’
y Theorem - Example
p

• A medical test for a rare disease (affecting 0.1% of the population

[ P( I )  0.001 ]) is imperfect:
When administered to an ill person, the test will indicate so with probability
0.92 [ P(Z I ) .92 P(Z I ) .08 ]
 The event (Z I ) is a false negative
When administered to a person who is not ill, the test will erroneously give a
positive result (false positive) with probability 0.04 [P(Z I )  0.04 P(Z I )  0.96]
 The event (Z I ) is a false positive. .
 2-36

Example
p ((continued))

P ( I )  0.001 P(I Z ) 
P(I  Z )
P(Z )
P(I  Z )

P(I  Z )  P(I  Z )
P ( I )  0.999
P(Z I ) P(I )

P(Z I ) P(I )  P(Z I ) P(I )
P ( Z I )  0.92 
( 9 2 )( 0.0 0 1)
(.
(.9 2 )( 0.0 0 1)  ( 0.0 4 )(.9 9 9 )
0.0 0 0 9 2 0.0 0 0 9 2
 
0.0 0 0 9 2  0.0 3 9 9 6
P ( Z I )  0.04  .0 2 2 5
.0 4 0 8 8
 2-37

Example
p ((Tree Diagram)
g )

Prior Conditional Joint

Probabilities Probabilities Probabilities

P( Z I )  092
. P( Z  I )  (0.001)(0.92) .00092

P( Z I )  008
. P( Z  I )  (0.001)(0.08) .00008
P( I )  0001
.

P( I )  0999
. P( Z I )  004
. P( Z  I )  (0.999)(0.04) .03996

P( Z I )  096
.

P( Z  I )  (0.999)(0.96) .95904
 2-38

Bayes’
y Theorem Extended

• Given a partition of events B1,B2 ,...,Bn:

P( A B )
P(B A)  1

P( A)
1

Applying the law of total

P( A B ) probability to the denominator
 1

 P( A B ) i
Applying the definition of
P( AB ) P(B ) conditional probability throughout
 1 1

 P( AB ) P(B )
i i
 2-39

Bayes’ Theorem Extended -

Example
 A scientist believes that during periods of high temperature,
temperature the Ecoli will
grow with probability 0.70; in periods of moderate temperature, the growth
occurs with probability 0.40; and during periods of low temperature, the
growth happens with probability 0.20.
 During any period of time, the probability of high temperature is 0.30, the
probability of moderate temperature is 0.50, and the probability of low
temperature
p ggrowth is 0.50.
 Suppose the bacterial growth occurs during the present period. What is the
probability we are experiencing a period of high temperature ?
Partition
Partition: Event A  Growth
H - High temperature P(H) = 0.30 P ( A H )  0.70
P ( A M )  0.40
M - Moderate temperature P(M) = 0.50
P ( A L)  0.20
L - Low temperature P(L) = 0.20
 2-40

Example
p ((continued))

P( H  A)
P( H A) 
P( A)
P( H  A)

P( H  A)  P( M  A)  P( L  A)
P( A H) P( H)

P( A H) P( H)  P( A M ) P( M )  P( A L) P( L)
( 0.70)( 0.30)

( 0.70)( 0.30)  ( 0.40)( 050
. )  ( 0.20)( 0.20)
0.21 0.21
 
0.21 0.20  0.04 0.45
 0.467
 2-41

Example
p ((Tree Diagram)
g )

Prior Conditional Joint

Probabilities Probabilities Probabilities
P ( A H )  0. 70 P ( A  H )  ( 0.30)( 0. 70)  0. 21

P ( A H )  0.30
P ( H )  0 .3 0 P ( A  H )  ( 0.30)( 0.30)  0. 09

P ( A M )  0. 40 P ( A  M )  ( 0.50 )( 0. 40)  0. 20

P ( M )  0.50

P ( A M )  0.60 P ( A  M )  ( 0.50 )( 0.60)  0.30

P ( A L )  0. 20
P ( L )  0.20 P ( A  L )  ( 0. 20)( 0. 20)  0. 04

P ( A L )  0.80 P ( A  L )  ( 0. 20 )( 0.80)  0.16

 2-42

2-8 The Joint Probability

y Table

 A joint
j i probability
b bili table
bl iis similar
i il to a contingency
i table
bl , except that
h iti
has probabilities in place of frequencies.
 The joint probability for Example 2-11 is shown below.
 The row totals and column totals are called marginal probabilities.
 2-43

The Joint Probability

y Table
 A joint probability table is similar to a contingency table , except that it
has probabilities in place of frequencies.
 The joint probability for the previous example is shown on the next
slide.
 The row totals and column totals are called marginal probabilities
probabilities.
 2-44

The Joint Probability Table:

 The
h joint
j i probability
b bili

High Medium Low Total

There is growth
0 21
0.21 02
0.2 0 04
0.04 0 45
0.45

No growth
0.09 0.3 0.16 0.55

Total 0.30 0.5 0.20 1.00

Marginal probabilities are the row totals and the column totals.
 3-45

Discrete and Continuous Random

Variables

A discrete random variable:

 has a countable number of possible values

 has discrete jumps (or gaps) between successive values

 has measurable probability associated with individual values

 counts

A continuous random variable:

 has an uncountably infinite number of possible values

 moves continuously from value to value

 has no measurable probability associated with each value

 measures ((e.g.:
g height,
g , weight,
g , speed,
p , value,, duration,, length)
g )
 3-46

Rules of Discrete Probability

Distributions

The probability distribution of a discrete random

variable X must satisfy the following two conditions.

1. P(x)  0 for all values of x.

2.  P(x)  1
all x

 Corollary: 0  P( X )  1
 3-47

Cumulative Distribution Function

The cumulative distribution function, F(x), of a discrete

random variable X is:
F(x)  P( X  x)   P(i)
all i  x

Cumulative Probability Distribution of the Number of mutations

x P(x) F(x)
1 .0
0 0.1 0.1 0 .9
0 .8
1 0.2 0.3 0 .7

2 03
0.3 06
0.6 0 .6
6
F(x)

0 .5
3 0.2 0.8 0 .4
0 .3
4 0.1 0.9 0 .2
0 .1
5 0.1 1.0 0 .0

1.00 0 1 2
x
3 4 5
 3-48

Cumulative Distribution Function

The probability that at most three mutations will occur:

Probability that at most 3 mutations will occur

x P(x) F(x)
0 0.1 0.1
1 0.2 0.3
2 0.3 0.6
3 0.2 0.8
4 0.1 0.9
5 01
0.1 10
1.0
1

Note: P(X < 3) = F(3) = 0.8 = P(0) + P(1) + P(2) + P(3)

 3-49

Using Cumulative Probability

Distributions

The probability that more than one mutation will occur:

Probability that more than 1 mutation will occur

x P(x) F(x)
0 0.1 0.1
1 0.2 0.3
2 0.3 0.6
3 0.2 0.8
4 0.1 0.9
5 0.1 1.0
1

N t P(X > 1) = P(X > 2) = 1 – P(X < 1) = 1 – F(1) = 1 – 0.3

Note: 0 3 = 00.7
7
 3-50

Using Cumulative Probability

Distributions

The probability that anywhere from one to three

mutations will occur:
Probability of having between 13 mutations to occur
x P(x) F(x)
0 0.1 0.1
1 0.2 0.3
2 03
0.3 06
0.6
3 0.2 0.8
4 0.1 0.9
5 0.1 1.0
1

Note: P(1
( < X < 3)) = P(X
( < 3)) – P(X
( < 0)) = F(3)
( ) – F(0)
( ) = 0.8 – 0.1 = 0.7
Binomial Distribution
n! X n  X
Probability P( X )  p q
X !n  X !

f ti
function
for 0  X  n, q  1  p
 Mean value   n p
 Variance and
Standard  2  n pq
Deviation
  2  n pq
Binomial Distribution:
Demonstration Problem
Accordingg to previous
p studies, approximately
pp y 6% of all AAs in
the Gprotein, are mutated. In analyzing a random Gprotein
molecule, what is the probability of getting two or fewer
mutation points in a sample of 20?
Binomial Distribution:
Demonstration Problem
 In this example,
 6% are mutated => p
 The sample size is 20 => n
 94% are left unchanged
g => q
 X is the number of successes desired
 What is the probability of getting 2 or fewer mutation points in the
sample of 20? => P(X≤2)
 The hard part of this problem is identifying p, n, and x – emphasize
this when studying the problems.
Binomial Distribution:
Demonstration Problem
n  20
p  .06
q  .94
P ( X  2 )  P ( X  0 )  P ( X  1)  P ( X  2 )
 .2901  .3703  .2246  .8850
20! 0 20  0
P ( X  0)   .06   .94   (1)(1)(.2901)  .2901
0!( 20  0)!

20! 1 20  1
P ( X  1)    
. 06 . 94  ( 20 )( )(.3086 )  .3703
)(. 06 )(
1!( 20  1)!

20! 2 20  2
P ( X  2)   .06   .94   (190 )(.0036 )(.3283 )  .2246
2!( 20  2)!
Binomial Distribution:
Demonstration Problem

 What
h are the
h mean andd standard
d d deviation
d i i off this
hi distribution?
di ib i

  n  p  ( 20 )(.
)( 06 )  1 .20
 2  n  p  q  ( 20 )( )(. 94 )  1 . 128
)(. 06 )(

   2  1 . 128  1 . 062
Binomial Distribution Table:
Demonstration Problem
n = 20 PROBABILITY
n  20
X 0.05 0.06 0.07 p  . 06
0 0.3585 0.2901 0.2342 q  . 94
1 0 3774
0.3774 0 3703
0.3703 0 3526
0.3526 P ( X  2 )  P ( X  0 )  P ( X  1)  P ( X  2 )
2 0.1887 0.2246 0.2521  . 2901  . 3703  . 2246  . 8850
3 0.0596 0.0860 0.1139

4 0.0133 0.0233 0.0364

5 0.0022 0.0048 0.0088

6 0.0003 0.0008 0.0017

7 0.0000 0.0001 0.0002

8 0.0000 0.0000 0.0000

… … … …

20 0.0000 0.0000 0.0000

Poisson Distribution

 The Poisson distribution focuses only on the number of discrete

occurrences over some interval
i t l or continuum
ti
 Poisson does not have a given number of trials (n)
as a binomial experiment does
 Occurrences are independent of other occurrences
 Occurrences occur over an interval
Poisson Distribution

 If Poisson distribution is studied over a long period

off time,
ti a long
l run average can be
b determined
d t i d
 The average is denoted by lambda (λ)
 Each Poisson distribution contains a lambda value from which the
probabilities are determined
 A Poisson distribution can be described by λ alone
Poisson Distribution

 Probability function

 X e 
P( X )  for X  0,1, 2,3,...
X!
where :
  longrun average
e  2 .718282 ... (the base of natural logarithms )
 Mean value  Variance  Standard deviation

  
Poisson Distribution:
Demonstration Problem 5
5.7
7
Hospital patients arrive randomly on weekday afternoons at
an average of 3.2 patients every 4 minutes. What is the
probability of having more than 7 patients in a 4-minute
interval on a weekday afternoon?
Poisson Distribution:
Demonstration Problem 5
5.7
7 - Solution
λ = 3.2 patients/4 minutes
We want to calculate P(X
( >7p patients/4 minutes))
The problem can either be solved as:
P(X>7) = P(X=8) + P(X=9) + …, or
P(X>7) = 1 – P(X≤7) = 1 – [P(X=7) + P(X=6) + … + P(X=0)]
The answer can be obtained directly or through software
The answer you get is 1.7% of the time.
Hospital officers could use these results to help them make staffing
decisions.