2-1

Chapter 5

Probability
 2-2

2 Probability


Using Statistics

Basic Definitions: Events, Sample Space, and
Probabilities

Basic Rules for Probability

Conditional Probability

Independence of Events

Combinatorial Concepts

The Law of Total Probability and Bayes’ Theorem

Joint Probability Table
 2-3

2
LEARNING OBJECTIVES

After studying this chapter, you should be able to:


Define probability, sample space, and event.

Distinguish between subjective and objective probability.

Describe the complement of an event, the intersection, and
the union of two events.

Compute probabilities of various types of events.

Explain the concept of conditional probability and how to
compute it.

Describe permutation and combination and their use in
certain probability computations.
 2-4

2-1 Probability is:


A quantitative measure of uncertainty

A measure of the strength of belief in the
occurrence of an uncertain event

A measure of the degree of chance or
likelihood of occurrence of an uncertain
event

Measured by a number between 0 and 1 (or
between 0% and 100%)
 2-5

Types of Probability

Objective or Classical Probability
 based on equally-likely events
 based on long-run relative frequency of events

 not based on personal beliefs

 is the same for all observers (objective)

 examples: toss a coin, throw a die, pick a card

 2-6

Types of Probability (Continued)


Subjective Probability
 based on personal beliefs, experiences, prejudices,
intuition - personal judgment
 different for all observers (subjective)

 examples: Super Bowl, elections, new product

introduction, snowfall
 2-7

2-2 Basic Definitions


Set - a collection of elements or objects of
interest
 Empty set (denoted by )

a set containing no elements
 Universal set (denoted by S)

a set containing all possible elements
 Complement (Not). The complement of A is  A 

a set containing all elements of S not in A
 2-8

Complement of a Set

Venn
VennDiagram
Diagramillustrating
illustratingthe
theComplement
Complementofofan
anevent
event
 2-9

Basic Definitions (Continued)

 Intersection (And)  A  B
–
a set containing all elements in both A and B
 Union (Or)  A  B
–
a set containing all elements in A or B or both
 2-10

Sets: A Intersecting with B

A
B

A B
 2-11

Sets: A Union B

A
B

A B
 2-12

Basic Definitions (Continued)

• Mutually exclusive or disjoint sets

–sets having no elements in common, having no
intersection, whose intersection is the empty set
• Partition
–a collection of mutually exclusive sets which
together include all possible elements, whose
union is the universal set
 2-13

Mutually Exclusive or Disjoint Sets

Sets have nothing in common

B
A
 2-14

Sets: Partition

S
A3
A1

A4
A2

A5
 2-15

Experiment
• Process that leads to one of several possible
outcomes *, e.g.:
 Coin toss
• Heads, Tails
 Throw die
• 1, 2, 3, 4, 5, 6
 Pick a card
 AH, KH, QH, ...
 Introduce a new product
• Each trial of an experiment has a single observed
outcome.
• The precise outcome of a random experiment is
unknown before a trial.
* Also called a basic outcome, elementary event, or simple event
* Also called a basic outcome, elementary event, or simple event
 2-16

Events : Definition

Sample Space or Event Set
 Set of all possible outcomes (universal set) for a given
experiment

E.g.: Roll a regular six-sided die
 S = {1,2,3,4,5,6}

Event
 Collection of outcomes having a common characteristic

E.g.: Even number
 A = {2,4,6}
 Event A occurs if an outcome in the set A occurs

Probability of an event
 Sum of the probabilities of the outcomes of which it consists

P(A) = P(2) + P(4) + P(6)
 2-17

Equally-likely Probabilities
(Hypothetical or Ideal Experiments)
• For example:
 Throw a die
• Six possible outcomes {1,2,3,4,5,6}
• If each is equally-likely, the probability of each is 1/6 = 0.1667 =
16.67%
1
 P ( e) 
n( S )
• Probability of each equally-likely outcome is 1 divided by the number
of possible outcomes
 Event A (even number)
• P(A) = P(2) + P(4) + P(6) = 1/6 + 1/6 + 1/6 = 1/2
• P( A)  P( e) for e in A

n( A ) 3 1
  
n( S ) 6 2
 2-18

Pick a Card: Sample Space

Hearts Diamonds Clubs Spades

A A A A
Union of Event ‘Ace’
K K K K

Events ‘Heart’ Q Q Q Q
n ( Ace ) 4 1
J J J J
P ( Ace )   
and ‘Ace’ 10 10 10 10
P ( Heart  Ace )  n(S ) 52 13
9 9 9 9

8 8 8 8
n ( Heart  Ace )
7 7 7 7

6 6 6 6
n(S )
5 5 5 5

4 4 4 4
16 4
3 3 3 3

2 2 2 2
52 13

The intersection of the

events ‘Heart’ and ‘Ace’

Event ‘Heart’
n ( Heart ) 13 1 comprises the single point
P ( Heart )   
circled twice: the ace of hearts
n(S ) 52 4 n ( Heart  Ace ) 1
P ( Heart  Ace )  
n(S ) 52
 2-19

2-3 Basic Rules for Probability


Range of
Range ofValues
Values for
forP(A):
P(A): 0 P( A) 1

Complements -- Probability
Complements Probability of
of not
not A
A
P( A ) 1  P( A)

 Intersection -- Probability
Intersection Probability of
of both
bothAA and
and BB
P( A  B) n( A  B)
n( S )
Mutuallyexclusive
Mutually exclusiveevents
events(A
(Aand
andC)
C)::
P( A  C) 0
 2-20

Basic Rules for Probability

(Continued)
•• Union
Union--Probability
Probabilityof
ofAAor
orBBor
orboth
both(rule
(ruleof
ofunions)
unions)

P( A  B) n( A  B) P( A)  P( B)  P( A  B)
n( S )

Mutually
 Mutuallyexclusive
exclusiveevents:
events:IfIfAAand
andBBare
aremutually
mutuallyexclusive,
exclusive,then
then

P( A  B) 0 so P( A  B) P( A)  P( B)
 2-21

Sets: P(A Union B)

A
B

P( A  B)
 2-22

2-4 Conditional Probability

•• Conditional
ConditionalProbability
Probability--Probability
Probabilityof
ofAAgiven
givenBB

P( A  B)
P( A B)  , where P( B) 0
P( B)

Independent
 Independentevents:
events:

P( A B) P( A)
P( B A) P( B)
 2-23

Conditional Probability (continued)

Rulesofofconditional
Rules conditionalprobability:
probability:

P( A B)  P( A  B) so P ( A  B ) P ( A B ) P ( B )
P( B)
P( B A) P( A)

If events A and D are statistically independent:

P ( A D )  P ( A)
so
P( A  D) P( A) P( D)
P ( D A)  P ( D )
 2-24

Contingency Table - Example 2-1

Counts

AT& T IBM Total

Telecommunication 40 10 50 Probability that a project is undertaken by

Computers 20 30 50 IBM given it is a telecommunications project:

Total 60 40 100

Probabilities
P ( IBM T )
AT& T IBM Total P ( IBM T ) 
P (T )
Telecommunication .40 .10 .50
0.10
 0.2
Computers .20 .30 .50
0.50

Total .60 .40 1.00

 2-25

2-5 Independence of Events

Conditions for the statistical independence of events A and B:

P ( A B )  P ( A)
P ( B A)  P ( B )
and
P ( A  B )  P ( A) P ( B )
P ( Ace  Heart ) P ( Heart  Ace )
P ( Ace Heart )  P ( Heart Ace ) 
P ( Heart ) P ( Ace )
1 1
1 1
 52   P ( Ace )  52   P ( Heart )
13 13 4 4
52 52

4 13 1
P ( Ace  Heart)  *  P ( Ace) P ( Heart)
52 52 52
 2-26

Independence of Events –
Example 2-2

Events Television (T) and Billboard (B) are

assumed to be independent.

a )P (T  B ) P (T ) P( B)
0.04 * 0.06 0.0024
b)P (T  B ) P (T )  P ( B )  P (T  B )
0.04  0.06  0.0024 0.0976
 2-27

Product Rules for Independent Events

The probability of the intersection of several independent events

is the product of their separate individual probabilities:

P( A  A  A  An ) P( A ) P( A ) P( A ) P( An )
1 2 3 1 2 3

The probability of the union of several independent events

is 1 minus the product of probabilities of their complements:

P( A  A  A  An ) 1  P( A ) P( A ) P( A ) P( An )
1 2 3 1 2 3

Example 2-7:
(Q Q Q Q ) 1 P(Q )P(Q )P(Q )P(Q )
1 2 3 10 1 2 3 10
1 0.9010 1 0.3487 0.6513
 2-28

2-6 Combinatorial Concepts

Consider a pair of six-sided dice. There are six possible outcomes

from throwing the first die {1,2,3,4,5,6} and six possible outcomes

from throwing the second die {1,2,3,4,5,6}. Altogether, there are

6*6 = 36 possible outcomes from throwing the two dice.

In general, if there are n events and the event i can happen in

Ni possible ways, then the number of ways in which the

...

Pick 5 cardssequence
from ofa ndeck of occur is N1N2Pick
events may Nn. 5 cards from a deck of


52 - with replacement 52 - without replacement

 52*52*52*52*52=525  52*51*50*49*48 =
380,204,032 different 311,875,200 different
possible outcomes possible outcomes
 2-29

More on Combinatorial Concepts

(Tree Diagram)
Order the letters: A, B, and C

. .
C
ABC
B

. . .
C B
ACB
A

.
B

.
A
. C

. BAC

. .
C A
C BCA

. . .
A B
CAB
B

. .
A
CBA
 2-30

Factorial

How many ways can you order the 3 letters A, B, and C?

There are 3 choices for the first letter, 2 for the second, and 1 for

the last, so there are 3*2*1 = 6 possible ways to order the three

letters A, B, and C.

How many ways are there to order the 6 letters A, B, C, D, E,

and F? (6*5*4*3*2*1 = 720)

Factorial: For any positive integer n, we define n factorial as:

n(n-1)(n-2)...(1). We denote n factorial as n!.

The number n! is the number of ways in which n objects can

be ordered. By definition 1! = 1 and 0! = 1.

 2-31

Permutations (Order is important)

What if we chose only 3 out of the 6 letters A, B, C, D, E, and F?

There are 6 ways to choose the first letter, 5 ways to choose the

second letter, and 4 ways to choose the third letter (leaving 3

letters unchosen). That makes 6*5*4=120 possible orderings or

permutations.

Permutations are the possible ordered selections of r objects out

of a total of n objects. The number of permutations of n objects

P  n!
taken r at a time is denoted by nPr, where

n r (n  r )!
Forexample:
6! 6! 6 * 5 * 4 * 3 * 2 *1
6 P3    6 * 5 * 4 120
(6  3)! 3! 3 * 2 *1
 2-32

Combinations (Order is not Important)

Suppose that when we pick 3 letters out of the 6 letters A, B, C, D, E, and F

we chose BCD, or BDC, or CBD, or CDB, or DBC, or DCB. (These are the

6 (3!) permutations or orderings of the 3 letters B, C, and D.) But these are

orderings of the same combination of 3 letters. How many combinations of 6

different letters, taking 3 at a time, are there?

 n
Combinations are the possible selections of r items from a group of n items  
 r
regardless of the order of selection. The number of combinations is denoted

and is read as n choose r. An alternative notation is nCr. We define the number

of combinations of r out of n elements as:

 n n!
  n C r 
r r! (n  r)!
Forexample:
 n 6! 6! 6 * 5 * 4 * 3 * 2 *1 6 * 5 * 4 120
  6 C3      20
r
  3! ( 6  3)! 3!3! (3 * 2 * 1)(3 * 2 * 1) 3 * 2 * 1 6
 2-33

2-7 The Law of Total Probability and

Bayes’ Theorem
The law of total probability:

P( A) P( A  B)  P( A  B )

In terms of conditional probabilities:

P( A) P( A  B)  P( A  B )
P ( A B ) P ( B )  P ( A B ) P ( B )

More generally (where Bi make up a partition):

P( A)  P( A  B )
i
 P( AB ) P( B )
i i
 2-34

The Law of Total Probability-

Example 2-3

Event U: Stock market will go up in the next year

Event W: Economy will do well in the next year

P(U W ) .75
P(U W ) 30
P(W ) .80  P(W ) 1 .8 .2

P(U ) P(U W )  P(U W )

P(U W ) P(W )  P(U W ) P(W )
(.75)(.80)  (.30)(.20)
.60.06 .66
 2-35

Bayes’ Theorem
• Bayes’ theorem enables you, knowing just a little
more than the probability of A given B, to find the
probability of B given A.
• Based on the definition of conditional probability
and the law of total probability.
P ( A B)
P ( B A) 
P ( A)
P ( A  B) Applying the law of total

P ( A B)  P ( A  B ) probability to the denominator

P ( A B) P ( B)
 Applying the definition of
P ( A B) P ( B)  P ( A B ) P ( B ) conditional probability throughout
 2-36

Bayes’ Theorem - Example 2-4

• A medical test for a rare disease (affecting 0.1% of the

population [ P ( I ) 0.001 ]) is imperfect:
When administered to an ill person, the test will indicate so
with probability 0.92 [ P( Z I ) .92  P( Z I ) .08 ]
 The event (Z I ) is a false negative
When administered to a person who is not ill, the test will
erroneously give a positive result (false positive) with
probability 0.04 [ P(Z I ) 0.04  P(Z I ) 0.96 ]
 The event (Z I ) is a false positive. .
 2-37

Example 2-4 (continued)

P ( I )  0.001 P ( I Z )
P( I Z ) 
P( Z )
P ( I Z )

P ( I Z )  P ( I Z )
P ( I )  0.999
P( Z I ) P( I )

P( Z I ) P( I )  P( Z I ) P( I )
P ( Z I )  0.92 
(.92)( 0.001)
(.92)( 0.001)  ( 0.04)(.999)
0.00092 0.00092
 
0.00092  0.03996 .04088
P ( Z I )  0.04 .0225
 2-38

Example 2-4 (Tree Diagram)

Prior Conditional Joint

Probabilities Probabilities Probabilities

P( Z I ) 0.92 P ( Z  I ) (0.001)(0.92) .00092

P( Z I ) 0.08 P ( Z  I ) (0.001)(0.08) .00008

P ( I ) 0.001

P( I ) 0.999 P ( Z I ) 0.04 P( Z  I ) (0.999)(0.04) .03996

P ( Z I ) 0.96

P ( Z  I ) (0.999)(0.96) .95904
 2-39

Bayes’ Theorem Extended

• Given a partition of events B1,B2 ,...,Bn:

P( A  B )
P ( B A)  1

P ( A)
1

Applying the law of total

P( A  B )
 1 probability to the denominator

 P( A  B ) i
Applying the definition of
P( A B ) P( B )
 1 1 conditional probability throughout

 P( A B ) P( B )
i i
 2-40

Bayes’ Theorem Extended -

Example 2-5

An economist believes that during periods of high economic growth, the U.S. dollar appreciates with probability 0.70; in periods of

moderate economic growth, the dollar appreciates with probability 0.40; and during periods of low economic growth, the dollar

appreciates with probability 0.20.


During any period of time, the probability of high economic growth is 0.30, the probability of moderate economic growth is 0.50,

and the probability of low economic growth is 0.50.


Suppose the dollar has been appreciating during the present period. What is the probability we are experiencing a period of high

economic growth?

Partition:
Event A  Appreciation
P ( A H )  0.70
H - High growth P(H) = 0.30
P ( A M )  0.40
M - Moderate growth P(M) = 0.50 P ( A L)  0.20
L - Low growth P(L) = 0.20
 2-41

Example 2-5 (continued)

P( H  A)
P( H A) 
P( A)
P( H  A)

P( H  A)  P( M  A)  P( L  A)
P( A H ) P( H )

P ( A H ) P ( H )  P ( A M ) P ( M )  P ( A L) P ( L)
( 0.70)( 0.30)

( 0.70)( 0.30)  ( 0.40)( 0.50)  ( 0.20)( 0.20)
0.21 0.21
 
0.21 0.20  0.04 0.45
 0.467
 2-42

Example 2-5 (Tree Diagram)

Prior Conditional Joint

Probabilities Probabilities Probabilities

P ( A H )  0.70 P ( A  H )  ( 0.30)( 0.70)  0.21

P ( A H )  0.30
P ( H )  0.30 P ( A  H )  ( 0.30)( 0.30)  0.09

P ( A M )  0.40 P ( A  M )  ( 0.50)( 0.40)  0.20

P ( M )  0.50

P ( A M )  0.60 P ( A  M )  ( 0.50)( 0.60)  0.30

P ( A L )  0.20
P ( L )  0.20 P ( A  L )  ( 0.20)( 0.20)  0.04

P ( A L )  0.80 P ( A  L )  ( 0.20)( 0.80)  0.16

 2-43

2-8 The Joint Probability Table


A joint probability table is similar to a contingency
table , except that it has probabilities in place of
frequencies.

The joint probability for Example 2-11 is shown
below.

The row totals and column totals are called
marginal probabilities.
 2-44

The Joint Probability Table



AAjoint
joint probability
probability table
table isis similar
similar toto aa contingency
contingency
table ,, except
table except that
that itit has
has probabilities
probabilities in in place
place of
of
frequencies.
frequencies.

 The joint
The joint probability
probability for for Example
Example 2-11 2-11 isis shown
shown on
on
the next
the next slide.
slide.

 The row
The row totals
totals and
and column
column totals
totals are
are called
called
marginal probabilities.
marginal probabilities.
 2-45

The Joint Probability Table:

Example 2-5

The joint probability table for Example 2-11
is summarized below.
High Medium Low Total
$
Appreciates
0.21 0.2 0.04 0.45
$Depreciates
0.09 0.3 0.16 0.55

Total 0.30 0.5 0.20 1.00

Marginal
Marginal probabilities
probabilities are
are the
the row
row totals
totals and
and the
the column
column totals.
totals.