Chapter 2

Probability

1
 Chapter Goals
After completing this chapter, you should be able to:
 Explain basic probability concepts and
definitions
 Use contingency tables to view a sample space
 Apply common rules of probability
 Compute conditional probabilities
 Determine whether events are statistically
independent
 Use Bayes’ Theorem for conditional probabilities
2
 Important Terms

 Probability – the chance that an uncertain event

will occur (always between 0 and 1)
 Event – Each possible type of occurrence or
outcome
 Simple Event – an event that can be described
by a single characteristic
 Sample Space – the collection of all possible
events
3
 Experiment
 A process that leads to possible outcomes.
 e.g.: Rolling a dice:

 e.g.: Choosing a card from a deck:

4
 Sample Spaces
 Collection of all possible outcomes
 e.g.: All six faces of a dice:

 e.g.: All 52 cards in a deck:

5
 Events
 Simple event
 An outcome from a sample space with one
characteristic
 e.g., A red card from a deck of cards

 Complement of an event A (denoted A’)

 All outcomes that are not part of event A
 e.g., All cards that are not diamonds

 Joint event
 Involves two or more characteristics
simultaneously
 e.g., An ace that is also red from a deck of cards
6
 Visualizing Events
 Contingency Tables
Ace Not Ace Total
Black 2 24 26
Red 2 24 26
Total 4 48 52

Ace Sample
 Tree Diagrams Red
Space

Full Cards Not an Ace

Deck
Black Ace
of 52 Cards
Cards Not an Ace 7
 Simple Events
The Event of a Triangle

There are 5 triangles in this collection of 18 objects

8
 Joint Events
The event of a triangle AND purple in color

Two triangles that are purple

9
 Mutually Exclusive Events
 Mutually exclusive events
 Events that cannot occur together

example:

A = queen of diamonds; B = queen of clubs

 Events A and B are mutually exclusive

10
11
 Collectively Exhaustive Events
 Collectively exhaustive events
 One of the events must occur
 The set of events covers the entire sample space

example: A = aces; B = black cards;

C = diamonds; D = hearts

 Events A, B, C and D are collectively exhaustive

(but not mutually exclusive – an ace may also be
a heart)
 Events B, C and D are collectively exhaustive
and also mutually exclusive
12
 Probability
 Probability is the numerical measure
of the likelihood that an event will
1 Certain
occur
 The probability of any event must
be between 0 and 1, inclusively
.5
0 ≤ P(A) ≤ 1 For any event A
 The sum of the probabilities of all
mutually exclusive and collectively
exhaustive events is 1
0 Impossible
P(A)  P(B)  P(C)  1
If A, B, and C are mutually exclusive and
collectively exhaustive 13
 Computing Joint and
Marginal Probabilities
 The probability of a joint event, A and B:
number of outcomes satisfying A and B
P( A and B) 
total number of elementary outcomes

 Computing a marginal (or simple) probability:

P(A)  P(A and B1 )  P(A and B2 )    P(A and Bk )
 Where B1, B2, …, Bk are k mutually exclusive and
collectively exhaustive events
14
Joint Probability Example
P(Red and Ace)
number of cards that are red and ace 2
 
total number of cards 52

Color
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52

15
Marginal Probability Example

P(Ace)
2 2 4
 P(Ace and Red)  P(Ace and Black )   
52 52 52

Color
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
16
 Joint Probabilities Using
Contingency Table

Event
Event B1 B2 Total
A1 P(A1 and B1) P(A1 and B2) P(A1)
A2 P(A2 and B1) P(A2 and B2) P(A2)

Total P(B1) P(B2) 1

Joint Probabilities Marginal (Simple) Probabilities

17
 General Addition Rule

General Addition Rule:

P(A or B) = P(A) + P(B) - P(A and B)

If A and B are mutually exclusive, then

P(A and B) = 0, so the rule can be simplified:

P(A or B) = P(A) + P(B)

For mutually exclusive events A and B
18
General Addition Rule Example
P(A or B) = P(A) + P(B) - P(A and B)

P(Red or Ace) = P(Red) +P(Ace) - P(Red and Ace)

= 26/52 + 4/52 - 2/52 = 28/52

Don’t count
the two red
Color aces twice!
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52

19
 Computing Conditional
Probabilities
 A conditional probability is the probability of one
event, given that another event has occurred:
P(A and B) The conditional
P(A | B)  probability of A given
P(B) that B has occurred

P(A and B) The conditional

P(B | A)  probability of B given
P(A) that A has occurred

Where P(A and B) = joint probability of A and B

P(A) = marginal probability of A
P(B) = marginal probability of B
20
 Conditional Probability
Example
 Of the cars on a used car lot, 70% have air
conditioning (AC) and 40% have a CD player
(CD). 20% of the cars have both.

 What is the probability that a car has a CD

player, given that it has AC ?

i.e., we want to find P(CD | AC)

21
 Conditional Probability Example

(continued)
 Of the cars on a used car lot, 70% have air
conditioning (AC) and 40% have a CD player (CD).
20% of the cars have both.
CD No CD Total
AC .2 .5 .7
No AC .2 .1 .3
Total .4 .6 1.0

P(CD and AC) .2

P(CD | AC)    .2857
P(AC) .7
22
 Conditional Probability Example

(continued)
 Given AC, we only consider the top row (70% of the cars). Of
these, 20% have a CD player. 20% of 70% is about 28.57%.

CD No CD Total
AC .2 .5 .7
No AC .2 .1 .3
Total .4 .6 1.0

P(CD and AC) .2

P(CD | AC)    .2857
P(AC) .7
23
 Conditional Probability Example

(continued)
 Given AC, we only consider the top row (70% of the cars). Of
these, 20% have a CD player. 20% of 70% is about 28.57%.

AC No AC Total
CD .2 .2 .4
No CD .5 .1 .6
Total .7 .3 1.0

P(CD and AC) .2

P(CD | AC)    .2857
P(AC) .7
P(AC and CD) = P(AC n CD) = P(CD n AC) 24
 Using Decision Trees

.2 P(AC n CD) = .2
P(CD|AC)=
.7 P(AC and CD) = .2
Given AC or
no AC:
P(AC and CD’) = .5
.5
.7 40%
All
Cars
.2
.3 P(AC’ and CD) = .2

.1 P(AC’ and CD’) = .1

.3 25
 Using Decision Trees

(continued)
P(AC|CD)=
.2 P(CD n AC) = .2
.4 P(CD and AC) = .2
Given CD or
no CD:
P(CD and AC’) = .2
.2
.4 70%
All
Cars
.5
.6 P(CD’ and AC) = .5

.1 P(CD’ and AC’) = .1

.6 26
 Statistical Independence
 Two events are independent if and only
if:

P(A | B)  P(A)

 Events A and B are independent when the

probability of one event is not affected by the other
event
27
 Multiplication Rules
P(A and B)
P(A | B) 
P(B)

 Multiplication rule for two events A and B:

P(A and B) P(A | B)P(B)

Note: If A and B are independent, then P(A | B)  P(A)
and the multiplication rule simplifies to
P(A and B)  P(A)P(B)

28
 Marginal Probability
 Marginal probability for event A:

P(A)  P(A | B1 )P(B1 )  P(A | B2 )P(B2 )    P(A | Bk )P(Bk )

 Where B1, B2, …, Bk are k mutually exclusive and

collectively exhaustive events

29
 Bayes’ Theorem
P(A)  P(A | B1 )P(B1 )  P(A | B2 )P(B2 )    P(A | Bk )P(Bk )

P(A and B)
P(A | B)  P(A and B) P(A | B)P(B)
P(B)
P(A | Bi )P(Bi )
P(Bi | A) 
P(A | B1 )P(B1 )  P(A | B2 )P(B2 )    P(A | Bk )P(Bk )
 where:
Bi = ith event of k mutually exclusive and
collectively exhaustive events
A = new event that might impact P(Bi)

30
 Bayes’ Theorem Example
P(A | Bi )P(Bi )
P(Bi | A) 
P(A | B1 )P(B1 )  P(A | B2 )P(B2 )    P(A | Bk )P(Bk )

31