Set Theory for Probability
Set Theory for Probability
1. Elementary probability
Combinatorics
Sample space
Probability
Equally likely outcomes
Objectives:
! To define events and sample spaces, describe them in
simple examples
! To list the axioms of probability, and use them to prove
simple results
! To use counting arguments to calculate probabilities when
there are equally likely outcomes
2 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Example
Rolling a die and flipping a coin can have a total of 6 · 2 = 12
different outcomes, combined.
3 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
1. Permutations
Definition
Let H = {h1 , h2 , . . . , hn } be a set of n different objects. The
permutations of H are the different orders in which one can
write all of the elements of H. There are n! = 1 · 2 · 3 · · · n of
them. We set 0! = 1.
"
4 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
1. Permutations
Definition
Let H = {h1 , h2 , . . . , hn } be a set of n different objects. The
permutations of H are the different orders in which one can
write all of the elements of H. There are n! = 1 · 2 · 3 · · · n of
them. We set 0! = 1.
"
Example
The results of a horse race with horses
H = {A, B, C, D, E, F, G} are permutations of H. A possible
outcome is (E, G, A, C, B, D, F) (E is the winner, G is second,
etc.). There are 7! = 5 040 possible outcomes.
4 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Definition
Let H = {h1 . . . h1 , h2 . . . h2 , . . . , hr . . . hr } be a set of r different
types of repeated objects: n1 many of h1 , n2 of h2 , . . . nr of hr .
The permutations with repetitions of H are the different orders
in which one can write all of the elements of H. There are
! "
n n!
:=
n1 , n2 , . . . , nr n1 ! · n2 ! · · · nr !
"
5 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Example
We can make
! "
11 11!
= = 83 160
5, 2, 2, 1, 1 5! · 2! · 2! · 1! · 1!
6 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
3. k-permutations
Definition
Let H = {h1 , h2 , . . . , hn } be a set of n different objects. The
k-permutations of H are the different ways in which one can
n!
pick and write k of the elements of H in order. There are (n−k )!
of these k-permutations.
"
7 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
3. k-permutations
Definition
Let H = {h1 , h2 , . . . , hn } be a set of n different objects. The
k-permutations of H are the different ways in which one can
n!
pick and write k of the elements of H in order. There are (n−k )!
of these k-permutations.
"
Example
The first three places of a horse race with horses
H = {A, B, C, D, E, F, G} form a 3-permutation of H. A
possible outcome is (E, G, A) (E is the winner, G is second, A
7!
is third.). There are (7−3)! = 210 possible outcomes for the first
three places.
7 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
"
8 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
"
(The case when the elements of H are only of finite supply is
much more complicated.)
8 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
"
(The case when the elements of H are only of finite supply is
much more complicated.)
Example
There are 263 = 17576 possible k = 3-letter words using the
r = 26 letters of the English alphabet.
8 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
5. k-combinations
Definition
Let H = {h1 , h2 , . . . , hn } be a set of n different objects. The
k-combinations of H are the different ways in which one can
pick k of the elements of H without order. There are
! "
n n!
:=
k k! · (n − k)!
"
9 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
5. k-combinations
Example
There are ! "
30 30!
= = 142 506
5 5! · (30 − 5)!
possible ways to form a committee of 5 students out of a class
of 30 students.
10 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
5. k-combinations
Example
There are ! "
30 30!
= = 142 506
5 5! · (30 − 5)!
possible ways to form a committee of 5 students out of a class
of 30 students.
Remark
In a similar way, there are
! "
n n!
:=
k1 , k2 , . . . , kr k1 ! · k2 ! · · · kr !
11 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
12 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Proof.
Either write out the factorials, or count the number of
k-combinations of n objects in two ways:
! the first object is chosen, and the remaining k − 1 objects
need to be picked out of n − 1, or
! the first object is not chosen, and all k objects need to be
picked out of n − 1.
12 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
13 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Proof.
In the product (x + y) · · · (x + y) on the left hand-side we need
to pick x or y from each parenthesis in all possible ways, and
multiply them. Picking x from k of these parentheses
#n $ and y
from the remaining n − k can be done in k ways, each of
which contributes x k · y n−k to the final sum.
13 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
14 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Sample space
Here we are (almost) going to define a mathematical model for
various experiments. To do it properly, we would need some
tools from measure theory. This will be skipped for now, but
you are welcome to revisit this point some time later during your
studies!
15 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Sample space
Here we are (almost) going to define a mathematical model for
various experiments. To do it properly, we would need some
tools from measure theory. This will be skipped for now, but
you are welcome to revisit this point some time later during your
studies!
! We always consider an experiment. Ω will denote the set of
all possible outcomes of this experiment.
15 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Sample space
Here we are (almost) going to define a mathematical model for
various experiments. To do it properly, we would need some
tools from measure theory. This will be skipped for now, but
you are welcome to revisit this point some time later during your
studies!
! We always consider an experiment. Ω will denote the set of
all possible outcomes of this experiment.
! An event will be a collection of possible outcomes.
Therefore, and event E will be considered a subset of Ω:
E ⊆ Ω.
15 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Sample space
Here we are (almost) going to define a mathematical model for
various experiments. To do it properly, we would need some
tools from measure theory. This will be skipped for now, but
you are welcome to revisit this point some time later during your
studies!
! We always consider an experiment. Ω will denote the set of
all possible outcomes of this experiment.
! An event will be a collection of possible outcomes.
Therefore, and event E will be considered a subset of Ω:
E ⊆ Ω.
! Sometimes Ω is too large, and not all its subsets can be
defined as events. This is where measure theory helps...
15 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Sample space
Here we are (almost) going to define a mathematical model for
various experiments. To do it properly, we would need some
tools from measure theory. This will be skipped for now, but
you are welcome to revisit this point some time later during your
studies!
! We always consider an experiment. Ω will denote the set of
all possible outcomes of this experiment.
! An event will be a collection of possible outcomes.
Therefore, and event E will be considered a subset of Ω:
E ⊆ Ω.
! Sometimes Ω is too large, and not all its subsets can be
defined as events. This is where measure theory helps...
! It makes perfect sense to define the union E ∪ F and the
intersection E ∩ F of two events, E and F .
15 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Sample space
Ω Ω
E F E F
E ∪F E ∩F
Notation: sometimes E ∪ F = E + F , E ∩ F = EF .
16 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
1. Examples
Example
Experiment: Is it going to rain today?
Sample space: Ω = {r, n}.
|Ω| = 2.
An event: E = {r}.
|E| = 1.
17 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
1. Examples
Example
Experiment: Finishing order of a race of 7 horses.
Sample space: Ω = {permutations of A, B, C, D, E, F, G}.
|Ω| = 7!.
An event: E = {horse B wins}
= {permutations that start with B}.
|E| = 6!.
Another event: F = {G wins, D is second}.
= {permutations starting as (G, D, . . . )}.
|F | = 5!.
1. Examples
Example
Experiment: Flipping two coins.
Sample space: Ω = {ordered pairs of the two outcomes}.
= {(H, H), (H, T ), (T , H), (T , T )}.
|Ω| = 4.
An event: E = {the two coins come up different}
= {(H, T ), (T , H)}.
|E| = 2.
Another event: F = {both flips come up heads}.
= {(H, H)}.
|F | = 1.
Notice: E ∪ F = {(H, T ), (T , H), (H, H)}
= {at least one H}.
19 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
1. Examples
Example
Experiment: Rolling two dice.
Sample space: Ω = {ordered pairs of the two outcomes}
= {(i, j) : i, j = 1 . . . 6}.
|Ω| = 36.
An event: E = {the sum of the rolls is 4}
= {(1, 3), (2, 2), (3, 1)}.
|E| = 3.
Another event: F = {the two rolls are the same}.
= {(i, i) : i = 1 . . . 6}.
|F | = 6.
Notice: E ∩ F = {(2, 2)}.
20 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
1. Examples
Example
Experiment: Repeatedly rolling a die until we first see 6.
Sample space: Ω = {sequences of numbers between 1}
and 5, and then a 6}. "
|Ω| = ∞.
An event: E = {roll 4 first, get 6 on the third roll}
= {(4, k, 6) : k = 1 . . . 5}.
|E| = 5.
21 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
1. Examples
Example
Experiment: Lifetime of a device (measured in years).
Sample space: Ω = [0, ∞)
|Ω| = ∞ (uncountable).
An event: E = {shouldn’t have bought it} = {0}
|E| = 1.
Another event: F = {device lasts for at least 5 years}
= [5, ∞).
|F | = ∞.
Another event: G = {device is dead by its 6th birthday}
= [0, 6).
|G| = ∞.
Notice: F ∩ G = [5, 6), F ∪ G = [0, ∞) = Ω.
22 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Remark
The union E ∪ F of events E and F always means E OR F .
The intersection E ∩ F of events E and F always means E
AND F .
Similarly:
Remark
&
The union i Ei of events Ei always means at least one of the
Ei ’s. '
The intersection i Ei of events Ei always means each of the
Ei ’s.
23 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Definition
If E ∩ F = ∅, then we say that the events E and F are
mutually exclusive events.
24 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Example
The experiment is rolling a die.
E = {rolling 1 on a die} ⊆ {rolling an odd no. on a die} = F .
Ω
F
2
3
6 E
1
5 4
E ⊆F
25 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
4. Complementary events
Definition
The complement of an event E is E c : = Ω − E. This is the
event that E does not occur.
Ω
E
Ec
Notice: E ∩ E c = ∅, E ∪ E c = Ω.
Notation: sometimes E c = Ē = E ∗ .
26 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
Commutativity: E ∪ F = F ∪ E,
E ∩ F = F ∩ E.
27 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
E ∩ (F ∩ G) = (E ∩ F ) ∩ G = E ∩ F ∩ G.
E F
28 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
G G
29 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
G G
E ∪F
29 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
G G
(E ∪ F ) ∩ G
29 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
G G
(E ∪ F ) ∩ G E ∩G
29 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
G G
(E ∪ F ) ∩ G F ∩G
29 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
G G
(E ∪ F ) ∩ G (E ∩ G) ∪ (F ∩ G)
29 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
G G
(E ∪ F ) ∩ G (E ∩ G) ∪ (F ∩ G)
(E ∩ F ) ∪ G = (E ∪ G) ∩ (F ∪ G).
E F E F
G G
29 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
G G
(E ∪ F ) ∩ G (E ∩ G) ∪ (F ∩ G)
(E ∩ F ) ∪ G = (E ∪ G) ∩ (F ∪ G).
E F E F
G G
E ∩F
29 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
G G
(E ∪ F ) ∩ G (E ∩ G) ∪ (F ∩ G)
(E ∩ F ) ∪ G = (E ∪ G) ∩ (F ∪ G).
E F E F
G G
(E ∩ F ) ∪ G
29 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
G G
(E ∪ F ) ∩ G (E ∩ G) ∪ (F ∩ G)
(E ∩ F ) ∪ G = (E ∪ G) ∩ (F ∪ G).
E F E F
G G
(E ∩ F ) ∪ G E ∪G
29 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
G G
(E ∪ F ) ∩ G (E ∩ G) ∪ (F ∩ G)
(E ∩ F ) ∪ G = (E ∪ G) ∩ (F ∪ G).
E F E F
G G
(E ∩ F ) ∪ G F ∪G
29 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
G G
(E ∪ F ) ∩ G (E ∩ G) ∪ (F ∩ G)
(E ∩ F ) ∪ G = (E ∪ G) ∩ (F ∪ G).
E F E F
G G
(E ∩ F ) ∪ G (E ∪ G) ∩ (F ∪ G)
29 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
30 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
E ∪F
30 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
(E ∪ F )c
30 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
(E ∪ F )c Ec
30 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
(E ∪ F )c Fc
30 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
(E ∪ F )c Ec ∩ F c
30 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
(E ∪ F )c Ec ∩ F c
(E ∩ F )c = E c ∪ F c .
E F E F
30 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
(E ∪ F )c Ec ∩ F c
(E ∩ F )c = E c ∪ F c .
E F E F
E ∩F
30 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
(E ∪ F )c Ec ∩ F c
(E ∩ F )c = E c ∪ F c .
E F E F
(E ∩ F )c
30 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
(E ∪ F )c Ec ∩ F c
(E ∩ F )c = E c ∪ F c .
E F E F
(E ∩ F )c Ec
30 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
(E ∪ F )c Ec ∩ F c
(E ∩ F )c = E c ∪ F c .
E F E F
(E ∩ F )c Fc
30 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
(E ∪ F )c Ec ∩ F c
(E ∩ F )c = E c ∪ F c .
E F E F
(E ∩ F )c Ec ∪ F c
30 / 280
Prob. Cond. Discr. Cont. Joint E, cov LLN, CLT Combi. Sample sp. Probability Equally l.
31 / 280