Probability Theory: Probability - Models For Random Phenomena
Probability Theory: Probability - Models For Random Phenomena
Deterministic Non-deterministic
Deterministic Phenomena
• There exists a mathematical model that allows
“perfect” prediction the phenomena’s
outcome.
• Many examples exist in Physics, Chemistry
(the exact sciences).
Non-deterministic Phenomena
• No mathematical model exists that allows
“perfect” prediction the phenomena’s
outcome.
Non-deterministic Phenomena
• may be divided into two groups.
1. Random phenomena
– Unable to predict the outcomes, but in the long-
run, the outcomes exhibit statistical regularity.
2. Haphazard phenomena
– unpredictable outcomes, but no long-run,
exhibition of statistical regularity in the
outcomes.
Phenomena
Non-deterministic
Deterministic
Haphazard
Random
Haphazard phenomena
– unpredictable outcomes, but no long-run,
exhibition of statistical regularity in the
outcomes.
– Do such phenomena exist?
– Will any non-deterministic phenomena exhibit
long-run statistical regularity eventually?
Random phenomena
– Unable to predict the outcomes, but in the long-
run, the outcomes exhibit statistical regularity.
Examples
1. Tossing a coin – outcomes S ={Head, Tail}
Unable to predict on each toss whether is Head or
Tail.
In the long run can predict that 50% of the time
heads will occur and 50% of the time tails will occur
2. Rolling a die – outcomes
S ={ , , , , , }
={1, 2, 3, 4, 5, 6}
An Event , E
The event, E, is any subset of the sample space,
S. i.e. any set of outcomes (not necessarily all
outcomes) of the random phenomena
Venn
S diagram
E
The event, E, is said to have occurred if after
the outcome has been observed the outcome lies
in E.
S
E
Examples
AB
A B
The event A B occurs if the event A occurs or
the event and B occurs .
AB
A B
Intersection
Let A and B be two events, then the intersection
of A and B is the event (denoted by AB) defined
by:
A B = {e| e belongs to A and e belongs to B}
AB
A B
The event A B occurs if the event A occurs and
the event and B occurs .
AB
A B
Complement
Let A be any event, then the complement of A
(denoted by A ) defined by:
A
A
The event A occurs if the event A does not
occur
A
A
In problems you will recognize that you are
working with:
A B
If two events A and B are are mutually
exclusive then:
1. They have no outcomes in common.
They can’t occur at the same time. The outcome
of the random experiment can not belong to both
A and B.
A B
Probability
Definition: probability of an Event E.
Suppose that the sample space S = {o1, o2, o3, …
oN} has a finite number, N, of oucomes.
Also each of the outcomes is equally likely
(because of symmetry).
Then for any event E
if A B = f
(A and B mutually exclusive)
If two events A and B are are mutually
exclusive then:
1. They have no outcomes in common.
They can’t occur at the same time. The outcome
of the random experiment can not belong to both
A and B.
A B
P[A B] = P[A] + P[B]
i.e.
P[A or B] = P[A] + P[B]
A B
Rule The additive rule
(In general)
or
P[A or B] = P[A] + P[B] – P[A and B]
Logic A B
A B
A B
Example:
Saskatoon and Moncton are two of the cities competing
for the World university games. (There are also many
others). The organizers are narrowing the competition to
the final 5 cities.
There is a 20% chance that Saskatoon will be amongst
the final 5. There is a 35% chance that Moncton will be
amongst the final 5 and an 8% chance that both
Saskatoon and Moncton will be amongst the final 5.
What is the probability that Saskatoon or Moncton will
be amongst the final 5.
Solution:
Let A = the event that Saskatoon is amongst the final 5.
Let B = the event that Moncton is amongst the final 5.
Given P[A] = 0.20, P[B] = 0.35, and P[A B] = 0.08
What is P[A B]?
Note: “and” ≡ , “or” ≡ .
P A B P A P B P A B
0.20 0.35 0.08 0.47
Rule for complements
2. P A 1 P A
or
P not A 1 P A
Complement
Let A be any event, then the complement of A
(denoted by A ) defined by:
A
A
The event A occurs if the event A does not
occur
A
A
Logic:
A and A are mutually exclusive.
and S A A
A
A
thus 1 P S P A P A
and P A 1 P A
Conditional Probability
Conditional Probability
• Frequently before observing the outcome of a random
experiment you are given information regarding the
outcome
• How should this information be used in prediction of
the outcome.
• Namely, how should probabilities be adjusted to take
into account this information
• Usually the information is given in the following
form: You are told that the outcome belongs to a
given event. (i.e. you are told that a certain event has
occurred)
Definition
Suppose that we are interested in computing the
probability of event A and we have been told
event B has occurred.
Then the conditional probability of A given B is
defined to be:
P A B if P B 0
P A B
P B
Rationale:
If we’re told that event B has occurred then the sample
space is restricted to B.
The probability within B has to be normalized, This is
achieved by dividing by P[B]
The event A can now only occur if the outcome is in of
A ∩ B. Hence the new probability of A is:
A
P A B B
P A B
P B A∩B
An Example
The academy awards is soon to be shown.
For a specific married couple the probability that
the husband watches the show is 80%, the
probability that his wife watches the show is
65%, while the probability that they both watch
the show is 60%.
If the husband is watching the show, what is the
probability that his wife is also watching the
show
Solution:
The academy awards is soon to be shown.
Let B = the event that the husband watches the show
P[B]= 0.80
Let A = the event that his wife watches the show
P[A]= 0.65 and P[A ∩ B]= 0.60
P A B 0.60
P A B 0.75
P B 0.80
Independence
Definition
Two events A and B are called independent if
P A B P A P B
Note if P B 0 and P A 0 then
P A B P A P B
P A B P A
P B P B
P A B P A P B
and P B A P B
P A P A
Thus in the case of independence the conditional probability of
an event is not affected by the knowledge of the other event
Difference between independence
and mutually exclusive
mutually exclusive
Two mutually exclusive events are independent only in
the special case where
P A 0 and P B 0. (also P A B 0
Mutually exclusive events are
A highly dependent otherwise. A
B
and B cannot occur
simultaneously. If one event
occurs the other event does not
occur.
Independent events
P A B P A P B
P A B P A
or P A
P B P S
S
A B
The ratio of the probability of the
A B set A within B is the same as the
ratio of the probability of the set
A within the entire sample S.
The multiplicative rule of probability
P A P B A if P A 0
P A B
P B P A B if P B 0
and
P A B P A P B
and
P[A B] = P[A] + P[B] if A B = f
The Rule for complements
for any event E
P E 1 P E
Conditional probability
P A B
P A B
P B
The multiplicative rule of probability
P A P B A if P A 0
P A B
P B P
A B
if P B 0
and
P A B P A P B
if A and B are independent.
Let A = A1 A2 A3 ….
A3
n3 A4
n4
Rule 2
Suppose we carry out two operations in sequence
Let
n1 = the number of ways the first
operation can be performed
n2 = the number of ways the second
operation can be performed once the
first operation has been completed.
Then N = n1 n2 = the number of ways the two
operations can be performed in sequence.
Diagram:
n2
n2
n
1
n2
n2
n2
Examples
1. We have a committee of 10 people. We
choose from this committee, a chairman and
a vice chairman. How may ways can this be
done?
Solution:
Let n1 = the number of ways the chairman can be
chosen = 10.
Let n2 = the number of ways the vice-chairman
can be chosen once the chair has been
chosen = 9.
Then N = n1n2 = (10)(9) = 90
2. In Black Jack you are dealt 2 cards. What is
the probability that you will be dealt a 21?
Solution:
The number of ways that two cards can be selected from
a deck of 52 is N = (52)(51) = 2652.
A “21” can occur if the first card is an ace and the
second card is a face card or a ten {10, J, Q, K} or the
first card is a face card or a ten and the second card is an
ace.
The number of such hands is (4)(16) +(16)(4) =128
Thus the probability of a “21” = 128/2652 = 32/663
The Multiplicative Rule of Counting
Suppose we carry out k operations in sequence
Let
n1 = the number of ways the first operation
can be performed
n2
n3
n1
n2
n2
Examples
1. Permutations: How many ways can you order n
objects
Solution:
Ordering n objects is equivalent to performing n operations in
sequence.
1. Choosing the first object in the sequence (n1 = n)
2. Choosing the 2nd object in the sequence (n2 = n -1).
…
k. Choosing the kth object in the sequence (nk = n – k + 1)
…
n. Choosing the nth object in the sequence (nn = 1)
The total number of ways this can be done is:
N = n(n – 1)…(n – k + 1)…(3)(2)(1) = n!
Example How many ways can you order the 4 objects
{A, B, C, D}
Solution:
N = 4! = 4(3)(2)(1) = 24
Here are the orderings.
n!
Pk =n n 1 n k 1
n
n k !
for k = n
n! n!
n Pn n!
0! 1
Example How many permutations of size 3 can be found in
the group of 5 objects {A, B, C, D, E}
5!
Solution: 5 P3 = 5 4 3 60
5 3 !
ABC ABD ABE ACD ACE ADE BCD BCE BDE CDE
ACB ADB AEB ADC AEC AED BDC BEC BED CED
BAC BAD BAE CAD CAE DAE CBD CBE DBE DCE
BCA BDA BEA CDA CEA DEA CDB CEB DEB DEC
CAB DAB EAB DAC EAC EAD DBC EBC EBD ECD
CAB DBA EBA DCA ECA EDA DCB ECB EDB EDC
Example We have a committee of n = 10 people and we
want to choose a chairperson, a vice-chairperson and a
treasurer
10! 10!
10 P3 = 10 9 8 720
10 3! 7!
Example We have a committee of n = 10 people and we want
to choose a chairperson, a vice-chairperson and a treasurer.
Suppose that 6 of the members of the committee are male and 4
of the members are female. What is the probability that the
three executives selected are all male?
Solution: Again we want to select 3 persons from the
committee of 10 in a specific order. (Permutations of size 3
from a group of 10).The total number of ways that this can be
done is:
10! 10!
10 P3 = 10 9 8 720
10 3! 7!
This is the size, N = n(S), of the sample space S. Assume all
outcomes in the sample space are equally likely.
Let E be the event that all three executives are male
6! 6!
n E 6 P3 = 6 5 4 120
6 3! 3!
Hence
nE 120 1
PE
n S 720 6
x y n C0 x 0 y n + n C1 x1 y n 1 + n C2 x 2 y n 2
n
+ n Ck x k y n k + + n Cn x n y 0
n 0 n n 1 n 1 n 2 n 2
x y + x y + x y +
0 1 2
n k nk n n 0
+ x y + + x y
k n
Proof: The term xkyn - k will arise when we select x from k
of the factors of (x + y)n and select y from the remaining n
– k factors. The no. of ways that this can be done is:
n
k
n
Hence there will be k terms equal to xkyn = k and
n 0 n n 1 n 1 n 2 n 2
x y
n
x y + x y + x y +
0 1 2
n k nk n n 0
+ x y + + x y
k n
Pascal’s triangle – a procedure for calculating binomial
coefficients
1
1 1
1 2 1
1 3 3 1
1 4 6 4 1
1 5 10 10 5 1
1 6 15 20 15 6 1
1 7 21 35 35 21 7 1
• The two edges of Pascal’s triangle contain 1’s
• The interior entries are the sum of the two
nearest entries in the row above
• The entries in the nth row of Pascals triangle
are the values of the binomial coefficients
2
x y x 2
2 xy y 2
x y x 3x y 3xy y
3 3 2 2 3
4
x y x 4
4 x 3
y 6 x 2 2
y 4 xy 3
y 3
x y
5
x 5 x y 10 x y 10 x y 5 xy y
5 4 3 2 2 3 4 5
x y
6
x 6 6 x5 y 15 x 4 y 2 20 x3 y 3 15 x 2 y 4 6 xy 5 y 6
Rule 2
N = n1 n2 = the number of ways that two operations can be
performed in sequence if
2. Permutations
n!
n Pk The number of ways that you can
n k ! choose k objects from n in a
specific order
3. Combinations
n n!
n Ck The number of ways that you
k k ! n k ! can choose k objects from n
(order of selection irrelevant)
Applications to some counting
problems
• The trick is to use the basic counting formulae
together with the Rules
• We will illustrate this with examples
• Counting problems are not easy. The more
practice better the techniques
Application to Lotto 6/49
Here you choose 6 numbers from the integers 1,
2, 3, …, 47, 48, 49.
Six winning numbers are chosen together with a
bonus number.
How many choices for the 6 winning numbers
49 49! 49 48 47 46 45 44
49 C6
6 6!43! 6 5 4 3 2 1
13,983,816
You can lose and win in several ways
1. No winning numbers – lose
2. One winning number – lose
3. Two winning numbers - lose
4. Two + bonus – win $5.00
5. Three winning numbers – win $10.00
6. Four winning numbers – win approx. $80.00
7. 5 winning numbers – win approx. $2,500.00
8. 5 winning numbers + bonus – win approx. $100,000.00
9. 6 winning numbers – win approx. $4,000,000.00
Counting the possibilities
1. No winning numbers – lose
All six of your numbers have to be chosen from the losing numbers
and the bonus.
43
6,096,454
6
2. One winning numbers – lose
One number is chosen from the six winning numbers and the
remaining five have to be chosen from the losing numbers and the
bonus.
6 43
6 962,598 = 5,775,588
1 5
3. Two winning numbers – lose
Two numbers are chosen from the six winning numbers and the
remaining four have to be chosen from the losing numbers (bonus
not included)
6 42
15 111,930 = 1,678,950
2 4
Three numbers are chosen from the six winning numbers and the
remaining three have to be chosen from the losing numbers + the
bonus number
6 43
20 12,341 = 246,820
3 3
Five numbers are chosen from the six winning numbers and the
remaining number has to be chosen from the losing numbers
(excluding the bonus number)
6 42
6 42 = 252
5 1
6 1
6 1 = 6
5 1
9. six winning numbers (no bonus) – win approx. $4,000,000.00
6
1
6
Summary
n Prize Prob
0 winning 6,096,454 nil 0.4359649755
1 winning 5,775,588 nil 0.4130194505
2 winning 1,678,950 nil 0.1200637937
2 + bonus 172,200 $ 5.00 0.0123142353
3 winning 246,820 $ 10.00 0.0176504039
4 winning 13,545 $ 80.00 0.0009686197
5 winning 252 $ 2,500.00 0.0000180208
5 + bonus 6 $ 100,000.00 0.0000004291
6 winning 1 $ 4,000,000.00 0.0000000715
Total 13,983,816
Summary of counting rules
Rule 1
n(A1 A2 A3 …. ) = n(A1) + n(A2) + n(A3) + …
if the sets A1, A2, A3, … are pairwise mutually exclusive
(i.e. Ai Aj = f)
Rule 2
N = n1 n2 = the number of ways that two operations can be
performed in sequence if
2. Permutations
n!
n Pk The number of ways that you can
n k ! choose k objects from n in a
specific order
3. Combinations
n n!
n Ck The number of ways that you
k k ! n k ! can choose k objects from n
(order of selection irrelevant)
Applications to some counting
problems
• The trick is to use the basic counting formulae
together with the Rules
• We will illustrate this with examples
• Counting problems are not easy. The more
practice better the techniques
Another Example
counting poker hands
A poker hand consists of five cards chosen at
random from a deck of 52 cards.
A A
A 6
A A A
A
6
6. Flush {x, y, z, u, v}
• Not all in sequence but all the same suit
7. Full House {x, x, x, y, y}
8. Four of a kind {x, x, x, x, y}
4
• Select the suits for the for x. 4
3
4
• Select the suits for the for y. 6
2