Lecture Stat 302

Introduction to Probability - Slides 1

AD

Jan. 2010

AD () Jan. 2010 1 / 18
Administrative details

Arnaud Doucet, O¢ ce LSK 308c, Department of Statistics & O¢ ce

ICCS 189, Department of Computer Science.
O¢ ce Hour, Department of Statistics: via appointment.
Textbook: A …rst course in probability, 8th edition by Sheldon Ross.
Website: http://people.cs.ubc.ca/~arnaud/stat302.html

AD () Jan. 2010 2 / 18
Notes, exercises and exams

Notes posted on the web + Additional material.

Exercises & solutions from textbook will be posted weekly.
3 assignements (20%).
1 mid-term (30%).
1 …nal exam (50%).

AD () Jan. 2010 3 / 18
Introduction to Probability

Probability theory yields mathematical tools to deal with uncertain

events.
Used everywhere nowadays and its importance is growing.
Applications include
Population genetics: tree-valued stochastic processes,
Web search engine: Markov chain theory,
Data mining, Machine learning: Stochastic gradient, Markov chain
Monte Carlo,
Image processing: Markov random …elds,
Design of wireless communication systems: random matrix theory,
Optimization of engineering processes: simulated annealing, genetic
algorithms,
Computer-aided design of polymers: Markov chain Monte Carlo.
Finance (option pricing, volatility models): Monte Carlo, dynamic
models,
Design of atomic bomb (Los Alamos): Markov chain Monte Carlo.

AD () Jan. 2010 4 / 18
Plan of the Course - Tentative Schedule

1 Combinatorial analysis; i.e counting (1 week)

2 Axioms of probability (1 week)
3 Conditional probability and inference (1 week)
4 Discrete & continuous random variables (2 weeks)
5 Multivariate random variables (1 week)
6 Properties of expectation, generating function (2 weeks)
7 Limit theorems: SLLN, CLT, inequalities (2 weeks)
8 Additional topics: Poisson and Markov processes (1 week)
9 Simulation and Monte Carlo methods (1 week)

AD () Jan. 2010 5 / 18
Combinatorial analyses aka Counting
Many basic probability problems are counting problems.
Example: Assume there are 1 man and 2 women in a room. You pick
a person randomly. What is the probability P1 that this is a man? If
you pick two persons randomly, what is the probability P2 that these
are a man and woman
Answer : You have the possible outcomes: (M), (W1), (W2) so
# “successful” events # boys 1
P1 = = = .
# events # boys + # girls 3
To compute P2 , you can think of all the possible events: (M,W1),
(M,W2), (W1,W2) so
# “successful” events 2
P2 = = .
# events 3
Both problems consists of counting the number of di¤erent ways that
a certain event can occur.
AD () Jan. 2010 6 / 18
Basic Principle of Counting

Basic Principle of Counting: Suppose that two experiments are to

be performed. Then if experiment 1 can results in any one of n1
possible outcomes and if, for each outcome of experiment 1, there are
n2 possible outcomes of experiment 2, then there are n1 n2 possible
outcomes of the two experiments.
Example: A football tournament consists of 14 teams, each of which
has 11 players. If one team and one of its players are to be selected as
team and player of the year, how many di¤erent choices are possible?
Answer : Selecting the team can be regarded as the outcome of the
…rst experiment and the subsequent choice of one of its player as the
outcome of the second experiment, so there are 14 11 = 154
possibilities.

AD () Jan. 2010 7 / 18
Generalized Principle of Counting
Generalized Principle of Counting: If r experiments that are to be
performed are such that the 1st one may result in any of n1 possible
outcomes; and if, for each of these n1 possible outcomes, there are n2
possible outcomes of the 2nd experiment; and if, for each of the
n1 n2 possible outcomes of the …rst two experiments, there are n3
possible outcomes of the 3rd experiment; and if..., then there is a
total of n1 n2 nr possible outcomes of the r experiments.
Example: A university committee consists of 4 undergrads, 5 grads, 7
profs and 2 non-university persons. A subcommitte of 4, consisting of
1 person from each category, is to be chosen. How many di¤erent
subcommittees are possible?
Answer: The choice of a subcommittee is the combined outcome of
the four separate experiments of choosing a single representative from
each of the categories. So it follows that, there are
4 5 7 2 = 280
possible subcommittees.
AD () Jan. 2010 8 / 18
Generalized Principle of Counting: More Examples
Example: How many di¤erent 6-place license plates are possible if the
…rst 3 places are to be occupied by letters and the …nal 3 by numbers
(BC format, http://www.canplates.com/bc.html)?
Answer: We have simply
26 26 26 10 10 10 = 17, 760, 000.
Example: How many functions de…ned on n points are possible if
each functional value is either 0 or 1?
Answer: We have
f = (f (1) , f (2) , ..., f (n ))
where f (i ) 2 f0, 1g so there are
2| 2 {z 2} = 2n
n times

possibilities.
AD () Jan. 2010 9 / 18
 Example: How many di¤erent 6-place license plates are possible if the
…rst 3 places are to be occupied by letters, the …nal 3 by numbers and
if repetition among letters were prohibited, repetition among numbers
were prohibited, repetition among both letters and numbers were
prohibited?
Answer: We have
letters prohib. 26 25 24 10 10 10 = 15, 600, 000
numbers prohib. 26 26 26 10 9 8 = 12, 654, 720
letters & numbers prohib. 26 25 24 10 9 8 = 11, 232, 200

AD () Jan. 2010 10 / 18
Permutations

Example: Consider the acronym UBC. How many di¤erent ordered

arrangements of the letters U, B and C are possible?
Answer: We have (B,C,U), (B,U,C), (C,B,U), (C,U,B), (U,B,C) and
(U,C,B); i.e. 6 possible arrangement. Each arrangement is known as
a permutation.
General Result. Suppose you have n objects. The number of
permutations of these n objects is given by

n (n 1) (n 2) 3 2 1 = n!

Remember the convention 0! = 1.

Example: Assume we have an horse race with 12 horses. How many
possible rankings are (theoretically) possible?

12! = 479, 001, 600

AD () Jan. 2010 11 / 18
Permutations: Examples

Example: A class in “Introduction to Probability” consists of 40 men

and 30 women. An examination is given and the students are ranked
according to their performance. Assume that no two students obtain
the same score.
How many di¤erent rankings are possible?
If the men are ranked among themselves and the women among
themselves, how many di¤erent rankings are possible?
Solution:
Each ranking corresponds to a particular ordered arrangement of the
40+30=70 people. So there are 70! = 1.1970 10100 possible rankings.
Number of ranking for men is 40! = 8.1592 1047 and for women
30! = 2.6525 1032 .

AD () Jan. 2010 12 / 18
Permutations: More Examples

Example: You have 10 textbooks that you want to order on your

bookshelf: 3 mathematics books, 3 physics books, 2 chemistry books
and 2 biology books. You want to arrange them so that all the books
dealing with the same subject are together on the shelf. How many
di¤erent arrangements are possible?
Solution: For each ordering of the subject, say M/P/C/B or
P/B/C/M, there are
3!3!2!2! = 144
arrangements. As there are 4! ordering of the subjects, then you have

144 4! = 3456

possible arrangements.

AD () Jan. 2010 13 / 18
Permutations: More Examples
Example: How many di¤erent letter arrangements can be formed from
the letters EEPPPR?
Solution: There are 6! possible permutations of letters E1 E2 P1 P2 P3 R
but the letters are not labelled so we cannot distinguish E1 and E2
and P1 , P2 and P3 ; e.g. E1 P1 E2 P2 P3 R cannot be distinguished from
E2 P1 E1 P2 P3 R and E2 P2 E1 P1 P3 R. That is if we permuted the E’s and
the P’s among themselves then we still have EPEPPR. We have 2!3!
permutations of the labelled letters of the form EPEPPR. Hence there
are
6!
= 60
2!3!
possible arrangements of the letters EEPPPR.
General Result. Suppose you have n objects. The number of
di¤erent permutations of these n objects of which n1 are alike, n2 are
alike,..., nr are alike is given by
n! n
:= .
n1 !n2 ! nr ! n1 , n2 , . . . , nr
AD () Jan. 2010 14 / 18
Permutations: More Examples

Example: A speed skating tournament has 4 competitors from South

Korea, 3 from Canada, 3 from China, 2 from the USA and 1 from
France. If the tournament result lists just the nationalities of the
players in the order in which they placed, how many outcomes are
possibles?
Answer: We simply have

(4 + 3 + 3 + 2 + 1) !
= 3, 603, 600.
4!3!3!2!1!

AD () Jan. 2010 15 / 18
Combinations
We want to determine the number of di¤erent groups of r objects
that could be formed from a total of n objects.
Example: How many di¤erent groups of 3 could be selected from
A,B,C,D and E?
Answer: There are 5 ways to select the 1st letter, 4 to select the 2nd
and 3 to select the 3rd so 5 4 3 = 60 ways to select WHEN the
order in which the items are selected is relevant. When it is not
relevant, then say the group BCE is the same as BEC, CEB, CBE,
EBC, ECB; there are 3! = 6 permutations. So when the order is
irrelevant, we have 60/6 = 10 di¤erent possible groups.
General result: When the order of selection is relevant, there are
n!
n (n 1) (n r + 1) =
(n r ) !
possible groups. When the order of selection is irrelevant, there are
n! n
:= Binomial coe¢ cient
(n r )!r ! r
possible
AD () groups. Jan. 2010 16 / 18
Combinations: Examples

Example: Assume we have an horse race with 12 horses. What is the

possible number of combinations of 3 horses when the order matters
and when it does not?
12!
Answer: When it matters, we have (12 3 )!
= 12 11 10 = 1320 and
12!
when it does not matter, we have (12 3 )!3!
= 220.
Example: From a group of 5 women and 7 men, how many di¤erent
committers consisting of 2 women and 3 men can be form? What is 2
of the men are feuding and refuse to be serve on the committee
together?
5 7
Answer: We have = 10 possible W groups and = 35
2 3
possible M groups, so 10 35 = 350 groups. In the 35 groups, we
2 5
have 5 = groups where the 2 feuding men can be so
2 1
there are 10 30 = 300 possible committees.
AD () Jan. 2010 17 / 18
Combinations: More Examples

Example: Assume we have a set of n antennas of which m are

defective. All the defectives and all the functionals are
indistiguishable. How many linear orderings are there in which no two
defectives are consecutives?
Answer: If two defectives antennas cannot be consecutive, then the
space among the n m functional antennas can contain at most one
defective antenna. There are n m + 1 positions possible and select
m of them to put the defective antennas, so that is

n m+1
m

con…gurations which is obviously equal to zero if m > n m + 1.

AD () Jan. 2010 18 / 18