Topic 1

Probability

Section 1 – Experiments and Sample Space

Definitions:

An experiment is defined to be any process which

generates well defined outcomes. By this we mean that on
any single repetition of the experiment one and only one
of the possible experimental outcomes will occur.

The sample space is defined as the set of all possible

experimental outcomes. Any one particular experimental
outcome is referred to as a sample point and is an
element of the sample space.

1
Section 2 – Simple and Joint Events

Definitions:

An event is a collection of one or more of the outcomes

of an experiment. Usually, the event is denoted by capital
letters such as A, B, C.

A simple event can be described by a single

characteristic.

A joint event is an event that has two or more

characteristic.

Section 2 – Simple and Joint Events

Example:

Suppose that a card is randomly selected from a deck.

The event that the selected card is black is a simple event.

The event that the selected card is a black ace is a joint

event.

2
Section 3 – Probability

Definitions:

Probability is a numerical measure of the likelihood

that a specific event will occur.

Let A be an event.
The probability that event A will occur is denoted by P ( A) .

Section 3 – Probability

Two Axioms of Probability

1. The probability of an event always lies in the range 0 to 1.

0 ≤ P( A) ≤ 1

Impossible event: An event that cannot occur. P( A) = 0

Sure event: An event that is certain to occur. P ( A) = 1

2. Let S be the sample space of an experiment.

P( S ) = 1

3
Section 4 – Three Conceptual Approaches to Probability

( 1 ) Classical Probability

Definition: Outcomes that have the same probability of

occurrence are called equally likely outcome.

The classical probability rule is applied to compute the

probabilities of events for an experiment all of whose
outcomes are equally likely.

Classical Probability Rule

Number of outcomes favorable to A

P ( A) =
Total number of outcomes for the experiment

Section 4 – Three Conceptual Approaches to Probability

Example:
Find the probability of obtaining an even number in one
roll of a die.

Solution:
6 outcomes: 1, 2, 3, 4, 5, 6 (equally likely)

Let A be an event that an even number is observed on the

die.
A = {2, 4,6}

3
P ( A) = = 0.50
6

4
Section 4 – Three Conceptual Approaches to Probability

( 2 ) Relative Frequency Concept of Probability

Suppose we want to calculate the following probabilities:

•The probability that an 80-year-old person will live for at

least one more year.

•The probability that the tossing of an unbalanced coin

will result in a head.

•The probability that we will observe a 1-spot if we roll a

loaded die.

Section 4 – Three Conceptual Approaches to Probability

These probabilities cannot be computed using the

classical probability rule because the various outcomes
for the corresponding experiments are not equally likely.

Relative Frequency as an Approximation of

Probability
If an experiment is repeated n times and an event A is
observed f times, then, according to the relative frequency
concept of probability:

f
P ( A) =
n

5
Section 4 – Three Conceptual Approaches to Probability

Example:

Ten of the 500 randomly selected cars manufactured at a

certain auto factory are found to be defective.

Assuming that the defective cars are manufactured

randomly, what is the probability that the next car
manufactured at this auto factory is defective?

Section 4 – Three Conceptual Approaches to Probability

Solution:

Let n denote the total number of cars in the sample and f

the number of defective cars in n.

f 10
P (next car is a defective) = = = 0.02
n 500

This probability is actually the relative frequency of

defective cars in 500 cars.

6
Section 4 – Three Conceptual Approaches to Probability

Note:
The relative frequencies are not probabilities but
approximate probabilities. However, if the experiment is
repeated again and again, this approximate probability of
an outcome obtained from the relative frequency will
approach the actual probability of that outcome. This is
called the Law of Large Numbers.

Law of Large Numbers

If an experiment is repeated again and again, the
probability of an event obtained from the relative
frequency approaches the actual or theoretical probability.

Section 4 – Three Conceptual Approaches to Probability

( 3 ) Subjective Probability

Many times we face experiments that neither have equally

likely outcomes nor can be repeated to generate data.

In such cases, we cannot compute the probabilities of

events using the classical probability rule or the relative
frequency concept.

7
Section 4 – Three Conceptual Approaches to Probability

Consider the following probabilities of events:

• The probability that Carol, who is taking statistics this

semester, will earn an A in this course.

• The probability that the Hang Seng Index will be higher

at the end of the next trading day.

• The probability that the Manchester United will win the

English Premier League next season.

Section 4 – Three Conceptual Approaches to Probability

Neither the classical probability rule nor the relative

frequency concept of probability can be applied to
calculate probabilities for these examples.

All these examples belong to experiments that have

neither equally likely outcomes nor the potential of being
repeated.

Definition: Subjective probability is the

probability assigned to an event based on subjective
judgment, experience, information, and belief.

8
Section 5 – Counting Rule
Section 5.1 Multiplicative Rule

You have k sets of different elements, n1 in the first set, n2

in the second set, …, nk in the kth set. Suppose you want to
form a sample of k elements by taking one element from
each of the k sets. The number of different samples that
can be formed is the product

n1 × n2 × ⋯× nk

Section 5 – Counting Rule

Section 5.1 Multiplicative Rule
Example:
Suppose that you have twenty candidates for three
different executive positions, E1, E2 and E3. How many
different ways could you fill the positions?

Solution:
Set 1: The candidates available to fill position E1 ⇒ n1 = 20
Set 2: The candidates remaining (after filling E1) that are
available to fill E2 ⇒ n2 = 19
Set 3: The candidates remaining (after filling E1 and E2)
that are available to fill E3 ⇒ n3 = 18
The number of different ways to fill
the three positions is 20 × 19 × 18 = 6840

9
Section 5 – Counting Rule
Section 5.2 Permutations Rule

Definition: A permutation of a set of objects in any

arrangement of these objects in a definite order.

Example:
S = { a , b, c }

Permutations of the elements of S:

abc, acb, bac, bca, cab, cba

Section 5 – Counting Rule

Section 5.2 Permutations Rule

Factorial – The symbol n! represents the product of all

integers from n to 1. That is,

n! = n( n − 1)( n − 2)⋯(3)(2)(1)

and by definition, 0! = 1

10
Section 5 – Counting Rule
Section 5.2 Permutations Rule

• The number of permutations (arrangements) of n

different elements is n!

• The number of permutations of n distinct objects taken r

at a time where 1 ≤ r ≤ n is

n!
P = n(n − 1)(n − 2)⋯(n − r + 1) =
(n − r )!
n r

Section 5 – Counting Rule

Section 5.2 Permutations Rule

Example:

Suppose that two letters are to be selected from A, B, C, D

and arranged in order.

( a ) How many permutations are possible?

( b ) What is the probability that the selection will

contain letter “A”?

11
Section 5 – Counting Rule
Section 5.2 Permutations Rule
Solution:
( a ) How many permutations are possible?
4!
4 P2 = = 12
(4 − 2)!
( b ) What is the probability that the selection will
contain letter “A”?

3! Required probability:
A P= =3
3 1
(3 − 1)!
6
3! = 0.5
A 3 P1 = =3 12
(3 − 1)!

Section 5 – Counting Rule

Section 5.3 Combinations Rule

Definition: A combination of a set of objects is a

group or subset of the objects disregarding their order.

That is, sampling is equivalent to partitioning a set of n

elements into 2 groups: elements that appear in the
sample and those that do not.

12
Section 5 – Counting Rule
Section 5.3 Combinations Rule

Let r be the number of elements in the sample, and n − r

be the number of elements remaining.

Then the number of different samples of r elements that

can be selected from n is

n n!
Cr or   =
 r  r !(n − r )!
n

Section 5 – Counting Rule

Section 5.3 Combinations Rule

Example:

How many ways can an executive committee of 5 be

chosen from a board of directors consisting of 15 members?

Solution:
15!
C5 = = 3003
5! (15 − 5)!
15

13
Section 5 – Counting Rule
Section 5.4 Venn Diagram

A Venn diagram shows a sample space and events

within the space.

The sample space is represented by a rectangle.

A simple event is shown as a shaded circle.

A: Simple event A
A
S S: Sample space

Section 5 – Counting Rule

Section 5.5 Basic Probability Laws

Definition: The union of two events

A and B is the set of all outcomes that B
A
are included in either A or B or both.
The union is denoted by A ∪ B .

Definition: The intersection of

two events A and B is the set of all
outcomes that are included in both A A B
and B.
The intersection is denoted by A ∩ B .

14
Section 5 – Counting Rule
Section 5.5 Basic Probability Laws

Example:

If event B is getting an even number on a die toss, and the

event A is getting a number 1 or 2.

Then,
A ∪ B = {1, 2, 4, 6}

A ∩ B = {2}

Section 5 – Counting Rule

Section 5.5 Basic Probability Laws

Definition: A set of events { A1 , A2 , …, An } is said to be

collectively exhaustive if one of the events must
occur.
That is, sample space S = A1 ∪ A2 ∪ … ∪ An

Example:

Suppose that a student is randomly selected from a class.

Let M be the event that the selected student is male.

Let F be the event that the selected student is female.

Then M and F are collectively exhaustive.

15
Section 5 – Counting Rule
Section 5.5 Basic Probability Laws

Definition: The probability of the intersection of two

events is called their joint probability.

It is written as P ( A and B ) or P ( A ∩ B ) .

Addition Rule

Consider two events A and B, the probability of the

union of A and B is
P ( A ∪ B ) = P ( A) + P( B ) − P ( A ∩ B )

Section 5 – Counting Rule

Section 5.5 Basic Probability Laws

Example:

A hamburger chain found that 65% of all customers order

french fries, 78% order soft drink, and 55% order both.

What is the probability that a customer will order at least

one of these?

16
Section 5 – Counting Rule
Section 5.5 Basic Probability Laws

Solution:
Let A be the event “Customer orders french fries” and
B be the event “Customer orders soft drink”.

From the given information,

P ( A) = 0.65, P ( B ) = 0.78, P ( A ∩ B ) = 0.55

Required probability:

P ( A ∪ B ) = P ( A) + P( B ) − P ( A ∩ B )
= 0.65 + 0.78 − 0.55 = 0.88

Section 5 – Counting Rule

Section 5.5 Basic Probability Laws
Definition: Two events are said to be
mutually exclusive if, when one
of the two events occurs in an A B
experiment, the other cannot occur.
That is, P ( A ∩ B ) = 0 .

Addition Rule – Mutually Exclusive

If the events, A and B, are mutually exclusive, the

probability that either event occurs is

P ( A ∪ B ) = P ( A) + P ( B )

17
Section 5 – Counting Rule
Section 5.5 Basic Probability Laws
Example:
A box contains two white balls, four yellow balls and five
red balls. A ball is selected from the box, what is the
probability that the selected ball is red or yellow?

Solution:
Let W be the event “the selected ball is white”, R be
the event “the selected ball is red” and Y be the event
“the selected ball is yellow”.
Required probability:
5 4 9
P ( R ∪ Y ) = P ( R ) + P (Y ) = + =
11 11 11

Section 5 – Counting Rule

Section 5.5 Basic Probability Laws

Definition: The complement of A,

denoted by A or A ' , is defined to be A
the event consisting of all sample A
points that are not in A.

For any event A,

P ( A) = 1 − P ( A)

18
Section 5 – Counting Rule
Section 5.5 Basic Probability Laws
Example:
If event A is getting a number 3 or 4 on a die toss, then the
complement of event A is A = {1, 2, 5, 6} .

Example:
A university is hiring candidates for four lecturer
positions. The candidates are five men and three women.
Assuming that every combination of men and women is
equally likely to be chosen, what is the probability that
at least one woman will be selected?

Section 5 – Counting Rule

Section 5.5 Basic Probability Laws

Solution:

P(at least one woman will be selected)

 5  3  5   3   5   3
 3  ×  1   2  ×  2   1  ×  3  13
   +   +    =
8 8 8 14
4  4  4
     

19
Section 5 – Counting Rule
Section 5.5 Basic Probability Laws

Alternatively,

Let A be the event that at least a woman is selected.

 5  3
 4×0 1
P ( A) =     =
8 14
 4
 

13
⇒ P( A) = 1 − P ( A) =
14

Section 5 – Counting Rule

Section 5.5 Basic Probability Laws
Example:
A director has two assistants helping her with her business.
The probability that the older of the two assistants will be
absent on any given day is 0.08, the probability that the
younger of the two will be absent on any given day is 0.05,
and the probability that they will both absent on any given
day is 0.02. Find the probabilities that
( a ) either or both of the assistants will be absent on any
given day;
( b ) at least one of the two assistants will not be absent on
any given day;
( c ) only one of the two assistants will be absent on any
given day.

20
Section 5 – Counting Rule
Section 5.5 Basic Probability Laws

Solution:

Let A be the event the older of the two assistants will

be absent on any given day, and
B be the event the younger of the two assistants will
be absent on any given day.

From the given information,

P ( A) = 0.08, P ( B ) = 0.05, P ( A ∩ B ) = 0.02

Section 5 – Counting Rule

Section 5.5 Basic Probability Laws

( a ) the probability that either or both of the assistants

will be absent on any given day

Use the addition rule, we have

P ( A ∪ B ) = P ( A) + P( B ) − P ( A ∩ B )

= 0.08 + 0.05 − 0.02 = 0.11

21
Section 5 – Counting Rule
Section 5.5 Basic Probability Laws

( b ) the probability that at least one of the two assistants

will not be absent on any given day

Use the complement rule, we have

P( A ∩ B) = 1 − P( A ∩ B)

= 1 − 0.02 = 0.98

Section 5 – Counting Rule

Section 5.5 Basic Probability Laws

( c ) the probability that only one of the two assistants

will be absent on any given day

The required probability is

P( A ∪ B) − P( A ∩ B)

= 0.11 − 0.02 = 0.09

22
Section 5 – Counting Rule
Section 5.6 Marginal Probability

Definition: Marginal probability is the probability

of a single event without consideration of any other event.
Marginal probability is also called simple
probability.

Section 5 – Counting Rule

Section 5.6 Marginal Probability
Example:
All 420 employees of a company were asked if they are
smokers or non-smokers and whether or not they are
university graduates. Based on this information, the
following two-way classification table was prepared.
Not a
University University
graduates graduates Total
Smoker 35 80 115
Non-smoker 130 175 305
Total 165 255 420

23
Section 5 – Counting Rule
Section 5.6 Marginal Probability
If one employee is selected at random from this company,
find the probability that this employee is

( a ) a university graduate;

( b ) a non-smoker;

( c ) a university graduate and a smoker; and

( d ) a university graduate or a non-smoker.

Note:
The probabilities in ( a ) and ( b ) are simple probabilities
while the probability in ( c ) is a joint probability.

Section 5 – Counting Rule

Section 5.6 Marginal Probability
Solution:

( a ) a university graduate

35 + 130 165
P (University graduate) = = = 0.3929
420 420

( b ) a non-smoker

130 + 175 305

P (Non-smoker) = = = 0.7262
420 420

24
Section 5 – Counting Rule
Section 5.6 Marginal Probability
(c) a university graduate and a smoker
35
P (University graduate and smoker) = = 0.8333
420
(d) a university graduate or a non-smoker

P (University graduate or Non-smoker)

= P (University graduate) + P (Non-smoker)

− P (University graduate and Non-smoker)
165 305 130
= + − = 0.8095
420 420 420

Section 5 – Counting Rule

Section 5.7 Conditional Probability and Independent

Definition: The probability of an event A given that an

event B has occurred, is called the conditional
probability of A given B and is denoted by the symbol
P( A | B) and read as ‘the probability of A given that B has
already occurred’.

B B
A
given B
A∩ B A∩ B

25
Section 5 – Counting Rule
Section 5.7 Conditional Probability and Independent

If A and B are two events with P ( A) ≠ 0 and P ( B ) ≠ 0 ,

then

P( A ∩ B) P( A ∩ B)
P( A B) = and P ( B A) =
P( B) P ( A)

The probability that both of the two events A and B occur is

P ( A ∩ B ) = P ( A) ⋅ P ( B A) and P( A ∩ B) = P( B) ⋅ P( A B)

Section 5 – Counting Rule

Section 5.7 Conditional Probability and Independent
Example:

Suppose a couple has two children.

The sample space is
S = {bb, bg , gb, gg}

where we assume an equiprobable space, that is, we assume

probability 1/4 for each point.

Find the probability that both children are boys if it is

known that:
( a ) At least one of the children is a boy.
( b ) The older child is a boy.

26
Section 5 – Counting Rule
Section 5.7 Conditional Probability and Independent

Solution:

( a ) The probability that both children are boys given that

at least one of the children is a boy

Here the reduced sample space consists of three elements

{bb, bg , gb} .
Let E denotes the event that both children are boys and F
the event that at least one of them is a boy, we have
P( E ∩ F ) 1 / 4 1
P( E | F ) = = =
P( F ) 3/ 4 3

Section 5 – Counting Rule

Section 5.7 Conditional Probability and Independent
Solution:

( b ) The probability that both children are boys given

that the older child is a boy

Here the reduced sample space consists of two elements

{bb, bg} .
Let E denotes the event that both children are boys and G
the event that the older child is a boy, we have
P( E ∩ G ) 1 / 4 1
P( E | G ) = = =
P (G ) 2/4 2

27
Section 5 – Counting Rule
Section 5.7 Conditional Probability and Independent

Example:

Suppose that an urn contains 8 red balls and 4 white balls.

We draw 2 balls from the urn without replacement.

If we assume that at each draw each ball in the urn is

equally likely to be chosen, what is the probability that both
drawn balls are red?

Section 5 – Counting Rule

Section 5.7 Conditional Probability and Independent
Solution:
Let R1 and R2 denote, respectively, the events that the first
and second ball drawn is red.

8 7
We have P ( R1 ) = and P ( R2 | R1 ) =
12 11
The desired probability is

P ( R1 ∩ R2 ) = P( R1 ) ⋅ P ( R2 | R1 )
8 7 14
= ⋅ = = 0.4242
12 11 33

28
Section 5 – Counting Rule
Section 5.7 Conditional Probability and Independent

Definition: Two events A and B are said to be

independent if the occurrence of one does not affect the
probability of the occurrence of the other.

In other words, A and B are independent events if

P ( A B ) = P ( A) and P ( B A) = P ( B )

If two events A and B are independent, then

P( A ∩ B) = P( A) ⋅ P( B)

Section 5 – Counting Rule

Section 5.7 Conditional Probability and Independent
Example:
A box contains a total of 100 DVDs that were
manufactured on two machines. Of them, 60 were
manufactured on Machine I. Of the total DVDs, 15 are
defective. Of the 60 DVDs that were manufactured on
Machine I, 9 are defective.

Let D be the event that a randomly selected DVD is

defective and A be the event that a randomly selected DVD
was manufactured on Machine I.

Are events D and A independent?

29
Section 5 – Counting Rule
Section 5.7 Conditional Probability and Independent
Solution:

From the given information,

15 9
P ( D) = = 0.15 and P ( D | A) = = 0.15
100 60

⇒ P( D ) = P ( D | A)

⇒ Events D and A are independent.

Section 5 – Counting Rule

Section 5.8 Law of Total Probability

Law of Total Probability

Assume that B1 , B2 , …, Bn are collectively exhaustive
events where P ( Bi ) > 0 , for i = 1, 2, … , n and Bi and B j
are mutually exclusive events for i ≠ j .

Then for any event A

P ( A) = P ( B1 ) P( A | B1 ) + P ( B2 ) P ( A | B2 ) + ⋯ + P( Bn ) P( A | Bn )

30
Section 5 – Counting Rule
Section 5.8 Law of Total Probability

For instance, when n = 3 , that is, B1 , B2 , B3 , are collectively

exhaustive and mutually exclusive events.

B1 B2 B3 B1 B2 B3

A B1 ∩ A B2 ∩ A B3 ∩ A

P ( A) = P ( B1 ∩ A) + P ( B2 ∩ A) + P ( B3 ∩ A)

= P ( B1 ) P ( A | B1 ) + P ( B2 ) P ( A | B2 ) + P( B3 ) P( A | B3 )

Section 5 – Counting Rule

Section 5.8 Law of Total Probability
Example:

In a college, the number of junior students is five times of

the number of senior students. The probability that a senior
student will be taking computer science is 0.20 and the
probability that a junior student will be taking computer
science is 0.30.

If a student is selected randomly, find the probability that

( a ) he/she is a senior computer science student?
( b ) he/she is a computer science major student?
( c ) he/she is a junior given that he/she is a computer
science student?

31
Section 5 – Counting Rule
Section 5.8 Law of Total Probability
Solution:
Let
A denotes the event that the selected student is a senior;
B denotes the event that the selected student is a junior;
C denotes the event that the selected student is a computer
science student.

From the given information:

P (C | A) = 0.20 P (C | B ) = 0.30

1 5
P ( A) = P( B) =
6 6

Section 5 – Counting Rule

Section 5.8 Law of Total Probability

( a ) the probability that he/she is a senior computer

science student

By the multiplicative rule,

1
P ( A ∩ C ) = P ( A) ⋅ P (C | A) = × 0.20 = 0.3333
6 =0.0333

32
Section 5 – Counting Rule
Section 5.8 Law of Total Probability

( b ) the probability that he/she is a computer science

major student

By the law of total probability,

P(C ) = P( A ∩ C ) + P( B ∩ C )

= P ( A) ⋅ P (C | A) + P ( B ) ⋅ P (C | B )

1 5
= × 0.20 + × 0.30 = 0.2833
6 6

Section 5 – Counting Rule

Section 5.8 Law of Total Probability

( c ) the probability that he/she is a junior given that

he/she is a computer science student

By the conditional probability,

P ( B ∩ C ) P ( B ) ⋅ P (C | B )
P( B | C ) = = = 0.8825
P (C ) P (C )

33
Section 5 – Counting Rule
Section 5.9 Bayes’ Theorem

Bayes’ Theorem
Suppose that B1 , B2 , …, Bn are n mutually exclusive and
exhaustive events, then

P ( Bk ∩ A)
P ( Bk | A) =
P ( A)
P ( Bk ) P ( A | Bk )
=
P ( B1 ) P ( A | B1 ) + P ( B2 ) P( A | B2 ) + ⋯ + P( Bn ) P( A | Bn )

Section 5 – Counting Rule

Section 5.9 Bayes’ Theorem

Definition: P ( Bk ) is called the prior probability in the

sense that it is assigned prior to the observation of any
empirical information.

Definition: P ( Bk | A) is called the posterior probability

or the revised probability. It is assigned after obtaining
additional information.

34
Section 5 – Counting Rule
Section 5.9 Bayes’ Theorem
Example:
In a certain assembly plant, three machines, B1, B2, and B3,
makes 30%, 45% and 25%, respectively, of the products.

It is known from past experience that 2%, 1% and 2% of

the products made by each machine, respectively, are
defective.

Now, suppose that a finished product is randomly selected.

What is the probability that it is defective? If a product
were chosen randomly and found to be defective, decide
which of the three machines is more likely to have
produced the defective product?

Section 5 – Counting Rule

Section 5.9 Bayes’ Theorem
Solution:
Let
A denotes the event that the product is defective;
B1 denotes the event that the product is made by machine B1;
B2 denotes the event that the product is made by machine B2;
B3 denotes the event that the product is made by machine B3

From the given information:

P ( B1 ) = 0.30 P ( B2 ) = 0.45 P ( B3 ) = 0.25

P ( A | B1 ) = 0.02 P ( A | B2 ) = 0.01 P( A | B3 ) = 0.02

35
Section 5 – Counting Rule
Section 5.9 Bayes’ Theorem

The probability that the product is defective

P ( A) = P ( A ∩ B1 ) + P ( A ∩ B2 ) + P ( A ∩ B3 )

= P ( B1 ) ⋅ P ( A | B1 ) + P( B2 ) ⋅ P( A | B2 ) + P( B3 ) ⋅ P( A | B3 )

= (0.30)(0.02) + (0.45)(0.01) + (0.25)(0.02) = 0.0155

Section 5 – Counting Rule

Section 5.9 Bayes’ Theorem

Given the product is defective, the probability that it is

made by machine B1

P ( B1 | A)

P ( B1 ) ⋅ P ( A | B1 )
=
P( B1 ) ⋅ P ( A | B1 ) + P ( B2 ) ⋅ P ( A | B2 ) + P ( B3 ) ⋅ P ( A | B3 )

(0.30)(0.02)
= = 0.3871
(0.30)(0.02) + (0.45)(0.01) + (0.25)(0.02)

36
Section 5 – Counting Rule
Section 5.9 Bayes’ Theorem

Similarly,
(0.45)(0.01)
P ( B2 | A) = = 0.2903
(0.30)(0.02) + (0.45)(0.01) + (0.25)(0.02)

(0.25)(0.02)
P ( B3 | A) = = 0.3226
(0.30)(0.02) + (0.45)(0.01) + (0.25)(0.02)

If a product were chosen randomly and found to be

defective, the machine B1 is more likely to have
produced the defective product.

37