ST2504 – INTRODUCTION TO PROBABILITY

THEORY
• Lecturer: Ms. Motena Mapesela
• Office: BTM217
Introduction
• Probability is a numerical value in the interval from 0 to 1 that is used
to express the likelihood or chance that a specific event will occur.
• Probability was developed in connection with games of chance,
games in which the outcomes cannot be predicted with certainty.
• In such games, the idea of some sort of trial or test or experiment is
important, e.g., one draws a card from a deck of cards or selects a
coloured ball from a jar.
• There is a well-defined trial and a set of possible outcomes.
Cont.
• It is helpful to realize that each of the situations above is equivalent
to drawing balls from a jar or urn. Each throw of a pair of dice, for
instance, results in identifiable outcome.
• Each such outcome can be replaced by a single ball, appropriately
marked.
Cont.
• The concept of the outcome of a trial, which is so natural to a game
of chance, is readily extended to more practical situations of scientific
importance, e.g. one may be sampling opinions or recording the ages
at death of members of a population. One may be making a physical
measurement in which the precise outcome is uncertain.
• The results observed may be considered to be the result of a trial
which yields one of a set of possible outcomes. This set of possible
outcomes is clearly defined
Basic Concepts
1. Experiment
2. Sample Space
3. Event
1. Experiment
• An experiment is an act or a process specially set up or occurring
naturally, which leads to some well-defined outcomes or results. It is
a process that generates a set of data.
• Example:
• If a coin is tossed, we observe whether a head or a tail is obtained.
We might describe this as an experiment of tossing a coin.
• Give at least 3 examples:
Trial
• Each repetition of an experiment is a trial.

• Example:
• If a coin is tossed three times, each single toss is a trial.
Outcome of an Experiment
• An outcome of an experiment is the result we observe when an
experiment is performed and need to be a numerical value.
• Example:
• A coin is tossed once and a head is observed.
• The process of tossing a coin would represent an experiment, while
the coin that is being tossed is the experimental unit and the
outcome is the head being observed
Solution
• Experiment Experimental Outcomes
• Toss a coin Head, Tail
• Select a part for inspection Defective, Non - defective
• Conduct a sales call Purchase, No Purchase
• Roll a die 1, 2, 3, 4, 5, 6
• Play a football game Win, Lose, Tie
Types of Experiments
• There are essentially two types of experiments.
• a) Deterministic Experiment
• b) Random/Stochastic Experiments
Types of Experiments
• There are essentially two types of experiments.
• a) Deterministic Experiment:
• An experiment is deterministic if its observed result is not subject to
chance. In a deterministic experiment, if the experiment is repeated a
number of times under exactly the same conditions, we expect the
same result.
• Example:
• If we measure the distance in kilometers between two towns, say
town A and town B many times, under the same conditions, we
expect to have the same results.
Cont.
• b) Random/Stochastic Experiments:
• An experiment is random/stochastic if its outcomes are uncertain. If a
random experiment is repeated under identical conditions, the
outcomes may be different as there may be some random
phenomena or chance mechanism at work affecting the outcome.
• Examples?
Example
• Tossing a coin or rolling a die is a random experiment since in each
case the process can lead to more than one possible outcomes.
• In the case of tossing a coin experiment the result will come up head
(H) or tail (T). And in the case of rolling a die the result will be the
numbers 1, 2, 3, 4, 5, and 6.
Properties of Random Experiments
• - The experiment should be repeatable under identical conditions;
• - The outcome on any particular trial is variable, dependent on some
chance or random mechanism;
• - The outcome of the experiment cannot be determined in advance.
2. Sample Space
• A set of all possible outcomes of a trial is called the sample space for
the experiment, usually denoted by S.
• Example……
• Sample Points – A particular outcome in the sample space S is called
a sample point.
• The number of sample points in S may be denoted by n(S) and is
called the size or cardinality of S.
• Example:
• A die is rolled once. There are six sample points in the sample space,
hence n(S)=6.
Sample Space and Events
• Example 1: Toss 3 fair coins once and note the faces up.

S =  HHH , HHT , HTT , HTH , THT , THH , TTH , TTT 

• Define A as an event that a head occurs in the first toss. What are the
elements of A?
A =  HHH , HHT , HTH , HTT 
3. Events
• An event is a subset of the sample space. Events are represented by
uppercase letters like A, B, etc.
• An event is said to have occurred if one of its elements is the
outcome observed.
• Example
• Roll a die once. Write the following events as sets:
• a) the number 4;
• b) A number greater than 4; and
• c) An odd number
Tree Diagram
• Suppose that three items are selected at random from a
manufacturing process. Each item is inspected and classified as
defective, D, or non-defective, N. To list the elements of the sample
space providing the most information, we construct the tree diagram.

• Construct the three diagram

• Also draw a tree diagram for tossing a coin three times.
Real World vs Mathematical model
• Real World Mathematical Model
• 1. Relative Frequency of occurrence Probability of events
• of events
• 2. Set of possible outcomes of a trial Sample space S of
elements
• 3. Events Subsets of S
• 4. Occurrence of an event Selection of an element
from the appropriate subset
Special Events and Combination of Events
• In probability theory we are concerned with various combinations of
events and relations between them.
• These combinations and relations are developed in set theory. Thus,
having introduced the concept of an event as a set, we have at our
disposal an important mathematical resource in the theory of sets.
Cont.
• When a trial or choice is made, one and only one element is chosen, i.e,
only one elementary outcome is observed – but a large number of events
may have occurred. Suppose in throwing a pair of dice, the pair numbers
(2-4) turn up. Only one of the possible outcomes of the trial has been
selected. But the following events (among others) have occurred:
• A “six” is thrown,
• A number less than 7 is thrown,
• An odd number is thrown etc.,
• These are distinct events, and not just different names for same events.
Union of Events
• An event is called the union of two events A and B if it occurs
whenever at least one of them occurs. This is denoted by A  B
• Example:
• A = {1,2}, B = {1,2,3},
A  B = 1, 2,3
Intersection of Events
• An event is called the intersection of two events A and B if it occurs
only whenever A and B occur simultaneously. In the language of sets
this event is denoted by A  B because it is the set containing
exactly the common points of A and B.
• Example
• A = {1,2 3,4,5}, B = {1,3,5,7,9}, A  B?
• When two events A and B have a relation that their joint occurrence
is impossible, they are said to be mutually exclusive events. That is,
• A  B =  . Such events are said to be disjoint.
Union and Intersection Vann Diagram
 Mutually Exclusive Events

• The events A and B are said to be mutually exclusive if

they have no outcomes in common,
• More generally, a collection of events 𝐴1 , 𝐴2 , . . . , 𝐴𝑛 is
said to be mutually exclusive if no two of them have
any outcomes in common.
Cont.
• If events A and B are mutually exclusive, the probability that one or
the other occurs is the sum of their individual probabilities.
• P(A or B) = P(A) + P(B)
• P(A or B or C) = ?
Sure/Certain Events
• An event is called certain if its occurrence is inevitable. Thus the
sample space is a certain event.
Impossible Event
• An event is called impossible if there is certainty in its nonoccurrence.
Therefore, the empty set  , which is S c is an impossible event.
• Complementary Event
• An event is called the complement of the event A if it only occurs
whenever A does not occur. The complement of A is denoted by Ac
or A
• Subset
• An event A is said to be a subset of the event B if, whenever A occurs,
B also occurs. This means that all of the sample points of A are
contained in B. This is denoted by A  B
Compliment Vann Diagram
Subset Vann Diagram
Events and sample space
• The relationship between events and the corresponding sample space
can be illustrated graphically by means of Venn Diagrams. In a Venn
diagram we let the sample space be a rectangle and represent events
by circles drawn inside the rectangle.
• Draw a Venn diagram to represent the following
•AꓵB=
•BꓵC=
•AUC=
• B’ ꓵ A =
Cont.
• AꓵBꓵC=
• (A U B) ꓵ C =
Example of Venn Diagram for 3 Events, A, B
and C
Solution
• A ꓵ B = A+B and A+B+C
• B ꓵ C = A+B+C and B+C
• A U C = Everything except B
• B’ ꓵ A = A and A+C
• A ꓵ B ꓵ C = A+B+C
• (A U B) ꓵ C’ = A, B and A+B+C
Some Laws of Set Theory
• 1. Commutative Law
A B = B  A
A B = B  A

• 2. Associative Laws
( A  B)  C = A  ( B  C )
( A  B)  C = A  ( B  C )
Cont.
• 3. Distributive Laws
( A  B)  C = ( A  C )  ( B  C )

( A  B)  C = ( A  C )  ( B  C )

• De Morgan’s Law
A B = A B
A B = A B
Exercise
• Proof the above laws using Venn diagrams
Exercises
• Give an example of an experiment. Identify the experimental units?
What are the possible outcomes of the experiment?
• What are the two types of experiments? Give an example of each.
• What are the properties of experiments?
• Think of any experiment/trial. What are the possible outcomes? What
are the sample points? From your trial, define the possible events and
write down their elements.
Exercises
• Suppose we ask the next person we meet on the street in which
month his/her birthday falls. What is the sample space?
• Let A be the event that the month has 31days. What are the elements
of A?
• Let B be the event that the month has a letter “r” in their full names.
What are the elements of B?
• What is A intersection B?
• What is the union of A and B?
• Is B a subset of A?
• Are A and B mutually exclusive?
Three Approaches to Probability
• 1. The relative frequency approach
• 2. The classical approach
• 3. The subjective approach
Relative Frequency Approach
• The outcomes of an experiment are not all equally likely under the
relative frequency approach, that is, the probabilities of the outcomes
differ.
• To estimate the probabilities, past data are used or new data are
generated by performing the experiment a large number of times and
then counting the number of times that event actually occurs.

𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑖𝑚𝑒𝑠 𝑒𝑣𝑒𝑛𝑡 𝐴 𝑜𝑐𝑐𝑢𝑟𝑟𝑒𝑑 𝑓

•𝑃 𝐴 = =
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑖𝑚𝑒𝑠 𝑡ℎ𝑒 𝑒𝑥𝑝𝑒𝑟𝑖𝑚𝑒𝑛𝑡 𝑖𝑠 𝑟𝑒𝑝𝑒𝑎𝑡𝑒𝑑 𝑛
Example
• In a sample of 500 families, 95 have an annual income of less than
M80 000, 272 families have an annual salary of M80 000 to M150 000
and the remaining families have an annual income of more than
M150 000. One family is randomly selected from these 500 families.
Find the probability that this family has an annual income of:
• A) less than M80 000
• B) more than M150 000
• C) M80 000 to M150 000 or more than M150 000
• D) Show that the probability of the sample space is equal to 1.
Solution
• Let
• A = less than M80 000
• B = M80 000 to M150 000
• C = more than M150 000
• n=500, f = 95
 Classical Approach
• An approach in which all outcomes in a sample space are equally
likely to occur, that is, all the outcomes have the same probability of
occurring.
• It is not necessary to perform the experiment repeatedly and observe
the results in order to determine the probability of events.
• If outcomes are mutually exclusive and equally likely, then
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑜𝑓 𝑡ℎ𝑒 𝑒𝑥𝑝𝑒𝑟𝑖𝑚𝑒𝑛𝑡 𝑓𝑎𝑣𝑜𝑢𝑟𝑎𝑏𝑙𝑒 𝑡𝑜 𝑡ℎ𝑒 𝑒𝑣𝑒𝑛𝑡
•𝑃 𝐴 =
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑜𝑓 𝑡ℎ𝑒 𝑒𝑥𝑝𝑒𝑟𝑖𝑚𝑒𝑛𝑡
Tips on solving problems
• - Write down the sample space;
• - Determine the probability of each outcome. Remember the
probabilities are the same for all the outcomes;
• - Denote the event with a capital letter and define the event as a
collection of outcomes from the sample space;
• - Find the probability of the event by obtaining the sum of the
probabilities of the outcomes that are contained in the event.
Example
• If two dice are rolled, find the probability that:
• A. both show an odd number
• B. a sum of 7 shows
• C. a sum of 4 or 11 shows
• D. only the second die shows a 3.
Example
• Moeketsi’s closet has three pairs of pants (black, white and green), two shirts
(green and white) and two pairs of shoes (black and white).
• a) How many different outfits can be made?

• b) Write down the sample space.

• c) What is the probability that if you close your eyes and choose randomly, you
would choose:
• (i) Pants and shoes with the same colour?
• (ii) Pants and shirts with the same colour?
• (iii) Pants, shoes and shirts with the same colour?
• (iv) Pants, shoes and shirts with the different colours?
Subjective Approach
• Sometimes experiments cannot be repeated a large number of times
to generate data, nor do they have equally likely outcomes.
Probabilities assigned to these events are based on personal belief,
intuition and experience of the individual who assigns them, as well
as their analysis of a particular situation.
• The probabilities assigned will therefore differ from individual to
individual.
Axioms of Probability
• A probability system or probability space consists of the triple:
• - A sample space S of elementary outcomes, w.
• - A class E of events; subsets of S.
• - A probability measure P(.) defined for each event in E and having the
following properties:
Axioms of Probability
• 1. P(S) = 1 (The probability of a sure event is 1),
• 2. P(A) ≥ 0, (The probability of an event is nonnegative)
• 3. If A and B are mutually exclusive/disjoint events,
•
• More generally, if 𝐴1 , 𝐴2 , … , 𝐴𝑛 are mutually exclusive events, then
 k  k
P ( A1  A2  ...  Ak ) = P  U Ai  =  P ( Ai )
 i =1  i =1
Theorems of Probability
• Axiom 3 states that the probability of the union of mutually exclusive
events is the sum of the probabilities of individual events.
• The following theorems of probability measures are consequences of
the axioms of probability.
• For any events A and B,
• 1.
• 2.
3. If A  B, then P ( A )  P ( B )
Proof the above theorems
Cont.

Proof. Since A  Ac = S and A  Ac = , we have

1 = P ( S ) = P ( A  Ac ) = P ( A ) + P ( Ac )
Cont…
• 2.
• To see this, note that Ø U S = S
• Therefore, P(ØUS) = P(S)
• P(Ø) + P(S) = P(S)
• P(Ø) = P(S) – P(S) = 0

• 3.If A  B, then P ( A )  P ( B )
• ( ) ( )
Write B = A  B  Ac , and notice that A  B  Ac = ;
•
•
( ( ) ) ( )
Thus P ( B ) = P A  B  Ac = P ( A ) + P B  Ac  P ( A )

P ( B  A )  0.
since c
Cont.
• 4. Consider the union of two events A and B, AUB.
• The union can be partitioned as follows: AUB = AU(AcꓵB)
• Similarly, the union can be written as AUB = BU(AꓵBc)
• Or as AUB = (AꓵBc)U(AcꓵB)U(AꓵB)
• 5. For any two events A and B, we can find the probability that A or B
occurs using the General Addition Rule
• P(AUB) = P(A) + P(B) – P(AꓵB)
• Note that the two events do not need to be mutually exclusive in order to use
this rule
Cont.
• 4. For any two events A and B, we can find the probability that A or B
occurs using the General Addition Rule
• P(AUB) = P(A) + P(B) – P(AꓵB)
• Note that the two events do not need to be mutually exclusive in order to use
this rule
Exercise
• Driving to work, a commuter passes through a sequence of three
intersections with traffic lights. At each light, she either stops, s, or
continues, c. The sample space is the set of all possible outcomes:

S = 
Exercise
• A person driving to work every day on a route with four traffic lights
believes the following to be suitable probabilities for the number of
red lights encountered on a trip. Let the random variable 𝑋 be the
number of red lights encountered. Let A be the event that no red light
is encountered with P(A) = 0.05, B be the event that one red light is
encountered with P(B) = 0.25, 𝐶 be the event that two red lights are
encountered with P(C) = 0.36, 𝐷 be the event that three red lights are
encountered with P(D) = 0.26, and E be the event that four red lights
are encountered with P(E) = 0.08.
• Does these probabilities satisfy the axioms of probability?
•
Cont.
• What is the probability of encountering at least one red traffic light on
a trip?
• What is the probability of encountering more than two red traffic
lights on a trip?
• What is the probability of encountering at the most two red traffic
lights on a trip?
•
Counting Techniques
• So far we have explained the basic concepts of probability using
examples for which the total number of outcomes is relatively small.
In practice this might not always be the case.
• The counting rules like permutations and combinations are
fundamental for calculation of various probabilities.
• The counting techniques will enable us to determine the number of
possible outcomes of a random experiment and cases of occurrence
of any event.
Multiplication Principle
• If one experiment can result in any one of the 𝒎 possible outcomes
and another experiment can result in any one of 𝑛 possible outcomes,
then there are 𝑚𝑛 possible outcomes for the two experiments.
• Example:
• If we can travel from town A to town B in 3 ways and from town B to
town C in 4 ways, then, in how many ways can we travel from town A
to C?
Cont.
• Answer: 3 x 4 = 12 ways
• Example
• A class has 12 boys and 18 girls. The teacher selects 1 boy and 1 girl
to act as representatives to the student government. He can do this in
any of the 12 x 18 = 216 different ways.
• The multiplication principle can be extended to more than two
experiments as follows:
• If there are k experiments and the first one has n1 possible outcomes,
the second has n2 possible outcomes, …, and the kth has nk possible
outcomes, then there are a total of
n1  n2  ...  nk
Example
• In an experiment, two coins are tossed and a die is rolled to make an
observation. How many outcomes are contained in the sample space?
Solution
• 𝑛1 =
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑝𝑜𝑠𝑠𝑖𝑏𝑙𝑒 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑤ℎ𝑒𝑛 𝑡ℎ𝑒 𝑓𝑖𝑟𝑠𝑡 𝑐𝑜𝑖𝑛 𝑖𝑠 𝑡𝑜𝑠𝑠𝑒𝑑 =
2, 𝑤ℎ𝑖𝑐ℎ 𝑎𝑟𝑒 {H, T}.
• 𝑛2 = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑝𝑜𝑠𝑠𝑖𝑏𝑙𝑒 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑤ℎ𝑒𝑛 𝑡ℎ𝑒 second 𝑐𝑜𝑖𝑛 𝑖𝑠
𝑡𝑜𝑠𝑠𝑒d =2, 𝑤ℎ𝑖𝑐ℎ 𝑎𝑟𝑒 {H, T}.
• 𝑛3 = 𝑡ℎ𝑒 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑝𝑜𝑠𝑠𝑖𝑏𝑙𝑒 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑤ℎ𝑒𝑛 𝑎 𝑑𝑖𝑒 𝑖𝑠 𝑟𝑜𝑙𝑙𝑒𝑑 =
6, 𝑤ℎ𝑖𝑐ℎ 𝑎𝑟𝑒 1,2,3,4,5,6
• Number of possible outcomes in the sample space = 𝑛1 𝑥 𝑛2 𝑥𝑛3 =
2 2 6 = 24
Factorial
• Sometimes it is necessary to obtain all possible different
arrangements of a number of objects in order to obtain the sample
space of an experiment.
• The total number of ways in which all of a number of distinct objects,
say 𝑟 distinct objects, can be arranged without repeating any of the
objects in an arrangement, is given by:
• 𝑟! = 𝑟 𝑟 − 1 𝑟 − 2 (𝑟 − 3) … (3)(2)(1)
The term distinct objects refers to the fact that all of the objects are
different and unique, that is, every other object occurs only once.
Example
• Four applicants apply for the same position in a company and are all
invited for an interview. Determine the number of different orders
(arrangements) in which they could be called in an interview.
Solution
• 4! = 4𝑥3𝑥2𝑥1 = 24 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡 𝑜𝑟𝑑𝑒𝑟𝑠 𝑎𝑟𝑟𝑎𝑛𝑔𝑒𝑚𝑒𝑛𝑡𝑠 𝑝𝑜𝑠𝑠𝑖𝑏𝑙𝑒
Permutations
• The number of 𝑟 arrangements we can make (𝑟 ≤ 𝑛), using 𝑛
different symbols (each once) is called the number of permutations of
𝑛 things 𝑟 at a time and is denoted by n Pr
• That is the total number of arrangements or permutations of 𝑛
objects taken 𝑟 at a time.
n!
n Pr =
( n − r )!
Exercise
• In Lesotho Premier Football League, there are 12 teams, and any
team can occupy any of the 12 positions at the end of the season.
• a) In how many ways can the 12 teams finish at the end of the
season?
• b) In how many ways can the top four positions be occupied at the
end of the season?
• c) Midway into the season, it is realized that the last four teams
cannot make it to the top 4 positions. In how many ways can the top
four positions be occupied at this stage?
Solution
• a) There are a total of 12! = 12x11x10x…x1=479, 001, 600 ways in
which the teams can finish the season.
12! 12!
• b) There are 12 P4 = = = 11,880 ways in which the 12 teams
(12 − 4 )! 8!
• can occupy the top four positions
• c) P =
8! 8!
= = 1,680
8 4
( 8 − 4 )! 4!
Combinations
• The number of distinct subsets, each of size r, that can be constructed
from a set with n elements is called the number of combinations of n
objects r at a time. This is represented by or nCr

• To evaluate remember that sets are not ordered and that r

different symbols can be used to construct r! permutations. Thus
since nCr is the number of different collections (subsets) of size r that
could be formed from a set with n objects, each of which leads to r!
permutations, it must be true that
Cont.

n n!
n Cr =   =
 r  ( n − r )!r!

Example:
A printed circuit board may be purchased from five suppliers. In how
many ways can three suppliers be chosen from the five?
Solution
• Since it is important to know only which three suppliers have been
chosen, not the order of selection, the number of ways is
 5  5!
3 = = 10
  3!2!
Class Exercise
• Refer to the teams that finish in the PSL.
• Teams that finish in the top four compete in the Top 4 tournament
while the last two teams are relegated to the first division.
• a) How many different team combinations are there for the Top 4
tournament?
• b) How many of these have your favourite team as a participant?
Solutions
• a) Since we are only interested in the teams that will take part in the
tournament (not their position on the log, we have

12  12!
• 4  = = 495 combinations
  8!4!

• b) If my favourite team is in the tournament, then there are three

places left which can be taken by the other eleven teams. Thus, there
are 11 = 11! = 165 possible combinations which include my team as a
3 
  8!3!

• participant
Example
• In how many ways can the letters of the word ABSENT be arranged?
• In how many ways can 3 balls be selected out of a total of 10 balls?
• Suppose a box contains 5 black balls and 6 red balls. In how many
ways can 2 black balls and 3 red balls be drawn from the box? (The
balls are replaced)
Solution
• The total number of ways in which letters of the word ABSENT can be
arranged is P =
6!
= 6! = 720 ways
6 6
( 6 − 6 )!
10  10! 10.9.8
3  = = = 120 ways
  3!7! 3.2

5
• Out of 5 black balls, 2 black balls can be drawn in ways. Out of 6
2
6
red balls, 3 can be drawn in ways.
3
Cont.
• Thus, number of ways in which 2 black balls and 3 red balls can be
5 6
drawn from a box of 5 black and 6 red balls is 𝑥
2 3
Counting Techniques - Summary
• Multiplication Principle
• There are two experiments resulting in 𝑚𝑛 possible outcomes.
• Factorial is the number of possible arrangements.
• A permutation is an arrangement of a given number of objects in a
particular order. 𝑟 ordered arrangements using 𝑛 symbols
• Combinations: 𝑟 arrangements are selected/chosen/drawn from 𝑛
symbols. These are problems in which order of selection does not
matter
Example
• Suppose that in a community of 400 adults, 300 bike or swim or do
both, 160 swim, and 120 swim and bike. What is the probability that
an adult, selected at random from this community, bikes?
• Let A be the event that the person swims and B be the event that
he/she bikes; then P(AUB)=300/400, P(A) = 160/140, and P(AB) =
120/140. Hence the relation P(AUB) = P(A) + P(B) – P(AB) implies that
• P(B) = P(AUB) + P(AB) – P(A)
• P(B) = 300/400 + 120/400 – 160/400 = 260/400 = 0.65
Exercise
• A commuter passes through three traffic lights on the way to work. Each
light is either red, yellow, or green. An experiment consists of observing the
colour of the three lights.
• i) List the 27 possible outcomes in the sample space.
• ii) Let A be the event that all the colours are the same. List the outcomes of
A.
• iii) Let B be the event that all the colours are different. List the outcomes of
B.
• iv) Let C be the event that at least two lights are green. List the outcomes
of C.
• v) List the outcomes in 𝐴 ‫𝐶 ځ‬.
Cont.
• vi) List the outcomes in 𝐴 ‫𝐵 ڂ‬.
• vii) List the outcomes in 𝐴 ‫ 𝑐 𝐶 ځ‬.
• viii) List the outcomes in 𝐴𝑐 ‫ 𝐶 ځ‬.
• ix) Are events A and C mutually exclusive? Explain.
• x) Are events B and C mutually exclusive? Explain.
Solution
• 𝑆𝑎𝑚𝑝𝑙𝑒 𝑆𝑝𝑎𝑐𝑒 =

𝑅𝑅𝑅, 𝑅𝑅𝐺, 𝑅𝑅𝑌, 𝑅𝐺𝑅, 𝑅𝐺𝐺, 𝑅𝐺𝑌, 𝑅𝑌𝑅, 𝑅𝑌𝐺, 𝑅𝑌𝑌, 𝐺𝑅𝑅, 𝐺𝑅𝐺, 𝐺𝑅𝑌
𝐺𝐺𝑅, 𝐺𝐺𝐺, 𝐺𝐺𝑌, 𝐺𝑌𝑅, 𝐺𝑌𝐺, 𝐺𝑌𝑌, 𝑌𝑅𝑅, 𝑌𝑅𝐺, 𝑌𝑅𝑌, 𝑌𝐺𝑅, 𝑌𝐺𝐺, 𝑌𝐺𝑌,
𝑌𝑌𝑅, 𝑌𝑌𝐺, 𝑌𝑌𝑌
𝐴 = 𝑅𝑅𝑅, 𝐺𝐺𝐺, 𝑌𝑌𝑌
• 𝐵 = 𝑅𝐺𝑌, 𝑅𝑌𝐺, 𝐺𝑅𝑌, 𝐺𝑌𝑅, 𝑌𝑅𝐺, 𝑌𝐺𝑅
• 𝐶 = 𝑅𝐺𝐺, 𝐺𝑅𝐺, 𝐺𝐺𝑅, 𝐺𝐺𝐺, 𝐺𝐺𝑌, 𝐺𝑌𝐺, 𝑌𝐺𝐺
• A and C are not mutually exclusive, they have some common elements.
• B and C are mutually exclusive, they have no elements in common, i.e.
they do not intersect.
Chapter 2 - Conditional Probability and
Independence
Conditional Probability
• A problem is known as a conditional probability problem if there is a
given condition that restricts the sample space.

• Let 𝐴 and 𝐵 be the two events in the sample space S with 𝑃 𝐵 > 0.
The probability that an event 𝐴 occurs given that event 𝐵 has already
occurred is called the conditional probability of 𝐴, given 𝐵 and is
denoted by 𝑃 𝐴Τ𝐵
Mathematical Definition of Conditional
Probability
• Let 𝐴 and 𝐵 be the two events. The conditional probability of 𝐴 given
𝐵 is defined as
𝑃(𝐴∩𝐵)
• 𝑃 𝐴 Τ𝐵 = , 𝑃(𝐵) ≠ 0
𝑃 𝐵
Cont.
• Let us consider three extreme cases, when the probability of event is
0, 1𝑜𝑟 𝑢𝑛𝑑𝑒𝑓𝑖𝑛𝑒𝑑.
• a) If 𝐴 𝑎𝑛𝑑 𝐵 are mutually exclusive events, then 𝐴 ∩ 𝐵 = ∅ and
hence
𝑃(𝐴∩𝐵)
• 𝑃 𝐴/𝐵 = =0
𝑃(𝐵)
• This is, if 𝐴 𝑎𝑛𝑑 𝐵 are disjoint events, then whenever 𝐵 occurs 𝐴
cannot occur and so 𝑃 𝐴/𝐵 = 0
• Are two disjoint events mutually exclusive?
Cont.
𝑃(𝐴∩𝐵) 𝑃(𝐵)
• b) If 𝐵 ⊆ 𝐴, then 𝑃 𝐴/𝐵 = = =1
𝑃(𝐵) 𝑃(𝐵)
• That is, if 𝐵 is contained in 𝐴, then whenever 𝐵 occurs A must
certainly occur.
• c) P A/B is undefined if 𝑃 𝐵 = 0
• Example: A fair die is tossed once
• (a) What is the probability that the number obtained is greater than
2?
• (b) If we are told that the throw resulted in an even number, find the
probability that that number is greater 2.
 Solution
• 𝑆 = 1, 2, 3, 4, 5, 6
• (a) Let 𝐴 denote an event greater than 2
• Then 𝐴 = 3, 4, 5, 6
4 2
• Hence 𝑃 𝐴 = =
6 3
• (b) Let 𝐵 be the event “even number occurred”
• Then 𝐵 = 2, 4, 6
• We are required to calculate 𝑃 𝐴/𝐵 .
3 1
• 𝑃 𝐵 = =
6 2
2 1
• 𝐴 ∩ 𝐵 = 4, 6 , 𝑃 𝐴 ∩ 𝐵 = =
6 3
𝑃(𝐴∩𝐵 1/3 2
• Hence 𝑃 𝐴/𝐵 = = =
𝑃(𝐵) 1/2 3
Multiplication Rule
• If 𝐴 𝑎𝑛𝑑 𝐵are two events in the sample space 𝑆, then the probability
of the joint occurrence of 𝐴 𝑎𝑛𝑑 𝐵 is given by
• 𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐵 𝑃 𝐴 𝐵 𝑖𝑓 𝑃 𝐵 ≠ 0
• = 𝑃 𝐴 𝑃 𝐵 𝐴 𝑖𝑓 𝑃(𝐴) ≠ 0
• The Multiplication rule states that the probability of the simultaneous
occurrence of two events equals the product of the probability of the
first event and the conditional probability of the second event given
that the first event has already occurred.
Example
• If we randomly pick two television tubes in succession from a
shipment of 240 television tubes of which 15 are defective, what is
the probability that they will both be defective?
Solution
• If we assume equal probabilities for each selection (which is what we
mean by “randomly” picking the tubes), the probability that the first
15
tube will be defective is , and the probability that the second tube
240
14
will be defective given that the first tube is defective is . Thus the
239
15 14 7
probability that both tubes will be defective is ∗ = . This
240 239 1912
assumes that we are sampling without replacement, that the first
tube is not replaced before the second tube is selected.
Total Probability
• The addition and multiplication laws are the basic tools of probability.
The application of these laws maybe demonstrated in the total
probability.
• To discuss total probability, we would need the following theorem.
• Theorem 2.2: If 𝐴1 , 𝐴2 , … , 𝐴𝑛 form a partition of the sample space 𝑆,
then for any event 𝐵 ⊆S, 𝑃 𝐵 > 0,
• 𝑃 𝐵 = 𝑃 𝐴1 ∩ 𝐵 + 𝑃 𝐴2 ∩ 𝐵 + ⋯ + 𝑃 𝐴𝑛 ∩ 𝐵 =
σ𝑛𝑖=1 𝑃(𝐴𝑖 ∩ 𝐵)
Theorem 2.3
• Suppose 𝐴1 , 𝐴2 , … , 𝐴𝑛 is a partition of the sample space 𝑆, and 𝐵 is
an event defined on the same sample space 𝑆 such that 𝑃 𝐵 >
0. Then
• 𝑃 𝐵 = σ 𝑃 𝐴𝑖 𝑃 𝐵 𝐴𝑖 = 𝑃 𝐴1 𝑃 𝐵 𝐴1 + 𝑃 𝐴2 𝑃 𝐵 𝐴2 +…+
𝑃 𝐴𝑛 𝑃 𝐵 𝐴𝑛
Example
• A group of visitors at NUL consists of 15 students from Oxford
University and 20 students from Leeds University. Among the
students from Oxford, 8 were females and among the Leeds students
5 were females. A student is selected (at random) to give a note of
thanks at the end of the visit. What is the probability that the student
is a female?
Solution
• Let 𝐹 = 𝑓𝑒𝑚𝑎𝑙𝑒 𝑠𝑡𝑢𝑑𝑒𝑛𝑡
• 𝐻1 = 𝑠𝑡𝑢𝑑𝑒𝑛𝑡 𝑖𝑠 𝑓𝑟𝑜𝑚 𝑂𝑥𝑓𝑜𝑟𝑑 𝑈𝑛𝑖𝑣𝑖𝑟𝑡𝑠𝑖𝑡𝑦
• 𝐻2 = 𝑠𝑡𝑢𝑑𝑒𝑛𝑡 𝑖𝑠 𝑓𝑟𝑜𝑚 𝐿𝑒𝑒𝑑𝑠 𝑈𝑛𝑖𝑣𝑒𝑟𝑠𝑖𝑡𝑦
• Then either the student comes from Oxford and she is a female or she
comes from Leeds and she is a female. This is a union of two disjoint
events which are 𝐻1 ∩ 𝐹 𝑎𝑛𝑑 𝐻2 ∩ 𝐹.
• Hence
• 𝑃 𝐹 = 𝑃 𝐻1 𝐹 𝐻1 + 𝑃 𝐻2 𝐹 𝐻2
15 8 20 5 13
•= + =
35 15 35 20 35
Example
• In a class, 70% of all students are boys and the rest are girls.
• 10% of the girls are smokers
• 20% of the boys are smokers
• Find the probability that a randomly selected student is a smoker.
Solution
• Given:
• P(B) = 0.70, P(G) = 0.30, P(S/G) = 0.10, P(S/B)= 0.20
• Required:
• P(S)
• Using law of total probability,
• P(S) = P(S/B)P(B) + P(S/G)P(G) = 0.17 = 17%
Bayes’ Rule/Theorem/Law
• The name is associated with the famous English statistician Thomas
Bayes (1702 – 1761).
• Bayes’ theorem is a simple mathematical formula used for calculating
special types of conditional probabilities.
• The theorem describes the probability of an event based on prior
knowledge of conditions that might be related to the event
Bayes’ Theorem
Suppose 𝐴1 , 𝐴2 , … , 𝐴𝑛 are events which form a partition of the sample
space 𝑆. Suppose also that the probabilities 𝑃 𝐴𝑖 ≠ 0, (𝑖 = 1, 2, … , 𝑛)
are known. Let 𝐵 be any event in 𝑆 such that 𝑃(𝐵) ≠ 0 and suppose
𝑃( 𝐵 𝐴 is also known. Then
𝑃 𝐴𝑖 𝑃 𝐵 𝐴 𝑖
𝑃 𝐴𝑖 𝐵 = σ
𝑃 𝐴𝑖 𝑃 𝐵 𝐴 𝑖
Example Cont.
• A group of visitors at NUL consists of 15 students from Oxford
University and 20 students from Leeds University. Among the
students from Oxford, 8 were females and among the Leeds students
5 were females. A student is selected (at random) to give a note of
thanks at the end of the visit. Suppose that a female student is
selected. What is the probability that the student is from Leeds
University?
Solution
• The required probability is given by
𝑃 𝐴2 𝑃 𝐵 𝐴 2
𝑃 𝐴2 𝐵 =
𝑃 𝐴1 𝑃 𝐵 𝐴1 +𝑃 𝐴2 𝑃 𝐵 𝐴2
• Using the previous notation in the previous example
𝑃 𝐻2 𝑃 𝐹 𝐻2
𝑃 𝐻2 𝐹 =
𝑃 𝐻1 𝑃 𝐹 𝐻1 +𝑃 𝐻 𝑃 𝐹 𝐻2
20 5
35 20 5
𝑃 𝐻2 𝐹 = 15 8 20 5 =
+ 13
35 15 35 20
Exercise
• Continuation of example under total probability
• Find the probability that the smoking student is a boy.
• Find the probability that the smoking student is a girl.

• Given:
• P(B) = 0.70, P(G) = 0.30, P(S/G) = 0.10, P(S/B) = 0.20
• P(S) = 0.17
Solution
• P(B/S) = P(S/B)P(B)/P(S) = 0.14/0.17 = 0.823
• P(G/S) = P(S/G)P(G)/P(S) = 0.03/0.17 = 0.176
Statistical Independence
• Events 𝐴 𝑎𝑛𝑑 𝐵 on the sample space 𝑆 are said to be independent if
the probability of the joint occurrence of 𝐴 𝑎𝑛𝑑 𝐵 is equal to the
product of their individual probabilities. 𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴 . 𝑃 𝐵 .

• Example: A box contains ten identical balls numbered 1 to 10. A ball is

picked at random with replacement form the box. Consider the
events:
• 𝐴 = 𝑥 𝑥 ≤ 4 𝑥 , 𝐵 = 𝑥 𝑥 𝑖𝑠 𝑒𝑣𝑒𝑛 Are the two events 𝐴 𝑎𝑛𝑑 𝐵
independent?
Solution
• 𝑆 = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 , 𝐴 = 1, 2, 3, 4 , 𝐵 =
2, 4, 6, 8, 10 ; 𝑎𝑛𝑑 𝐴 ∩ 𝐵 = 2, 4
2 1 4 2 5 1
• So 𝑃 𝐴 ∩ 𝐵 = = ,𝑃 𝐴 = = ,𝑃 𝐵 = =
10 5 10 5 10 2
2 5 1
•𝑃 𝐴 𝑃 𝐵 = ∗ = = 𝑃(𝐴 ∩ 𝐵)
5 10 5
• Yes the two events are independent.

• Note: If 𝐴 𝑎𝑛𝑑 𝐵 are independent, then 𝐴 𝑎𝑛𝑑 𝐵 ′ are also

independent.
Definition 1.1
• Events 𝐴1 , 𝐴2 , … , 𝐴𝑘 are independent if and only if (iff) the probability
of their intersection of any 2, 3, …, or k of these events equals the
product of their respective probabilities.
• For any three events 𝐴, 𝐵 𝑎𝑛𝑑 𝐶, for example, independence requires
that
•𝑃 𝐴∩𝐵 =𝑃 𝐴 𝑃 𝐵
• 𝑃 𝐴 ∩ 𝐶 = 𝑃 𝐴 𝑃(𝐶)
• 𝑃 𝐵 ∩ 𝐶 = 𝑃 𝐵 𝑃 𝐶 , and
• 𝑃 𝐴 ∩ 𝐵 ∩ 𝐶 = 𝑃 𝐴 𝑃 𝐵 𝑃(𝐶)
Homework
• A box contains ten identical balls numbered 1 to 10. A ball is picked at
random with replacement form the box. Consider the events:
𝑛(𝑛+1)
• 𝐴 = 𝑥 𝑥 ≤ 4 𝑥 , 𝐵 = 𝑥 𝑥 𝑖𝑠 𝑒𝑣𝑒𝑛 , 𝐶 = 𝑥𝑥= Are
2
the three events 𝐴 𝑎𝑛𝑑 𝐵 independent?
Solution
• 𝑆 = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 , 𝐴 = 1, 2, 3, 4 ,
• 𝐵 = 2, 4, 6, 8, 10 ; 𝐶 = 1, 3, 6, 10 ; 𝑎𝑛𝑑 𝐴 ∩ 𝐵 = 2, 4 ,
• 𝐴 ∩ 𝐶 = 1, 3 , 𝐵 ∩ 𝐶 = 6, 10 , 𝐴 ∩ 𝐵 ∩ 𝐶 = ∅
2 1 4 2 5 1
• So 𝑃 𝐴 ∩ 𝐵 = = ,𝑃 𝐴 = = ,𝑃 𝐵 = =
10 5 10 5 10 2
4
• , P©= ,
10
4 5 4 1
• 𝑃(𝐴 ∩ 𝐵 ∩ 𝐶) =0, but P 𝐴 𝑃 𝐵 𝑃(𝐴) = ∗ * = ≠0
10 10 10 5
• Hence 𝐴, 𝐵, 𝑎𝑛𝑑 𝐶 𝑎𝑟𝑒 𝑛𝑜𝑡 𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡
Note
• (a) Three or more events can be pairwise independent even though
they are not statistically independent.
• (b) If 𝑃 𝐴 ∩ 𝐵 ∩ 𝐶 = 𝑃 𝐴 𝑃 𝐵 𝑃 𝐶 , it does not necessarily imply
that the events 𝐴, 𝐵 𝑎𝑛𝑑 𝐶 are pairwise independent.
• © If 𝐴 is independent of 𝐵, and 𝐴 is independent of 𝐶, then 𝐵 is not
necessarily independent of C.
• (d) If 𝐴 is independent of 𝐵, and 𝐴 is independent of 𝐶, then A is not
necessarily independent of 𝐵 ∪ 𝐶.