LECTURE NO 16:

IN THE FIRST PART OF THIS COURSE,

YOU LEARNT:

• Techniques for summarizing and describing collected data (Descriptive Statistics):

• Univariate situation
• Regression and Correlation (bivariate situation)
IN THIS (SECOND) PART OF THE COURSE,

YOU WILL LEARN:

• Basic Probability Theory

• Discrete Probability Distributions
• Continuous Probability Distributions
As the basic concepts of Probability Theory are very clearly explained through Set Theory, we will begin with a review of basic
set theory. In addition, we will review Counting Rules which are required for solving various problems in probability.

TOPICS OF LECTURE NO 16:

 Set Theory
 Counting Rules:
 The Rule of Multiplication

“SET”:

A set is any well-defined collection or list of distinct objects, e.g. a group of students, the books in a library, the integers
between 1 and 100, all human beings on the earth, etc. The term well-defined here means that any object must be classified
as either belonging or not belonging to the set under consideration, and the term distinct implies that each object must
appear only once. The objects that are in a set, are called members or elements of that set. Sets are usually denoted by
capital letters such as A, B, C, Y, etc., while their elements are represented by small letters such as, a, b, c, y, etc.

Elements are enclosed by parentheses to represent a set.

For example:

Examples of Sets:

A = {a, b, c, d} or

B = {1, 2, 3, 7}

The Number of a set A, written as n(A), is defined as the number of elements in A.

If x is an element of a set A, we write x Î A which is read as “x belongs to A” or x is in A. If x does not belong to A, i.e. x is not an
element of A, we write x Ï A.

A set that has no elements is called an empty or a null set and is denoted by the symbolf. (It must be noted that {0} is not an
empty set as it contains an element 0.)

If a set contains only one element, it is called a unit set or a singleton set.

It is also important to note the difference between an element “x” and a unit set {x}.

A set may be specified in two ways:

1. We may give a list of all the elements of a set (the “Roster” method),
e.g.

A = {1, 3, 5, 7, 9, 11} ;
B = {a book, a city, a clock, a teacher};

2. We may state a rule that enables us to determine whether or not a given object is a member of the set(the “Rule” method
or the “Set Builder” method),
e.g.

A = {x | x is an odd number and x < 12} meaning that A is a set of all elements x such that x is an odd number and x is less than
12. (The vertical line is read as “such that”.)

An important point to note is that:

The repetition or the order in which the elements of a set occur, does not change the nature of the set. The size of a set is
given by the number of elements present in it.
This number may be finite or infinite. Thus a set is finite when it contains a finite number of elements; otherwise it is an
infinite set.
The Empty set is regarded as a Finite set.

Examples of finite sets:

i) A = {1, 2, 3, …, 99, 100};

ii) B = {x | x is a month of

the year};

iii) C = {x | x is a printing

mistake in a book};

iv) D = {x | x is a living citizen

of Pakistan};

Examples of infinite sets:

i) A = {x | x is an even

integer};

ii) B = {x | x is a real number

between 0 and 1 inclusive},

i.e. B = (x | x 0 < x < 1}

iii) C = {x | x is a point on a line};

iv) D = {x | x is a sentence in a

English language}; etc.

SUBSETS:
 A set that consists of some elements of another set, is called a subset of that set.

For example, if B is a subset of A, then every member of set B is also a member of set A.

If B is a subset of A, we write:

B Ì A or equivalently:
AÉB

‘B is a subset of A’ is also read as ‘B is contained in A’,

or ‘A contains B’.

EXAMPLE:

If A = {1, 2, 3, 4, 5, 10}
and B {1, 3, 5}

then B Ì A,
i.e. B is contained in A.

It should be noted that any set is always regarded a subset of itself.

and an empty set f is considered to be a subset of every set.

Two sets A and B are Equal or Identical, if and only if they contain exactly the same elements.

In other words, A = B if and only if A Ì B and B Ì A.

Proper Subset:

If a set B contains some but not all of the elements of another set A, while A contains each element of B, i.e. if

B Ì A and B ¹ A

then the set B is defined to be a proper subset of A.

Universal Set:

The original set of which all the sets we talk about, are subsets, is called the universal set (or the space) and is generally
denoted by S or W.

The universal set thus contains all possible elements under consideration.

A set S with n elements will produce 2n subsets, including S and f.

EXAMPLE;

Consider the set A = {1, 2, 3}.

All possible subsets of this set are:

f, {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3} and {1, 2, 3}.

Hence, there are 23 = 8 subsets of the set A.

VENN DIAGRAM.:

A diagram that is understood to represent sets by circular regions, parts of circular regions or their complements
with respect to a rectangle representing the space S is called a Venn diagram, named after the English logician John Venn
(1834-1923).
The Venn diagrams are used to represent sets and subsets in a pictorial way and to verify the relationship among sets and
subsets.

A Simple Venn diagram:

Disjoint Sets

S B

A
B

OPERATIONS ON SETS:

Let the sets A and B be the subsets of some universal set S. Then these sets may be combined and operated on in
various ways to form new sets which are also subsets of S.

The basic operations are union, intersection, difference and complementation.

UNION OF SETS:

The union or sum of two sets A and B, denoted by A È B, and read as “A union B”, means the set of all elements that
belong to at least one of the sets A and B, that is

A È B = { x | x Î A or x Î B}

By means of a Venn Diagram, A È B is shown by the shaded area as below:

A
B

A B is shaded

Example:

Let A = {1, 2, 3, 4} and B = {3, 4, 5, 6}

Then A È B = {1, 2, 3, 4, 5, 6}

INTERSECTION OF SETS:

The intersection of two sets A and B, denoted by A Ç B, and read as “A intersection B”, means that the set of all
elements that belong to both A and B; that is

A Ç B = {x | x Î and x Î B}.

Diagrammatically, A Ç B is shown by the shaded area as below:

 B
A
S

A B is shaded

Example:

Let A = {1, 2, 3, 4} and B = {3, 4, 5, 6}

Then A Ç B = {3, 4}

The operations of union and intersection that have been defined for two sets may conveniently be extended to any finite
number of sets.

DISJOINT SETS:

Two sets A and B are defined to be disjoint or mutually exclusive or non-overlapping when they have no elements
in common, i.e. when their intersection is an empty set

i.e. A Ç B = f.

On the other hand, two sets A and B are said to be conjoint when the have at least one element in common.

SET DIFFERENCE:

The difference of two sets A and B, denoted by A – B or by A – (A Ç B), is the set of all elements of A which do not
belong to B.

Symbolically,

A – B = {x | x Î A and x Ï B}

It is to be pointed out that in general A – B ¹ B – A.

The shaded area of the following Venn diagram shows the difference A – B:
 B
A
S

Difference A – B is shaded

It is to be noted that A – B and B are disjoint sets. If A and B are disjoint, then the difference A – B coincides with the set A.

COMPLEMENTATION:

The particular difference S – A, that is, the set of all those elements of S which do not belong to A, is called
the complement of A and is denoted by`A or by Ac.

In symbols:

`A = {x | x Î S and s Ï A}

The complement of S is the empty set f.

The complement of A is shown by the shaded portion in the following Venn diagram.

S A B

A is shaded
It should be noted that A – B and A Ç `B, where `B is the complement of set B, are the same set. Next, we consider the
Algebra of Sets. The algebra of sets provides us with laws which can be used to solve many problems in probability
calculations.

Let A, B and C be any subsets of the universal set S. Then, we have:

Commutative laws:

A È B = B È A and

AÇB=BÇA

Associative laws:

(A È B) È C = A È (B È C)

and

(A Ç B) Ç C = A Ç (B Ç C)

Distributive laws

A Ç (B È C) = (A Ç B) È (A Ç C)

and A È (B Ç C) = (A È B) Ç (A È C)

Idempotent laws

A È A = A and

AÇA=A

Identity laws

A È S = S,

A Ç S = A,

A È f = A, and

A Ç f = f.

Complementation laws

A È `A = S,

A `ÇA = f,

(`A ) = A,

`S = f, and

f` = S

De Morgan’s laws:
PARTITION OF SETS:

A partition of a set S is a sub-division of the set into non-empty subsets that are disjoint and exhaustive, i.e.
their union is the set S itself.

This implies that:

i)Ai Ç Aj = f, where i ¹ j;

ii)A1 Ç A2 È … È An = S.

The subsets in a partition are called cells.

EXAMPLE:

Let us consider a set S = {a, b, c, d, e}.

Then {a, b}, and {c, d, e} is a partition of S as each element of S belongs to exactly one cell.

CLASS OF SETS:

A set of sets is called a class. For example, in a set of lines, each line is a set of points.

POWER SET:

The class of ALL subsets of a set A is called the Power Set of A and is denoted by P(A).

For example, if A = {H, T}, then P(A) = {f, {H}, {T}, {H, T}}.

CARTESIAN PRODUCT OF SETS:

The Cartesian product of sets A and B, denoted by A ´ B, (read as “A cross B”), is a set that contains all ordered pairs (x, y),
where x belongs to A and y belongs to B.

Symbolically, we write

A ´ B = {(x, y) | x Î A and y Î B}

This set is also called the Cartesian set of A and B set of A and B, named after the French mathematician Rene’ Descartes
(1596-1605).

The product of a set A by itself is denoted by A2.

This concept of product may be extended to any finite number of sets.

EXAMPLE:

Let A = {H, T} and B = {1, 2, 3, 4, 5, 6}.

Then the Cartesian product set is the collection of the following twelve (2 ´ 6) ordered pairs:

A´B = {(H, 1); (H, 2);(H, 3); (H, 4);(H, 6); (H, 6);(T, 1); (T, 2); (T, 3); (T, 4); (T, 5); (T, 6) }

Clearly, these twelve elements together make up the universal set S when a COIN and a DIE are tossed together.

A die is a cube of wood or ivory whose six faces are marked with dots are shown below:
The plural of the word ‘die’ is ‘dice’.

The product A ´ B may conveniently be found by means of the so-called

tree diagram shown below:

Tree Diagram
A B AB
1(H, 1)
1(H, 2)
1(H, 3)
H 1(H,4 )
1(H, 5)
1(H, 6)
1(T, 1)
2(T, 2)
3(T, 3)
T 4(T, 4)
5(T, 5)
6(T, 6)

TREE DIAGRAM:

The “tree” is constructed from the left to the right. A “tree diagram” is a useful device for enumerating all the
possible outcomes of two or more sequential events.

The possible outcomes are represented by the individual paths or branches of the tree.

It is relevant to note that, in general

A ´ B ¹ B ´ A.
Having reviewed the basics of set theory, let us now review the COUNTING RULES that facilitate the computation of
probabilities in a number of problems.

RULE OF MULTIPLICATION:

If a compound experiment consists of two experiments which that the first experiment has exactly m distinct
outcomes and, if corresponding to each outcome of the first experiment there can be n distinct outcomes of the second
experiment, then the compound experiment has exactly mn outcomes.

EXAMPLE:

The compound experiment of tossing a coin and throwing a die together consists of two experiments:

The coin-tossing experiment consists of two distinct outcomes (H, T), and
the die-throwing experiment consists of six distinct outcomes (1, 2, 3, 4, 5, 6).

The total number of possible distinct outcomes of the compound experiment is therefore 2 ´ 6 = 12 as each of the two
outcomes of the coin-tossing experiment can occur with each of the six outcomes of die-throwing experiment. As stated
earlier, if A = {H, T} and B = {1, 2, 3, 4, 5, 6}, then the Cartesian product set is the collection of the following twelve (2 ´ 6)
ordered pairs:

A´B = { (H, 1); (H, 2);(H, 3); (H, 4);

(H, 5); (H, 6);(T, 1); (T, 2);

(T, 3); (T, 4); (T, 5); (T, 6) }

Tree Diagram
A B AB
1(H, 1)
1(H, 2)
1(H, 3)
H 1(H,4 )
1(H, 5)
1(H, 6)
1(T, 1)
2(T, 2)
3(T, 3)
T 4(T, 4)
5(T, 5)
6(T, 6)

The rule of multiplication can be readily extended to compound experiments consisting of any number of experiments
performed in a given sequence.
This rule can also be called the Multiple Choice Rule, as illustrated by the following example:

EXAMPLE:

Suppose that a restaurant offers three types of soups, four types of sandwiches, and two types of desserts. Then, a
customer can order any one out of 3 ´ 4 ´ 2 = 24 different meals.

2. Basic Principles of Counting

Counting
An efficient way of counting is necessary to handle large masses of statistical data
(e.g. the level of inventory at the end of a given month, or the number of production runs
on a given machine in a 24 hour period, etc.), and for an understanding of probability.

In this section, we shall develop a few counting techniques. Such techniques will enable
us to count the following, without having to list all of the items:

 the number of ways,

 the number of samples, or
 the number of outcomes.

Before we learn some of the basic principles of counting, let's see some of the notation
we'll need.

Number of Outcomes of an Event

As an example, we may have an event E  defined as

E = "day of the week"

We write the "number of outcomes of event E" as n(E).

So in the example,

n(E)=7,
since there are 7 days in the week.
Addition Rule

Let E1 and E2 be mutually exclusive events (i.e. there are no common outcomes).

Let event E describe the situation where either event E1 or event E2 will occur.

The number of times event E will occur can be given by the expression:

n(E) = n(E1) + n(E2)

where

n(E) = Number of outcomes of event E

n(E1) = Number of outcomes of event E1

n(E2) = Number of outcomes of event E2

Example 1

Consider a set of numbers S={−4,−2,1,3,5,6,7,8,9,10}

Let the events E1, E2 and E3 be defined as:

E = choosing a negative or an odd number from S;

E1= choosing a negative number from S;

E2 = choosing an odd number from S.

Find n(E).

Solution:

E  and E  are mutually exclusive events (i.e. no common outcomes).

1 2

n(E) = n(E1) + n(E2)
=2+5
 =7

Example 2

In how many ways can a number be chosen from 1 to 22 such that

(a) it is a multiple of 3 or 8?

(b) it is a multiple of 2 or 3?

Solution:

Part (a)

Here, E1 = multiples of 3:

E1 = {3, 6, 9,12, 15, 18, 21}

n(E1) = 7

E  = multiples of 8:
2

E2 = {8, 16}

n(E2) = 2

Events E1 and E2 are mutually exclusive.

n(E) = n(E1) + n(E2) = 7 + 2 = 9

Part (b)

Here, E1 = multiples of 2:

E1 = {2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22}

n(E1) = 11

E  = multiples of 3:
2
E2 = {3, 6, 9,12, 15, 18, 21}
n(E2) = 7

Events E1 and E2 are not mutually exclusive.

We could proceed as follows:

n(E) = n(E1) + n(E2) − n(E1 ∩ E2) = 11 + 7 − 3 = 15

where E1 ∩ E2 means "the intersection of the sets E1 and E2".

Multiplication Rule

Now consider the case when two events E1 and E2 are to be performed and the
events E1 and E2 are independent events i.e. one does not affect the other's outcome.

[We see more on independent events later in this chapter.]

Example 3

Say the only clean clothes you've got are 2 t-shirts and 4 pairs of jeans. How many
different combinations can you choose?

Solution:

We have 2 t-shirts and with each t-shirt we could pick 4 pairs of jeans. Altogether there
are

2×4=8 possible combinations.
We could write

E1 = "choose t-shirt" and

E2 = "choose jeans"

Multiplication Rule in General

Suppose that event E1 can result in any one of n(E1) possible outcomes; and for each
outcome of the event E1, there are n(E2) possible outcomes of event E2.
Together there will be n(E1) × n(E2) possible outcomes of the two events

Tip:

In counting and probability, "AND" usually requires us to MULTIPLY.

That is, if event E is the event that both E1 and E2 must occur, then

n(E) = n(E1) × n(E2)

In our example above,

n(E1) = 2 (since we had 2 t-shirts)

n(E2) = 4 (since there were 4 pairs of jeans)

So total number of possible outcomes is given by:

n(E) = n(E1) × n(E2) = 2 × 4 = 8

Example 4

What is the total number of possible outcomes when a pair of coins is tossed?

Solution:

The events are described as:

E1 = toss first coin (2 outcomes, so n(E1) = 2.)

E2 = toss second coin (2 outcomes, so n(E2) = 2.)

They are independent, since neither toss affects the outcome of the other toss.

So n(E) = n(E1) × n(E2) = 2 × 2 = 4

[We could list the outcomes: HH HT TH TT].

Example 5

The life insurance policies of an insurance company are classified by:

 age of the insured:

 under 25 years,
 between 25 years and 50 years,
 over 50 years old;
 Gender;
 marital status:
 single or
 married.

What is the total number of classifications?

Solution:

The events are described as:

E1 = age of the insured: 3 age divisions, so n(E1) = 3.

E2 = Gender: 2 possibilities, so n(E2) = 2.

E3 = marital status: 2 possibilities, so n(E3) = 2.

Each event is independent, so

n(E) = n(E1) × n(E2) × n(E3) = 3 × 2 × 2 = 12

Example 6

For our clothes problem above, say we found 3 caps that we could wear with our 2 t-
shirts and 4 pairs of jeans. How many different combinations could we choose from
now?

Solution:

We have 2 choices in the first row, 4 in the second row and 3 in the third row. Together,
we will have

n(E) = n(E1) × n(E2) = 2 × 4 × 3 = 24 combinations