Theory of Probability

Sample Spaces and Events

A set S of outcomes (events) of an experiment is called a sample space for the experiment.

Each element in S is called simple outcome, or, simple event.

An event E is defined to be any subset of S (including the empty set and the sample space set S).
Event E is a simple event if it contains only one element and a compound event if it contains
more than one element.

We say that an event E occurs if any of the simple events in E occurs.

Probability of an Event
Given a simple space
S = {e1 , e2 , · · · , en }
with n simple events, to each simple event ei we assign a real number, denoted by P (ei ), called
the probability of the event ei .

These numbers can be assigned in a arbitrary manner as long as the following two conditions are
satisfied:

1. The probability of a simple event is a number between 0 and 1, inclusive. That is,

0 ≤ P (ei ) ≤ 1

2. The sum of the probabilities of all simple events in the sample space is 1. That is,

P (e1 ) + P (e2 ) + · · · + P (en ) = 1

Any probability assignment that meets conditions 1 and 2 is said to be an acceptable proba-
bility assignment.

Probability of an Event E

Given an acceptable probability assignment for the simple events in a sample space, S, we define
the probability of an arbitrary event E, denoted by P (E), as follows:

(A) If E is the empty set, then P (E) = 0.

(B) If E is a simple event, then P (E) is already been assigned,
(C) If E is a compound event, then P (E) is the sum of the probabilities of all the simple events
in E.
(D) If E is the sample space, then P (E) = P (S) = 1.
Empirical Probability Approximation

Empirical probability is based on the concept of relative frequency.

If an experiment is conducted n times and event E occurs with frequency f (E), then the ratio
f (E)/n is called relative frequency of the occurrence of event E in n trials.

The empirical probability of E, denoted by P (E), by the number (if it exists) by the relative
frequency.
Frequency of occurrence of n f (E)
P (E) = =
Total number of trials n

Equally Likely Assumption

By assigning the same probability to each of simple event in S, we are actually making the as-
sumption that each simple event is likely to occur as any other. This is referred to as an equally
likely assumption.

Probability of a Simple Event under an Equally Likely Assumption

If, in a sample space

S = {e1 , e2 , · · · , en }
with n elements, we assume each simple event ei is as likely to occur as any other, then we assign
the probability 1/n to each. That is,
1
P (ei ) =
n

Probability of a Arbitrary Event under an Equally Likely Assumption

If we assume each simple event in sample space S is as likely to occur as any other, then the
probability of an event E in S is given by
Number of elements in E n(E)
P (E) = =
Number of elements in S n(S)

Union, Intersection, and Complement of Events

The union of two events A and B is the event that occurs if either A or B or both occur on a
single performance of the experiment.

The union of events A and B is denoted by the symbol A ∪ B.

A ∪ B consists of all the sample points that belong to A or B or both (Figs. 1a and 1d).

A ∪ B = {e ∈ S | e ∈ A or e ∈ B}
The intersection of two events A and B is the event that occurs if both A and B occur on a
single performance of the experiment.

The intersection of events A and B is denoted by the symbol A ∩ B.

A ∩ B consists of all the sample points that belong to A and B (Figs. 1b and 1c).

A ∩ B = {e ∈ S | e ∈ A and e ∈ B}

Example

Application - Survey and Marketing

The results (in percentage) from electronic survey for marketing new product on ebay are given
in Table I. Find (i). P (A) (ii). P (B) (iii). P (A ∪ B) and (iv). P (A ∩ B) given that

Table 1: Response to electronic survey

Age (years) Income
< $25, 000 $25, 000 − $50, 000 > $50, 000
< 30 5% 12% 10%
30 − 50 14% 22% 16%
> 50 8% 10% 3%

A : {A respondent’s income is more than $50,000}

B : {A respondent’s age is 30 or more}

Solution

The data is characterized in terms of age and income of the respondents to the ebay survey. Thus
the experiment consists of selecting a respondent from collection of all respondents and observing
which income and age group he/she occupies.

The sample events are the six different age - income classification:
E1 : {< 30yrs, < $25, 000}, E4 : {< 30yrs, $25, 000 − $50, 000}
E2 : {30 − 50yrs, < $25, 000}, E5 : {30 − 50yrs, $25, 000 − $50, 000}
E3 : {> 50yrs, < $25, 000}, E6 : {> 50yrs, $25, 000 − $50, 000}
E7 : {< 30yrs, > $50, 000}, E8 : {30 − 50yrs, > $50, 000}
E9 : {> 50yrs, > $50, 000},
Assign the probabilities to the sample points. Randomly selecting one of the respondents, the
probability that he/she will occupy a particular age-income classification is just the relative fre-
quency (proportion) of the respondent in the classification. Therefore the corresponding proba-
bilities are
P (E1 ) = 0.05, P (E2 ) = 0.14
 P (E3 ) = 0.08 P (E4 ) = 0.12
P (E5 ) = 0.22 P (E6 ) = 0.10
P (E7 ) = 0.10, P (E8 ) = 0.16
P (E9 ) = 0.03

Since the event A consists of income classification of more than ($50, 000) in all age classifications,
the probability of A is the sum of the probabilities of the sample points in A:

P (A) = P (E7 ) + P (E8 ) + P (E9 ) = 0.10 + 0.16 + 0.03 = 0.29

Similarly B = (≥ 30)yrs consists

P (B) = P (E2 ) + P (E3 ) + P (E5 ) + P (E6 ) + P (E8 ) + P (E9 )

= 0.14 + 0.08 + 0.22 + .10 + 0.16 + 0.03 = 0.73

The union of A and B consists of all respondents whose income exceeds $50, 000 or whose age is
30 or more.

P (A ∪ B) = 0.10 + 0.14 + 0.22 + 0.16 + 0.08 + 0.10 + 0.03 = 0.83

The intersection of events A and B consists of all sample points in both A and B.

P (A ∩ B) = 0.16 + 0.03 = 0.19

Complement of an Event
Consider a finite sample space
S = {e1 , e2 , · · · , en }
0
that is divided into two subsets E and E such that
0
E∩E =Ø
0
that is, E and E are mutually exclusive, and
0
E∪E =S
0 0
Then E is called the complement of E relative to S. Thus E contains all the elements of S
that are not in E. Also,
0
P (S) = P (E ∪ E )
0
= P (E) + P (E )
= 1

Thus
0 0
P (E) = 1 − P (E ), P (E ) = 1 − P (E)

Example - Birthday Problem

In a group on n people, what is the probability that at least 2 people have the same birthday (the
same month and day, excluding February 29)?

Solution
We will assume that the simple events in S are equally likely (i.e., that for any person in the
group, any birthday is as likely as any other).

Since any person could have any one of 365 birthdays (excluding February 29), the multiplication
principle implies that the number of simple events in S is
n(S) = 365
|{z} · 365
|{z} ··· 365
|{z} ··· 365
|{z}
Ist person 2nd person 2nd person nth person
= 365n

0
Now, let E be the event that at least 2 people in the group have the same birthday. Then E is
the event that no 2 people have the same birthday. The multiplication principle also can be used
0
to determine the number of simple events in E :
Ist person 2nd person 2nd person nth person
0
z}|{ z}|{ z}|{ z }| {
n(E ) = 365 · 364 · 363 · · · (366 − n)
[365 · 364 · 363 · · · (366 − n)](365 − n)!
=
(365 − n)!
365!
=
365 − n
Since we have assumed that S is an equally likely sample space,
0
0 n(E )
P (E ) =
n(S)
365!
(365−n)!
=
365n
365!
=
365n (365 − n)!
Thus
0
P (E) = 1 − P (E )
365!
= 1−
365n (365 − n)!
This equation is valid for any n satisfying 1 ≤ n ≤ 365. For example, in a group of 6 people,
365!
P (E) = 1 −
3656 (359)!
= 0.04
It is interesting to note that as the size of the group increases, P (E) increases more rapidly than
one might expect.
Rules of Probability

In the study of probability it is often necessary to combine the probabilities of events. This is
accomplished through both rules of addition and rules of multiplication. There are two rules for
addition,
⇒ the special rule of addition and
⇒ the general rule of addition.
Special Rule of Addition

The special rule of addition states that the probability of the event A or the event B occurring is
equal to the probability of event A plus the probability of event B. The rule is expressed by

P (A or B) = P (A) + P (B)

• To apply the special rule of addition the events must be mutually exclusive. This means that
when one event occurs none of the other events can occur at the same time (Fig. 2b).

General Rule of Addition

What if the events are not mutually exclusive? In that case the general rule of addition is used.
The probability is computed using the formula

P (A or B) = P (A) + P (B) − P (A and B)

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

For example, a study showed 15 percent of the work force to be unemployed, 20 percent of the
work force to be minorities, and 5 percent to be both unemployed and minorities. What percent of
the work force are either minorities or unemployed? Note that if P (unemployed) and p (minority)
are totaled, the 5 percent who are both minorities and unemployed are counted in both group C
that is, they are double-counted. They must be subtracted to avoid this double counting. Hence,

P(unemployed or minority) = P(unemployed) + P (minority)

−P(unemployed and minority)
= 0.15 + 0.20 − 0.05 = 0.30

These two events are not mutually exclusive (Fig. 2a).

Corollarry

If A1 , A2 , · · · An are mutually exclusive, then

P (A1 ∪ A2 ∪ · · · ∪ An ) = P (A1) + P (A2) + · · · + P (An ).

If A1 , A2 , · · · An is a partition of a sample space S, then

P (A1 ∪ A2 ∪ · · · ∪ An ) = P (A1 ) + P (A2 ) + · · · + P (An )

= P (S)
= 1
Theorem
For three events A, B and C,

P (A ∪ B ∪ C) = P (A) + P (B) + P (C) − P (A ∩ B) − P (A ∩ C)

−P (B ∩ C) + P (A ∩ b ∩ C)

• When we want to find the probability that two events will both happen, we use the concept
known as joint probability.

Joint probability: A probability that measures the likelihood that two or more events will happen
concurrently (at the same time).