Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Probability Theory

Dr. Sthitadhi Das

Department of Mathematics, Brainware University
West Bengal, Kolkata - 700125
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Introduction
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

What is probability?

Probability is a measure of chance that a particular incident

will happen, e.g . whether head will turn up after tossing a
coin, whether one shall get ‘4’ after rolling a die once,
whether the batsman A will score a century today or not, etc.
Probability can also be treated as the degree of belief of a
particular statement or assertion, e.g .
- Team India will win the next Cricket match against Australia,
- there will be no rainfall today,
- a green ball will be drawn at random from a bucket of balls
with different colours.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Applications of probability

Probability theory is the basic ground of Statistics.

It is useful in statistical modelling, drawing inferences about
the population or a part of the population under study by
conducting random experiment(s) etc.
Probability theory has vast application in the fields like
business, healthcare, finance, agriculture, meteorology etc.

Defining probability is the main aspect in this

session.

Before going to define probability, the elementary ideas on set

theory needs to be discussed in brief.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Elementary ideas on set theory

Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Definition of set

Set theory is the sole basis of probability theory.

A set is a collection of distinct objects which satisfy

some well specified property/properties.
For example, the set of first 10 prime numbers is
P∗ = {1, 3, 5, 7, 11, 13, 17, 19, 23, 29}.
A set of five flowers is
F = {sunflower , rose, lotus, marigold, tulip}.
A set can be expressed in either roster method or set-builder
method.
- The sets P∗ and F are expressed by roster method.
- Examples of sets expressed by set-builder method are
S = {x : x 2 − 8x − 3 = 0}, M = {y : y is a fruit} etc.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Types of set

One can classify sets into two types, viz., finite set and
infinite set.
- A set having finite number of elements is called a finite set.
For example, a set of first 25 perfect square numbers i.e.
A = {1, 4, 9, . . . , 625}.
- A set having infinite number of elements is termed as infinite
set. For example, B = {j: j is a natural number}.
Each of both finite and infinite sets are further categorized
into two types; (i) countable set, (ii) uncountable set.
- A set is countable when it has one to one correspondence with
the set of natural numbers. For example,
A = {k : k is a positive integer} is a countable set.
- A set which does not satisfy the definition of a countable set is
called √
an uncountable set. For example,
B = { x : x is not a perfect square number} is an
uncountable set.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Operations on sets

Union
S of sets: Union of two arbitrary sets A1 and A2 is
A1 A2 = {u : u belongs to either A1 or A S2 }. Similarly,
S
union of k sets A1 , . . . , Ak is defined as A1 . . . Ak whose
elements are the elements S of either A1 or A2 or . . . Ak .
For example, {8, 6, 3} {−5, 6, −7, 3} = {8, 6, 3, −5, −7}.
Intersection
T of sets: Intersection of two arbitrary sets A1 and
A2 is A1 A2 = {u : u belongs to both A1 andTA2 }.T
Similarly, intersection of k sets A1 , . . . , Ak is A1 . . . Ak
whose elements are theTcommon elements of A1 , . . . , Ak .
For example, {8, 6, 3} {−5, 6, −7, 3} = {6, 3}.
Difference of two sets: The difference of two arbitrary sets C
and D is C \D = {e : e is in C but not in D}. For example,
{8, 6, 3}\{−5, 6, −7, 3} = {8}.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Operations on sets

Universal set & Null set: An universal set is a set containing

all the elements related to an experiment. Hence, it is a set by
experimenter’s choice. For example, the set of positive real
numbers R+ = {u : u > 0 and u is a real number} can be
chosen as an universal set by an experimenter, whereas
another experimenter can take
C = {z : z is a complex number} as his/her universal set.
On the other hand, a null set is the set having no element,
generally denoted as {} or ϕ.
Complement of a set: The complement of a set S is defined as
S c = {r : r is not an element of S}. If O is the universal set
of consideration, then S c = O\ S. For example, if
O = {x : x is an even number} and S = {0, 2, 4, 6, 10, 12},
then S c = {8, 14, 16, . . .}.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Properties of set
[ [ \ \
Commuativity: A B=B A, A B=B A.
[  [   [ [
Associativity: A B C = A B C,
\  \   \ \
A B C = A B C.
[  \   [ \  [ 
Distributivity: A B C = A B A C ,
\  [   \ [  \ 
A B C = A B A C .
[ \
Idempotency: A A = A, A A = A.
\
A\ B = A Bc.
If A ⊂ B, then B c ⊂ Ac .
[ [ \ \
A Ω = A, A ϕ = A; A Ω = A, A ϕ = ϕ.
 \ [ \ 
For any two sets A and B, A = A B A Bc .
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Properties of set

De Morgan’s first law:

 [ c \
(i) For two sets A and B, A B = Ac Bc .
k
!c k
!
[ \
c
(ii) For k arbitrary sets A1 , . . . , Ak , Ai = Ai .
i=1 i=1
De Morgan’s second law:
 \ c [
(i) For two sets A and B, A B = Ac Bc .
k
!c k
!
\ [
c
(ii) For k arbitrary sets A1 , . . . , Ak , Ai = Ai .
i=1 i=1
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Random experiment
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

What is random experiment?

An experiment, whose outcomes are already known but

cannot be predicted beforehand during its performance, is
called a random experiment.
By tossing a coin, one can get one of the two possible
outcomes, namely ‘Head’ (H) and ‘Tail’ (T), but it cannot be
asserted beforehand that either H or T will turn up.
A random experiment can be performed repeatedly under
similar conditions.
The relative frequency of an outcome of random experiment
converges to its probability as the number of repetitions is
large enough. For example, after 1000 tossing of a coin, the
relative frequency of H is expected to be very close to its
actual probability.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Terminologies related to probability theory

Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Terminologies

Sample space: The set of all possible outcomes of a random

experiment E constitutes the sample space of E. For example,
S = {H, T } is the sample space for “Tossing a coin”.
Trial: A typical random experiment which after performance
yields one of two possible results; either success or failure. For
example, if H occurs after tossing a coin then it is labelled as
“success” and if T appears then it is labelled as “failure”.
Event: A subset of the sample space S is called an event. For
example, tossing a coin twice at a time has the sample space
Ω = {HH, HT , TH, TT }. The set Ω1 = {HT , TT } = sample
space of “at most one head at first draw and no head in
second draw” is a subset of Ω and is an event therefore.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Terminologies

Mutually exclusive events: Suppose A1 , A2 , . . . , An be n

events
T of a random experiment E with sample space S. If
Ai Aj for all i ̸= j = 1, . . . , n i.e. the events are mutually
disjoint to each other, then all the n events are called
mutually exclusive events for E .
Exhaustive events: S TheSevents S A1 , . . . , An are called mutually
exhaustive if A1 A2 . . . An = Ω, i.e. at least one of the
events A1 , A2 , . . . , An will occur whenever the experiment E
is performed.
Equally likely events: The events of a random experiment
which are equally probable to occur, i.e. same probability of
occurrence of those events are observed are said to be equally
likely events.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Classical definition of probability

Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Definition

If E be a random experiment with sample space Ω, then the

probability of an event A is defined as

n(A)
P(A) =
n(Ω)
where n(A) denotes the number of elements in A and n(Ω) is
the number of elements in Ω.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Example

Suppose a die is thrown once to check its face value. The

sample space associated with the experiment is
Ω = {1, 2, 3, 4, 5, 6}. Let us consider an event A such that

A = “ 5 will not occur ”

and it can be noted that A = {1, 2, 3, 4, 6}. Then, by the

definition of classical probability, one can compute the
probability of A as

n(A) 5
P(A) = = .
n(Ω) 6
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Limitations

The classical definition of probability assumes all the events of

E to be equally likely. However, the definition does not work
when the events are not equally likely.
When the number of all possible cases is too high, the
definition breaks down.
An improvement on the classical definition of probability can
be done by considering relative frequency of an event A,
known as empirical definition of probability, discussed next.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Empirical definition of probability

Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Definition

Suppose a random experiment E is repeatedly conducted

under similar circumstances a large number of times, say N,
and N(A) be the frequency of event A in this process. Then,
the probability of occurrence of A is defined as

N(A)
lim
N→∞ N
provided that the above limit exists.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Example

Suppose a coin with sample space Ω = H, T is tossed 1 time,

40 times, 100 times, 500 times and 1000 times successively
and the number of tails (T) after each experimentation are
recorded as n1 (T ) = 0, n2 (T ) = 14, n3 (T ) = 42,
n4 (T ) = 218, n5 (T ) = 407 respectively. Then, the relative
frequencies of occurrence of tails are calculated as
0 14 42
f1 (T ) = = 0, f2 (T ) = = 0.35, f3 (T ) = = 0.42
1 40 100
218 407
f4 (T ) = = 0.436, f5 (T ) = = 0.407.
500 1000
As the number of tossing the coin increases, the relative
frequency of occurrence of tail gets stabilized to 0.4. So, it
can be said that the probability of getting a tail is 0.4.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Limitations

Since the number of conducting the experiment becomes high,

the assumption of similar circumstances may be violated.
N(A)
The limit lim may not exist always.
N→∞ N
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Axiomatic definition of probability

Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Algebra of set

Prior to provide the axiomatic definition of probability

measure, one should know about the algebra involving sets.
Class of sets: A collection of sets which is closed under various
set operations (e.g. union, intersection, complementation
etc.) is called a class of sets.
Field: A non-empty class of sets C, which is
1 closed under complementation, i.e. if A ∈ C then Ac ∈ C.
2 closed under finite union (or, intersection), i.e. if
k
[ k
\
A1 , . . . , Ak ∈ C then Ai (or , Ai ) ∈ C.
i=1 i=1
For example, S1 = {ϕ, Ω}, S2 = {ϕ, A, Ac , Ω} are two fields,
where Ω is the universal set.
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Algebra of set

σ-Field: A non-empty class of sets C, which is

1 closed under complementation, i.e. if A ∈ C then Ac ∈ C.
2 closed under countable[ union (or, intersection),
\ i.e. if
A1 , A2 , . . . ∈ C then Ai (or , Ai ) ∈ C.
i=1,2,... i=1,2,...
Suppose A be an arbitrary class of sets. Then, the σ-field
generated by A, denoted as σ(A), has the following definition
[
σ(A) = {A1 , A2 , . . . : Ai ’s ∈ A, Ai ∈ A, Aci ∈ A ∀ i = 1, 2, . . .}.
i

If σ1 (A), σ2 (A), . . . be the σ-fields generated by the class of

sets A, then\ the minimal σ-field generated by A is
σ0 (A) = σi (A).
i=1,2,...
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Definition

A probability measure P, defined on σ-algebra (or, field)

generated by the class of subsets of Ω denoted by A, satisfies
the following three properties:
(a) P(ϕ) = 0,
(b) P(Ω) = 1,
(c) If {An }∞
n=1 be a sequence of mutually disjoint events in A, then

∞
[ ∞
X
P( Ai ) = P(Ai ).
i=1 i=1
Introduction Elementary ideas on set theory Random experiment Terminologies related to probability theory Classical definition of

Thank you