Probability Measure and Probability Space

Md Aktar Ul Karim
Probability - Theory and Applications
Symbiosis Statistical Institute, Pune

Contents

1 Introduction to Sigma Fields 1

2 Probability Measure 2
2.1 Properties of a Probability Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Solved Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 Continuity of probability measure: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3 Probability Space 14
3.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.2 Properties of a Probability Space: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.3 Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.4 Solved Problems: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.5 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

1. Introduction to Sigma Fields

A sigma field (or σ field) is a collection of sets that is closed under certain operations. It forms the foundation
for probability theory, as it determines the events for which we can assign probabilities.
Definition
Let Ω be a set. A collection F of subsets of Ω is called a sigma field if:

• Ω, ∅ ∈ F (The entire space is in the sigma field).

• If A ∈ F, then Ac ∈ F (Closed under complements).

S∞
• If A1 , A2 , · · · ∈ F, then i=1 Ai ∈ F (Closed under countable unions).

By De Morgan’s laws, a sigma field is closed under countable intersections.

 Discrete Sigma Field
Let Ω = {1, 2, 3}. The discrete sigma field on Ω is the power set of Ω, denoted as:

F = {∅, {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, Ω}

This sigma field contains all the subsets of Ω.

Trivial Sigma Field
For the same Ω = {1, 2, 3}, the trivial sigma field is:

F = {∅, Ω}

Sigma Field Generated by an Interval Consider Ω = R and the interval A = (0, 1]. Then we can
generate a sigma field by A and will be denoted by σ(A), which contains all sets that can be formed using A
and the operations in the sigma field definition.
In this case, it includes A, Ac , ∅, and R.

2. Probability Measure

A probability measure is a function P : F → [0, 1] on a sigma field F that satisfies the following axioms:

• Non-negativity: P (A) ≥ 0 for all A ∈ F.

• Normalization: P (Ω) = 1.

• Countable Additivity: For any disjoint sets A1 , A2 , · · · ∈ F,

∞ ∞
!
[ X
P Ai = P (Ai ).
i=1 i=1

2.1. Properties of a Probability Measure

• Monotonicity: If A ⊆ B, then P (A) ≤ P (B).

Proof: Let B = A ∪ (B \ A), and since A ∩ (B \ A) = ∅, by additivity:

P (B) = P (A) + P (B \ A)

Since P (B \ A) ≥ 0, we have P (A) ≤ P (B).

• For any events A1 , A2 , . . . , An , not necessarily disjoint:

n n
!
[ X
P Ai ≤ P (Ai )
i=1 i=1

2
 This follows from countable additivity.
Proof: We prove this using mathematical induction. For n = 1, the statement is trivially true because:

P (A1 ) = P (A1 )

Assume the inequality holds for n = k, i.e.,

k k
!
[ X
P Ai ≤ P (Ai )
i=1 i=1

Now, we need to prove that the inequality holds for n = k + 1. Consider:

k+1 k
! ! !
[ [
P Ai = P Ai ∪ Ak+1
i=1 i=1

Using the sub-additivity property of probability, we know that:

k k
! ! !
[ [
P Ai ∪ Ak+1 ≤ P Ai + P (Ak+1 )
i=1 i=1

By the inductive hypothesis:

k k
!
[ X
P Ai ≤ P (Ai )
i=1 i=1
Therefore, we have:
k+1 k k+1
!
[ X X
P Ai ≤ P (Ai ) + P (Ak+1 ) = P (Ai )
i=1 i=1 i=1
Thus, by induction, the inequality holds for any finite n. Therefore, the inequality
n n
!
[ X
P Ai ≤ P (Ai )
i=1 i=1

2.2. Solved Problems

1. Let Ω = {1, 2, 3}, and suppose P ({1}) = 0.3, P ({2}) = 0.4, and P ({3}) = 0.3. Show that for A = {1, 2}
and B = {2, 3}, P (A ∪ B) = P (A) + P (B) − P (A ∩ B).

Ans: We have:

P (A ∪ B) = P ({1, 2, 3}) = 1, P (A) = P ({1, 2}) = 0.7, P (B) = P ({2, 3}) = 0.7

Since A ∩ B = {2}, we subtract P ({2}) = 0.4:

P (A ∪ B) = 0.7 + 0.7 − 0.4 = 1

Hence, additivity holds.

3
2. Let Ω = [0, 1], and define a sequence of sets An = [1/n, 1]. Compute limn→∞ P (An ) assuming P is the
Lebesgue measure.
1
Sol: The probability of An = [1/n, 1] is the length of the interval, P (An ) = 1 − n. As n → ∞,
P (An ) → 1 − 0 = 1. Therefore, limn→∞ P (An ) = 1.

3. Let P be a probability measure defined on Ω = {1, 2, 3, 4, 5, 6} and F = 2Ω is the power set of Ω, and P
is a probability measure defined as follows:

P ({1}) = 0.1, P ({2}) = 0.2, P ({3}) = 0.1, P ({4}) = 0.2, P ({5}) = 0.1, P ({6}) = 0.3

Verify the following properties of the probability measure P :

(a) P (Ω) = 1.

(b) P (∅) = 0.

(c) For disjoint events A = {1, 2, 3} and B = {4, 5}, verify that P (A ∪ B) = P (A) + P (B).

Ans: (a.) Verify P (Ω) = 1:

Ω = {1, 2, 3, 4, 5, 6} is the entire set, and the value of P for each outcome is given.

We need to check for P (ω) = 1

P (Ω) = P ({1}) + P ({2}) + P ({3}) + P ({4}) + P ({5}) + P ({6})

Substitute the given values:

P (Ω) = 0.1 + 0.2 + 0.1 + 0.2 + 0.1 + 0.3 = 1.0

Since P (Ω) = 1, the first property holds.

(b.) Verify P (∅) = 0:

By definition, a probability measure assigns zero probability to the empty set ∅. So,

P (∅) = 0

(c.) Here, sets A and B are disjoint, meaning A ∩ B = ∅. So, we need to prove that:

P (A ∪ B) = P (A) + P (B)

P (A) = P ({1, 2, 3}) = P ({1}) + P ({2}) + P ({3}) = 0.1 + 0.2 + 0.1 = 0.4

4
 P (B) = P ({4, 5}) = P ({4}) + P ({5}) = 0.2 + 0.1 = 0.3

Now, A ∪ B = {1, 2, 3, 4, 5}, so:

P (A ∪ B) = P ({1}) + P ({2}) + P ({3}) + P ({4}) + P ({5}) = 0.1 + 0.2 + 0.1 + 0.2 + 0.1 = 0.7

Therefore,
P (A) + P (B) = 0.4 + 0.3 = 0.7 = P (A ∪ B)

4. Let Ω = [0, 1] be the unit interval and F the Borel sigma algebra B([0, 1]). Define a probability measure
P on (Ω, F) such that for any interval [a, b] ⊆ [0, 1],

P ([a, b]) = b − a.

(a) Verify that P is a valid probability measure by checking the following properties:

i. P (Ω) = 1.

ii. For any disjoint intervals A = [a1 , b1 ] and B = [a2 , b2 ], verify that P (A ∪ B) = P (A) + P (B).

(b) Show that P ([0, 0.5]) = 0.5 and P ([0.25, 0.75]) = 0.5.

Ans: (a.) To verify that P is a valid probability measure we have to show that P (Ω) = 1.

The sample space Ω = [0, 1], and we are given that for any interval [a, b] ⊆ [0, 1],

P ([a, b]) = b − a

. For Ω = [0, 1], we have a = 0 and b = 1, so:

P (Ω) = P ([0, 1]) = 1 − 0 = 1.

Thus, P (Ω) = 1

(b.) Let A = [a1 , b1 ] and B = [a2 , b2 ] be two disjoint intervals such that A ∩ B = ∅. We have to show that:

P (A ∪ B) = P (A) + P (B).

Since the intervals A and B are disjoint, the union A ∪ B is another interval. If we consider A = [a1 , b1 ]
and B = [a2 , b2 ] where b1 < a2 , then A ∪ B = [a1 , b1 ] ∪ [a2 , b2 ]

By the definition of P , we know that:

P ([a, b]) = b − a.

5
 Now,
P (A) = P ([a1 , b1 ]) = b1 − a1 , P (B) = P ([a2 , b2 ]) = b2 − a2

Then
P (A) + P (B) = (b1 − a1 ) + (b2 − a2 ).

Since b1 < a2 , the total length of the union is:

P (A ∪ B) = P ([a1 , b1 ] ∪ [a2 , b2 ])

Since A ∩ B = ∅ the total length will be

P (A ∪ B) = (b1 − a1 ) + (b2 − a2 ).

Thus, P (A ∪ B) = P (A) + P (B).

Now, we have to show that P ([0, 0.5]) = 0.5 and P ([0.25, 0.75]) = 0.5.

We are given P ([a, b]) = b − a. For [0, 0.5], we have:

P ([0, 0.5]) = 0.5 − 0 = 0.5.

Similarly, for the interval [0.25, 0.75], we have:

P ([0.25, 0.75]) = 0.75 − 0.25 = 0.5.

Thus, P ([0, 0.5]) = 0.5 and P ([0.25, 0.75]) = 0.5.

5. Let Ω = N be the set of natural numbers and F = 2Ω be the sigma algebra of all subsets of N. Consider
the probability measure P defined as:

1
P ({n}) = , for all n ∈ N.
2n

(a) Show that P (Ω) = 1.

(b) Prove that P is countably additive, i.e., if A1 , A2 , . . . are disjoint sets in F, then P ( ∞
S
n=1 An ) =
P∞
n=1 P (An ).

(c) Calculate P (A) where A = {2, 3, 5, 7}.

Ans: (a.) We are given that Ω = N, and for each n ∈ N, the probability of the singleton set {n} is given
by:
1
P ({n}) = .
2n

6
To show that P (Ω) = 1, we need to sum the probabilities of all singleton sets in N:
∞ ∞
X X 1
P (Ω) = P ({1}) + P ({2}) + P ({3}) + · · · = P ({n}) = .
2n
n=1 n=1
P∞ 1 1
The series n=1 2n is a geometric series with the first term a = 2 and the common ratio r = 12 . The sum
of an infinite geometric series with |r| < 1 is given by:
∞
X a
arn−1 = .
1−r
n=1
P∞ 1
For the series n=1 2n , the sum is:
∞ 1
X 1 2
= 1 = 1.
2n 1− 2
n=1

Thus, P (Ω) = 1.

(b.) To prove that P is countably additive, we need to show that for any collection of disjoint sets
A1 , A2 , . . . in F, the following holds:
∞ ∞
!
[ X
P An = P (An ).
n=1 n=1

Let A1 , A2 , . . . be disjoint sets in F = 2Ω , where each An ⊆ N.

1
Since P is defined for individual elements of N as P ({n}) = 2n , for any disjoint sets A1 , A2 , . . ., the
probability measure P (An ) can be written as:
X X 1
P (An ) = P ({k}) = .
2k
k∈An k∈An
S∞
Now, consider the union A = n=1 An .

Since the sets A1 , A2 , . . . are disjoint, the union will contain all elements from each set without overlap,
so:
∞ ∞ X
!
[ X X 1
P (A) = P An = P ({k}) = .
2k
n=1 k∈A n=1 k∈An

Since the sets are disjoint, this is equivalent to:

∞ ∞
!
[ X
P An = P (An ).
n=1 n=1

Thus, P is countably additive.

(c.) Now we will calculate P (A) where A = {2, 3, 5, 7}

7
 The probability measure for any finite set A is the sum of the probabilities of the individual elements:

P (A) = P ({2}) + P ({3}) + P ({5}) + P ({7}).

1
Using the given formula P ({n}) = 2n , we compute:

1 1 1 1 1 1 1 1 53
P (A) = 2
+ 3+ 5+ 7 = + + + = .
2 2 2 2 4 8 32 128 128
53
Thus, P (A) = 128 .

6. Let Ω = {1, 2, 3} and let F be a sigma algebra on Ω that contains the following sets: F = {∅, {1}, {2, 3}, Ω}.
Define a probability measure P on F such that:

P ({1}) = 0.2, P ({2, 3}) = 0.8

(a) Show that P (Ω) = 1.

(b) Verify the additivity property for disjoint sets in F.

(c) Determine the probability for all Sets in F.

Sol. Given:

• Sample space Ω = {1, 2, 3}

• sigma algebra F = {∅, {1}, {2, 3}, Ω}.

• Probability measure P on F is defined as:

P ({1}) = 0.2, P ({2, 3}) = 0.8.

We need to prove that P is a valid probability measure and calculate P for all sets in F.

The given values of the measure are:

P ({1}) = 0.2, P ({2, 3}) = 0.8.

Both values are non-negative, so the non-negativity condition is satisfied.

We need to prove that P (A ∪ B) = P (A) + P (B) for any disjoint sets A, B ∈ F.

We consider A = {1} and B = {2, 3}. Since A ∩ B = ∅, we have:

P (A ∪ B) = P ({1} ∪ {2, 3}) = P (Ω) = 1.

8
 Also:
P (A) + P (B) = P ({1}) + P ({2, 3}) = 0.2 + 0.8 = 1.

Therefore, countable additivity holds for these sets.

To check if the normality, we have to show P (Ω) = 1.

The set Ω can be written as the union of the disjoint sets {1} and {2, 3}:

Ω = {1} ∪ {2, 3}.

Using countable additivity:

P (Ω) = P ({1}) + P ({2, 3}) = 0.2 + 0.8 = 1.

Thus, the measure is normalized.

Calculation of P for All Sets in F:

For ∅: Since ∅ is the empty set, by the property of probability measures, we have:

P (∅) = 0.

For {1}, {2, 3}: , the value is already given.

P ({1}) = 0.2, P ({2, 3}) = 0.8.

For Ω = {1, 2, 3}, we have already calculated

P (Ω) = 1.

Thus the values of P for the sets in F are:

P (∅) = 0, P ({1}) = 0.2, P ({2, 3}) = 0.8, P (Ω) = 1.

7. Let Ω = [0, 1] and let F be the Borel sigma algebra on [0, 1]. Define P as the Lebesgue measure on [0, 1],
where P ([a, b]) = b − a for any interval [a, b] ⊆ [0, 1].

(a) Show that P satisfies the properties of a probability measure.

(b) Consider the set A = ∞

S  n−1 n 
n=1 2n , 2n . Find P (A).

(c) Prove that P ([a, b]) is countably additive over disjoint intervals.

9
Sol. (a.) To prove P is a valid probability measure, we have to show it satisfies all three properties:
non-negativity, normalization, and countable additivity.
For any interval [a, b] ⊆ [0, 1], the measure is given by:

P ([a, b]) = b − a.

Since b ≥ a, we have b − a ≥ 0, which ensures that P ([a, b]) ≥ 0. This confirms that P is non-negative.
For the whole space Ω = [0, 1], we need to prove P (Ω) = 1.
Since Ω = [0, 1], we compute:
P ([0, 1]) = 1 − 0 = 1.

Thus, P (Ω) = 1, satisfying the normalization property.

To show the countably additive property of P , we consider a countable collection of disjoint intervals
{Ai }∞
i=1 where Ai ⊆ [0, 1] and Ai ∩ Aj = ∅ for i ̸= j.

Now, we need to verify that

∞ ∞
!
[ X
P Ai = P (Ai ).
i=1 i=1

Since P ([a, b]) = b − a for any interval [a, b], this property holds by the definition of the Lebesgue measure.
Therefore, P is countably additive.
Thus, P satisfies the properties of a probability measure.

(b.) Consider the set A = ∞

S  n−1 n 
n=1 2n , 2n . Find P (A).

The set A is the union of the intervals n−1 n

 
2n , 2n for n = 1, 2, 3, . . ..

We calculate the Lebesgue measure of each interval:

 
n−1 n n n−1 1
P , = n − n = n.
2n 2n 2 2 2

Now, by using the countable additivity, the total measure of A is given by

∞   X ∞
X n−1 n 1
P (A) = P , = .
2n 2n 2n
n=1 n=1

This is a geometric series with the sum:

∞
X 1
P (A) = = 1.
2n
n=1

Thus, P (A) = 1.

(c.) Let {[ai , bi ]}∞

i=1 be a countable collection of disjoint intervals in [0, 1], where [ai , bi ] ∩ [aj , bj ] = ∅ for

i ̸= j.

10
 We want to prove that:
∞ ∞
!
[ X
P [ai , bi ] = P ([ai , bi ]).
i=1 i=1

The measure of each interval [ai , bi ] is given by:

P ([ai , bi ]) = bi − ai .

S∞
Since the intervals are disjoint, the union i=1 [ai , bi ] forms a new set of intervals with a total length equal
to the sum of the individual lengths.
Thus, we have:
∞ ∞
!
[ X
P [ai , bi ] = (bi − ai ).
i=1 i=1
But the right-hand side is just the sum of the individual measures:
∞ ∞
!
[ X
P [ai , bi ] = P ([ai , bi ]).
i=1 i=1

Therefore, P is countably additive over disjoint intervals.

This completes the proof.
P∞
Theorem 2.1. First Borel-Cantelli Lemma: If n=1 P (An ) < ∞, then the probability that infinitely many An
occur is 0, i.e., P (lim sup An ) = 0.

Proof. The event lim sup An means a set of outcomes in Ω that belong to infinitely many of the events An . So
we can write:
lim sup An = {ω ∈ Ω : ω ∈ An for infinitely many n} .
This can be written as:
∞ [
\ ∞
lim sup An = Ak .
N =1 k=N
This means that for every N , there is some k ≥ N such that the event Ak occurs.
Now, we define a sequence of events BN as:
∞
[
BN = Ak .
k=N
T∞
Notice that lim sup An = N =1 BN , which implies:
∞
!
\
P (lim sup An ) = P BN .
N =1

For each N , the probability of the event BN can be bounded using the union bound (also known as Boole’s
inequality):
∞ ∞
!
[ X
P (BN ) = P Ak ≤ P (Ak ).
k=N k=N

11
 By assumption, ∞
P P∞
n=1 P (An ) < ∞. Therefore, the tail sum k=N P (Ak ) converges to zero as N → ∞.
Specifically, for any ϵ > 0, there exists an N such that:
∞
X
P (Ak ) < ϵ.
k=N

Hence, for large N , we have:

P (BN ) ≤ ϵ.
Since P (BN ) ≤ ϵ for large N , and ϵ can be made arbitrarily small, it follows that:

lim P (BN ) = 0.
N →∞

Consequently, by the continuity of probability, we conclude that:

∞
!
\
P BN = lim P (BN ) = 0.
N →∞
N =1

Thus, we have shown that:

P (lim sup An ) = 0.
This completes the proof of the First Borel-Cantelli Lemma.

P∞
Theorem 2.2. Second Borel-Cantelli Lemma: If A1 , A2 , . . . are independent and n=1 P (An ) = ∞, then the
probability that infinitely many An occur is 1, i.e., P (lim sup An ) = 1.

The proof is left for the readers.

2.3. Continuity of probability measure:

The continuity of a probability measures the behaviour of the probabilities with respect to increasing or
decreasing sequences of events.

1. Continuity from Below:

For an increasing sequence of sets A1 ⊆ A2 ⊆ . . ., we have:
∞
!
[
A1 ⊆ A2 ⊆ · · · =⇒ lim P (An ) = P An .
n→∞
n=1

This property is known as continuity from below.

2. Continuity from Above:

For a decreasing sequence of sets B1 ⊇ B2 ⊇ . . ., we have:
∞
!
\
B1 ⊇ B2 ⊇ · · · =⇒ lim P (Bn ) = P Bn .
n→∞
n=1

This property is known as continuity from above.

12
 Example

1. Let Ω = [0, 1] and define a sequence of increasing sets An = [0, 1 − n1 ]. Then:

1
P (An ) = 1 − , lim P (An ) = P ([0, 1]) = 1.
n n→∞

This illustrates the continuity from the below property.

2. Let An = 0, n1 for n = 1, 2, . . .. As n → ∞, An → (0, 0] = ∅, and thus:



1
P (An ) = , lim P (An ) = 0.
n n→∞

This illustrates the continuity of the above property.

3. Let Ω = [0, 1] and define a sequence of decreasing sets Bn = [ n1 , 1]. Then:

1
P (Bn ) = 1 − , lim P (Bn ) = P (∅) = 0.
n n→∞

This illustrates the continuity from the below property.

Theorem 2.3. Continuity of Probability Measure: Let {An }∞

n=1 be a decreasing sequence of sets, i.e., An ⊇
An+1 , and P (A1 ) < ∞. Then:
∞
!
\
P An = lim P (An )
n→∞
n=1

Proof. Define Bn = An \ An+1 , which are disjoint. By countable additivity:

∞ ∞
! !
[ \
P (A1 ) = P Bn + P An
n=1 n=1

Since Bn → ∅ as n → ∞, we have:
∞
!
\
P An = lim P (An )
n→∞
n=1

Countable Additivity vs. Continuity

The difference between countable additivity and continuity lies in the nature of the sets being considered. While
countable additivity holds for disjoint sets, continuity applies to sequences of increasing or decreasing sets, even
when they are not disjoint.

13
3. Probability Space

A probability space is a mathematical framework, used to model random experiments. It consists of three
main components: a sample space, a sigma field, and a probability measure.
Hence, a probability space is a triple (Ω, F, P ), where:

• Ω is the sample space, the set of all possible outcomes of a random experiment.

• F is a sigma field (or sigma algebra) on Ω, which contains all the events we are interested in (subsets of
Ω).

• P is a probability measure, a function that assigns a probability to each event in F, satisfying the axioms
of probability.

3.1. Example

1. For a continuous random variable like the uniform distribution on [0, 1], we can write:

• Ω = [0, 1],

• F is the Borel sigma field on[0, 1],

• If P is a Lebesgue measure, so that for any interval (a, b] ⊂ [0, 1],

P ((a, b]) = b − a.

In this way, we can define a probability space for a continuous random variable.

2. Define the probability space for the result of tossing a fair coin.

• Sample space: Ω = {H, T } , where H is heads and T is tails.

• Sigma field: F = {∅, {H}, {T }, {H, T }}, the power set of Ω.

• Probability measure: P ({H}) = 0.5, P ({T }) = 0.5, and P (Ω) = 1.

Thus, the probability space is (Ω, F, P ).

3. Problem: Define the probability space for rolling a fair six-sided die.

• Sample space: Ω = {1, 2, 3, 4, 5, 6}.

• Sigma field: The power set of Ω, F.

• Probability measure: Since the die is fair, P ({i}) = 1

6 for each i = 1, 2, 3, 4, 5, 6.

14
 The probability space is (Ω, F, P ), where P is uniform over Ω.

Real-life Example: Stock Prices

Let Ω represent the set of all possible stock prices in a given market.
The sigma field F represents different price intervals or events.
P is defined as a probability measure based on a normal distribution (or other models).
This model can simulate stock prices for risk management or investment strategies.

3.2. Properties of a Probability Space:

• Non-negativity:
P (A) ≥ 0 for all events A ∈ F.

• Total Probability:
The probability of the entire sample space is 1, i.e., P (Ω) = 1.

• Additivity:
If A1 , A2 , . . . , An ∈ F are mutually disjoint (i.e., Ai ∩ Aj = ∅ for i ̸= j), then:
n n
!
[ X
P Ai = P (Ai )
i=1 i=1

• Monotonicity:
If A ⊆ B, then P (A) ≤ P (B).

3.3. Theorems
Theorem 3.1. Complement Rule: For any event A ∈ F, the probability of its complement is:
P (Ac ) = 1 − P (A)
Proof. Since A ∪ Ac = Ω and A ∩ Ac = ∅, by additivity:
P (A) + P (Ac ) = P (Ω) = 1
Thus, P (Ac ) = 1 − P (A).

Theorem 3.2. Inclusion-Exclusion Principle: For any two events A, B ∈ F, the probability of their union
is:
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
Proof. By considering the union A ∪ B, we can write:
P (A ∪ B) = P (A) + P (B \ A)
Since B \ A is the part of B not in A, then:
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

15
Theorem 3.3. Total Probability: Let {Ai }ni=1 be a partition of Ω, where each Ai ∈ F and Ai ∩ Aj = ∅ for
i ̸= j. Then, for any event B ∈ F, we have:
n
X
P (B) = P (B ∩ Ai )
i=1

Proof. Since {Ai } is a partition of Ω, B can be written as:

n
[
B= (B ∩ Ai )
i=1

Then by countable additivity, we get:

n
X
P (B) = P (B ∩ Ai )
i=1

3.4. Solved Problems:

1. Consider a company that insures two types of devices: smartphones and laptops. The probability that a
smartphone is insured is 0.7, and the probability that a laptop is insured is 0.6. What is the probability
that a randomly chosen smartphone is not insured?

Sol.
By using the complement rule:

P (Smartphone not insured) = 1 − P (Smartphone insured) = 1 − 0.7 = 0.3

2. Let P (A) = 0.4, P (B) = 0.5, and P (A ∩ B) = 0.2. Compute P (A ∪ B).

Sol. Using the inclusion-exclusion principle:

P (A ∪ B) = P (A) + P (B) − P (A ∩ B) = 0.4 + 0.5 − 0.2 = 0.7

3. Let P (A) = 0.65. Find P (Ac ).

Sol. Using the complement rule:

P (Ac ) = 1 − P (A) = 1 − 0.65 = 0.35

4. Let Ω = R, and let F = B(R) be the Borel sigma algebra. Consider the probability measure P defined
as:
Z b
1 −x2
P ([a, b]) = √ e 2 dx,
2π a

and prove all the properties.

16
 Sol. First, we have to verify P (R) = 1
The probability measure P is defined as the standard normal distribution, and we know that the total
probability over the real line is 1, i.e.,:
Z ∞
1 x2
P (R) = √ e− 2 dx = 1.
2π −∞

Now, we prove the countable additivity

Let A1 , A2 , . . . be disjoint sets in F.
By the properties of the normal distribution, the measure P satisfies countable additivity.
For disjoint sets:
∞ ∞
!
[ X
P An = P (An ),
n=1 n=1

where each An ∈ B(R).

Now we calculate P (An ) for intervals
We define intervals An = [n, n + 1] for n ≥ 1.
The probability of these intervals is given by:
Z n+1
1 x2
P (An ) = √ e− 2 dx.
2π n

As n → ∞, this probability decreases rapidly since the tail of the normal distribution decays exponentially.

3.5. Advanced Problems

1. Consider a biased coin that lands on heads with probability p and tails with probability 1 − p. Instead
of focusing on individual coin tosses, imagine you are interested in sequences of outcomes over an infinite
number of tosses.

• Sample Space: The sample space Ω consists of all possible infinite sequences of heads and tails:

Ω = {(H, T, H, . . . ), (T, H, T, . . . ), . . . }.

• Sigma Algebra: F is the Borel sigma algebra on infinite sequences, which also allows us to consider
complex events like ”the sequence has infinitely many heads or tails.”

• Probability Measure: The probability measure assigns probabilities to these sequences. For example,
the probability of obtaining a sequence where the first toss is head is:

P (the first toss is head) = p.

17
 2. Model a sequence of biased coin tosses where each toss has a different bias. What would the probability
space look like, and how would you define the measure?

Sol. We are modeling a sequence of biased coin tosses where each toss has a different bias. The triplets
of probability space will be defined by:

(a) Sample Space (Ω): Each toss results in either a Head (H) or Tail (T ).
For an infinite sequence of tosses, the sample space is Ω = {H, T }∞ , representing all possible infinite
sequences of heads and tails.
Each element in Ω represents a sequence like (H, T, H, H, . . . ).

(b) Sigma Algebra (F): The sigma algebra (F) is the Borel sigma algebra on Ω.
It contains all events (sets of sequences) that we are interested in. This includes sets like ”the first
toss is a head” or ”the second toss is a tail”, as well as more complex sets involving multiple tosses.

(c) Probability Measure (P ): Now we define the measure for a biased coin for each toss. Let pn be the
probability of getting a head (H) on the n-th toss.
Then the probability of a sequence ω = (ω1 , ω2 , . . . ), where ωn ∈ {H, T }, can be written as:
∞
Y
P (ω) = [pn · 1(ωn = H) + (1 − pn ) · 1(ωn = T )]
n=1

Here, 1(ωn = H) is an indicator function that equals 1 if ωn = H and 0 otherwise.

This product defines the probability of observing any particular sequence of heads and tails, given
that the probability of heads is pn for the n-th toss.

Exercise
Consider a continuous probability space (Ω, F, P ), where Ω = [0, 1] and F is the Borel sigma algebra B([0, 1]).
The probability measure P is given by the Lebesgue measure, so for any interval [a, b] ⊆ [0, 1], we have
P ([a, b]) = b − a.
h i S∞
n−1 n
1. Define An = n , n+1 . Calculate P (An ) and find P ( n=1 An ).

 1 1
 S∞
2. Let Bn = 2n , 2n−1 . Compute P ( n=1 Bn ), and verify whether countable additivity holds for this
sequence of disjoint sets.

18