36-752 Advanced Probability Overview Spring 2018

2. Measurable Functions, Random Variables, and Integration

Instructor: Alessandro Rinaldo

Associated reading: Sec 1.5 of Ash and Doléans-Dade; Sec 1.3 and 1.4 of Durrett.

1 Measurable Functions

1.1 Measurable functions

Measurable functions are functions that we can integrate with respect to measures in much
the same way that continuous functions can be integrated “dx”. Recall that the Riemann
integral of a continuous function f over a bounded interval is defined as a limit of sums of
lengths of subintervals times values of f on the subintervals. The measure of a set generalizes
the length while elements of the σ-field generalize the intervals. Recall that a real-valued
function is continuous if and only if the inverse image of every open set is open. This
generalizes to the inverse image of every measurable set being measurable.

Definition 1 (Measurable Functions). Let (Ω, F) and (S, A) be measurable spaces. Let
f : Ω → S be a function that satisfies f −1 (A) ∈ F for each A ∈ A. Then we say that f is
F/A-measurable. If the σ-field’s are to be understood from context, we simply say that f is
measurable.

Example 2. Let F = 2Ω . Then every function from Ω to a set S is measurable no matter

what A is.

Example 3. Let A = {∅, S}. Then every function from a set Ω to S is measurable, no
matter what F is.

Proving that a function is measurable is facilitated by noticing that inverse image commutes
with union, complement, and intersection. That is, f −1 (AC ) = [f −1 (A)]C for all A, and for
arbitrary collections of sets {Aα }α∈ℵ ,
!
[ [
f −1 Aα = f −1 (Aα ),
α∈ℵ α∈ℵ
!
\ \
f −1 Aα = f −1 (Aα ).
α∈ℵ α∈ℵ

1
Exercise 4. Is the inverse image of a σ-field is a σ-field? That is, if f : Ω → S and if A is
a σ-field of subsets of S, then f −1 (A) is a σ-field of subsets of Ω.

Proposition 5. If f is a continuous function from one topological space to another (each

with Borel σ-field’s) then f is measurable.

The proof of this makes use of Lemma 7.

Definition 6 (σ-filed generated by measurable functions). Let f : Ω → S, where

(S, A) is a measurable space. The σ-field f −1 (A) is called the σ-field generated by f . The
σ-field f −1 (A) is sometimes denoted σ(f ).

It is easy to see that f −1 (A) is the smallest σ-field C such that f is C/A-measurable. We
can now prove the following helpful result.

Lemma 7. Let (Ω, F) and (S, A) be measurable spaces and let f : Ω → S. Suppose that C
is a collection of sets that generates A. Then f is measurable if f −1 (C) ⊆ F.

Exercise 8. Prove Lemma 7.

Exercise 9 (A common method to prove measurability). Prove the following. Let

S = IR in Lemma 7. Let D be a dense subset of IR, and let C be the collection of all intervals
of the form (−∞, a), for a ∈ D. To prove that a real-valued function is measurable, one need
only show that {ω : f (ω) < a} ∈ F for all a ∈ D. Similarly, we can replace < a by > a or
≤ a or ≥ a.

Exercise 10. Show that a monotone increasing function is measurable.

Example 11. Suppose that f : Ω → IR takes values in the extended reals. Then f −1 ({−∞, ∞}) =
[f −1 ((−∞, ∞))]C . Also
\∞
−1
f ({∞}) = {ω : f (ω) > n},
n=1

and similarly for −∞. In order to check whether f is measurable, we need to see that the
inverse images of all semi-infinite intervals are measurable sets. If we include the infinite
endpoint in these intervals, then we don’t need to check anything else. If we don’t include
the infinite endpoint, and if both infinite values are possible, then we need to check that at
least one of {∞} or {−∞} has measurable inverse image.

Here are some simple properties of measurable functions.

Theorem 12 (Properties of Measurable Functions). Let (Ω, F), (S, A), and (T, B) be
measurable spaces.

2
 1. If f is an extended real-valued measurable function and a is a constant, then af is
measurable.

2. If f : Ω → S and g : S → T are measurable, then g(f ) : Ω → T is measurable.

3. If f and g are measurable real-valued functions, then f + g and f g are measurable.

Proof: For a = 0, part 1 is trivial. Assume a 6= 0. Because


{ω : f (ω) < c/a} if a > 0,
{ω : af (ω) < c} =
{ω : f (ω) > c/a} if a < 0,

we see that af is measurable.

For part 2, just notice that [g(f )]−1 (B) = f −1 (g −1 (B)).
For part 3, let h : IR2 → IR be defined by h(x, y) = x + y. This function is continuous, hence
measurable. Then f + g = h(f, g). We now show that (f, g) : Ω → IR2 is measurable, where
(f, g)(ω) = (f (ω), g(ω)). To see that (f, g) is measurable, look at inverse images of sets that
generate B 2 , namely sets of the form (−∞, a] × (−∞, b], and apply Lemma 7. We see that

(f, g)−1 ((−∞, a] × (−∞, b]) = f −1 ((−∞, a]) ∩ g −1 ((−∞, b]),

which is measurable. Hence, (f, g) is measurable and h(f, g) is measurable by part 2. Simi-
larly f g is measurable.

You can also prove that f /g is measurable when the ratio is defined to be an arbitrary
constant when g = 0. Similarly, part 3 can be extended to extended real-valued functions
so long as care is taken to handle cases of ∞ − ∞ and ∞ × 0.

Theorem 13. Let fn : Ω → IR be measurable for all n. Then the following are measurable:

1. lim supn→∞ fn ,

2. lim inf n→∞ fn ,

3. {ω : limn→∞ fn exists}.

limn→∞ fn where the limit exists,
4. f =
0 elsewhere.

Exercise 14. Prove Theorem 13.

3
2 Random Variables and Induced Measures
Definition 15 (Random Variables). If (Ω, F, P ) is a probability space and X : Ω → IR
is measurable, then X is called a random variable. In general, if X : Ω → S, where (S, A)
is a measurable space, we call X a random quantity.

Example 16. Let Ω = (0, 1) with the Borel σ-field, and let µ be Lebesgue measure, a
probability. Let Z(ω) = ω and X(ω) = b2ωc. Both X and Z are random variables. X takes
only two values, 0 and 1. It is easy to see that µ({ω : X(ω) = 1}) = 1/2. It is also easy to
see that µ({ω : Z(ω) ≤ c}) = c for 0 ≤ c ≤ 1.

Each measurable function from a measure space to another measurable space induces a
measure on its range space.

Lemma 17 (Induced Measure). Let (Ω, F, µ) be a measure space and let (S, A) be a
measurable space. Let f : Ω → S be a measurable function. Then f induces a measure on
(S, A) defined by ν(A) = µ(f −1 (A)) for each A ∈ A.

Proof: Clearly, ν ≥ 0 and ν(∅) = 0. Let {An }∞

n=1 be a sequence of disjoint elements of A.
Then
∞
! "∞ #!
[ [
ν An = µ f −1 An
n=1 n=1
∞
!
[
−1
= µ f [An ]
n=1
∞
X
= µ(f −1 [An ])
n=1
∞
X
= ν(An ).
n=1

The measure ν in Lemma 17 is called the measure induced on (S, A) from µ by f . This
measure is only interesting in special cases. First, if µ is a probability then so is ν.

Definition 18 (Probability Distribution). Let (Ω, F, P ) be a probability space and let

(S, A) be a measurable space. Let X : Ω → S be a random quantity. Then the measure
induced on (S, A) from P by X is called the distribution of X.

We typically denote the distribution of X by µX . In this case, µX is a measure on the space

(S, A).

4
Exercise 19. Consider the random variables in Example 16. The distribution of X is
the Bernoulli distribution with parameter 1/2. The distribution of each Z is the uniform
distribution on the interval (0, 1). Write down the corresponding (S, A, ν) in Lemma 17 for
X and Z, respectively.

If µ is infinite and f is not one-to-one, then the induced measure may be of no interest at
all.

Exercise 20. Either prove or create a counterexample to the following conjecture: If µ is a

σ-finite measure on some measurable space (Ω, F), then for any measurable function f from
Ω to S, the induced measure is also σ-finite.

Example 21 (Jacobians). If Ω = S = IRk and f is one-to-one with a differentiable inverse,

then ν is the measure you get from the usual change-of-variables formula using Jacobians.

We have just seen how to construct the distribution from a random variable. Oddly enough,
the opposite construction is also available. First notice that every probability ν on (IR, B 1 )
has a distribution function F defined by F (x) = ν((−∞, x]). Now, we can construct a
probability space (Ω, F, P ) and a random variable X : Ω → IR such that ν = P (X −1 ).1
Indeed, just let Ω = IR, F = B 1 , P = ν, and X(ω) = ω.

3 Simple Functions
Definition 22 (Simple Functions). A measurable function that takes finitely many values
is called a simple function.

Example 23. Let (Ω, F) be a measurable space and let A1 , . . . , An be disjoint elements of
F, and let a1 , . . . , an be real numbers. Then f = ni=1 ai IAi defines a simple function since
P

f −1 ((−∞, a)) is a union of at most finitely many measurable sets.

Definition 24 (Canonical Representation of A Simple Function). Let f be a simple

function whose distinct values are a1 , . . . , an , and let Ai = {ω : f (ω) = ai }. Then f =
Pn
i=1 ai IAi is called the canonical representation of f .

Lemma 25 (Monotone Approximation). Let f be a nonnegative measurable extended

real-valued function from Ω. Then there exists a sequence {fn }∞n=1 of nonnegative (finite)
simple functions such that fn ≤ f for all n and limn→∞ fn (ω) = f (ω) for all ω.
1
Notation: When X is a random quantity and B is a set in the space where X takes its values, we use
the following two symbols interchangeably: X −1 (B) and X ∈ B. Both of these stand for {ω : X(ω) ∈ B}.
Finally, for all B,
µX (B) = Pr(X ∈ B) = P (X −1 (B)).

5
Proof: For each n, define An,k = f −1 ([k/n, (k + 1)/n)) for k = 0, 1, . . . , n2 − 1 and An,∞ =
P 2 −1
f −1 ([n, ∞]). Define fn (ω) = n1 nk=0 kIAn,k (ω) + nIAn,∞ (ω). The proof is easy to complete
now.

Lemma 25 says that each nonnegative measurable function f can be approximated arbi-
trarily closely from below by simple functions. It is easy to see that if f is bounded the
approximation is uniform once n is greater than the bound.
Many theorems about real-valued functions are easier to prove for nonnegative measurable
functions. This leads to the common device of splitting a measurable function f as follows.

Definition 26 (Splitting Measurable Functions). Let f be a real-valued function. The

positive part f + of f is defined as f + (ω) = max{f (ω), 0}. The negative part f − of f is
f − (ω) = − min{f (ω), 0}.

Notice that both the positive and negative parts of a function are nonnegative. It follows
easily that f = f + − f − . It is easy to prove that the positive and negative parts of a
measurable function are measurable.

4 Integration
Let (Ω, F, µ) be a measure space. The definition of integral is done in three stages. We start
with simple functions.
+0
Definition 27 (Integral of Simple Functions). Let f : Ω → IR be a simple function
with canonical representation f (ω) = ni=1 ai IAi (ω) The integral of f with respect to µ is
P
Pn R R
defined to be i=1 ai µ(Ai ). The integral is denoted variously as f dµ, f (ω)µ(dω), or
R
f (ω)dµ(ω).

The values ±∞ are allowed for an integral.

We use the following convention whenever necessary in defining an integral: ±∞ × 0 = 0.
This applies to both the case when the function is 0 on a set of infinite measure and when
the function is infinite on a set of 0 measure.
R R
Proposition 28. If f ≤ g and both are nonegative and simple, then f dµ ≤ gdµ.

Definition 29 (Integrable (Simple) Functions). We say that tf (simple) is integrable

R
with respect to µ if f dµ is finite.

Example 30. A real-valued simple function is always integrable with respect to a finite
measure.

The second step in the definition of integral is to consider nonnegative measurable functions.

6
Definition 31 (Integral for General Nonnegative Functions). For nonnegative mea-
surable f , define the integral of f with respect to µ by
Z Z
f dµ = sup gdµ.
nonnegative finite simple g ≤ f
R
That is, if f is nonnegative and measurable, f dµ is the least upper bound (possibly infinite)
of the integrals of nonnegative finite simple functions g ≤ f . An equivalent but perhaps more
transparent definition of the integral of a non-negative function f with respect to the measure
µ is the following: Z X 
f dµ = sup inf f (ω) µ(Ai ),
ω∈Ai
i
where the supremum ranges over all finite partitions A1 , . . . , An of Ω.
Proposition 28 helps to show that Definition 27 is a special case of Definition 31, so the two
definitions do not conflict when they both apply.
Finally, for arbitrary measurable f , we first split f into its positive and negative parts,
f = f + − f −.
Definition 32 (Integral of General Measurable Functions). Let f be measurable. If
either f + or f − is integrable with respect to µ, we define the integral of f with respect to µ
to be f + dµ − f − dµ, otherwise the integral does not exist.
R R

It is easy to see that Definition 31 is a special case of Definition 32, so the two definitions
do not conflict when they both apply. The reason for splitting things up this way is to avoid
ever having to deal with ∞ − ∞.
One unfortunate consequence of this three-part definition is that many theorems about
integrals must be proven in three steps. One fortunate consequence is that, for most of these
theorems, at least some of the three steps are relatively straightforward.
R R
Definition 33 (Integration Over A Set). If A ∈ F, we define A f dµ by IA f dµ.
Proposition 34 (Monotonicity of Integral). If f ≤ g and both integrals are defined,
R R
then f dµ ≤ gdµ.
Exercise 35. Show that if f and g are integrable, then
Z Z  Z
 
 f dµ − gdµ ≤ |f − g| dµ.
 

Example 36 (Integration with Counting Measure). Let µ be counting measure on a

set Ω. (This measure is not σ-finite unless Ω is countable.) If A ⊆ Ω, then µ(A) = #(A),
the number of elements in A. If f is a nonnegative simple function, f = ni=1 ai IAi , then
P

Z Xn X
f dµ = ai #(Ai ) = f (ω).
i=1 All ω

7
It is not difficult to see that the equality of the first and last terms above continues to hold
for all nonnegative functions, and hence for all integrable functions.
Definition 37 (Expectation and Variance of Random Variables). If P is a probability
R
and X is a random variable, then XdP is called the mean of X, expected value of X, or
expectation of X and denoted E(X). If E(X) = µ is finite, then the variance of X is
Var(X) = E[(X − µ)2 ].

4.1 Riemann and Lebesgue Integrals

Before we study integration in detail, we should note that integration with respect to
Lebesgue measure is the same as the Riemann integral in many cases.
Theorem 38. Let f be a continuous function on a closed bounded interval [a, b]. Let µ be
Rb R
Lebesgue measure. Then the Riemann integral a f (x)dx equals [a,b] f dµ.
Exercise 39. Prove Theorem 38.
Example 40. A case in which the Riemann integral differs from the Lebesgue integral is that
of “improper” Riemann integrals. These are defined as limits of Riemann integrals that are
each defined in the usual way. For example, integrals of unbounded functions and integrals
over unbounded regions cannot be defined in the usual way because the Riemann sums would
always be ∞ or undefined. Consider the function f (x) = sin(x)/x over the interval [1, ∞).
It is not difficult to see that neither f + nor f − is integrable with respect to Lebesgue measure.
Hence, the integral that we have defined here does not exist. However, the improper Riemann
RT
integral is defined as limT →∞ 1 f (x)dx, if the limit exists. In this case, the limit exists.

4.2 Properties of Integrals

Definition 41 (Almost sure/almost everywhere). Suppose that some statement about
elements of Ω holds for all ω ∈ AC where µ(A) = 0. Then we say that the statement holds
almost everywhere, denoted a.e. [µ]. If P is a probability, then almost everywhere is often
replaced by almost surely, denoted a.s. [P ].

Some simple properties of integrals include the following:

R R
• For c a constant, cf dµ = c f dµ if the latter exists.
R
• If f ≥ 0, then f dµ ≥ 0.
R R
• if f = g for all ω ∈ Ω except on a set with zero measure, and if either f dµ or gdµ
exists, then so does the other, and they are equal. Similarly, if one of the integrals
doesn’t exist, then neither does the other.

8
Intuitively, the following two results seem obvious. However, in order to rigorously prove
them, we need some additional tools on integrals and limits, described in the next set of
lecture notes.
R R R
Theorem 42 (Additivity). (f + g)dµ = f dµ + gdµ whenever at least two of them
are finite.

Theorem 43 (Change of Variable). Let (Ω, F, µ) be a measure space and let (S, A) be a
measurable space. Let f : Ω → S be a measurable function. Let ν be the measure induced on
(S, A) by f from µ. (See Definition 18.) Let g : S → IR be A/B 1 measurable. Then
Z Z
gdν = g(f )dµ, (1)

if either integral exists.

Although it is not clear at this point how to prove the previous results for general measurable
functions, it is not hard to verify them for simple functions.

Exercise 44. Let f and g be nonnegative simple functions defined on a measure space
R R R
(Ω, F, µ). Then (f + g)dµ = f dµ + gdµ. Furthermore, if g is the indicator function of
a measurable subset of S, then (1) holds.