Random Variables: An overview

master INVESMAT 2018–2019

Unit 1

Random Differential Equations and Applications Random Variables: An overview 1

 MAIN IDEA

In this session we will remember the main definitions, concepts, properties and results
related to univariate random variables (r.v.’s).
To be equipped with a good background of r.v.’s is crucial in dealing with random
differential equations (r.d.e.’s) since their inputs, such as, initial and/or boundary
conditions, source terms and coefficients can be r.v.’s rather than constants.

Random Differential Equations and Applications Random Variables: An overview 2

 Part I

Univariate Random Variables

Random Differential Equations and Applications Random Variables: An overview 3

 Basics on Probability

Example 1: Tossing coins

Random experiment1 : Toss a pair of fair coins (or equivalently, flipping a coin
twice).
Sample space:
Ω = {ω1 = (h, h); ω2 = (h, t); ω3 = (t, h); ω4 = (t, t)}

Elementary events:
E1 = {ω1 } ; E2 = {ω2 } ; E3 = {ω3 } ; E4 = {ω4 }

Compounded events:
Events Mathematical description
A = to get at least a head A = E1 ∪ E2 ∪ E3 = (E4 )c
B = to get just a head B = E2 ∪ E3 = (E1 ∪ E4 )c
C = to get 3 heads C = 0/ = Ωc
D = to get at least a head or a tail D = Ω = 0/ c

notice that knowing set theory becomes very useful!

1: If we know a lot of Physics, it would not be a random experiment!

Random Differential Equations and Applications Random Variables: An overview 4
Example 1: Tossing coins (continuation)
Random variables (r.v.’s):

X: Ω → R
X = number of heads ⇒
ωi → X (ωi ) ∈ {0, 1, 2}

X (ω1 ) = 2, X (ω2 ) = X (ω3 ) = 1, X (ω4 ) = 0.

Y = difference in absolute value between heads and tails

Y: Ω → R
ωi → Y (ωi ) ∈ {0, 2}

Y (ω1 ) = Y (ω4 ) = 2, Y (ω2 ) = Y (ω3 ) = 0.

Hence, we say, informally, that a real r.v. is a function with domain Ω and codomain
or support (S ) the set of the real numbers: R. In the previous examples:

S (X ) = {0, 1, 2} , S (Y ) = {0, 2} .

Random Differential Equations and Applications Random Variables: An overview 5

Example 2: Trading assets
Random experiment1 : The value of a stock traded in the Spanish index
IBEX-35. For instance, ACS company.
Sample space:
Ω = { every social, economical event that determines
each of the possible values for ACS}

Let us assume that ACS’s values lie on [0, 10].

Elementary events:
Ei = {ωi }∞
i=1 , ωi ∈ Ω, such as it determines the ACS’s values

Compounded events:
Events Mathematical description
S
A = ACS’s value is greater than 6 A = j∈J ωj , unknown
S
B = ACS’s value is between 5.3 and 6.2 B = k∈K ωk , unknown
C = ACS’s value is greater than 20 C = 0/
D = ACS’s value is less than 20 D =Ω

again we use set theory to represent events!

1:If we know how change the variables which determine the value of the shares of this
Spanish financial index, it would not be a random experience!
Random Differential Equations and Applications Random Variables: An overview 6
Example 2: Trading assets (continuation)
Random variable (r.v.):

X = value of a share of ACS tomorrow

We will be interested in evaluating probabilities like:

P[{ω ∈ Ω : 6 < X (ω) ≤ 10}], P[{ω ∈ Ω : 5.3 ≤ X (ω) ≤ 6.9}]

To achieve this goal, we need Probability Theory since both experiments are not
predictable in the sense that they appear according to a random mechanism that is
too complex to be understood using deterministic tools.
In this example, we observe that often we do not need know Ω as we are just
interested in the codomain of the r.v. X .

Random Differential Equations and Applications Random Variables: An overview 7

To study problems associated with the r.v. X , one first collects relevant subsets of Ω,
the events, in a class FΩ called a σ –field or σ –algebra. In order for FΩ to contain all
relevant events, it is natural to include all the ω in the event space Ω and also the
union, difference, and intersection of any events in FΩ , the set Ω and its complement,
the empty set 0.
/
Coming back the example of market share, if we consider a share price X of a stock,
not only the events {ω ∈ Ω : X (ω) = c} should belong to FΩ but also

P[{ω ∈ Ω : a < X (ω) < b}], P[{ω ∈ Ω : b < X (ω)}], P[{ω ∈ Ω : X (ω) ≤ a}],

and many more events that canTbe relevant. So, it is natural to require that
elementary operations such as , ∪, c on the events of FΩ will not land outside the
class FΩ . This is the intuitive meaning of a σ –algebra.

Random Differential Equations and Applications Random Variables: An overview 8

Definition: σ –field or σ -algebra and measurable space
A σ -field F ≡ FΩ is a collection of subsets of Ω satisfying the following conditions:
/ Ω ∈ F (we really only need to impose that either 0/ ∈ F or Ω ∈ F ).
0,
If A ∈ F , then Ac ∈ F .
∞
If A1 , A2 , . . . ∈ F , then Ai ∈ F .
[

i=1
The pair (Ω, FΩ ) is called measurable space (or probabilizable space). Its elements are
referred to as measurable sets.

Using adequately the previous conditions one can deduce that many other sets lie on
F:
Finite or countable infinite intersections:
!c
∞ ∞
A1 ∩ A2 = ((A1 )c ∪ (A2 )c )c ∈ F , (Ai )c ∈ F.
\ [
Ai =
i=1 i=1

.
Differences: A \ B = A ∩ B c .

A σ -algebra is closed under finite and countable infinite unions, intersections,

differences, etc.

Random Differential Equations and Applications Random Variables: An overview 9

Example 3: Some elementary σ –fields
F1 = {0,
/ Ω}, the smallest σ -field.
F2 = {0,
/ Ω, A, Ac }.
F3 = 2Ω = {A : A ⊂ Ω}, the biggest σ –field (called the power set of Ω).

In general the power set is unnecessarily too big. This motivates the concept of
σ –field generated by a collection of sets. One can prove that, given a collection C of
subsets of Ω, there exists a smallest σ -field σ (C) on Ω containing C. We call σ (C) the
σ -field generated by C.

Example 4: σ –field generated by a collection of sets

F1 = σ (0).
/
F2 = σ ({A}).
F3 = σ (F3 ).

Exercise 1: σ –field generated by a collection of sets

Consider the random experience of flipping a coin twice (see previous Example 1 for
notation). Compute:
The smallest σ –algebra containing {ω1 } and {ω2 }: σ ({ω1 } , {ω2 }).
The biggest σ –algebra containing {ω1 } and {ω2 }.

Random Differential Equations and Applications Random Variables: An overview 10

An important σ –algebra, called the σ –algebra of Borel generated by a set is the
following:

Example 5: σ –field of Borel

FR = BR = σ ((a, b]), a < b.

To this σ –algebra belongs all intervals of the form (a, b], unions of such intervals and
intersections and complements of all the resulting sets.
Notice that a large class of sets are included in BR . For instance:
S∞ 1
(a, b) = n=1 (a, b − n ].
T∞ 1
{x} = n=1 (x − n , x].
Although it is possible to construct odd sets that do not belong to BR .

So far we have introduced the concept of r.v. in a rough sense. Next we formalize its
definition.

Definition of real r.v.

X : Ω → R is a real r.v. ⇔ X −1 (A) ∈ FΩ , ∀A ∈ BR .

In words of Measure Theory:

X : Ω → R is a real r.v. ⇔ X is Borel measurable.

Random Differential Equations and Applications Random Variables: An overview 11

The concept of probability is used to measure the likelihood of the occurrence of
certain events. In the first random experiment (toss a pair of fair coins):
P [{ω ∈ Ω : X (ω1 ) = 2}] = P [{ω ∈ Ω : X (ω4 ) = 0}] = 1/4.
This assignment is based on empirical evidence by flipping the coins a large number of
times. The Law of the Large Numbers gives the theoretical justification of such
assignment. In this notes we skip the formal details which are closed to Measure
Theory. A very good account of the foundations can be found in Ref. [1, chap.1–4].

Probability Measure: definition

Given Ω a set and a σ –algebra FΩ of Ω, a probability measure is a map

P : (Ω, FΩ ) → ([0, 1], B[0,1] ), such that P−1 (A) ∈ FΩ , ∀A ∈ B[0,1] ,

(this means that P is B[0,1] –measurable and usually we write P : Ω → [0, 1]) satisfying
the following conditions:
P [Ω] = 1.
P [Ac ] = 1 − P [A] (⇒ P [0]
/ = 0).
" #
∞
[ ∞
P An = ∑ P[An ] if An ∩ Am = 0,/ n 6= m : n, m ≥ 1.
n=1 n=1

From these conditions can be derived many other such as:

P [A ∪ B] = P [A] + P [B] − P [A ∩ B] , ∀A, B ∈ F .

Random Differential Equations and Applications Random Variables: An overview 12

Probability space
The triplet (Ω, F , P) where:
Ω is a countable event space,
F ⊂ 2Ω is the σ -field of Ω,
P : Ω → [0, 1] is a probability measure,
is called a probability space.

Example 6: Flipping a coin until a tail shows

Ω = {ω1 , ω2 , ω3 , . . .}, ωi = the outcome where i − 1 tosses are heads and the i–th
toss is a tail: ω1 = T , ω2 = (H, T ), ω3 = (H, H, T ), etc.
FΩ = {0,/ {ω1 }, {ω2 }, . . . , {ω1 , ω2 }, {ω1 , ω3 }, . . .} is a σ –algebra of Ω such that
{ωi } ∈ FΩ (is the power set of Ω).
Event: B = {the first tail occurs after an odd number of tosses}, i.e.,
B = {ω1 , ω3 , ω5 , . . .} ∈ FΩ .

i
P [{ωi }] = 21 define a probability measure in Ω.
On account of the previous theory one gets:
∞ ∞  2i−1
1 2
P [B] = ∑ P [{ω2i−1 }] = ∑ = .
n=1 n=1 2 3

Random Differential Equations and Applications Random Variables: An overview 13

Distribution function (d.f.)
The collection of the probabilities
Z x
FX (x) = P [X ≤ x] = P [{ω ∈ Ω : X (ω) ≤ x}] = dFX (y ) ∈ [0, 1], ∀x ∈ R
−∞

is the distribution function FX (x) of r.v. X . It yields the following key probabilities:
P [{ω ∈ Ω : a < X (ω) ≤ b}] = FX (b) − FX (a), a < b.
P [{ω ∈ Ω : X (ω) = x}] = FX (x) − lim FX (x − ε).
ε→0+
With these probabilities we can approximate the probability of the event
{ω ∈ Ω : X (ω) ∈ B} for very complicated subsets B of R.

Distribution of a r.v.
The collection of the probabilities
Z
PX (B) = P [X ∈ B] = P [{ω ∈ Ω : X (ω) ∈ B}] = dFX (x)
B

for suitable subsets B ⊂ R is the distribution of r.v. X .

The suitable subsets of R are called Borel sets. They are sets from the so-called
σ -algebra de Borel generated by the semi-open intervals:

BR = σ ({(a, b] : −∞ < a < b < ∞}).

Random Differential Equations and Applications Random Variables: An overview 14

Discrete distributions

 0 ≤ pk ≤ 1,
∞
FX (x) = ∑ pk , x ∈ R, where pk = P [X = xk ] ,
k: xk ≤x
 ∑ pk = 1.
k=1

The d.f. FX (x) can have jumps (it is continuous at the right). Its plot is a step
function.
The d.f. and the corresponding distribution are discrete. A r.v. with such a
distribution is a discrete r.v.
A discrete r.v. assumes only a finite or countably infinite number of values:
x1 , x2 , . . . with probabilities pk = P [X = xk ], respectively. pk is usually referred to
as the probability mass function (p.m.f.).

Example 7: Binomial and Poisson distributions

Random Variable Distribution
n
k n−k , k = 0, 1, . . . , n
X ∼ Bi(n; p), n ∈ N, 0 ≤ p ≤ 1 P [X = k] = k p (1 − p)
−λ λk
X ∼ Po(λ ), λ > 0 P [X = k] = e k! , k = 0, 1, . . .

Binomial: X = the number of ill patients living in separated rooms of a hospital.

Assume that 30% are ill and the hospital has 50 patients. P [X = 15]? Answer:
X ∼ Bi(n = 50; p = 0.3).
Poisson: X = the number of cars arriving to a petrol station in each minute.
Assume that, in average, 3 cars per minute used to arrive there. P [X = 5]?
Answer: X ∼ Po(λ = 3).
Random Differential Equations and Applications Random Variables: An overview 15
 X ∼ Bi(n = 5; p = 0.6)

d.f. (left) and p.d.f. (right)

Random Differential Equations and Applications Random Variables: An overview 16

Continuous distributions
Most common continuous distributions have a probability density function (p.d.f.)
X (x)
fX (x), i.e., a function such that: dFdx = fX (x) satisfying that:

fZX (x) ≥ 0, ∀x ∈ R,
Z x Z x
(
∞
FX (x) = dFX (y ) = fX (x) dx, x ∈ R, where
−∞ −∞ fX (x) dx = 1.
−∞

The d.f. FX (x) does not have any jump, hence P [X = x] = 0, ∀x ∈ R since
lim FX (x − ε) = FX (x).
ε→0+
The d.f. and the corresponding distribution are continuous. A r.v. with such a
distribution is a continuous r.v.

Example 8: Uniform and Gaussian distributions

Random Variable  Distribution
1
b−a , a ≤ x ≤ b,
X ∼ U(a, b), b > a fX (x) =
0 , otherwise,
 2
X ∼ N(µ; σ 2 ), µ ∈ R, σ > 0 fX (x) = √ 1
2
exp − 21 x−µ
σ
2πσ

Uniform: It is to be used when there is no information unless X lies on [a, b].

Gaussian: X = the tall of a male student of the UPV. Assume that the average
tall of males is 175cm. and the standard deviation is 5cm. P [170 ≤ X ≤ 185]?.
Answer: X ∼ N(µ = 175; σ 2 = 25).
Random Differential Equations and Applications Random Variables: An overview 17
 X ∼ N(µ = 0; σ 2 = 1)

d.f. (left) and p.d.f. (right)

Random Differential Equations and Applications Random Variables: An overview 18

 Some remarks about d.f.’s and p.d.f.’s

The d.f. always exists.

The p.d.f. could not exist for odd continuous r.v.’s, so it is more general deals
with the d.f. Whereas its discrete analogous, the mass probability function, does
not have such problem.
p.d.f. of discrete and continuous r.v.’s can be treated all together with the help
of the Dirac delta function. A consistent definition for a discrete r.v. X with
p.d.f. (mass probability function) pk = P[X = xk ], k = 1, 2, . . . is:

∞
fX (x) = ∑ pk δ (x − xk ), x ∈ R.
k=1

In this way, it satisfies:

fX (x) ≥ 0, ∀x ∈ R,
Z ∞ ∞ Z ∞ ∞
fX (x) dx = ∑ pk δ (x − xk ) dx = ∑ pk = 1.
−∞ k=1 −∞
| {z } k=1
=1

Random Differential Equations and Applications Random Variables: An overview 19

Moments of a r.v.
Important characteristics of a r.v. are given in terms of its moments:

Denomination Discrete Continuous Remarks

∞ Z ∞
Mean
∑ xk pk x fX (x) dx 1
µX = E [X ] k=1 −∞
∞ Z ∞
Variance
σX2 = E (X − µX )2
  ∑ (xk − µX )2 pk −∞
(x − µX )2 fX (x) dx 2
k=1
Moment ∞ Z ∞
w.r.t. the origin ∑ (xk )m pk −∞
x m fX (x) dx 3
αm = E [X m ] k=1
∞ Z ∞
Central moment
µm = E [(X − µX )m ] ∑ (xk − µX )m pk −∞
(x − µX )m fX (x) dx 4
k=1
Moment of a ∞ Z ∞
real function g ∑ g (xk )pk g (x)fX (x) dx 5
−∞
E [g (X )] k=1

1 It is the center of gravity of the distribution and the most likely value of X
although when X is Rdiscrete could not take it. To keep as general as possible we
∞
should write: µX = −∞ x dFX .
2 It provides a measure of the dispersion around µX . The standard deviation σX
has the same units as the r.v. X .
3 m = 1 ⇒ α1 = µX .
4 m = 2 ⇒ µ2 = σX2 = V [X ].
5 Generalizes all the previous moments: g (x) = x ⇒ µX ; g (x) = (x − µX )2 ⇒ σX2 ;
etc. Differential Equations and Applications
Random Random Variables: An overview 20
Using the binomial theorem it is easy to show that:

Relationship between moments w.r.t. the origin and central moments

Let X be a r.v. such as αk , 1 ≤ k ≤ m, exist. Then
m  
m
µm = ∑ (−1)k αk (α1 )m−k , m ≥ 0.
k=0 k

For m = 2 one obtains the so–called König theorem:

µ2 = σX2 = E X 2 − (µX )2 = α2 − (α1 )2 .
 

It is also possible to express αm as a function of µk , 0 ≤ k ≤ m:

m  
m
αm = ∑ µk (α1 )m−k , m ≥ 0.
k=0 k

Random Differential Equations and Applications Random Variables: An overview 21

For significant r.v.’s one can give a closed formula for the moments. For example:

Exercise 2: Moments of Gaussian and Lognormal r.v.’s

Prove that the following formulas for the moments of a Gaussian r.v.
X ∼ N(µ; σ 2 ), µ ∈ R, σ > 0:

µX = µ.
σX2 = σ 2.
2m−1
 
 − µ) 2m 
E (X = 0, m = 1, 2, . . .
E (X − µ) = 1 · 3 · 5 · · · (2m − 1) · σ 2m , m = 1, 2, . . .

Prove the following formula for the exponential moment of a Gaussian r.v. (or
equivalently, for a Lognormal r.v.):

λ2
h i
E eλ Z = e 2 , Z ∼ N(0, 1), λ ∈ R.

This formula will be used in Unit 3 when studying an important stochastic

process called the Geometric Brownian Motion or Log-normal stochastic process.

However, an analogous formula for other r.v.’s like X ∼ Bi(n; p) is not available.

Random Differential Equations and Applications Random Variables: An overview 22

The so-called absolute moments w.r.t. the origin play a key role in Probability Theory:

absolute moments w.r.t. the origin

Denomination Discrete Continuous Remarks
Absolute moment ∞ Z ∞
w.r.t. the origin ∑ |xk |m pk −∞
|x|m fX (x) dx 1
βm = E [|X |m ] k=1

1 The great advantage of dealing with βm is that whenever E[|X |m ] exist then the
smaller absolute moments w.r.t. the origin also exist. Indeed, let us assume that
βm < ∞ and let k : k ≤ m. Then

|x| < 1 ⇒ |x|k ≤ 1

 h i
⇒ |x|k ≤ 1+|x|m ⇒ βk = E |X |k ≤ 1+E [|X |m ] = βm +1 < ∞.
|x| ≥ 1 ⇒ |x|k ≤ |x|m
As a consequence, many results are stated assuming the existence of the
involved absolute moments rather than moments.
This summarizes as:
h i
If E[|X |m ] < +∞ ⇒ E X k < +∞, ∀ k ≤ m.

Remarks:
If β2 < ∞ (βk < ∞) then X is called a 2–r.v. (k–r.v.).
In an analogous way one defines the absolute moments w.r.t. the mean:
γm = E [|X − µX |m ] .

Random Differential Equations and Applications Random Variables: An overview 23

As a consequence of the algebraic properties of the sums (for discrete r.v.’s) and
integrals (for continuous r.v.’s) one can state a large number of properties regarding
both: mean and variance. Here we show some of them although later, once the
concept of independence has been introduced, this list will be extended:

Some properties of the mean and variance

Let k, a, b, {ci }ni=1 are constants and {Xi }ni=1 r.v.’s and, f (x) a function. Then:
1 If k is a constant: E [k] = k.
2 If a ≤ X ≤ b ⇒ a ≤ E [X ] ≤ b. (Although if X is a discrete r.v. taking values
{x1 , x2 , . . . , }, it may occur that E [X ] 6= xi , ∀i ∈ {1, 2, . . .}).
" #
n n
3 Linearity: E k ± ∑ ci f (Xi ) = k ± ∑ ci E [f (Xi )]. In particular, taking f (x) = x:
" # i=1 i=1
n n
E k ± ∑ ci Xi = k ± ∑ ci E [Xi ] .
i=1 i=1
4 V [X ] = 0 ⇔ P [X = k] = 1.
5 Invariance under translations: V [aX ± b] = a2 V [X ] .

Random Differential Equations and Applications Random Variables: An overview 24

Moment generating function (m.g.f.) of a r.v. X
Let us assume that X is a (discrete or continuous) r.v. with p.d.f. (pk or fX (x),
respectively). One defines its moment generating function as:
 ∞


 ∑ e uxk pk if X is a discrete r.v., pk = P[X = xk ], k ≥ 1,
k=1
h i 
aX (u) = E e uX =

 Z ∞
e uX fX (x) dx if X is a continuous r.v.


−∞
This real function may not exist. One requires to prove that the series/integral
converges. When it exists, the m.g.f. is an alternative way to characterize the
probability distribution of a r.v.
Notice that if X is a positive r.v., then m.g.f. aX (−u) is just the Laplace
transform of the p.d.f. fX (x).

Let X be a continuous r.v. such as αn exist n ≥ 0 and having a p.d.f. fX (x), then:
Z ∞ ∞ n n
u x
Z ∞
E e uX e ux fX (x) dx
 
aX (u) = = = ∑ fX (x) dx
−∞ −∞ n=0 n!
∞
un ∞ ∞
un u2
Z
= ∑ x n fX (x) dx = ∑ αn = α0 + uα1 + α2 + · · ·
n=0 n! −∞ n=0 n! 2

Relating m.g.f. and moments w.r.t. the origin (this justifies its name!)
dn aX (u) 

αn = n
, n = 0, 1, 2, . . .
du 
u=0

Random Differential Equations and Applications Random Variables: An overview 25

Exercise 3: m.g.f. of a zero–mean Gaussian r.v. X ∼ N(0; σ 2 ), σ > 0
σ 2 u2
aX (u) = e 2 .
Obtain from it, α1 and α2 . Do they match?

m.g.f. has a lot of appealing properties:

Some properties of m.g.f.

If aX (u) = aY (u), ∀u ∈ R, then X and Y have the same probability distribution.
If Y = X + k, k ∈ R then aY (u) = e ku aX (u).
If Y = kX , k ∈ R then aY (u) = aX (ku).

Sometimes aX (t) is more specifically referred to as the m.g.f. w.r.t. the origin since
one can also define the m.g.f. w.r.t. the mean:
h i
mX (u) = E e u(X −µX ) ,

which has the following property:

Relating m.g.f. w.r.t. the mean and moments w.r.t. the origin
dn mX (u) 

µn = , n = 0, 1, 2, . . .
du n u=0

Random Differential Equations and Applications Random Variables: An overview 26

Characteristic function (c.f.) of a r.v. X
Let us assume that X is a (discrete or continuous) r.v. with p.d.f. (pk or fX (x),
respectively). One defines its characteristic function as (i denote the imaginary unit):
 ∞

i 

 ∑ e iuxk pk if X is a discrete r.v., pk = P[X = xk ], k ≥ 1,
k=1
h 
iuX
ϕX (u) = E e =

 Z ∞
e iux fX (x) dx if X is a continuous r.v.,


−∞

where u ∈ R.
This complex function always exists.
ϕX (u) is just the Fourier transform the p.d.f. fX (x). By the Fourier inversion
theorem, one can recover the p.d.f.:

1
Z ∞
fX (x) = e −iux ϕX (u) du.
2π −∞

Relating c.f. and moments w.r.t. the origin

dn ϕX (u) 

= i n αn , n = 0, 1, 2, . . .
du n u=0

Random Differential Equations and Applications Random Variables: An overview 27

Some properties of c.f.
If ϕX (u) = ϕY (u), ∀x ∈ R, then X and Y have the same probability distribution.
ϕX (0) = 1.
|ϕX (u)| ≤ 1.
ϕX (u) is uniformly continuous.
If Y = a + bX , a, b ∈ R then ϕY (u) = e iua ϕX (bu).

Relevant properties related to independence of r.v.’s will be shown later that make
really useful c.f.

Exercise 4: c.f. of a uniform r.v.

Let X ∼ Un([0, 1]) be a uniform r.v. on the interval [a, b]. Check that its c.f. is given
by:
e iub − e iua
ϕX (u) = .
iu(b − a)
Obtain from it, α1 and α2 . Do they match?

Random Differential Equations and Applications Random Variables: An overview 28

Some significant inequalities (I)
Chebyshev (general): If g is measurable and nonnegative and E (g (X ))k
 
exists for some k ≥ 0, then
E (g (X ))k
 
P [g (X ) > t] ≤ , ∀t > 0.
tk

When k = 1, g (X ) = |X | is usually called the Markov inequality:

E [|X |]
P [|X | > t] ≤ , ∀t > 0.
t

Chebyshev: If X is a 2-r.v. then:

1
P [|X − µX | ≥ ασX ] ≤ , α > 0.
α2

It is obtained taking: t = ασX > 0, k = 2 and g (X ) = |X − µX | in the

generalized Chebyshev inequality.

α = 2 ⇒ P [µX − 2σX ≤ X ≤ µX + 2σX ] ≥ 34 = 0.75.

α = 3 ⇒ P [µX − 3σX ≤ X ≤ µX + 3σX ] ≥ 89 = 0.88.
This justify the wide use (and abuse) of 3σ -confidence intervals. But we know:

P [µX − 2σX ≤ X ≤ µX + 2σX ] ≈ 0.95,
X ∼ N(µ; σ 2 ) ⇒
P [µX − 3σX ≤ X ≤ µX + 3σX ] ≈ 0.99,
The relevance of Chebyshev inequality lies on its generality, i.e., it holds for any k–r.v.!
Random Differential Equations and Applications Random Variables: An overview 29
Some significant inequalities (II)
Hölder: If X and Y are r.v.’s such that E [|X |m ] and E [|Y |n ] exist, then
1 1
E [|XY |] ≤ (E [|X |n ])1/n (E [|Y |m ])1/m , m, n ≥ 1 : + = 1.
m n

Schwarz: If X and Y are 2-r.v.’s then


1/2  2 
1/2
E [|XY |] ≤ E |X |2

E |Y | .

It is a particular case of Hölder inequality m = n = 2.

Liapunov: If X is a r.v.’s such that E [|X |n ] exists, then

(E [|X |m ])1/m ≤ (E [|X |n ])1/n , 0 < m < n.


1/2
A significant case (m = 1 < 2 = n): E [|X |] ≤ E |X |2

.
Jensen: Let f be a convex function on R and assume that the following
involved expectations exist, then
f (E [X ]) ≤ E [f (X )]) .

Remark: Taking f (x) = x 2 one gets: σX2 ≥ 0.

Exercise 5: Linking Liapunov and Jensen inequalities

Obtain the Liapunov inequality by applying Jensen inequality.

Random Differential Equations and Applications Random Variables: An overview 30

Some significant inequalities (III)
Minkowski: If E [|X |n ] and E [|Y |n ] exist, then

(E [|X + Y |n ])1/n ≤ (E [|X |n ])1/n + (E [|Y |n ])1/n , n ≥ 1.

Derived from Minkowski’s inequality: If E |Xj |m , 1 ≤ j ≤ n exist, then

 

"  m #
1  n  1 n  m 
 ∑ Xj  ≤ ∑ E Xj  , m ≥ 1.

E
n j=1  n j=1

cs –inequality: If E [|X |s ] and E [|Y |s ] 1 ≤ j ≤ n exist, then


1 if s ≤ 1,
E [|X + Y |s ] ≤ cs (E [|X |s ] + E [|Y |s ]) , cs =
2s−1 if s ≥ 1.

More useful inequalities exist but these notes do not pursue to provide a
comprehensive survey of them.

Random Differential Equations and Applications Random Variables: An overview 31

 Random Variable Transformation (R.V.T.)

The problem can be stated as follows:

Given a (continuous) r.v. X with d.f. FX (x) (p.d.f. fX (x)) and a transformation of it,
Y = r (X ). What is the d.f. GY (y ) (p.d.f. gY (y )) of the new r.v. Y ?

Assuming that r is bijective and denoting by X = s(Y ) its inverse map, one gets
GY (y ) = P[Y ≤ y ] = P[r (X ) ≤ y ] = P[X ≤ s(y )] = FX (s(y )),
dGY (y )
and taking into account that dy = gY (y )
 
 ds(y ) 
gY (y ) = fX (s(y ))  .
dy 
Notice that the modulus assures that gY (y ) ≥ 0.

R.V.T. method: scalar version

H : Let X be a continuous r.v. with p.d.f. fX (x) with support S (X ) and Y = r (X )

being r a bijective mapping.
T : The p.d.f. of Y , gY (y ), is given by:
 
 ds(y ) 
gY (y ) = fX (x = s(y ))  , y ∈ S (r (X )).
dy 

The result can also be stated when X is a discrete r.v.

Random Differential Equations and Applications Random Variables: An overview 32
Example 9: Splitting the domain to compute the p.d.f. of a r.v.
Let X ∼ Un([0, a]), a > 0 and Y = X 2 . Prove the p.d.f. of r.v. Y is given by

1
fY (y ) = √ , 0 < y ≤ a2 .
2a y

We use the d.f.:

GY (y ) = P[Y ≤ y ] = P[X 2 ≤ y ]
√ √ √ √
= P[− y ≤ X ≤ y ] = P[X ≤ y ] − P[X ≤ − y ]
√ √
= FX ( y ) − FX (− y ),
computing the derivative:

1 √ 1 √ 1 √
gY (y ) = √ fX ( y ) + √ fX (− y ) = √ , 0< y ≤ a ≡ 0 < y ≤ a2 .
2 y | {z } 2 y | {z } 2a y
√ =0
= 1a , 0< y ≤a

Exercise 6: Computing the p.d.f. of a Log-normal r.v.

Let X ∼ N(µ; σ 2 ) and Y = e X . Prove the p.d.f. of r.v. Y , usually called log–normal, is
given by
1 (log(Y )−µ)2
−
fY (y ) = √ e 2σ 2 , y > 0.
2πσ y
Check the result obtained in the second part of the Exercise 2.
Random Differential Equations and Applications Random Variables: An overview 33
 Part II

The Hilbert Space of Random Variables

L2

Random Differential Equations and Applications Random Variables: An overview 34

Now we are going to introduce the space L2 of r.v.’s that have finite second moment:
E[X 2 ] < ∞ usually referred to as 2–r.v.’s or second–order r.v.’s. Its presentation is
made by completing an inner product linear space (pre–Hilbert linear space). The
ingredients are:
Let (Ω, FΩ , P) be a probability space and for each A ∈ FΩ let us consider the indicator
function for A (which is a r.v.):

1 if ω ∈ A,
IA : Ω → {0, 1}, IA (ω) = E[IA ] = P[A].
0 if ω ∈ / A,

The linear combinations of indicator functions are called simple r.v.’s:

n n
X (ω) = ∑ ci IAi (ω), E[X ] = ∑ ci P[Ai ].
i=1 i=1

The pre–Hilbert space SRV

SRV = {X : X is a simple r.v. defined on (Ω, FΩ , P)}

SRV is a linear space.

Inner product:
" #
n n n n  
hX , Y i = E[XY ] = E ∑ ∑ c i IA i d j IB j = ∑ ∑ ci dj P Ai ∩ Bj , X , Y ∈ SRV .
i=1 j=1 i=1 j=1

Norm: kX kRV = (E[X 2 ])1/2 .

This space is not complete.
Random Differential Equations and Applications Random Variables: An overview 35
 The Hilbert space HRV = L2

The space SRV can be completed (in many ways) in such a way that SRV be dense in
the new resulting space HRV = L2 . This means:
Completeness: Given a Cauchy sequence in HRV , i.e., such that, for all ε > 0
there exists an integer N so that kXn − Xm kRV < ε when m, n > N. Then, there
is a r.v. X ∈ HRV such that kXn − X kRV −−−→ 0.
n→∞
Dense: Given ε > 0, there is a simple r.v. Y ∈ SRV such that: kX − Y k < ε.
Inner product: hX , Y i = E[XY ].
Norm: kX kRV = (E[X 2 ])1/2 . This norm depends on the probability distribution
through the expectation.

HRV = L2 (Ω) = {X : Ω → R such that kX kRV = kX k2 = (E[X 2 ])1/2 }

The functions of the space HRV are called 2–r.v.’s or second–order r.v.’s.
Remark: If X : Ω → C, then kX k2 = (E[X X ])1/2 = (E[|X |2 ])1/2 , where X stands for
the complex conjugate of X .

Random Differential Equations and Applications Random Variables: An overview 36

Example 9: The Hilbert space L2 ([0, 1])
Sample space: Ω = [0, 1].
σ –algebra: FΩ = σ ((a, b]), (a, b] ⊂ [0, 1].
Probability measure: P[A] = b − a if A = (a, b] ∈ FΩ (Lebesgue measure).
SRV is the space of all simple r.v. defined on FΩ :
n 
1 if x ∈ Ai ,
X (x) = ∑ ci IAi (x), Ai ∈ FΩ , IAi (x) =
i=1
0 if x∈/ Ai .

X (x) = x ⇒ X ∼ Un([0, 1]).

HRV is the completion of SRV and it is usually referred to as
 Z 1 
HRV = L2 ([0, 1]) = Lebesgue measurable functions f on [0, 1] : |f (x)|2 dx < ∞
0

Many of the functions needed in the completion of HRV may be highly

discontinuous and Lebesgue integral must be considered rather than Riemann.
To this space belong continuous r.v.’s on [0, 1]. As the set of piecewise
continuous functions which are square integrable are dense in L2 ([0, 1]), this
space can be roughly considered as the set of the piecewise continuous functions
which are square integrable in [0, 1].
Z 1
Inner product: hX , Y i = X (x)Y (x) dx.
0
Z 1 1/2
Norm: kX kRV = |X (x)|2 dx .
0

Random Differential Equations and Applications Random Variables: An overview 37

Example 10: A weighted normed Hilbert space
Sample space: Ω = {x : −∞ < x < ∞}.
σ –algebra: FΩ = σ ((a, b]), (a, b] ⊂ R.
Probability measure:
(s−µ) 2
1
Z
−
P[A] = p(s)ds, p(s) = √ e 2σ 2 , ∀A ∈ FΩ , µ ∈ R, σ 2 > 0.
A 2πσ 2

SRV is the space of all simple r.v. defined on FΩ :

n 
1 if x ∈ Ai ,
X (x) = ∑ ci IAi (x), Ai ∈ FΩ , IAi (x) =
i=1
0 if x∈/ Ai .

X (x) = x ⇒ X ∼ N(µ; σ 2 ).
HRV is the completion of SRV .
Z ∞
Inner product: hX , Y i = E[XY ] = X (s)Y (s)p(s) ds, X , Y ∈ HRV .
−∞
1
Z ∞ (s−µ)2
−
Norm: (kX kRV )2 = √ |X (s)|2 e 2σ 2 ds, X ∈ HRV .
2πσ 2 −∞

The above integrals are considered in the Lebesgue sense. It is useful to note that the
set of piecewise continuous r.v. X such that
Z ∞
|X (s)|2 p(s) ds < ∞, is dense in HRV .
−∞

Random Differential Equations and Applications Random Variables: An overview 38

Exercise 7: A sequence of convergent r.v.’s in HRV = L2 ([0, 1])
Let Y ∼ Un([0, 1]) and

1 1

{Xn }∞ 2 Y (x) if n ≤ Y (x) ≤ 1, ∀x ∈ [0, 1].
n=1 : Xn (x) =
0 otherwise,

1
Prove that {Xn }∞
n=1 ⊂ HRV is a Cauchy sequence such that it converges to 2Y as
n → ∞.

Exercise 8: A nonconvergent sequence of r.v.’s in HRV = L2 ([0, 1])

In the context of the previous example. Let {Yn }∞ n=1 be a sequence of independent
r.v.’s such as Yn ∼ Un([0, 1]) for each n ≥ 1. Let X = 1 and {Xn }∞
n=1 be the sequence

√1

1 if n
≤ Yn (x) ≤ 1,
Xn (x) = ∀x ∈ [0, 1].
1 + nYn (x) otherwise,

Prove that {Yn }∞ ∞

n=1 ⊂ HRV and {Xn }n=1 is not a Cauchy sequence and

n→∞
kXn − X kRV −−−→ ∞.

Random Differential Equations and Applications Random Variables: An overview 39

 The Banach space Lp (Ω), p ≥ 1

So far we have introduced the Hilbert space HRV = L2 (Ω) on a probability space
(Ω, FΩ , P). This space has the nice property that one can define on it the norm:

kX kRV = (E[X 2 ])1/2 ,

which comes from the inner product

 p 
hX , Y i = E[XY ] ⇒ kX kRV = + hX , X i = (E[X 2 ])1/2 .

Now, we introduce the following space (for r.v.’s X such as the involved expectation
exists):

Lp (Ω) = {X : Ω → R such that kX kp = (E[X p ])1/p < ∞}, p ≥ 1.

One can demonstrate that:

Lp (Ω) is a linear space (by Minkowsky inequality).
kX kp is a norm. This norm only comes from an inner product when p = 2. This
means that we can only takes advantage of orthogonality in L2 (Ω).
(Lp (Ω), k · kp ) is a Banach space and (L2 (Ω), k · k2 = k · kRV ) is a Hilbert space.

Random Differential Equations and Applications Random Variables: An overview 40

Next, we state some significant properties and facts that will be play a relevant role in
some sessions of this course.
Lq (Ω) ⊂ Lp (Ω) for 1 ≤ p ≤ q. (It is a consequence of Liapunov inequality).
(Lp (Ω), k · kp ) is not a Banach algebra: kXY kp  kX kp kY kp unless X and Y are
independent r.v.’s (independence concept will be introduced in the Unit 2). In
fact, let Z be a positive r.v. and p = 2, then
 
√ √  (kXY k2 )2 = E[(XY )2 ] = E[Z 2 ], 
X = Z,Y = Z ⇒
(kX k2 kY k2 )2 = E[X 2 ]E[Y 2 ] = E[Z ]E[Z ] = (E[Z ])2
 

Taking into account that: V[Z ] = E[Z 2 ] − (E[Z ])2 ≥ 0, one deduces that:

(kXY k2 )2 ≥ (kX k2 kY k2 )2 ⇒ kXY k2 ≥ kX k2 kY k2 just the reverse!

Unfortunately independence can be an unacceptable hypothesis in many cases

where we want to bound the p–norm of the product of two or more r.v.’s!
We can overcome this drawback using the Schwarz inequality is easy to obtain
that:
kXY k2 ≤ kX k4 kY k4 , ∀X , Y ∈ L4 .
In general,
kXY kp ≤ kX k2p kY k2p , ∀X , Y ∈ L2p , p ≥ 1.

Random Differential Equations and Applications Random Variables: An overview 41

We close this Part of UNIT 1, pointing out that even simplest algebraic properties
involving m.s. convergence (this is the convergence associated to kX k2 ) require to be
proven. For example:
)
A ∈ L2 , m.s.
m.s.
{Xn : n ≥ 0} ⊂ L2 : Xn −−−→ X ∈ L2 ; AXn −−−→ AX .
n→∞
n→∞

Notice that if A and {Xn : n ≥ 0} are assumed to be independent for each n, then
above property holds. Unfortunately, often independence hypothesis cannot be
embraced in our context. Nevertheless, we can impose conditions involving
information belonging to L4 (specifically related to the so-called mean fourth
convergence, i.e., the convergence associated to kX k4 ) to establish this basic property.
)
A ∈ L4 , m.s.
m.f. ; AXn −−−→ AX .
{Xn : n ≥ 0} ⊂ L4 : Xn −−−→ X ∈ L4 n→∞
n→∞

Random Differential Equations and Applications Random Variables: An overview 42

 Part III

Convergence of Random Variables

Random Differential Equations and Applications Random Variables: An overview 43

Convergence in p–th mean
A sequence {Xn }n≥1 ⊂ Lp (Ω) is said to be p–th mean convergent to X ∈ Lp (Ω) if one
holds:  
p−th
Xn −−−→ X ⇐⇒ kXn − X kp −−−→ 0. ≡ E[|Xn − X |p ] −−−→ 0 .
n→∞ n→∞ n→∞

It is assumed in this definition that E[|Xn |p + |X |p ] < ∞, ∀n ≥ 1.

Particular cases (and assuming the existence of the involved expectations):

Convergence in mean: When p = 1
m
Xn −−−→ X ⇐⇒ E[|Xn − X |] −−−→ 0.
n→∞ n→∞

Convergence in mean square: When p = 2

m.s.
Xn −−−→ X ⇐⇒ l.i.m. Xn = X ⇐⇒ E[|Xn − X |2 ] −−−→ 0.
n→∞ n→∞ n→∞

As a consequence of Schwartz inequality one gets:

m.s. m
Xn −−−→ X =⇒ Xn −−−→ X
n→∞ n→∞

or more generally (by Liapunov inequality):

p−th q−th
Xn −−−→ X ⇒ Xn −−−→ X , p ≥ q ≥ 1.
n→∞ n→∞

Random Differential Equations and Applications Random Variables: An overview 44

Exercise 9: Studying the m.s. convergence of some sequences of r.v.’s
1 Let {Xn }n≥1 be a sequence of r.v.’s defined by:

1 n
Xn = ∑ Yj , {Yn }n≥1 i.i.d. r.v.’s of mean µ and variance σ 2 > 0.
n j=1

Prove that {Xn }n≥1 ⊂ L2 (Ω) is a Cauchy sequence and its m.s. limit is µ.
Remember that {Xn }n≥1 is a Cauchy sequence if for all ε > 0 there exists an
integer N so that kXn − Xm k2 < ε when m, n > N.
2 Let {Xn }n≥1 be a sequence of i.i.d. r.v.’s. Prove that {Xn }n≥1 ⊂ L2 (Ω) is m.s.
convergent to 0.
1 if P[Xn (ω) = 1] = n1 ,

Xn (ω) = Xn = n ≥ 1.
0 if P[Xn (ω) = 0] = 1 − n1 ,

3 Let X ∼ Un([0, 1]) and {Xn }n≥1 be a sequence of r.v.’s. Prove that
{Xn }n≥1 ⊂ L2 (Ω) is m.s. convergent to X.

0 ≤ X (ω) ≤ n12 ,

0 if
Xn (ω) = 1 n ≥ 1.
X (ω) if n2
≤ X (ω) ≤ 1,
4 Let {Xn }n≥1 be a sequence of i.i.d. r.v.’s. Prove that {Xn }n≥1 ⊂ L2 (Ω) is not
m.s. convergent. (Hint: 
Check it is not a Cauchy sequence).
n if P[Xn (ω) = n] = n12 ,
Xn (ω) = Xn = n ≥ 1.
0 if P[Xn (ω) = 0] = 1 − n12 ,

Random Differential Equations and Applications Random Variables: An overview 45

The following properties make very attractive m.s. convergence.

Significant properties of m.s. convergence

H : Let {Xn } ⊂ L2 (Ω) be a sequence such that

m.s.
Xn −−−→ X .
n→∞

T : Then
E[Xn ] −−−→ E[X ], V[Xn ] −−−→ V[X ].
n→∞ n→∞

Try to provide a proof of this important result!

Exercise 10: Checking m.s. properties about some m.s. convergence sequences of r.v.’s
Check this pair of properties on the m.s. convergent sequences 2 and 3 of the previous
exercise.

Random Differential Equations and Applications Random Variables: An overview 46

Convergence almost surely or with probability 1
A sequence {Xn }n≥1 is said to be almost surely convergent to X (or to converge with
probability 1 (w.p. 1)) if one holds:

a.s.
Xn −−−→ X ⇐⇒ P[{ω ∈ Ω : lim |Xn (ω) − X (ω)| = 0}] = 1.
n→∞ n→∞

An equivalent definitionof a.s. convergence

"( is the following: )#
 \

 lim P ω ∈ Ω : |Xj (ω) − X (ω)| ≤ ε = 1,
 n→∞


 j≥n
a.s.
Xn −−−→ X ⇐⇒
n→∞ 

"( )#
 [
lim ∈ Ω : |X (ω) − X (ω)| > = 0.



 n→∞ P ω j ε
j≥n

The following result is very useful in applications:

Sufficient condition to a.s. convergence

H : Let {Xn }n≥1 be a sequence such that ∀ ε > 0 holds:

∞
∑ P[{ω ∈ Ω : |Xn (ω) − X (ω) ≥ ε|] < ∞.
n=1

T : Then a.s.
Xn −−−→ X .
n→∞

Random Differential Equations and Applications Random Variables: An overview 47

Exercise 11: An example of a.s. convergent sequence of r.v.’s
Let X ∼ Un([0, 1]) and {Xn }n≥1 be a sequence of r.v.’s. Prove that {Xn }n≥1 ⊂ L2 (Ω) is
a.s. convergent to X.
if 0 ≤ X (ω) ≤ n12 ,

0
Xn (ω) = n ≥ 1.
X (ω) if n12 ≤ X (ω) ≤ 1,

Example 11: m.s. convergence (and hence convergence in probability) does not imply
a.s. convergence
Let {Xn }n≥1 be a sequence of i.i.d. r.v.’s. It was already shown that it converges in
m.s. sense to the zero r.v. Prove that it is not a.s. convergent to X ≡ 0.

1 if P[Xn (ω) = 1] = n1 ,

Xn (ω) = Xn = n ≥ 1.
0 if P[Xn (ω) = 0] = 1 − n1 ,

   
 \   \ 
lim P  ω ∈ Ω : |Xj (ω) − X (ω)| ≤ ε  = lim P  ω ∈ Ω : Xj (ω) = 0 
n→∞   n→∞  
j≥n   j≥n 
1 1
= lim 1 − 1− ···
n→∞ n n + 1
∞  1
= lim ∏ 1 −
n→∞
j=0 n+j
!
∞ 1
= lim exp − ∑ = 0 6= 1.
n→∞
j=0
n + j

Random Differential Equations and Applications Random Variables: An overview 48

Convergence in probability
A sequence {Xn }n≥1 is said to converge in probability to X if ∀ ε > 0 one holds:

p
Xn −−−→ X ⇐⇒ lim P[{ω ∈ Ω : |Xn (ω) − X (ω)| > ε}] = 0.
n→∞ n→∞

Exercise 12: An example of sequence of r.v.’s convergent in probability

Prove that the following sequence of r.v.’s {Xn }n≥1 converges in probability to
X (ω) ≡ −1, but it does not converge to this value in the m.s. sense.
if P[{ω ∈ Ω : Xn (ω) = n}] = n1 ,

n
Xn (ω) = n ≥ 1.
−1 if P[{ω ∈ Ω : Xn (ω) = −1}] = 1 − n1 ,

Check that: lim E[Xn ] 6= E[X ].

n→∞

Remark: This exercise shows that convergence in probability does not imply m.s.
convergence. This is usually stated as m.s. convergence is a strong type of
convergence whereas c.p. is of weak type.

A useful result that links convergence in probability and a.s. convergence is:
p a.s.
Xn −−−→ X ⇒ ∃ a subsequence {Xnk } : Xnk −−−→ X .
n→∞ k→∞

Random Differential Equations and Applications Random Variables: An overview 49

Convergence in distribution
A sequence {Xn }n≥1 is said to converge in distribution to X if for all bounded and
continuous function f one holds:
d
Xn −−−→ X ⇐⇒ lim E[f (Xn )] = E[f (X )].
n→∞ n→∞

This is equivalent to:

lim FXn (x) = F (x), ∀x where the d.f. FX (x) is continuous,

n→∞

where FX (x) and FXn (x) are the d.f.’s of X and Xn , respectively.

Sufficient condition to convergence in distribution

H : Let {Xn }n≥1 and X r.v.’s such as their m.g.f. are given by {αn (t)}n≥1 and αn (t),
respectively. Let us assume that
lim αn (t) = α(t), ∀t.
n→∞

T : Then d
Xn −−−→ X .
n→∞

Random Differential Equations and Applications Random Variables: An overview 50

Example 12: An example of sequence of r.v.’s convergent in distribution
Let (Ω, FΩ , P) a probability space where:
Ω = {x : 0 ≤ x ≤ 1}.
FΩ the σ –algebra of Borel sets generated by intervals of the form: (a, b] ⊂ [0, 1].
P is the probability measure defined in the following.
In this probability space, we consider the sequence of r.v.’s {Xn }n≥3 ,
Xn ∼ Un([1/n, 1 − 1/n]) and the r.v. X ∼ Un([0, 1]) so that Xn is independent of X for
each n ≥ 3. Then:
Z x  n
n−2 if s ∈ [1/n, 1 − 1/n],
FXn (x) = pn (s) ds, pn (s) =
0 0 otherwise,
Z x 
1 if s ∈ [0, 1],
FX (x) = p(s) ds, p(s) =
0 0 otherwise,

Then, for any f ∈ C ([0, 1]) one gets:

Z 1−1/n Z 1
lim f (x)pn (x) dx = f (x)p(x) dx.
n→∞ 1/n 0

d
This proves that: Xn −−−→ X .
n→∞

Exercise 13: Convergence in distribution does not imply m.s. convergece

Prove that the sequence defined in the previous example does not converge in m.s.
since
E[|Xn − X |2 ] −−−→ 1/6.
n→∞
Random Differential Equations and Applications Random Variables: An overview 51
 Relationship between the different types of convergence for r.v.’s

This ranking can be justified intuitively: convergence in distribution needs only a

knowledge of the univariate distribution; convergence in mean square sense and
probability need only a knowledge of the bivariate distribution and convergence a.s.
needs the knowledge of all multivariate distributions.

Random Differential Equations and Applications Random Variables: An overview 52

Some of the arrows of the previous diagram can be established easily:

1 : It is a direct application of Chebyshev inequality (try it!). But the converse

implication is not true, in general, as it was shown in a previous exercise.
2 : See Pag. 83–84 of reference [3] for the proof, for example. The same
example referred above serves to show that the converse is not true, in general.
3 : See Pag. 85–86 of reference [3] for the proof, for example. The converse
implication is not true: A sequence of i.i.d. r.v.’s obviously converge in
distribution but fails to converge in probability.
4 : It was shown in a previous counterexample.

5 : See next example.

It must be underlined that one can establish relationship between all types of
convergence but adding very technical (and sometimes restrictive) hypotheses.

Random Differential Equations and Applications Random Variables: An overview 53

Example 13: a.s. convergence (and hence convergence in probability) does not imply
m.s. convergence
Let {Xn }n≥1 be a sequence of i.i.d. r.v.’s. Prove that {Xn }n≥1 ⊂ L2 (Ω) is not m.s.
convergent. (Hint: Check it is not a Cauchy sequence).
n if P[Xn (ω) = 1] = n12 ,

Xn (ω) = Xn = n ≥ 1.
0 if P[Xn (ω) = 0] = 1 − n12 ,

"( )# "( )#
[ [
lim P ω ∈Ω: |Xj (ω) − X (ω)| > ε = lim P ω ∈Ω: Xj (ω) = j
n→∞ n→∞
j≥n j≥n
∞
1
≤ lim ∑ j2
n→∞
j=n
= 0.

Random Differential Equations and Applications Random Variables: An overview 54

 References
1 V. Quesada, A. Garcı́a (1988): Lecciones de Cálculo de Probabilidades, Ed.
Dı́az de Santos, Madrid. (Spanish)
2 M. Loève (1977): Probability Theory, Vol. I-II, Graduate Texts in Mathematics,
Springer Verlag, New York.
3 T.T. Soong (1973): Random Differential Equations in Science and Engineering,
Academic Press, New York.

Random Differential Equations and Applications Random Variables: An overview 55