Random variables Specifying the probability distribution Expected value of a random variable

Statistics (Part II)

WS 2016/17
Random variables Specifying the probability distribution Expected value of a random variable

Outline

1 Random variables (p. 86 ff. (Chap. 3+4))

Basic definitions

2 Specifying the probability distribution

Discrete RV
Continuous RV
Determining the probability of an event for a RV

3 Expected value of a random variable

Expected value of a RV
Expected value of a function of a RV
Properties of the expected value
Variance of a (function of a) RV

Source of images: Wackerly, Mendenhall and Scheaffer: Mathamatical Statistics with Applications. Thomson
Brooks/Cole
Random variables Specifying the probability distribution Expected value of a random variable

Basic definitions

A random variable is a function of the simple simple events, i.e. a

real-valued function defined over the sample space:
f : Ω→R
ω → f (ω)
A random variable is represented by upper-case letters Y = f (ω), the
particular value of the random variable is denoted as a lower-case letter y .
Remark: The function f is not random!
Example: In a game with a balanced die the player wins 10 e for a face
number larger than 4. He loses 5 e otherwise.
The player is not interested in the probability of the simple events of
tossing the die (1,2,. . . ,6) but whether he wins or loses money. However,
what is random is the face number that occurs after tossing the die. The
win or loss as a result of a particular face number is deterministic. The
value of f is only random, since we do not know which simple event
(sample point, argument of f ) will occur. ⇒ Only the arguments of f are
random!
What is the probability of winning 10 e?
2
P(Y = 10) = P(ω = 5 or ω = 6) =
6
Random variables Specifying the probability distribution Expected value of a random variable

Basic definitions

A random variable (RV) is discrete if it can assume only a finite or

countably infinite number of discrete values. If the RV can take a value in
an interval it is said to be continuous. (Countably infinite means that
there is a one-to-one correspondence to the positive integers.)
The collection of the probabilities of a discrete RV is called probability
distribution of the RV, i.e. a probability distribution of a discrete random
variable is the assignment of probabilities to each of the possible values of
the RV. The sum of these probabilities must equal 1.
Since it is mathematically impossible to assign nonzero probabilities to
each value on a line interval while satisfying the requirements that the sum
of these probabilities equals 1, the probability distribution of continuous
RV is described by a nonnegative function denoted as probability density
function.
Random variables Specifying the probability distribution Expected value of a random variable

Specifying the probability distribution of a discrete RV

Probability mass function (→ graphically: barplot): function that assigns

probabilities to the values that Y can assume. P(Y = y ) = p(y ) for all y ,
where p(y ) is the sum of the probabilities of all sample points that have
the same value y = f (ω).
Cumulative distribution function (cdf) denoted as F (y ) (→ graphically:
step-function)
A cdf of a discrete random variable is defined as
F (y ) = P(Y ≤ y ) ∀y ∈ R .
It jumps at yi , where pi = P(Y = yi ) > 0, and the size of the jump is pi .
Properties of the cdf:
(a) lim F (y ) = 1 and lim F (y ) = 0
y →∞ y →−∞

(b) F non − decreasing

(for continuous RV see p.160 in textbook)
Indepence: Two random variables X and Y are independent when
P(X = xi , Y = yj ) = P(X = xi ) · P(Y = yj )
Random variables Specifying the probability distribution Expected value of a random variable

Specifying the probability distribution of a continuous RV (p. 158 ff)

Probability density function (→ graphically: density function f (y )):

function which can be seen as a theoretical model of the relative frequency
distribution (as given by a histogram) where n → ∞.
Properties of a density function

Cumulative distribution function (cdf) denoted as F (y ) (→ graphically:

continuous, non-decreasing function)
A cdf of a continuous random variable is defined as

F (y ) = P(Y ≤ y ) ∀y ∈ R .

In contrast to a cdf of a discrete random variable that increases only at

the finite (or countable) number of points with positive probability, the cdf
of a continuous RV is a monotonic, non-decreasing continuous function.
Random variables Specifying the probability distribution Expected value of a random variable

Specifying the probability distribution of a continuous RV (p. 158 ff)

Properties of the cdf:

A random variable with cdf F (y ) is continuous if F (y ) is continuous for

−∞ < y < ∞.
If F (y ) is the cdf of a continuous random variable. Then f (y ), given as
dF (y )
f (y ) = = F ′ (y )
dy
wherever the derivative exists, is the probability density function of Y .
Thus, the cdf is found as
Z ∞
F (y ) = f (t) dt
−∞

Remark: The probability density function of a continuous RV need not be

continuous.
Random variables Specifying the probability distribution Expected value of a random variable

Quantiles (p. 164)

Let 0 < p < 1 be the p-th quantile of a random variable. It is defined as

the smallest value yp such that P(Y ≤ yp ) = F (yp ) ≥ p. If Y is
continuous, yp is the smallest value such that F (yp ) = F (Y ≤ yp ) = p.
If p = 0.5, the quantile is denoted as median.
For continuous RV: yp = F −1 (p).
Example: A discrete RV Y represents three different incomes: small,
medium and large, with probabilities
yi small medium large
pi 0.2 0.5 0.3
What is the 0.25-Quantile? → y0.25 =medium
In a sample containing the following observations: s,s,s,s,s,m,m,m,l,l
(ordered sample), what is the median (y0.5 )?
Empirical distribution:
yi small medium large
hi 0.5 0.3 0.2
Smallest value for which F (yp ) ≥ 0.5 holds is yp = small.
Random variables Specifying the probability distribution Expected value of a random variable

Determining the probability of an event of a discrete RV

Using the probability mass function: “add corresponding probabilities pi ”

Example: Tossing unbalanced die
yi 1 2 3 4 5 6
pi 0.3 0.2 0.1 0.2 0.15 0.05
P(face number odd) = P(Y ∈ {1, 3, 5}) = 0.3 + 0.1 + 0.15 = 0.55
Using the cdf: (Attention when RV is discrete!!)
Example above:
yi 1 2 3 4 5 6
pi 0.3 0.2 0.1 0.2 0.15 0.05
Fi = P(Y ≤yi ) 0.3 0.5 0.6 0.8 0.95 1

P(face number ≤ 4) = P(Y ∈ {1, 2, 3, 4}) = 0.8 = F (4)

P(face number ≥ 4) = 1 − P(Y < 4) = 1 − F (3) = 1 − 0.6 = 0.4
P(2 ≤ face number ≤ 5) = P(2 ≤ Y ≤ 5) = P(Y ≤ 5) − P(Y < 2) =
| {z } | {z }
=F (5) =F (1)

= F (5) − F (1) = 0.95 − 0.3 = 0.6

Random variables Specifying the probability distribution Expected value of a random variable

Determining the probability of an event of a continuous RV

Using the probability density function: “area under the density”

Z b
P(Y ∈ [a, b]) = f (t) dt
a

Since P(Y = a) = 0, there is no matter whether the interval is closed or

open, i.e.

P(Y ∈ [a, b]) = P(Y ∈ ]a, b]) = P(Y ∈ [a, b[) = P(Y ∈ ]a, b[ )

Using the cdf:

P(Y ∈ [a, b]) = F (b) − F (a)

Random variables Specifying the probability distribution Expected value of a random variable

Expected value of a RV

The expected value is a measure to describe a probability mass function or

a probability density function.
P
Provided that i |yi |pi < ∞ for a discrete RV:

The expected value of a discrete RV is a weighted average. It represents

the “center of mass”.
R∞
Similarly, provided that |y | f (y ) dy < ∞ for a continuous RV:
−∞
Random variables Specifying the probability distribution Expected value of a random variable

Determining the expected value of a function of a RV

Discrete random variable

Proof: Let Y be a discrete RV with values y1 , . . . , yn and corresponding

probabilities P(Y = yi ) = pi , i = 1, . . . , n. Let g (Y ) be a function of Y
with values g1 , . . . , gm (g need not be a one-to-one mapping, i.e. m ≤ n).
Consequently, g (Y ) is a RV such that for i = 1, . . . , m

And thus, we obtain

Random variables Specifying the probability distribution Expected value of a random variable

Expected value of a function of a RV

Continuous random variable

Example: Find the expected value of the following distributions:

P(Y = k) = q k−1 p k = 1, 2, . . . , where q = 1 − p, p ∈ [0, 1],

k
λ −λ
P(X = k) = e , k = 0, 1, 2, . . . , λ > 0
k!
1 (x−µ)2
−
f (x) = √ e 2σ2 −∞<x <∞
2 πσ
1 1
f (x) = −∞<x <∞
π 1 + x2
Random variables Specifying the probability distribution Expected value of a random variable

Properties of the expected value

Let c be a constant and p(y ) a probability mass function. and g (Y ),

g1 (Y ), . . . , gk (Y ) be a functions of the RV Y .
E (c) = c.
Proof:

E (cg (Y )) = cE (g (Y ))
Proof:
Random variables Specifying the probability distribution Expected value of a random variable

Properties of the expected value

E (g1 (Y ) + · · · + gk (Y )) = E (g1 (Y )) + · · · + E (gk (Y ))

Proof:

The same holds for continuous RVs where the sum is replaced by the
corresponding integral.
Let g be a linear function. Then the following properties hold:

E (a Y + b) = a E (Y ) + b
Random variables Specifying the probability distribution Expected value of a random variable

Determining the variance of a RV

Discrete random variable

The variance σ 2 (or the standard deviation σ) is a measure of risk.

Theorem of Steiner
Var (Y ) = E (Y 2 ) − µ2

Let g be a linear function. Then the following properties hold:

Var (a Y + b) = a2 Var (Y )