Chapter 2.

Random Variables
2.1
2.2
2.3
2.4
2.5
2.6

NIPRL

Discrete Random Variables

Continuous Random Variables
The Expectation of a Random Variable
The Variance of a Random Variable
Jointly Distributed Random Variables
Combinations and Functions of Random Variables

2.1 Discrete Random Variable

2.1.1 Definition of a Random Variable (1/2)

Random variable
A numerical value to each outcome of a particular experiment
S

-3
NIPRL

-2

-1

2.1.1 Definition of a Random Variable (2/2)

Example 1 : Machine Breakdowns
Sample space : S {electrical , mechanical , misuse}
Each of these failures may be associated with a repair cost
State space : {50, 200,350}
Cost is a random variable : 50, 200, and 350

NIPRL

2.1.2 Probability Mass Function (1/2)

Probability Mass Function (p.m.f.)
pi
A set of probability value
assigned
to each of the values taken
by the discrete random variable
xi
0 pi 1 and
i pi 1
Probability : P ( X xi ) pi

NIPRL

2.1.2 Probability Mass Function (1/2)

Example 1 : Machine Breakdowns
P (cost=50)=0.3, P (cost=200)=0.2,
P (cost=350)=0.5
0.3 + 0.2 + 0.5 =1
f ( x)

xi

50

200

350

pi

0.3

0.2

0.5

0.5
0.3
0.2

50
NIPRL

200

350

Cost($)

2.1.3 Cumulative Distribution Function (1/2)

Cumulative Distribution Function
Function : F ( x) P ( X x) F ( x)
Abbreviation : c.d.f

P( X y )

y: y x

F ( x)
1.0
0.5
0.3
0
NIPRL

50

200

350

x($cost)

2.1.3 Cumulative Distribution Function (2/2)

Example 1 : Machine Breakdowns

x 50 F ( x) P (cost x) 0
50 x 200 F ( x) P(cost x) 0.3
200 x 350 F ( x) P(cost x) 0.3 0.2 0.5
350 x F ( x) P(cost x) 0.3 0.2 0.5 1.0

NIPRL

2.2 Continuous Random Variables

2.2.1 Example of Continuous Random Variables (1/1)

Example 14 : Metal Cylinder Production

Suppose that the random variable
is the diameter of a
X
randomly chosen cylinder manufactured by the company. Since
this random variable can take any value between 49.5 and
50.5, it is a continuous random variable.

NIPRL

2.2.2 Probability Density Function (1/4)

Probability Density Function (p.d.f.)
Probabilistic properties of a continuous random variable

f ( x) 0

statespace

NIPRL

f ( x)dx 1

2.2.2 Probability Density Function (2/4)

Example 14
Suppose that the diameter of a metal cylinder has a p.d.f

f ( x) 1.5 6( x 50.2) 2 for 49.5 x 50.5

f ( x) 0, elsewhere
f ( x)

49.5
NIPRL

50.5

2.2.2 Probability Density Function (3/4)

This is a valid p.d.f.

50.5

49.5

(1.5 6( x 50.0) 2 ) dx [1.5 x 2( x 50.0)3 ]50.5

49.5
[1.5 50.5 2(50.5 50.0)3 ]
[1.5 49.5 2(49.5 50.0)3 ]
75.5 74.5 1.0

NIPRL

2.2.2 Probability Density Function (4/4)

The probability that a metal cylinder has a diameter between 49.8
and 50.1 mm can be calculated to be

50.1

49.8

(1.5 6( x 50.0) 2 ) dx [1.5 x 2( x 50.0) 3 ]50.1

49.8
[1.5 50.1 2(50.1 50.0) 3 ]
[1.5 49.8 2(49.8 50.0) 3 ]
75.148 74.716 0.432

f ( x)

49.5
NIPRL

49.8

50.1

50.5

2.2.3 Cumulative Distribution Function (1/3)

Cumulative Distribution Function

F ( x) P( X x)

f ( y )dy

dF ( x )
f ( x)
dx

P ( a X b) P ( X b) P ( X a )
F (b) F (a )
P ( a X b) P (a X b)
NIPRL

2.2.2 Probability Density Function (2/3)

Example 14
x

F ( x) P( X x) (1.5 6( y 50.0) 2 )dy

49.5

x
[1.5 y 2( y 50.0)3 ]49.5

[1.5 x 2( x 50.0)3 ] [1.5 49.5 2(49.5 50.0)3 ]

1.5 x 2( x 50.0)3 74.5

P (49.7 X 50.0) F (50.0) F (49.7)

(1.5 50.0 2(50.0 50.0)3 74.5)
(1.5 49.7 2(49.7 50.0)3 74.5)
0.5 0.104 0.396
NIPRL

2.2.2 Probability Density Function (3/3)

P(49.7 X 50.0) 0.396

1

P ( X 50.0) 0.5

F ( x)

P( X 49.7) 0.104
49.5

NIPRL

49.7

50.0

50.5

2.3 The Expectation of a Random Variable

2.3.1 Expectations of Discrete Random Variables (1/2)

Expectation of a discrete random variable with p.m.f

P( X xi ) pi
E( X )

px
i

Expectation of a continuous random variable with p.d.f f(x)

E( X )

state space

xf ( x ) dx

The expected value of a random variable is also called the mean of

the random variable
NIPRL

2.3.1 Expectations of Discrete Random Variables (2/2)

Example 1 (discrete random variable)
The expected repair cost is

E (cost) ($50 0.3) ($200 0.2) ($350 0.5) $230

NIPRL

2.3.2 Expectations of Continuous Random Variables (1/2)

Example 14 (continuous random variable)
The expected diameter of a metal cylinder is

E( X )

50.5

49.5

x(1.5 6( x 50.0) 2 ) dx

Change of variable: y=x-50

E ( x)

0.5

0.5

0.5

0.5

( y 50)(1.5 6 y 2 )dy
(6 y 3 300 y 2 1.5 y 75)dy

[3 y 4 / 2 100 y 3 0.75 y 2 75 y ]0.5

0.5
[25.09375] [ 24.90625] 50.0
NIPRL

2.3.2 Expectations of Continuous Random Variables (2/2)

Symmetric Random Variables
If x has a p.d.f f ( xthat
is
)
symmetric about a point
so that

f ( x)

E( X )

f ( x) f ( x)

Then,

E( X )

(why?)

So that the expectation of the

random variable is equal to the
point of symmetry

NIPRL

E ( X ) xf ( x)dx

xf ( x)dx + xf ( x)dx

y 2 x

xf ( x)dx +

NIPRL

yf ( y )dy

2.3.3 Medians of Random Variables (1/2)

Median
Information about the middle value of the random variable

F ( x) 0.5
Symmetric Random Variable
If a continuous random variable is symmetric about a point ,
then both the median and the expectation of the random variable

are equal to

NIPRL

2.3.3 Medians of Random Variables (2/2)

Example 14

F ( x) 1.5 x 2( x 50.0)3 74.5 0.5

x 50.0

NIPRL

2.4 The variance of a Random Variable

2.4.1 Definition and Interpretation of Variance (1/2)

Variance( )
A positive quantity that measures the spread of the distribution
of the random variable about its mean value
Larger values of the variance indicate that the distribution is
more spread out
2

Definition:

Var( X ) E (( X E ( X )) 2 )
E ( X 2 ) ( E ( X )) 2

Standard Deviation
The positive square root of the variance
Denoted by

NIPRL

2.4.1 Definition and Interpretation of Variance (2/2)

Var( X ) E (( X E ( X )) 2 )
E ( X 2 2 XE ( X ) ( E ( X )) 2 )
E ( X 2 ) 2 E ( X ) E ( X ) ( E ( X )) 2
E ( X 2 ) ( E ( X )) 2

f ( x)

Two distribution with

identical mean values but
different variances

x
NIPRL

2.4.2 Examples of Variance Calculations (1/1)

Example 1

Var( X ) E (( X E ( X )) 2 ) pi ( xi E ( X )) 2
i

0.3(50 230) 2 0.2(200 230) 2 0.5(350 230) 2

17,100 2

17,100 130.77

NIPRL

2.4.3 Chebyshevs Inequality (1/1)

Chebyshevs Inequality
If a random variable has a mean

a variance
and

P ( c X c ) 1
For example, taking

1
c2
for c 1

c gives
2

1
P ( 2 X 2 ) 1 2 0.75
2

NIPRL

, then
2

 Proof

2
(
x

)
f ( x)dx

( x ) 2 f ( x) dx c 2 2

| x | c

f ( x)dx.

| x | c

P(| x | c ) 1/ c 2
P (| x | c ) 1 P(| x | c ) 1 1/ c

NIPRL

2.4.4 Quantiles of Random Variables (1/2)

Quantiles of Random variables
The pth quantile of a random variable X

F ( x) p

A probability of pthat the random variable takes a value less

p
than the th quantile
Upper quartile
The 75th percentile of the distribution
Lower quartile
The 25th percentile of the distribution
Interquartile range
The distance between the two quartiles

NIPRL

2.4.4 Quantiles of Random Variables (2/2)

Example 14

F ( x) 1.5 x 2( x 50.0)3 74.5 for 49.5 x 50.5

Upper quartile :

F ( x) 0.75 x 50.17

Lower quartile :

F ( x) 0.25

Interquartile range :

NIPRL

x 49.83

50.17 49.83 0.34

2.5 Jointly Distributed Random Variables

2.5.1 Jointly Distributed Random Variables (1/4)

Joint Probability Distributions

Discrete

P ( X xi , Y y j ) pij 0
satisfying

ij

Continuous

f ( x, y ) 0 satisfying

NIPRL

state space

f ( x, y )dxdx 1

2.5.1 Jointly Distributed Random Variables (2/4)

Joint Cumulative Distribution Function
Discrete

F ( x, y ) P ( X xi , Y y j )
Continuous

F ( x, y )

i: xi x j: y j y

F ( x, y )

NIPRL

pij

f ( w, z )dzdw

2.5.1 Jointly Distributed Random Variables (3/4)

Example 19 : Air Conditioner Maintenance
A company that services air conditioner units in residences and
office blocks is interested in how to schedule its technicians in
the most efficient manner
The random variable X, taking the values 1,2,3 and 4, is the
service time in hours
The random variable Y, taking the values 1,2 and 3, is the
number of air conditioner units

NIPRL

2.5.1 Jointly Distributed Random Variables (4/4)

Y=
number
of units

Joint p.m.f

X=service time

ij

0.12 0.18

0.12

0.08

0.07

0.05

Joint cumulative distribution

function

0.08

0.15

0.21

0.13

F (2, 2) p11 p12 p21 p22

NIPRL

0.01

0.01

0.02

0.07

L 0.07 1.00

0.12 0.18 0.08 0.15

0.43

2.5.2 Marginal Probability Distributions (1/2)

Marginal probability distribution
Obtained by summing or integrating the joint probability
distribution over the values of the other random variable
Discrete

P ( X i ) pi pij
j

Continuous

f X ( x)

NIPRL

f ( x, y )dy

2.5.2 Marginal Probability Distributions (2/2)

Example 19
Marginal p.m.f of X
3

P ( X 1) p1 j 0.12 0.08 0.01 0.21

j 1

Marginal p.m.f of Y
4

P (Y 1) pi1 0.12 0.08 0.07 0.05 0.32

i 1

NIPRL

 Example 20: (a jointly continuous case)

Joint pdf:

f ( x, y )

Marginal pdfs of X and Y:

f X ( x) f ( x, y )dy
fY ( y ) f ( x, y )dx

NIPRL

2.5.3 Conditional Probability Distributions (1/2)

Conditional probability distributions
The probabilistic properties of the random variable X under the
knowledge provided by the value of Y
Discrete

P( X i, Y j ) pij
pi| j P( X i | Y j )

P (Y j )
p j

Continuous

f ( x, y )
f X |Y y ( x)
fY ( y )
The conditional probability distribution is a probability
distribution.
NIPRL

2.5.3 Conditional Probability Distributions (2/2)

Example 19
Marginal probability distribution of Y

P (Y 3) p3 0.01 0.01 0.02 0.07 0.11

Conditional distribution of X

p1|Y 3

NIPRL

p13 0.01
P ( X 1| Y 3)

0.091
p3 0.11

2.5.4 Independence and Covariance (1/5)

Two random variables X and Y are said to be independent if
Discrete

pij pi p j for all values iof X and jof Y

Continuous

f ( x, y ) f X ( x) fY ( y ) for all x and y

How is this independency different from the independence among
events?

NIPRL

2.5.4 Independence and Covariance (2/5)

Covariance

Cov( X , Y ) E (( X E ( X ))(Y E (Y )))

E ( XY ) E ( X ) E (Y )

Cov( X , Y )

E (( X E ( X ))(Y E (Y )))
E ( XY XE (Y ) E ( X )Y E ( X ) E (Y ))
E ( XY ) E ( X ) E (Y ) E ( X ) E (Y ) E ( X ) E (Y )
E ( XY ) E ( X ) E (Y )

May take any positive or negative numbers.

Independent random variables have a covariance of zero
What if the covariance is zero?
NIPRL

2.5.4 Independence and Covariance (3/5)

Example 19 (Air conditioner maintenance)

E ( X ) 2.59,
4

E (Y ) 1.79

E ( XY ) ijpij
i 1 j 1

(11 0.12) (1 2 0.08)

L (4 3 0.07) 4.86

Cov( X , Y ) E ( XY ) E ( X ) E (Y )
4.86 (2.59 1.79) 0.224

NIPRL

2.5.4 Independence and Covariance (4/5)

Correlation:

Cov( X , Y )
Corr( X , Y )
Var( X )Var(Y )
Values between -1 and 1, and independent random variables
have a correlation of zero

NIPRL

2.5.4 Independence and Covariance (5/5)

Example 19: (Air conditioner maintenance)

Var( X ) 1.162, Var(Y ) 0.384

Cov( X , Y )
Corr( X , Y )
Var( X )Var(Y )
0.224

0.34
1.162 0.384

NIPRL

 What if random variable X and Y have linear relationship, that is,

Y aX b

a0
Cov( X , Y ) E[ XY ] E[ X ]E[Y ]
E[ X (aX b)] E[ X ]E[aX b]

where

aE[ X 2 ] bE[ X ] aE 2 [ X ] bE[ X ]

a ( E[ X 2 ] E 2 [ X ]) aVar ( X )

Cov( X , Y )
aVar ( X )
Corr ( X , Y )

Var ( X )Var (Y )
Var ( X )a 2Var ( X )
That is, Corr(X,Y)=1 if a>0; -1 if a<0.
NIPRL

2.6 Combinations and Functions of Random

Variables
2.6.1 Linear Functions of Random Variables (1/4)

Linear Functions of a Random Variable

If X is a random variable and
Y aX b
for some numbers a, b Rthen
E (Y ) aE ( X ) b
2
and Var(Y ) a Var( X )
Standardization
X an expectation of
-If a random variable has
2
of ,
X 1

has an expectation of zero and a variance of one.

NIPRL

a variance
and

2.6.1 Linear Functions of Random Variables (2/4)

Example 21:Test Score Standardization
Suppose that the raw score X from a particular testing procedure
are distributed between -5 and 20 with an expected value of 10
and a variance 7. In order to standardize the scores so that they
lie between 0 and 100, the linear transformation
is applied to the scores.
Y 4 X 20

NIPRL

2.6.1 Linear Functions of Random Variables (3/4)

For example, x=12 corresponds to a standardized score of
y=(412)+20=68

E (Y ) 4 E ( X ) 20 (4 10) 20 60

Var(Y ) 42 Var( X ) 42 7 112

Y 112 10.58

NIPRL

2.6.1 Linear Functions of Random Variables (4/4)

Sums of Random Variables
If X 1 and X 2are two random variables, then

E ( X 1 X 2 ) E ( X 1 ) E ( X 2 ) ( why ?)
and

Var( X 1 X 2 ) Var( X 1 ) Var( X 2 ) 2Cov( X 1 , X 2 )

If X 1 and X 2are independent, so that
then

Cov( X 1 , X 2 ) 0

Var( X 1 X 2 ) Var( X 1 ) Var( X 2 )

NIPRL

 Properties of

Cov( X 1 , X 2 )

Cov( X 1 , X 2 ) E[ X 1 X 2 ] E[ X 1 ]E[ X 2 ]

Cov( X 1 , X 2 ) Cov( X 2 , X 1 )

Cov( X 1 , X 1 ) Var ( X 1 ) Cov( X 2 , X 2 ) Var ( X 2 )

Cov( X 1 X 2 , X 1 ) Cov ( X 1 , X 1 ) Cov ( X 2 , X 1 )

Cov( X 1 X 2 , X 1 X 2 ) Cov ( X 1 , X 1 ) Cov( X 1 , X 2 )

Cov( X 2 , X 1 ) Cov( X 2 , X 2 )

Var ( X 1 X 2 ) Var ( X 1 ) Var ( X 2 ) 2Cov( X 1 , X 2 )

NIPRL

2.6.2 Linear Combinations of Random Variables (1/5)

Linear Combinations of Random Variables
If X 1 , K , X n is a sequence of random variables and
areb constants, then

a1 , Kand
, an

E (a1 X 1 L an X n b) a1 E ( X 1 ) L an E ( X n ) b
If, in addition, the random variables are independent, then
2

Var(a1 X 1 L an X n b) a1 Var( X 1 ) L an Var( X n )

NIPRL

2.6.2 Linear Combinations of Random Variables (2/5)

Averaging Independent Random Variables
Suppose that X 1 , K , X nis a sequence of independent random

variables with an expectation and a variance

.
2

Let
Then
and

X1 L X n
n

E( X )

2
Var( X )
n

What happened to the variance?

NIPRL

2.6.2 Linear Combinations of Random Variables (3/5)

1
1
X 1 L X n
n
n
1
1
L
n
n

1
1
E ( X1 ) L E ( X n )
n
n

E( X ) E

1
1
Var( X ) Var X 1 L X n
n
n
2

2
1
1
2
2
L
n
n
n

NIPRL

1
1
Var( X 1 ) L
n
n

Var( X n )

2.6.2 Linear Combinations of Random Variables (4/5)

Example 21
The standardized scores of the two tests are
10
5
50
Y1 X 1 and Y2 X 2
3
3
3
The final score is

2
1
20
5
50
Z Y1 Y2
X1 X 2
3
3
9
9
9

NIPRL

2.6.2 Linear Combinations of Random Variables (5/5)

The expected value of the final score is
20
5
50
E (Z )
E ( X1 ) E ( X 2 )
9
9
9
50
20
5

18
30

9
9
9
62.22
The variance of the final score is
5
50
20
Var( Z ) Var
X1 X 2
9
9
9
2

20
5
Var( X 1 ) Var( X 2 )
9
9

NIPRL

20
5

24
60 137.04

9
9

2.6.3 Nonlinear Functions of a Random Variable (1/3)

Nonlinear function of a Random Variable
A nonlinear function of a random variable X is another random
variable Y=g(X) for some nonlinear function g.
Y X 2, Y X , Y eX
There are no general results that relate the expectation and
variance of the random variable Y to the expectation and variance
of the random variable X

NIPRL

2.6.3 Nonlinear Functions of a Random Variable (2/3)

For example,

f(x)=1

E(x)=0.5

f X ( x) 1 for 0 x 1
f ( x) 0 elsewhere
0

FX ( x) x for 0 x 1
f(y)=1/y

E(y)=1.718

Consider

Y eX
where 1 Y 2.718
What is the pdf of Y?
NIPRL

1.0

2.718

2.6.3 Nonlinear Functions of a Random Variable (3/3)

CDF methd

FY ( y ) P(Y y ) P (e X y ) P( X ln( y )) Fx (ln( y )) ln( y )

The p.d.f of Y is obtained by differentiation to be

fY ( y )

dFY ( y ) 1

dy
y

for 1 y 2.718

Notice that

E (Y )

2.718

z 1

zfY ( z )dz

2.718

z 1

E (Y ) e E ( X ) e 0.5 1.649

NIPRL

1dz 2.718 1 1.718

 Determining the p.d.f. of nonlinear relationship between r.v.s:

) is
Given f X ( x)and Y g ( X
,what
fY ?( y )
If

x1 , x2 ,L , xnare all its real roots, that is,

y g ( x1 ) g ( x2 ) L g ( xn )

then

where

NIPRL

f X ( xi )
fY ( y )
i 1 | g '( xi ) |
dg ( x)
g '( x)
dx

 Example: determine

fY ( y )

f X ( x) 1 for 0 x 1
f ( x) 0 elsewhere

y g ( x) e x
ln y x
dg
ex y
dx
1
1
fY ( y )

| y| y
NIPRL

Y eX
where 1 Y 2.718

-> one root is possible in

0 x 1