Chapter 2.

Random
Variables

2.1 Discrete Random Variables

2.2 Continuous Random Variables
2.3 The Expectation of a Random Variable
2.4 The Variance of a Random Variable
2.5 Jointly Distributed Random Variables
2.6 Combinations and Functions of Random
Variables

NIPRL
 2.1 Discrete Random Variable
2.1.1 Definition of a Random Variable (1/2)

• Random variable
– A numerical value to each outcome of a particular
experiment
S

-3 -2 -1 0 1 2 3

NIPRL
 2.1.1 Definition of a Random Variable (2/2)

• Example 1 : Machine Breakdowns

– Sample space : S {electrical , mechanical , misuse}
– Each of these failures may be associated with a repair
cost {50, 200,350}
– State space :
– Cost is a random variable : 50, 200, and 350

NIPRL
 2.1.2 Probability Mass Function (1/2)

• Probability Mass Function (p.m.f.)

– A set of probability valuepi assigned to each of the
values taken by the discrete randomxi variable
– 0  pi 1 and  i pi 1
– Probability : P ( X  xi )  pi

NIPRL
 2.1.2 Probability Mass Function (1/2)

• Example 1 : Machine Breakdowns

– P (cost=50)=0.3, P (cost=200)=0.2,
P (cost=350)=0.5
– 0.3 + 0.2 + 0.5 =1 xi 50 200 350

pi 0.3 0.2 0.5

f ( x)
0.5

0.3
0.2

50 200 350 Cost($)

NIPRL
 2.1.3 Cumulative Distribution Function (1/2)

• Cumulative Distribution Function

– Function :F ( x) P( X x) F ( x)   P ( X  y )
y: y x
– Abbreviation : c.d.f

F ( x)
1.0

0.5

0.3

0 50 200 350 x($cost)

NIPRL
 2.1.3 Cumulative Distribution Function (2/2)

• Example 1 : Machine Breakdowns

   x  50  F ( x) P(cost  x) 0
50 x  200  F ( x) P(cost x) 0.3
200 x  350  F ( x) P(cost x) 0.3  0.2 0.5
350 x    F ( x) P(cost x) 0.3  0.2  0.5 1.0

NIPRL
 2.2 Continuous Random Variables
2.2.1 Example of Continuous Random Variables
(1/1)
• Example 14 : Metal Cylinder Production
– Suppose that the random variableX is the diameter
of a randomly chosen cylinder manufactured by the
company. Since this random variable can take any
value between 49.5 and 50.5, it is a continuous
random variable.

NIPRL
 2.2.2 Probability Density Function (1/4)

• Probability Density Function (p.d.f.)

– Probabilistic properties of a continuous random variable

f ( x) 0
statespace
f ( x)dx 1

NIPRL
 2.2.2 Probability Density Function (2/4)

• Example 14
– Suppose that the diameter of a metal cylinder has a p.d.f
f ( x) 1.5  6( x  50.2) 2 for 49.5  x 50.5
f ( x) 0, elsewhere

f ( x)

49.5 50.5 x
NIPRL
 2.2.2 Probability Density Function (3/4)

• This is a valid p.d.f.

50.5

49.5
(1.5  6( x  50.0) 2 )dx [1.5 x  2( x  50.0)3 ]50.5
49.5

[1.5 50.5  2(50.5  50.0)3 ]

 [1.5 49.5  2(49.5  50.0)3 ]
75.5  74.5 1.0

NIPRL
 2.2.2 Probability Density Function (4/4)

• The probability that a metal cylinder has a diameter

between 49.8 and 50.1 mm can be calculated to be
50.1
49.8
(1.5  6( x  50.0) 2 ) dx [1.5 x  2( x  50.0) 3 ]50.1
49.8

[1.5 50.1  2(50.1  50.0)3 ]

 [1.5 49.8  2(49.8  50.0)3 ]
f ( x) 75.148  74.716 0.432

49.5 49.8 50.1 50.5 x

NIPRL
 2.2.3 Cumulative Distribution Function (1/3)

• Cumulative Distribution Function

x
F ( x) P ( X  x )  f ( y )dy


dF ( x)
 f ( x) 
dx

P(a  X b) P ( X b)  P ( X a )

F (b)  F (a )

P(a  X b) P (a  X b)

NIPRL
 2.2.2 Probability Density Function (2/3)

• Example 14
x
F ( x) P( X x)  (1.5  6( y  50.0) 2 )dy
49.5

[1.5 y  2( y  50.0)3 ]49.5

x

[1.5 x  2( x  50.0)3 ]  [1.5 49.5  2(49.5  50.0)3 ]

1.5 x  2( x  50.0)3  74.5
P(49.7  X 50.0) F (50.0)  F (49.7)
 (1.5 50.0  2(50.0  50.0)3  74.5)
 (1.5 49.7  2(49.7  50.0)3  74.5)
0.5  0.104 0.396

NIPRL
 2.2.2 Probability Density Function (3/3)

P(49.7  X 50.0) 0.396

1

P( X 50.0) 0.5
F ( x)

P( X 49.7) 0.104

49.5 49.7 50.0 50.5 x

NIPRL
 2.3 The Expectation of a Random Variable
2.3.1 Expectations of Discrete Random Variables
(1/2)
• Expectation of a discrete random variable with p.m.f
P( X  xi )  pi

E ( X )  pi xi
i

• Expectation of a continuous random variable with p.d.f f(x)

E ( X )  xf ( x ) dx
state space

• The expected value of a random variable is also called the

mean of the random variable

NIPRL
 2.3.1 Expectations of Discrete Random Variables
(2/2)

• Example 1 (discrete random variable)

– The expected repair cost is
E (cost) ($50 0.3)  ($200 0.2)  ($350 0.5) $230

NIPRL
2.3.2 Expectations of Continuous Random Variables
(1/2)

• Example 14 (continuous random variable)

– The expected diameter of a metal cylinder is
50.5
E ( X )  x(1.5  6( x  50.0) 2 )dx
49.5
– Change of variable: y=x-50
0.5
E ( x)  ( y  50)(1.5  6 y 2 )dy
 0.5
0.5
 ( 6 y 3  300 y 2  1.5 y  75)dy
 0.5

[ 3 y 4 / 2  100 y 3  0.75 y 2  75 y ]0.5

 0.5

[25.09375]  [ 24.90625] 50.0

NIPRL
2.3.2 Expectations of Continuous Random Variables
(2/2)

• Symmetric Random Variables

– If x has a p.d.f f ( x) that is f ( x) E ( X ) 
symmetric about a point
so that
f (   x)  f (   x)

– Then, E ( X )  (why?)

– So that the expectation of

the random variable is
equal to the point of
 x
symmetry

NIPRL
 E ( X ) xf ( x)dx
 
 xf ( x)dx +xf ( x)dx
- 

 y 2   x
 
 xf ( x)dx +  yf ( y )dy
- -



NIPRL
 2.3.3 Medians of Random Variables (1/2)

• Median
– Information about the “middle” value of the random
variable
F ( x) 0.5

• Symmetric Random Variable 

– If a continuous random variable is symmetric about a
 median and the expectation of
point , then both the
the random variable are equal to

NIPRL
 2.3.3 Medians of Random Variables (2/2)

• Example 14

F ( x) 1.5 x  2( x  50.0)3  74.5 0.5

x 50.0

NIPRL
 2.4 The variance of a Random Variable
2.4.1 Definition and Interpretation of Variance (1/2)

• Variance( 
2
)
– A positive quantity that measures the spread of the
distribution of the random variable about its mean value
– Larger values of the variance indicate that the
distribution is more spread out

– Definition: Var( X ) E (( X  E ( X )) 2 )
E ( X 2 )  ( E ( X )) 2

• Standard Deviation
– The positive square root of the variance
– Denoted by 

NIPRL
 2.4.1 Definition and Interpretation of Variance (2/2)

Var( X ) E (( X  E ( X )) 2 )
E ( X 2  2 XE ( X )  ( E ( X )) 2 )
E ( X 2 )  2 E ( X ) E ( X )  ( E ( X )) 2
E ( X 2 )  ( E ( X )) 2

f ( x)
Two distribution with
identical mean values
but different variances

x
NIPRL
 2.4.2 Examples of Variance Calculations (1/1)

• Example 1

Var( X ) E (( X  E ( X )) 2 )  pi ( xi  E ( X ))2
i

0.3(50  230) 2  0.2(200  230) 2  0.5(350  230)2

17,100  2

  17,100 130.77

NIPRL
 2.4.3 Chebyshev’s Inequality (1/1)

• Chebyshev’s Inequality

– If a random variable has a mean 2
and a variance ,
then 1
P (   c  X   c ) 1 
c2
for c 1
c 2
– For example, taking gives
1
P (   2  X   2 ) 1  2 0.75
2

NIPRL
 • Proof

 2  ( x   ) 2 f ( x)dx   ( x   ) 2 f ( x)dx c 2 2  f ( x)dx.
  
| x  | c  
| x  | c

 P(| x   | c ) 1/ c 2

 P (| x   |c ) 1  P (| x   | c ) 1  1/ c 2

NIPRL
 2.4.4 Quantiles of Random Variables (1/2)

• Quantiles of Random variables

– The p th quantile of a random variable X
F ( x)  p
– A probability ofp that the random variable takes a
p the th quantile
value less than
• Upper quartile
– The 75th percentile of the distribution
• Lower quartile
– The 25th percentile of the distribution
• Interquartile range
– The distance between the two quartiles

NIPRL
 2.4.4 Quantiles of Random Variables (2/2)

• Example 14
F ( x) 1.5 x  2( x  50.0)3  74.5 for 49.5  x 50.5
– Upper quartile : F ( x) 0.75 x 50.17

– Lower quartile : F ( x) 0.25 x 49.83

– Interquartile range 50.17

:  49.83 0.34

NIPRL
 2.5 Jointly Distributed Random Variables
2.5.1 Jointly Distributed Random Variables (1/4)

• Joint Probability Distributions

– Discrete
P ( X  xi , Y  y j )  pij 0
satisfying  p
i j
ij 1
– Continuous

f ( x, y ) 0 satisfying 
state space
f ( x, y )dxdx 1

NIPRL
 2.5.1 Jointly Distributed Random Variables (2/4)

• Joint Cumulative Distribution Function

– Discrete

F ( x, y ) P ( X  xi , Y  y j )
– Continuous

F ( x, y )    pij
i: xi x j: y j y

x y
F ( x , y )   f ( w, z )dzdw
w   z  

NIPRL
 2.5.1 Jointly Distributed Random Variables (3/4)

• Example 19 : Air Conditioner Maintenance

– A company that services air conditioner units in
residences and office blocks is interested in how to
schedule its technicians in the most efficient manner
– The random variable X, taking the values 1,2,3 and 4, is
the service time in hours
– The random variable Y, taking the values 1,2 and 3, is
the number of air conditioner units

NIPRL
 2.5.1 Jointly Distributed Random Variables (4/4)

X=service time
• Joint p.m.f
Y=
numbe  p
i j
ij 0.12  0.18
r of
1 2 3 4
units    0.07 1.00

1 0.12 0.08 0.07 0.05 • Joint cumulative

distribution function

2 0.08 0.15 0.21 0.13 F (2, 2)  p11  p12  p21  p22

0.12  0.18  0.08  0.15
3 0.01 0.01 0.02 0.07 0.43

NIPRL
 2.5.2 Marginal Probability Distributions (1/2)

• Marginal probability distribution

– Obtained by summing or integrating the joint probability
distribution over the values of the other random variable
– Discrete
P ( X i )  pi   pij
j

– Continuous

f X ( x)  f ( x, y )dy


NIPRL
 2.5.2 Marginal Probability Distributions (2/2)

• Example 19
– Marginal p.m.f of X
3
P ( X 1)  p1 j 0.12  0.08  0.01 0.21
j 1
– Marginal p.m.f of Y
4
P (Y 1)  pi1 0.12  0.08  0.07  0.05 0.32
i 1

NIPRL
 • Example 20: (a jointly continuous case)

• Joint pdf: f ( x, y )
• Marginal pdf’s of X and Y:

f X ( x) f ( x, y )dy

fY ( y ) f ( x, y )dx

NIPRL
 2.5.3 Conditional Probability Distributions (1/2)

• Conditional probability distributions

– The probabilistic properties of the random variable X
under the knowledge provided by the value of Y
– Discrete
P( X i, Y  j ) pij
pi| j P ( X i | Y  j )  
P(Y  j ) p j
– Continuous

f ( x, y )
f X |Y y ( x) 
fY ( y )
– The conditional probability distribution is a probability
distribution.

NIPRL
 2.5.3 Conditional Probability Distributions (2/2)

• Example 19
– Marginal probability distribution of Y

P (Y 3)  p3 0.01  0.01  0.02  0.07 0.11

– Conditional distribution of X
p13 0.01
p1|Y 3 P ( X 1| Y 3)   0.091
p3 0.11

NIPRL
 2.5.4 Independence and Covariance (1/5)

• Two random variables X and Y are said to be independent if

– Discrete
pij  pi  p j for all values iof X andjof Y
– Continuous

f ( x, y )  f X ( x) fY ( y ) for all x and y

– How is this independency different from the

independence among events?

NIPRL
 2.5.4 Independence and Covariance (2/5)

• Covariance
Cov( X , Y ) E (( X  E ( X ))(Y  E (Y )))
 E ( XY )  E ( X ) E (Y )

Cov( X , Y )  E (( X  E ( X ))(Y  E (Y )))

 E ( XY  XE (Y )  E ( X )Y  E ( X ) E (Y ))
 E ( XY )  E ( X ) E (Y )  E ( X ) E (Y )  E ( X ) E (Y )
 E ( XY )  E ( X ) E (Y )

– May take any positive or negative numbers.

– Independent random variables have a covariance of zero
– What if the covariance is zero?

NIPRL
 2.5.4 Independence and Covariance (3/5)

• Example 19 (Air conditioner maintenance)

E ( X ) 2.59, E (Y ) 1.79
4 3
E ( XY )  ijpij
i 1 j 1

(11 0.12)  (12 0.08)

  (4 3 0.07) 4.86
Cov( X , Y ) E ( XY )  E ( X ) E (Y )
4.86  (2.59 1.79) 0.224

NIPRL
 2.5.4 Independence and Covariance (4/5)

• Correlation:

Cov( X , Y )
Corr( X , Y ) 
Var( X )Var(Y )

– Values between -1 and 1, and independent random

variables have a correlation of zero

NIPRL
 2.5.4 Independence and Covariance (5/5)

• Example 19: (Air conditioner maintenance)

Var( X ) 1.162, Var(Y ) 0.384

Cov( X , Y )
Corr( X , Y ) 
Var( X )Var(Y )
0.224
 0.34
1.162 0.384

NIPRL
 • What if random variable X and Y have linear relationship, that
is,Y aX  b
a 0
where
Cov( X , Y ) E[ XY ]  E[ X ]E[Y ]
E[ X (aX  b)]  E[ X ]E[aX  b]
aE[ X 2 ]  bE[ X ]  aE 2 [ X ]  bE[ X ]
a ( E[ X 2 ]  E 2 [ X ]) aVar ( X )
Cov( X , Y ) aVar ( X )
Corr ( X , Y )  
Var ( X )Var (Y ) Var ( X )a 2Var ( X )

That is, Corr(X,Y)=1 if a>0; -1 if a<0.

NIPRL
 2.6 Combinations and Functions of Random
Variables
2.6.1 Linear Functions of Random Variables (1/4)
• Linear Functions of a Random Variable
– If X is a random variable and Y aX  b
for some numbersa, b  R thenE (Y ) aE ( X )  b
and Var(Y ) a 2 Var( X )

• Standardization
-If a random variableX has an expectation of and a
variance of ,
2
X  1  
Y  X   
   
has an expectation of zero and a variance of one.

NIPRL
 2.6.1 Linear Functions of Random Variables (2/4)

• Example 21:Test Score Standardization

– Suppose that the raw score X from a particular testing
procedure are distributed between -5 and 20 with an
expected value of 10 and a variance 7. In order to
standardize the scores so that they lie between 0 and
100, the linear transformation
Y 4 X  20 is applied to
the scores.

NIPRL
 2.6.1 Linear Functions of Random Variables (3/4)

– For example, x=12 corresponds to a standardized score

of y=(4ⅹ12)+20=68

E (Y ) 4 E ( X )  20 (4 10)  20 60

Var(Y ) 42 Var( X ) 42 7 112

 Y  112 10.58

NIPRL
 2.6.1 Linear Functions of Random Variables (4/4)

• Sums of Random Variables

– If X 1 and X 2 are two random variables, then
E ( X 1  X 2 ) E ( X 1 )  E ( X 2 ) ( why ?)
and
Var( X 1  X 2 ) Var( X 1 )  Var( X 2 )  2Cov( X 1 , X 2 )
– If X 1 and X 2 are independent, so thatCov( X 1 , X 2 ) 0
then
Var( X 1  X 2 ) Var( X 1 )  Var( X 2 )

NIPRL
 • Properties of Cov( X 1 , X 2 )
Cov( X 1 , X 2 ) E[ X 1 X 2 ]  E[ X 1 ]E[ X 2 ]

 Cov( X 1 , X 2 ) Cov( X 2 , X 1 )
Cov( X 1 , X 1 ) Var ( X 1 ) Cov( X 2 , X 2 ) Var ( X 2 )
Cov( X 1  X 2 , X 1 ) Cov ( X 1 , X 1 )  Cov ( X 2 , X 1 )
Cov( X 1  X 2 , X 1  X 2 ) Cov( X 1 , X 1 )  Cov( X 1 , X 2 ) 
Cov( X 2 , X 1 )  Cov( X 2 , X 2 )

 Var ( X 1  X 2 ) Var ( X 1 )  Var ( X 2 )  2Cov( X 1 , X 2 )

NIPRL
 2.6.2 Linear Combinations of Random Variables
(1/5)

• Linear Combinations of Random Variables

– If X 1 ,  , X n is a sequence of random variables anda1 ,  , an
and b are constants, then
E (a1 X 1   an X n  b) a1 E ( X 1 )    an E ( X n )  b

– If, in addition, the random variables are independent,

then 2 2
Var(a1 X 1   an X n  b) a1 Var( X 1 )    an Var( X n )

NIPRL
 2.6.2 Linear Combinations of Random Variables
(2/5)
• Averaging Independent Random Variables
– Suppose thatX 1 ,  , X n is a sequence of independent
random variables with an expectation  and a variance
2
.

X1   X n
– Let X
n
E ( X ) 
– Then
2
and Var( X ) 
n

– What happened to the variance?

NIPRL
 2.6.2 Linear Combinations of Random Variables
(3/5)

1 1  1 1
E ( X ) E  X 1   X n   E ( X 1 )    E ( X n )
n n  n n
1 1
     
n n

2 2
1 1  1 1
Var( X ) Var  X 1    X n    Var( X 1 )     Var( X n )
n n   n  n
2 2
1 2 
2
1 2
         
 n  n n

NIPRL
 2.6.2 Linear Combinations of Random Variables
(4/5)

• Example 21
– The standardized scores of the two tests are
10 5 50
Y1  X 1 and Y2  X 2 
3 3 3
– The final score is
2 1 20 5 50
Z  Y1  Y2  X 1  X 2 
3 3 9 9 9

NIPRL
 2.6.2 Linear Combinations of Random Variables
(5/5)

– The expected value of the final score is

20 5 50
E (Z )  E( X1 )  E( X 2 ) 
9 9 9
 20  5  50
 18    30  
 9  9  9
 62.22

– The variance of the final score is

 20 5 50 
Var( Z ) Var  X1  X 2  
 9 9 9 
2 2
 20   5
  Var( X 1 )    Var( X 2 )
 9   9
2 2
 20   5
  24    60 137.04
 9   9

NIPRL
 2.6.3 Nonlinear Functions of a Random Variable
(1/3)

• Nonlinear function of a Random Variable

– A nonlinear function of a random variable X is another
random variable Y=g(X) for some nonlinear function g.
Y  X 2 , Y  X , Y e X

– There are no general results that relate the expectation

and variance of the random variable Y to the
expectation and variance of the random variable X

NIPRL
 2.6.3 Nonlinear Functions of a Random Variable
(2/3)
– For example, f(x)=
1 E(x)=0.
5
f X ( x) 1 for 0  x 1
f ( x) 0 elsewhere

0 1 x
FX ( x)  x for 0  x 1
f(y)=1/
y E(y)=1.71
– Consider 8

Y e X
where 1 Y  2.718

– What is the pdf of Y?

1.0 2.718 y

NIPRL
 2.6.3 Nonlinear Functions of a Random Variable
(3/3)
• CDF methd

FY ( y ) P (Y  y ) P (e X  y ) P ( X ln( y )) Fx (ln( y )) ln( y )

– The p.d.f of Y is obtained by differentiation to be

dFY ( y ) 1
fY ( y )   for 1  y  2.718
dy y
• Notice that
2.718 2.718
E (Y )  zfY ( z )dz  1dz 2.718  1 1.718
z 1 z 1

E (Y ) e E ( X ) e0.5 1.649

NIPRL
 • Determining the p.d.f. of nonlinear relationship between
r.v.s: f ( x ) Y g ( X ) fY ( y )
X
Given and ,what is ?
x1 , x2 , , xn
If are all its real roots, that is,
y  g ( x1 )  g ( x2 )    g ( xn )
n
f X ( xi )
then fY ( y ) 
i 1 | g '( xi ) |

where g '( x )  dg ( x )
dx

NIPRL
 • Example: determinefY ( y )

f X ( x) 1 for 0  x 1 Y e X
f ( x) 0 elsewhere where 1 Y  2.718

y  g ( x) e x
 ln y  x -> one root is possible in 0  x 1
dg
e x  y
dx
1 1
fY ( y )  
| y| y

NIPRL