Module 25

CONDITIONAL EXPECTATIONS

() Module 25 CONDITIONAL EXPECTATIONS 1 / 23

X = (Y , Z ) : a p-dimensional r.v. with joint p.m.f./p.d.f. fY ,Z (·) and
support SX , where

Y = (Y1 , . . . , Yp1 ): a p1 -dimensional r.v. with p.m.f./p.d.f. fY (·)

and support SY ,

Z = (Z1 , . . . , Zp2 ): a p2 -dimensional r.v. with p.m.f./p.d.f. fZ (·)

and support SZ ,
and p = p1 + p2 .

() Module 25 CONDITIONAL EXPECTATIONS 1 / 23

For given z ∈ SZ (with fZ (z) > 0) the conditional p.m.f/p.d.f. of Y
given Z = z is

fY ,Z (y , z)
fY |Z (y |z) =
fZ (z)
= k(z)fY ,Z (y , z), y ∈ Rp ,

where, for fixed z ∈ SZ (with fZ (z) > 0), k(z) = [fZ (z)]−1 is a
normalizing constant. Thus the conditional p.m.f./p.d.f. is
proportional to the joint p.d.f.

() Module 25 CONDITIONAL EXPECTATIONS 2 / 23

Definition 1:

Let ψ : Rp1 → R be a given function and let z ∈ SZ (with fZ (z) > 0)

be given.
(a) The conditional expectation of ψ(Y ) given that Z = z is defined by
Z
E (ψ(Y )|Z = z) = ψ(y )fY |Z (y |z)dy ,
R p1

provided the expectation is finite.

(b) The conditional variance of ψ(Y ) given that Z = z is defined by

 

 2
Var(ψ(Y )|Z = z) = E ψ(Y ) − E ψ(Y )|Z = z |Z = z
  
2
= E ψ 2 (Y )|Z = z − E ψ(Y )|Z = z .

Note that E (ψ(Y )|Z = z) is a function of z ∈ SZ .

() Module 25 CONDITIONAL EXPECTATIONS 3 / 23
Notation:

E (ψ(Y )|Z ) = ψ ∗ (Z ),
where
ψ ∗ (z) = E (ψ(Y )|Z = z).

Similarly we define Var(ψ(Y )|Z ), Cov(Y1 , Y2 |Z ) and ρ(Y1 , Y2 |Z ).

() Module 25 CONDITIONAL EXPECTATIONS 4 / 23

Result 1 :
Under the above notations


(a) E (ψ(Y )) = E E (ψ(Y )|Z ) .

(b) Var(ψ(Y )) = Var E (ψ(Y )|Z ) + E Var(ψ(Y )|Z ) .

Proof : For A.C. case.

(a) Note that
E E (ψ(Y )|Z ) = E (ψ ∗ (Z )),

where, for z ∈ SZ ,

ψ ∗ (z) = E (ψ(Y )|Z = z)

Z
= ψ(y )fY |Z (y |z)dy .
Rp1

Thus
() Module 25 CONDITIONAL EXPECTATIONS 5 / 23
E E (ψ(Y )|Z ) = E (ψ ∗ (Z ))


Z
= ψ ∗ (z)fZ (z)dz
Rp2
Z Z 
= ψ(y )fY |Z (y |z)dy fZ (z)dz
Rp2 Rp1
Z
= ψ(y )fY |Z (y |z)fZ (z)dy dz
Rp
Z
= ψ(y )fY ,Z (y , z)dy
Rp
= E (ψ(Y )).

() Module 25 CONDITIONAL EXPECTATIONS 6 / 23

(b) Let ψ ∗ (Z ) = E ψ(Y )|Z . Then by (a)


2 
Var(ψ(Y )) = E ψ(Y ) − E (ψ(Y ))
  

2 
= E E ψ(Y ) − E (ψ(Y )) |Z

= E (ψ1 (Z ), ) (1)

where
 

2 
ψ1 (Z ) = E ψ(Y ) − E ψ(Y ) Z



= E ψ(Y ) − E ψ(Y )|Z


2 
+ E ψ(Y )|Z − E (ψ(Y ))  Z

() Module 25 CONDITIONAL EXPECTATIONS 7 / 23

    2

2 

= E ψ(Y ) − E ψ(Y )|Z Z + E ψ(Y )|Z − E (ψ(Y ))
   



+2 E ψ(Y )|Z − E (ψ(Y )) E ψ(Y ) − E ψ(Y )|Z Z


  2
= Var(ψ(Y )|Z ) + E ψ(Y )|Z − E E (ψ(Y )|Z ) + 0.

Thus from (1)

  2 
 

Var(ψ(Y )) = E Var(ψ(Y ) Z ) + E E ψ(Y )|Z − E E (ψ(Y )|Z )

   

= E Var(ψ(Y )Z ) + Var E ψ(Y )|Z .

() Module 25 CONDITIONAL EXPECTATIONS 8 / 23

Remark 1:

If Y and Z are independent then

E (ψ(Y )|Z ) = E (ψ(Y ))

and
Var(ψ(Y )|Z ) = Var(ψ(Y )).

() Module 25 CONDITIONAL EXPECTATIONS 9 / 23

Example 1:

Let X = (X1 , X2 , X3 )0 be A.C. r.v.with joint p.d.f.

(
1
, if 0 < x3 < x2 < x1 < 1
fX (x1 , x2 , x3 ) = x1 x2 .
0, otherwise

Let Y1 = 2X1 − X2 + 3X3 , Y2 = X1 − 2X2 + X3 and Y = X1 X2 X3 .

(a) Find ρ(Y1 , Y2 ).

(b) For a fixed x2 ∈ (0, 1) find E (Y |X2 = x2 ), Var(Y |X2 = x2 ) and

Cov(X1 , X3 |X2 = x2 ).

() Module 25 CONDITIONAL EXPECTATIONS 10 / 23

Solution :

(a)

Cov(Y1 , Y2 ) = Cov(2X1 − X2 + 3X3 , X1 − 2X2 + X3 )

= 2Var(X1 ) − 5Cov(X1 , X2 ) + 5Cov(X1 , X3 )
+2Var(X2 ) − 7Cov(X2 , X3 ) + 3Var(X3 )
Var(Y1 ) = 4Var(X1 ) + Var(X2 ) + 9Var(X3 ) − 4Cov(X1 , X2 )
+12Cov(X1 , X3 ) − 6Cov(X2 , X3 )
Var(Y2 ) = Var(X1 ) + 4Var(X2 ) + Var(X3 ) − 4Cov(X1 , X2 )
+2Cov(X1 , X3 ) − 4Cov(X2 , X3 )
Cov(Y1 , Y2 )
ρ(Y1 , Y2 ) = p
Var(Y1 )Var(Y2 )

() Module 25 CONDITIONAL EXPECTATIONS 11 / 23

For any function ψ(·) : R3 → R
Z
E (ψ(X1 , X2 , X3 )) = ψ(x1 , x2 , x3 )fX (x1 , x2 , x3 )dx
R3
Z1 Zx1 Zx2
ψ(x1 , x2 , x3 )
= dx3 dx2 dx1
x1 x2
0 0 0

Thus,

Z1 Zx1 Zx2
1 1
E (X1 ) = dx3 dx2 dx1 = ;
x2 2
0 0 0
Z1 Zx1 Zx2
x1 1
E (X12 ) = dx3 dx2 dx1 = ;
x2 3
0 0 0

() Module 25 CONDITIONAL EXPECTATIONS 12 / 23

 Z1 Zx1 Zx2
1 1
E (X2 ) = dx3 dx2 dx1 = ;
x1 4
0 0 0
Z1 Zx1 Zx2
x2 1
E (X22 ) = dx3 dx2 dx1 = ;
x1 9
0 0 0
Z1 Zx1 Zx2
x3 1
E (X3 ) = dx3 dx2 dx1 = ;
x1 x2 8
0 0 0
Z Zx1 Zx2
1
x32 1
E (X32 ) = dx3 dx2 dx1 = ;
x1 x2 27
0 0 0
Z Zx1 Zx2
1
1
E (X1 X2 ) = dx3 dx2 dx1 = ;
6
0 0 0

() Module 25 CONDITIONAL EXPECTATIONS 13 / 23

 Z1 Zx1 Zx2
x3 1
E (X1 X3 ) = dx3 dx2 dx1 = ;
x2 12
0 0 0
Z1 Zx1 Zx2
x3 1
E (X2 X3 ) = dx3 dx2 dx1 = ;
x1 18
0 0 0
1
Var(X1 ) = E (X12 ) − (E (X1 ))2 = ;
12
7
Var(X2 ) = E (X22 ) − (E (X2 ))2 = ;
144
37
Var(X3 ) = E (X32 ) − (E (X3 ))2 = ;
1728
1
Cov(X1 , X2 ) = E (X1 X2 ) − E (X1 )E (X2 ) = ;
24

() Module 25 CONDITIONAL EXPECTATIONS 14 / 23

 1
Cov(X1 , X3 ) = E (X1 X3 ) − E (X1 )E (X3 ) = ;
48
7
Cov(X2 , X3 ) = E (X2 X3 ) − E (X2 )E (X3 ) = .
228

Therefore
1 5 5 7 49 37
Cov(Y1 , Y2 ) = − + + − +
6 24 48 72 288 1728
31
= ;
576
Var(Y1 ) = 4Var(X1 ) + Var(X2 ) + 9Var(X3 ) − 4Cov(X1 , X2 )
+12Cov(X1 , X3 ) − 6Cov(X2 , X3 )
1 7 37 1 1 7
= + + − + −
3 144 192 6 4 48
295
= ;
576

() Module 25 CONDITIONAL EXPECTATIONS 15 / 23

 Var(Y2 ) = Var(X1 ) + 4Var(X2 ) + Var(X3 ) − 4Cov(X1 , X2 )
+2Cov(X1 , X3 ) − 4Cov(X2 , X3 )
1 7 37 1 1 7
= + + − + −
12 36 1728 6 24 72
133
= ;
1728
Cov(Y1 , Y2 )
ρ(Y1 , Y2 ) = p
Var(Y1 )Var(Y2 )
= 0.2710.

(b) Clearly, given X2 = x2 , X1 and X3 are independent with p.d.f.s

() Module 25 CONDITIONAL EXPECTATIONS 16 / 23

 (
c1 (x2 )
fX1 |X2 (x1 |x2 ) = x1 , if x2 < x1 < 1
,
0, otherwise
(
−1
(ln x2 )x1 , if x2 < x1 < 1
= ,
0, otherwise
and (
c2 (x2 ), if 0 < x3 < x2
fX3 |X2 (x3 |x2 ) =
0, otherwise
(
1
= x2 , if 0 < x3 < x2 .
0, otherwise
Also, for 0 < x2 < 1,
Z1 Zx2
1
fx2 (x2 ) = dx3 dx1
x1 x2
x2 0
= − ln x2 ;
() Module 25 CONDITIONAL EXPECTATIONS 17 / 23
 (
− ln x2 , if 0 < x2 < 1
fx2 (x2 ) = .
0, otherwise

E (Y |X2 = x2 ) = E (X1 X2 X3 |X2 = x2 )

= x2 E (X1 X3 |X2 = x2 )
= x2 E (X1 |X2 = x2 )E (X3 |X2 = x2 );

Z∞
E (X1 |X2 = x2 ) = x1 fX1 |X2 (x1 |x2 )dx1
−∞
Z1
−1
= dx1
ln x2
x2
x2 − 1
= ;
ln x2
() Module 25 CONDITIONAL EXPECTATIONS 18 / 23
 Z∞
E (X3 |X2 = x2 ) = x3 fX3 |X2 (x3 |x2 )dx3
−∞
Zx2
x3
= dx3
x2
0
x2
= ;
2

x22 (x2 − 1)
E (Y |X2 = x2 ) = ;
2 ln x2

E (Y 2 |X2 = x2 ) = E (X12 X22 X32 |X2 = x2 )

= x22 E (X12 |X2 = x2 )E (X32 |X2 = x2 );

() Module 25 CONDITIONAL EXPECTATIONS 19 / 23

 Z1
−x1
E (X12 |X2 = x2 ) = dx1
ln x2
x2
1 − x22
= − ;
2 ln x2
Zx2
x32
E (X32 |X2 = x2 ) = dx3
x2
0
x22
= .
3

Var(X1 |X2 = x2 ) = E (X12 |X2 = x2 ) − (E (X1 |X2 = x2 ))2 .

Since, given X2 = x2 , X1 and X3 are independent we have
Cov(X1 , X3 |X2 = x2 ) = 0.
() Module 25 CONDITIONAL EXPECTATIONS 20 / 23
Take Home Problem

Let X = (X1 , X2 , X3 )0 be a discrete r.v. with joint p.m.f.

(
x1 x2 x3
fX (x1 , x2 , x3 ) = 72 , if x1 = 1, 2, x2 = 1, 2, 3, x3 = 1, 3 .
0, otherwise

Let Y1 = 2X1 − X2 + 3X3 , Y2 = X1 − 2X2 + X3 and Y = X1 X2 X3 .

(a) Find ρ(Y1 , Y2 );
(b) For fixed x2 ∈ {1, 2, 3} find E (Y |X2 = x2 ) and Var(Y |X2 = x2 ).

() Module 25 CONDITIONAL EXPECTATIONS 21 / 23

Abstract of Next Module

We will introduce the joint m.g.f. of a random vector and study its
properties.

() Module 25 CONDITIONAL EXPECTATIONS 22 / 23

 Thank you for your patience

() Module 25 CONDITIONAL EXPECTATIONS 23 / 23