Use of moment generating

functions
1. Using the moment generating functions of
X, Y, Z, …determine the moment
generating function of W = h(X, Y, Z, …).
2. Identify the distribution of W from its
moment generating function
This procedure works well for sums, linear
combinations etc.
Therorem
Let X and Y denote a independent random variables
each having a gamma distribution with parameters
(,1) and (,2). Then W = X + Y has a gamma
distribution with parameters (, 1 + 2).
Proof:

1 2
     
mX  t     and mY  t    
  t    t 
Therefore mX Y  t   mX  t  mY  t 

1 2 1  2
        
     
  t    t    t 

Recognizing that this is the moment generating

function of the gamma distribution with parameters
(, 1 + 2) we conclude that W = X + Y has a
gamma distribution with parameters (, 1 + 2).
Therorem (extension to n RV’s)
Let x1, x2, … , xn denote n independent random variables each
having a gamma distribution with parameters (,i), i = 1, 2, …,
n.
Then W = x1 + x2 + … + xn has a gamma distribution with
parameters (, 1 + 2 +… + n).
Proof:

i
  
mxi  t     i  1, 2..., n
  t 
Therefore
mx1  x2 ... xn  t   mx1  t  mx2  t  ...mxn  t 
1 2 n 1  2 ... n
           
    ...    
  t    t    t    t 

Recognizing that this is the moment generating

function of the gamma distribution with parameters
(, 1 + 2 +…+ n) we conclude that
W = x1 + x2 + … + xn has a gamma distribution with
parameters (, 1 + 2 +…+ n).
Therorem
Suppose that x is a random variable having a
gamma distribution with parameters (,).
Then W = ax has a gamma distribution with
parameters (/a, ).
Proof: 
  
mx  t    
  t 

   
   a 
then max  t   mx  at      
   at    t 
 a 
Special Cases
1. Let X and Y be independent random variables
having a 2 distribution with 1 and 2 degrees of
freedom respectively then X + Y has a 2
distribution with degrees of freedom 1 + 2.
2. Let x1, x2,…, xn, be independent random variables
having a 2 distribution with 1 , 2 ,…, n degrees
of freedom respectively then x1+ x2 +…+ xn has a
2 distribution with degrees of freedom 1 +…+ n.

Both of these properties follow from the fact that a

2 random variable with  degrees of freedom is a
 random variable with = ½ and  = /2.
Recall
If z has a Standard Normal distribution then z2 has a
2 distribution with 1 degree of freedom.

Thus if z1, z2,…, z are independent random variables

each having Standard Normal distribution then
U  z  z  ...  z
2
1
2
2
2


has a 2 distribution with  degrees of freedom.

Therorem
Suppose that U1 and U2 are independent random variables and
that U = U1 + U2 Suppose that U1 and U have a 2 distribution
with degrees of freedom 1and respectively. (1 < )
Then U2 has a 2 distribution with degrees of freedom 2 = -1
Proof:

v1 v

 12  2
 12  2

Now mU1  t    1  and mU  t    1 

 2 t   2 t 
Also mU  t   mU1  t  mU 2  t 

mU  t 
Hence mU 2  t  
mU1  t 
v

 12  2

1   12 
v  v1

2 t 
2 2

  v1   
 2 t 
1
 12  2

1 
 2 t 
Q.E.D.
Distribution of the sample
variance
n

 (x  x )
i
2

s2  i 1
n 1
 Properties of the sample variance
n n

 i
( x
i 1
 x ) 2
  i
( x
i 1
 a ) 2
 n ( x  a ) 2

Proof:
n n

 i
( x  x
i 1
) 2
  i
( x  a
i 1
 a  x ) 2

n
  ( xi  a ) 2  2( xi  a)( x  a )  ( x  a ) 2 
i 1
 n n
  ( xi  a)  2( x  a) ( xi  a)  n( x  a )
2 2

i 1 i 1

n
  ( xi  a) 2  2n( x  a) 2  n( x  a) 2
i 1

n
  ( xi  a)  n( x  a )
2 2

i 1
 n n

 i
( x
i 1
 x ) 2
  i
( x  a
i 1
) 2
 n ( x  a ) 2

Special Cases
2
1. Setting a = 0.  n

n n n   xi 
 i 1 

i 1
( xi  x ) 2
 
i 1
xi
2
 nx 2
 
i 1
xi
2

n
Computing formula
2. Setting a = .
n n

 i
( x
i 1
 x ) 2
  i
( x   )
i 1
2
 n ( x   ) 2

n n
or  (x  )   (x  x)
i 1
i
2

i 1
i
2
 n( x   ) 2
 Distribution of the sample variance

Let x1, x2, …, xn denote a sample from the normal

distribution with mean  and variance 2.
Let
x1   xn  
z1  , , z n 
 
n
Then
 x  
2
i
z   z 
2
1
2
n
i 1
 2

has a 2 distribution with n degrees of freedom.

Note:
n n

 i
( x   ) 2
 i
( x  x ) 2
n( x   ) 2
i 1
 i 1

 2
2 2
or U = U2 + U1
n

 (x  ) i
2

U i 1
 2

has a 2 distribution with n degrees of freedom.

We also know that x
has normal distribution with mean  and
variance 2/n
Thus x 
z

n
has a Standard Normal distribution and
n x  
2

U1  z 
2

 2

has a 2 distribution with 1 degree of freedom.

If we can show that U1 and U2 are independent
then n

 ( xi  x ) 2
(n  1) s 2
U2  i 1

 2
2
has a 2 distribution with n - 1 degrees of freedom.

The final task would be to show that

n

 (x  x )
i 1
i
2
and x

are independent
 Summary

Let x1, x2, …, xn denote a sample from the normal

distribution with mean  and variance 2.

1. than x has normal distribution with

mean  and variance 2/n
n

2.  n  1 s 2  (x  x )
i
2

U  i 1
 2
 2

has a 2 distribution with  = n - 1 degrees of

freedom.
 The Transformation Method
Theorem
Let X denote a random variable with
probability density function f(x) and U =
h(X).
Assume that h(x) is either strictly increasing
(or decreasing) then the probability density of
U is: 1
dh (u ) dx
 1

g  u   f h (u )
du
 f  x
du
Proof
Use the distribution function method.
Step 1 Find the distribution function, G(u)
Step 2 Differentiate G (u ) to find the
probability density function g(u)
G  u   P  U  u   P  h  X   u 

 P  X  h 1 (u )  h strictly increasing
  

 P  X  h (u )  h strictly decreasing
 1
 
 F h 1 (u )
  h strictly increasing

1
1  F h (u )   h strictly decreasing

hence
g  u   G  u 
 dh  u  1



 F h  u
1

du
 h strictly increasing

 F  h 1 u dh  u 
1

  
du
 h strictly decreasing
or
1
dh (u ) dx

g  u   f h (u )
1
 du
 f  x
du
 Example
Suppose that X has a Normal distribution
with mean  and variance 2.
Find the distribution of U = h(x) = eX.
Solution:
 x 
2

1 
f  x  e 2 2

2
dh  u  d ln  u  1
1

h  u   ln  u  and
1
 
du du u
hence
1
dh (u ) dx

g  u   f h (u )
1
 du
 f  x
du

 ln  u    
2

1 1 
 e 2 2
for u  0
2 u
This distribution is called the log-normal
distribution
 log-normal distribution
0.1

0.08

0.06

0.04

0.02

0
0 10 20 30 40
 The Transfomation Method
(many variables)
Theorem
Let x1, x2,…, xn denote random variables with
joint probability density function
f(x1, x2,…, xn )
Let u1 = h1(x1, x2,…, xn).
u2 = h2(x1, x2,…, xn).

un = hn(x1, x2,…, xn).
define an invertible transformation from the x’s to the u’s
Then the joint probability density function of
u1, u2,…, un is given by:
d  x1 , , xn 
g  u1 , , un   f  x1 , , xn 
d  u1 , , un 
 f  x1 , , xn  J
 dx1 dx1 
 du 
dun 
d  x1 , , xn   1 
where J   det    
d  u1 , , un   
 dxn 
dxn 
Jacobian of the transformation  du1 dun 
 Example
Suppose that x1, x2 are independent with density
functions f1 (x1) and f2(x2)
Find the distribution of
u1 = x1+ x2
u2 = x 1 - x 2
Solving for x1 and x2 we get the inverse transformation
u1  u2
x1 
2
u1  u2
x2 
2
The Jacobian of the transformation
 dx1 dx1 
 du du2 
d  x1 , x2   det  1

J
d  u1 , u2   dx2 dx2 
 du 
du2 
 1
1 1 
2 2    1   1    1  1    1
 det         
1 
1   2  2   2  2  2
 2 2 
The joint density of x1, x2 is
f(x1, x2) = f1 (x1) f2(x2)
Hence the joint density of u1 and u2 is:
g  u1 , u2   f  x1 , x2  J

 u1  u2   u1  u2  1
 f1   f2  
 2   2 2
From
 u1  u2   u1  u2  1
g  u1 , u2   f1   f2  
 2   2 2
We can determine the distribution of u1= x1 + x2

g1  u1    g  u , u  du
1 2 2


 u1  u2   u1  u2  1
 

f1   f2 
 2   2 2
 du2

u1  u2 u1  u2 dv 1
put v  then  u1  v, 
2 2 du2 2
Hence

 u1  u2   u1  u2  1
g1  u1    f1   f2   du2
  2   2 2

  f  v f  u

1 2 1  v  dv

This is called the convolution of the two

densities f1 and f2.
Example: The ex-Gaussian distribution
Let X and Y be two independent random
variables such that:
1. X has an exponential distribution with
parameter .
2. Y has a normal (Gaussian) distribution with
mean  and standard deviation .
Find the distribution of U = X + Y.
This distribution is used in psychology as a
model for response time to perform a task.
  e   x x0
Now f1  x   
 0 x0

 x 
2

1 
f2  y   e 2 2

2
The density of U = X + Y is :.

g  u   f  v  f  u  v  dv
1 2


  u v   
2

1 
  e v
e 2 2
dv
0 2
or   u v   
2

  v
g  u   e 2 2
dv
2 0
  u  v     2 2  v
2

 
 
2 0
e 2 2
dv

v 2  2 u    v   u     2 2  v
2

 
 
2 0
e 2 2
dv

 u 
2
 v 2  2  u     2   v
  
 e
2
2 2 2
e dv
2 0
  u  2 v 2  2  u     2   v
or  


 e
2
2 2 2
e dv
2 0

2 2
 u    2   u     2   v 2  2  u     2  v   u     2 
  

2
e 2 2
e
0
2 2
dv

2 2
 u    2   u     2   v 2  2  u     2  v    u     2 
 1 
 e 2 2

0 2
e 2 2
dv

2
 u    2   u     2 

 e 2 2
P  V  0
Where V has a Normal distribution with mean

V  u        2

and variance 2.

Hence

   u    
 2 
       2   u  
g  u   e  2  1       
   2

  
Where (z) is the cdf of the standard Normal
distribution
 The ex-Gaussian distribution
0.09

g(u)
0.06

0.03

0
0 10 20 30