Gamma Distribution

Continuous Data Sampling Distribution (STU 07215)

Leguma Bakari
Email: [email protected]
Email: [email protected]
Phone:+255 762 760 095
September 3, 2022
Eastern Africa Statistical Training Center (EASTC)

1
Outline

1 Definition

2 Facts of Gamma Function

3 Moment Generating Function

4 Expected Value and Variance

5 Incomplete Gamma Function

6 Distribution Function

7 Gamma Function with Rate Parameter

2
Definition

• In the approximate of Poisson process with mean 𝜆, the waiting

time X until the next(first) event occurs follows an Exponential
distribution .
• If now we let the random variable X be the waiting time until the
𝛼th event occurs,
• The distribution for a random variable X is now follow a Gamma
distribution , 𝛼th represents a total of Poisson arrivals in a given
time X .
• A random variable X is said to follow a Gamma distribution is it
satisfies the following density function
•
𝛼
 Γ(𝜆 𝛼) x 𝛼−1 e−𝜆x ; x > 0



f ( x; 𝛼, 𝜆) =
 0, otherwise


3
Standard Gamma Distribution and Gamma Function

• For 𝜆 = 1, then the Gamma density function will be reduced into

standard Gamma distribution with the following density function
•
 1 x 𝛼−1 e−x ; x > 0



f ( x; 𝛼) = Γ( 𝛼)
 0, otherwise


• The are under the curve for standard Gamma distribution will be
∫∞
• f ( x; 𝛼) = 1, therefore
∫0∞ 1
• x 𝛼− 1 e − x = 1, which is equivalent to
∫0∞ Γ ( 𝛼)
• 0
x 𝛼− 1 e − x dx = Γ(𝛼)
• where Γ(𝛼) is an important quantity known as Gamma function.
∫∞
• Γ(𝛼) = 0 x 𝛼−1 e−x dx

4
Outline

1 Definition

2 Facts of Gamma Function

3 Moment Generating Function

4 Expected Value and Variance

5 Incomplete Gamma Function

6 Distribution Function

7 Gamma Function with Rate Parameter

5
Properties (Facts) of Gamma Function

i. Γ( 1) = 1
ii. Γ(𝛼) = (𝛼 − 1)Γ(𝛼 − 1), and for 𝛼 equals to integer n + 1, then
Γ( n+1 )
iii. Γ( n + 1) = nΓ( n), which is the same as, n = Γ (n)
iv. Γ( n) = ( n − 1) !, which is the same as, Γ( n + 1) = n!
  √
v. Γ 12 = 𝜋

6
Proof of Γ( 1) = 1

• Recall the Gamma function,

∫∞
• Γ(𝛼) = 0 x 𝛼−1 e−x dx
• at 𝛼 = 1,
∫∞
• Γ( 1) = 0 x 1−1 e−x dx ,
∫∞
• Γ( 1) = 0 x 0 e−x dx
∫∞
• Γ( 1) = 0 ( 1) e−x dx
∫∞
• Γ( 1) = 0 e−x dx = −e−x | xx ==∞
0
• Γ( 1) = −e−∞ − (−e0 ) = 0 − (−1) = 1
• Γ( 1) = 1, hence proved.

7
Proof of Γ(𝛼) = (𝛼 − 1)Γ(𝛼 − 1)

• by using integration by parts

∫ ∫
udv = uv − vdu
∫∞
• Γ(𝛼) = 0 x 𝛼−1 e−x dx
• let u = x 𝛼−1 and dv = e−x dx ,
• du
∫ ∫
= (𝛼 − 1) x ( 𝛼−2 ) ,du = (𝛼 − 1) x 𝛼−2 dx and dv = e−x dx ,
dx
v = −e − x
∫∞
• Γ(𝛼) = 0 x 𝛼−1 e−x dx =
∞ ∫ ∞
)(−e−x ) 0 − 0 (−e−x ) (𝛼 − 1) x 𝛼−2 dx
 𝛼−1
(x
∫∞ ∞ ∫∞
• Γ(𝛼) = 0 x 𝛼−1 e−x dx = −x 𝛼−1 e−x 0 + (𝛼 − 1) 0 x 𝛼−2 e−x dx


• Γ(𝛼) =
∞ ∫∞
(−∞ 𝛼−1 e−∞ ) − (−0 𝛼−1 e−0 ) 0 + (𝛼 − 1) 0 x ( 𝛼−1 ) −1 e−x dx


• Γ(𝛼) = [−∞( 0) + 0 ( 1)] + (𝛼 − 1)Γ(𝛼 − 1) = 0 + (𝛼 − 1)Γ(𝛼 − 1)

• Γ(𝛼) = (𝛼 − 1)Γ(𝛼 − 1), hence proved.
8
Proof of Γ( n) = ( n − 1) !

• For integer n, ie 𝛼 = n,
• Γ(𝛼) = (𝛼 − 1)Γ(𝛼 − 1) becomes,
• Γ( n) = ( n − 1)Γ( n − 1), by using the recurrence method,
• the expression, Γ( n − 1) can be written as,
• Γ( n − 1) = [( n − 1) − 1]Γ[( n − 1) − 1] = Γ( n − 1) = ( n − 2)Γ( n − 2)
• Γ( n) = ( n − 1)Γ( n − 1) = ( n − 1) [( n − 2)Γ( n − 2)]
• Γ( n) = ( n − 1)( n − 2) ( n − 3)Γ( n − 3)
• Γ( n) = ( n − 1)( n − 2) ( n − 3) · · · Γ( 1)
• but Γ( 1) = 1, therefore
• Γ( n) = ( n − 1)( n − 2) ( n − 3) · · · ( 1) = ( n − 1) !
• Γ( n) = ( n − 1) !, or Γ( n + 1) = n!

9
 Γ( n+1)
Proof of n = Γ( n)

• Recall Γ( n) = ( n − 1)Γ( n − 1)
• which also implies Γ( n + 1) = nΓ( n)
Γ( n+1 )
• therefore n = Γ( n )

10
 √
Proof of Γ( 12 ) = 𝜋

∫∞
• Γ(𝛼) = 0 x 𝛼−1 e−x dx
• at 𝛼 = 21
∫∞ 1 ∫∞ 1
• Γ 12 = 0 x 2 −1 e−x dx = 0 x − 2 e−x dx
2
• let x = z2 , dx
dz
= z , dx = zdz
z2
√
• for x = 2 it imply z = 2x , the change of limits are,
• at x = 0 , z = 2 ( 0) = 0 and at x = ∞ , z = 2 (∞) = ∞,
√︁ √︁

therefore
  ∫ ∞  2  − 12 z2
• Γ 21 = 0 z2 e − 2 zdz
z2 z2
  ∫∞ 1 1 ∫∞ 1
• Γ 12 = 0 ( z 2 ) − 2 ( 2−1 ) − 2 e− 2 zdz = 0 z −1 z ( 2 2 ) e− 2 dz
  1 ∫ ∞ z2 √ ∫ ∞ z2
• Γ 21 = 2 2 0 z 0 e− 2 dz = 2 0 e− 2 dz
  √ √ ∫∞ z2
• Γ 12 = 2 2𝜋 0 √1 e− 2 dz
2𝜋
  √ ∫ ∞ 1 − z2
• Γ 2 = 2 𝜋 0 √ e 2 dz
1
11
2𝜋
• from standard normal distribution
∫∞ z2
• 0 √1 e− 2 dz = 21
2𝜋
  √ ∫∞ z2 √  
• Therefore, Γ 12 = 2 𝜋 0 √1 e− 2 dz = 2 𝜋 21
2𝜋
  √
• Γ 2 = 𝜋, hence proved.
1

12
Outline

1 Definition

2 Facts of Gamma Function

3 Moment Generating Function

4 Expected Value and Variance

5 Incomplete Gamma Function

6 Distribution Function

7 Gamma Function with Rate Parameter

13
Moment Generating Function

• Mx ( t ) = E ( etx ) = x etx f ( x ) dx
∫
∫∞ ∫∞
• Mx ( t ) = 0 etx Γ(𝜆 𝛼) x 𝛼−1 e−𝜆x dx =
𝛼 𝜆𝛼 𝛼− 1 e −𝜆x etx dx
Γ ( 𝛼) ∫0 x
∞ ∞
• Mx ( t ) = Γ𝜆( 𝛼) 0 x 𝛼−1 e−𝜆x +tx dx =
𝛼 ∫ 𝛼
𝜆 𝛼− 1 e − (𝜆− t ) x dx
Γ( 𝛼) 0 x
y
• Let, y = (𝜆 − t ) x , then x = 𝜆− , dy = 𝜆 − t , dx =
t dx
dy
𝜆− t .
• x = 0, y = (𝜆 − t )( 0) = 0, and
• x = ∞, y = (𝜆 − t ) (∞) = ∞
𝛼 ∫ ∞ y
𝛼− 1 − y dy
• Mx ( t ) = Γ𝜆( 𝛼) 0 𝜆− t e 𝜆− t
∞ 𝛼− 1
• 𝜆𝛼
∫ 1 1
Mx ( t ) = Γ ( 𝛼) 0 y (𝜆− t ) 𝛼− 1
e − y 𝜆− t
dy
∞
•
𝛼 ∫
𝜆 1 −y
Mx ( t ) = (𝜆− t ) 𝛼 Γ( 𝛼) 0 y 𝛼− e dy
∫∞
• but 0 y 𝛼− 1 −y
e dy = Γ(𝛼)
•
𝛼 𝜆𝛼
therefore Mx ( t ) = (𝜆−t 𝜆) 𝛼 Γ( 𝛼) Γ(𝛼) = (𝜆−

t)
𝛼
𝜆− t − 𝛼 t −𝛼
• 𝜆 𝛼



Mx ( t ) = 𝜆− t = 𝜆 = 1− 𝜆
t −𝛼
•


Mx ( t ) = 1 − 𝜆
14
Outline

1 Definition

2 Facts of Gamma Function

3 Moment Generating Function

4 Expected Value and Variance

5 Incomplete Gamma Function

6 Distribution Function

7 Gamma Function with Rate Parameter

15
Expected Value

• E ( X ) = Mx′ ( t )| t =0
𝜆− t − 𝛼

−𝛼
• Mx ( t ) = 𝜆−𝜆 𝛼
= 1 − 𝜆t



t
= 𝜆
 
t − 𝛼− 1

− 𝛼−1
• Mx′ ( t ) = −𝛼 1 − − 𝜆1 = 𝛼𝜆 1 − 𝜆t


𝜆
  − 𝛼−1
• E ( X ) = Mx′ ( t )| t =0 = 𝛼
𝜆 1− 0
𝜆 = 𝛼
𝜆 ( 1)
− 𝛼− 1

• E (X ) = 𝛼
𝜆

16
Variance

• Var ( X ) = E ( X 2 ) − [ E ( X )] 2
• E ( X 2 ) = Mx′′ ( t )| t =0
t − 𝛼− 1
• recall Mx′ ( t ) = 𝛼


𝜆 1− 𝜆
 
t − 𝛼− 2
• M ′′ ( t ) − 𝜆1
𝛼


x = 𝜆 (−𝛼 − 1) 1 − 𝜆
t − 𝛼− 2
• Mx′′ ( t ) = 𝛼


𝜆2
(𝛼 + 1) 1 − 𝜆
  − 𝛼−2
• E ( X 2 ) = Mx′′ ( t )| t =0 = 𝛼
𝜆2
(𝛼+ 1) 1− 0
𝜆 = 𝛼
𝜆2
(𝛼+ 1) ( 1) − 𝛼−2
𝛼2 +𝛼
• E (X 2) = 𝜆2
𝛼2 +𝛼 𝛼 2
• Var ( X ) = E ( X 2 ) − [ E ( X )] 2 =


𝜆2
− 𝜆
𝛼2 +𝛼 𝛼2 𝛼2 +𝛼− 𝛼2
• Var ( X ) = 𝜆2
− 𝜆2
= 𝜆2
• Var ( X ) = 𝛼
𝜆2

17
Outline

1 Definition

2 Facts of Gamma Function

3 Moment Generating Function

4 Expected Value and Variance

5 Incomplete Gamma Function

6 Distribution Function

7 Gamma Function with Rate Parameter

18
Incomplete Gamma Function

• recall the Gamma function

∫∞ ∫a ∫∞
• Γ(𝛼) = 0 x 𝛼−1 e−x dx = 0 x 𝛼−1 e−x dx + a x 𝛼−1 e−x dx
• Γ(𝛼) = 𝛾( a, 𝛼) + Γ( a, 𝛼), where
∫a
• 𝛾( a, 𝛼) = 0 x 𝛼−1 e−x dx , and
∫∞
• Γ( a, 𝛼) = a x 𝛼−1 e−x dx
• 𝛾( a, 𝛼) and Γ( a, 𝛼) are both known as incomplete Gamma
function,
• recall
• 𝛾( a, 𝛼) + Γ( a, 𝛼) = Γ(𝛼),
• upon the normalization,
𝛾 ( a, 𝛼) Γ ( a, 𝛼) Γ ( 𝛼)
• Γ ( 𝛼) + Γ ( 𝛼) = Γ ( 𝛼)
• P ( a, 𝛼) + Q ( a, 𝛼) = 1, therefore
𝛾 ( a, 𝛼)
• P ( a, 𝛼) = Γ( 𝛼)
Γ( a, 𝛼)
• Q ( a, 𝛼) = Γ ( 𝛼) 19
Outline

1 Definition

2 Facts of Gamma Function

3 Moment Generating Function

4 Expected Value and Variance

5 Incomplete Gamma Function

6 Distribution Function

7 Gamma Function with Rate Parameter

20
Distribution Function

• For Gamma distribution the distribution function is given by

∫a
• F ( x ) = P ( X < a) = 0 f ( x ) dx
∫ a 𝜆𝛼
• F ( a) = 0 Γ ( 𝛼) x 𝛼−1 e−𝜆x dx
𝛼 ∫ a
• F ( a) = Γ(𝜆 𝛼) 0 x 𝛼−1 e−𝜆x dx
• let y = 𝜆x ; x = 𝜆y ;dy=𝜆dx ; dx = 𝜆1 dy
change of limits
• at x = a; y = 𝜆( a) = 𝜆a
at x = 0, y = 𝜆( 0) = 0
𝛼 ∫ 𝜆a
𝛼−1 −y 1 𝛼 ∫ 𝜆a y 𝛼− 1
• F ( a) = Γ(𝜆 𝛼) 0 𝜆y e 𝜆 dy = Γ(𝜆 𝛼) 0 𝜆 𝛼−1 e − y 𝜆1 dy
∫ 𝜆a
• F ( a) = 𝜆 𝛼𝜆Γ( 𝛼) 0 y 𝛼−1 e−y dy = Γ(1𝛼) 𝛾(𝛼, 𝜆a) = 𝛾 (𝜆 a, 𝛼)
𝛼
Γ( 𝛼)
𝛾 (𝜆a, 𝛼)
• F ( a) = P ( X < a) = Γ ( 𝛼) = P (𝛼, 𝜆a)
• which also implies,
Γ(𝜆a, 𝛼)
• F ( a) = P ( X > a) = Γ( 𝛼) = 1 − P (𝜆a, 𝛼) = Q (𝜆a, 𝛼)
21
Outline

1 Definition

2 Facts of Gamma Function

3 Moment Generating Function

4 Expected Value and Variance

5 Incomplete Gamma Function

6 Distribution Function

7 Gamma Function with Rate Parameter

22
Gamma Function with Rate Parameter

• Recall f ( x; 𝛼, 𝜆) = 𝜆𝛼
Γ( 𝛼) x
𝛼− 1 e −𝜆x ; x >0
• where
• 𝛼 is shape parameter and
• 𝜆 is a scale parameter
• For some of the text use rate parameter; ie 𝛽 = 1
𝜆
x
• f (x ) = 1 𝛼− 1 e − 𝛽
𝛽 𝛼 Γ( 𝛼) x
• where
• 𝛼 is shape parameter and
• 𝛽 is a scale parameter
• E ( x ) = 𝛼𝛽
• E ( x ) = 𝛼𝛽2

23
Practical Examples

Recall EXERCISES FOR SECTION 4-10, page 132 download this

book from here.

24