STAS 221 Statistical Methods for

Computer Engineering
Spring 2018-2019

Lecture 3

Families of discrete distributions

Prof. Dr. Orhan Gemikonaklı
 Last Week
• Discrete Random Variables and Their
Distributions
– Distribution of a random variable
– Distribution of a vector
– Expectation and Variance
– Covariance and Correlation

CNG222
 This Week
• Chapter 3: Families of Discrete Distributions
– Uniform Distribution
– Bernoulli distribution
– Binomial distribution
– Geometric distribution
– Negative Binomial distribution
– Poisson distribution

CNG222
 Discrete Distributions
• Discrete random variables are used to
describe random phenomena in which only
discrete values can occur.
• In this section, we will learn about:
– Uniform distribution
– Bernoulli trials and Bernoulli distribution
– Binomial distribution
– Geometric and negative binomial distribution
– Poisson distribution

O Gemikonakli, METU 4
 Discrete Uniform Distribution
• A random variable X follows the discrete uniform
distribution on the interval [a, a+1,...,b], if it may attain
each of these values with equal probability.
1
𝑃 𝑋 = 𝑥 = 𝑛, 𝑎 ≤ 𝑥 ≤ 𝑏
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
0, 𝑥<𝑎
𝑥−𝑎+1
𝐹 𝑋 = 𝑎≤𝑥≤𝑏
𝑛
1, 𝑥>𝑏

𝑛 =𝑏−𝑎+1

CNG222
Discrete Uniform Distribution
1 𝑛2 − 1
𝐸 𝑋 = 𝑎 + 𝑏 and 𝑉 𝑋 =
2 12

CNG222
 Discrete Uniform Example:
Throwing a die:
𝑛 =𝑏−𝑎+1=6−1+1=6
1 1
𝐸 𝑋 = 𝑎 + 𝑏 = 1 + 6 = 3.5
2 2
1 6 1+2+3+4+5+6
(Alternatively 𝐸 𝑋 = 𝑖= = 3.5)
𝑛 𝑖=1 6
𝑛 𝑛(𝑛+1) 6(6+1)
Note: 𝑖=1 𝑖 = , for n=6, = 21.
2 2
𝑛 2 − 1 62 − 1
𝑉𝑎𝑟 𝑋 = = ≈ 2.92
12 12

CNG222
 Discrete Uniform - Example

• Repeat the previous example for the case of

flipping a coin.
– 𝑛 =?
– 𝐸 𝑋 =?
– 𝑉𝑎𝑟 𝑋 =?

CNG222
 Bernoulli Trials
and Bernoulli Distribution
• A random variable with two possible values, 0
and 1, is called a Bernoulli variable
• The distribution of such a r.v. is called the
Bernoulli distribution
• Any random experiment with a binary
outcome is called a Bernoulli trial
• Generic outcome names: successes and
failures

CNG222
 Not equally likely Outcomes
• In general, f(1) = f(0) = 0.5 does NOT hold
when the binary outcomes are not equally
likely
• If f(1) = p, what is E(X) and Var(X)?

CNG222
 What about “non 0-1”, binary
outcomes?

• Example:
– What if the two possible outcomes are 5 and 9
with f(5) = 0.3 and f(9) = 0.7?
– What is the expected value?

CNG222
 What about “non 0-1”, binary
outcomes?
• Example:
– What if the two possible outcomes are 5 and 9
with f(5) = 0.3 and f(9) = 0.7?
– What is the expected value?
– It is just a shifted and rescaled standard Bernoulli
trial.
• X = 4B + 5
• E(X) = E(4B + 5) = 4E(B)+5 = 4∙0.7+5 = 7.8

CNG222
 Bernoulli
• Bernoulli Trials:
– Consider an experiment consisting of n trials, each can be a success or
a failure.
• Let Xj = 1 if the jth experiment is a success
• and Xj = 0 if the jth experiment is a failure
– The Bernoulli distribution (one trial):
 p, x j  1, j  1,2,..., n

p j ( x j )  p( x j )  1  p  q, x j  0,j  1,2,...,n
0,
 otherwise
– where E(Xj) = p and V(Xj) = p (1-p) = p q
• Bernoulli process:
– The n Bernoulli trials where trials are independent:
p(x1,x2,…, xn) = p1(x1) p2(x2) … pn(xn)

O Gemikonakli, METU 13
 Binomial Distribution
• The number of successes in n Bernoulli trials, X,
has a binomial distribution.
– n: number of trials
– p: probability of success
𝑛 𝑥 𝑛−𝑥
𝑓𝑋 𝑥 = 𝑃 𝑋 = 𝑥 = 𝑝 𝑞
𝑥
• Expected value and variance:
– A binomial variable X is a sum of n independent
Bernoulli trials.
– E(X) = np, Var(X) = npq

O Gemikonakli, METU
 Binomial Distribution

 n  x n  x
  p q , x  0,1,2,..., n
p( x)   x 
0,
 otherwise
The number of
Probability that
outcomes having the
there are
required number of
x successes and
successes and
(n-x) failures
failures

O Gemikonakli, METU 15
 Using Distribution Tables
• Table A2, cdf of Binomial distribution
• pdf can be obtained by difference of two
consecutive entries
• Example 3.16
• Example 3.17

O Gemikonakli, METU
 Geometric Distribution

• The number of Bernoulli trials, X, needed to get

the first success

𝑓𝑥 𝑥 = 𝑃 𝑋 = 𝑥 = 𝑝𝑞 𝑥−1 , 𝑥 = 0, 1, 2, … .

∞ 𝑥−1
Check: 𝑥 𝑓𝑥 𝑥 = 𝑥=1 𝑝 1−𝑝 =1

• Expected value and variance

– E(x) = 1/p, and V(X) = q/p2

O Gemikonakli, METU 17
 Geometric Distribution
• Example 3.20 St. Petersburg Paradox
• Gambling with a guaranteed strategy to win a
desired amount
– Even when p is less then 0.5!
– Start with the desired amount
– Double betting amount every time you loose
– Stop when you win the first time
– E.g if p=0.2 the expected number of bets to win is
5 (five)!

O Gemikonakli, METU
 Geometric Distribution
• So what’s the paradox?
• What is the amount of money, Y, needed to
follow the strategy?
– 𝑌 = 𝐷2𝑋−1 where D is the desired amount and X
is the number of bets needed to win.
– E(Y) = infinity when p≤0.5 (the paradox)

CNG222
 Negative Binomial
• In a sequence of independent Bernoulli trials,
the number of trials needed to obtain k
successes
– It can be considered as the inverse of the
Binomial, where, we now fix the number of
successes and count the number of trials n to
reach that number of successes
• It is a generalization of the Geometric
distribution
CNG222
 Negative binomial distribution
• The number of Bernoulli trials, X, until the kth
success
• If Y is a negative binomial distribution with
parameters p and k, then:

 y  1 y  k k
  q p , y  k , k  1, k  2,...
p( x)   k  1
0,
 otherwise

• E(Y) = k/p, and V(X) = kq/p2

O Gemikonakli, METU
 Poisson Distribution
• Poisson distribution describes many random
processes quite well and is mathematically quite
simple.
– where a > 0, pdf and cdf are:
x
e a a i
e a F ( x)  
a x

p( x)   x! , x  0,1,... i 0 i!
0, otherwise

– E(X) = a = V(X)

O Gemikonakli, METU 22
 Poisson Distribution
• Example: A computer repair person is “beeped” each
time there is a call for service. The number of beeps
per hour ~ Poisson(a = 2 per hour).

– The probability of three beeps in the next hour:

p(3) = e-223/3! = 0.18
also, p(3) = F(3) – F(2) = 0.857-0.677=0.18

– The probability of two or more beeps in a 1-hour period:

p(2 or more) = 1 – p(0) – p(1)
= 1 – F(1)
= 0.594
O Gemikonakli, METU 23
 Poisson approximation of Binomial
distribution
• Poisson distribution can be used to
approximate Binomial distribution when n is
large and p is small
– E.g., 𝑛 ≥ 30 and 𝑝 ≤ 0.05
– 𝑛𝑝 = 𝜆
• Example 3.25 The Birthday Problem

CNG222
 Summary
• Chapter 3: Families of Discrete Distributions
– Uniform distribution
– Bernoulli distribution
– Binomial distribution
– Geometric distribution
– Negative Binomial distribution
– Poisson distribution

CNG222
 Next Lecture – Chapter 4
• Probability density
• Families of continuous distributions
– Uniform distribution
– Exponential distribution
– Gamma distribution
– Normal distribution
– Central Limit Theorem

CNG222
 Example 3.16
(Table A2, cdf of Binomial distribution)
As part of a business strategy, randomly selected
20% of new internet service subscribers receive
a special promotion from the provider. A group
of 10 neighbors signs for the service. What is the
probability that at least 4 of them get a special
promotion?

CNG222
 Example 3.16
• We need to find the probability P {X ≥ 4},
where X is the number of people, out of 10,
who receive a special promotion. This is the
number of successes in 10 Bernoulli trials,
therefore, X has Binomial distribution with
parameters n = 10 and p = 0.2.
𝑛 𝑥 𝑛−𝑥
𝑓𝑋 𝑥 = 𝑃 𝑋 = 𝑥 = 𝑝 𝑞
𝑥

CNG222
 Example 3.16
Use Table A2 for CDF

P {X ≥ 4} = 1 − F(3) = 1 − 0.8791 = 0.1209.

Bonus question: Can you obtain E(X) and Var(X)?

CNG222
 Example 3.17
An exciting computer game is released. Sixty
percent of players complete all the levels. Thirty
percent of them will then buy an advanced
version of the game. Among 15 users, what is
the expected number of people who will buy the
advanced version? What is the probability that
at least two people will buy it?

CNG222
 Example 3.17
• p = P {buy advanced}
= P {buy advanced | complete all levels}P
{complete all levels}
= (0.30)(0.60) = 0.18.
• Then we have
E(X) = np = (15)(0.18) = 2.7 and
P {X ≥ 2} = 1 − P(0) − P(1)
= 1 − (1 − p)n − np(1 − p)n−1 = 0.7813.
• The last probability was computed directly by
formula (3.9) because the probability of success,
0.18, is not in Table A2.
CNG222
 Example 3.20
(St. Petersburg Paradox). This paradox was
noticed by a Swiss mathematician Daniel
Bernoulli (1700–1782), a nephew of Jacob. It
describes a gambling strategy that enables one
to win any desired amount of money with
probability one.

CNG222
 Example 3.20
• Consider a game that can be played any number of times.
• Rounds are independent, and each time your winning
probability is p.
• The game does not have to be favorable to you or even fair;
this p can be any positive probability. For each round, you
bet some amount x.
• In case of a success, you win x. If you lose a round, you lose
x.
• The strategy is simple. Your initial bet is the amount that
you desire to win eventually.
• Then, if you win a round, stop. If you lose a round, double
your bet and continue.

CNG222
 Example 3.20
• Let the desired profit be $100. The game will
progress as follows:

CNG222
 Example 3.20
• How many rounds should be played?
• How much money does one need to have in
order to be able to follow this strategy?

CNG222