Chapter 1

Probability
1.1 Definition
prop If E ∩ F = ∅ then P(E ∪ F ) = P(E ) + P(F )

prop Inclusion-exclusion principle:

n
! n n
!
[ X X X \
P Ai = P(Ai ) − P(Ai ∩ Aj ) + P(Ai ∩ Aj ∩ Ak ) − · · · + (−1)n+1 P Ai
i=1 i=1 i<j i<j<k i=1

1.2 Conditional probability and Bayes’ theorem

def The conditional probability of event E given event F is defined as:
P(E ∩ F )
P(E | F ) = .
P(F )
Multiplication rule

P(E1 E2 E3 · · · En ) = P(E1 )P(E2 | E1 )P(E3 | E1 E2 ) · · · P(En | E1 E2 · · · En−1 )

Total probability rule

Let E1 , E2 , E3 , . . . , En be a collection of mutually exclusive events whose union is sample space

S. Let E be any event, then
n
X
P(E ) = P(E | Ej )P(Ej )
j=1

Bayes’ Theorem

Let E1 , E2 , E3 , . . . , En be a collection of mutually exclusive events whose union is sample space

S. Let E be any event such that P(E ) ̸= 0. Then for any event Ek , k = 1, 2, 3, . . . , n,

P(E | Ek )P(Ek ) P(E | Ek )P(Ek )

P(Ek | E ) = Pn =
j=1 P(E | Ej )P(Ej ) P(E )

1
Chapter 2
Discrete Distributions
2.1 Random variables
def A random variable is any rule that associates a number with each outcome in a sample
space.

Random variables are customarily denoted by uppercase letters. A particular value of the ran-
dom variable is denoted by a lowercase letter.

def A Bernoulli random variable is a rv whose only possible values are 0 and 1.

2.2 Probability distributions

def The probability distribution of X gives how the total probability of 1 is allocated to each
value of the rv. Also known as the probability mass function (pmf).

p(x) = P(X = x) = P (∀s ∈ S, X (s) = x)

def A parameter is a quantity that can be assigned any one of a number of possible values,
with each different value determining a different probability distribution.

The collection of all probability distributions for different values of the parameter is called a
family of probability distributions.

def The cumulative distribution function (cdf) of a discrete random variable X is the probability
that X will take a value less than or equal to x.

X
F (x) = P(X ≤ x) = p(y )
y :y ≤x

The graph of the cdf of a discrete random variable is a step function.

prop P(a ≤ X ≤ b) = F (b) − F (a−)

where ”a−” represents the largest possible value of the rv X strictly less than a.

2
2.3 Expected values
def The expected value or mean value of a rv X is the average value of X on performing repeated
trials of an experiment.

The expectation of a discrete random variable is the weighted average of all possible outcomes,
where the weights are the probabilities of realizing each given value.
X
E (x) = µX = x · p(x)
x∈D

The expected value of X describes where the probability distribution is centered.

P
prop For any function h(X ), E [h(X )] = h(x) · p(x), where D is the set of possible values for
D
the rv X and p(x) is the pmf.

prop For a linear function, E (aX + b) = a · E (X ) + b.

prop E [X + Y ] = E [X ] + E [Y ]

prop E [X Y ] = E [X ] · E [Y ] if X and Y are independent variables.

def If an rv X has a pmf p(x) and expected value µ, the variance (V(X) or σX 2 ) is given by:
X
V (X ) = (x − µ)2 · p(x) = E [(X − µ)2 ] = E [X 2 ] − (E [X ])2
D

prop V (cX ) = c 2 V (X )

Proof. V (cX ) = E [(cX )2 ] − (E [cX ])2 = E [c 2 X 2 ] − c 2 (E [X ])2 = c 2 V (X )

2.4 Moment generating functions

def The k th moment of X is defined as E [X k ].

def The k th central moment of X is defined as E [(X − µ)k ].

def The moment generating function for X is denoted by mX (t) and defined as mX (t) = E [e tX ]

prop The k th moment of a random variable X is given by:

dk mX (t)
E [X k ] =
dt k t=0

If the Taylor expansion of the mgf of X is given by M(t) = a0 + a1 t + a2 t 2 + . . . + an t n + . . . ,

then E [X n ] = n!an .

3
prop if the moments of a specified order exist, then all the lower order moments automatically
exist.

prop MX +a (t) = e at MX (t)

prop MbX (t) = MX (bt)

prop The mgf of the sum of a number of independent random variables is equal to the product
of their respective mgfs.

MX1 +X2 +...+Xn (t) = MX1 (t) · MX2 (t) · . . . · MXn (t)

2.5 Binomial random variable

def A Bernoulli trial is a random experiment or a trial whose outcome can be classified as either
a success or a failure.

def The binomial random variable X denotes the number of successes that occur in n indepen-
dent Bernoulli trials. It takes the parameters n and p, where p is the probability of success,
which remains same for every trial.

If X is a binomial random variable with parameters (n, p), then we write it as X ∼ Bin(n, p).

The pmf of a binomial random distribution having parameters (n, p) is given by:
 
 n p x q n−x , x = 0, 1, . . . , n
b(x; n, p) = x
0, otherwise


where q = 1 − p.

prop E [X ] = np

Proof.

prop V (X ) = np(1 − p)

prop mX (t) = p(e t + 1 − p)n

prop If Xi , (i = 1, 2, . . . , k) are independent random variables with parameters (ni , p), (i =

1, 2, . . . , k), then their sum
k k
!
X X
Xi ∼ B ni , p
i=1 i=1

4
2.6 Geometric random variable
def A geometric random variable X is one which has a geometric distribution with parameter
p, 0 < p < 1.

The density function (pmf) of a geometric rv is given by:

f (x) = (1 − p)x−1 p = q x−1 p for x = 1, 2, 3, . . .

The cumulative density function of a geometric rv is given by:

(
0, if x < 1
F (x) =
1 − q [x] , if x ≥ 1

1
prop E [X ] =
p
q
prop V (X ) =
p2
pe t
prop mX (t) =
1 − qe t

2.7 Poisson random variable

def The Poisson random variable X is one which has a Poisson distribution with parameter k.

The pmf of a Poisson distribution is given as:

e −k k x
p(x; k) = ; for x = 0, 1, 2, . . . and k > 0.
x!
∞ ∞
k
X kx X e −k · k x
e = ; and so =1
x=0
x! x=0
x!

prop E [X ] = k

prop V [X ] = k
t −1)
prop mX (t) = e k(e ∀t ∈ R

prop For any binomial experiment in which n is large (¿50) and p is small, b(x; n, p) ≈ p(x; k),
where k = np.

5
2.7.1 Poisson process

A Poisson process is a counting process with rate λ, such that Xt is the Poisson random variable
(with parameter λt), or in other words the number of events that occur during the interval
[0, t).

2.8 Hypergeometric random variable

A random sample of size n is drawn from a collection of N objects. Of the N objects, r objects
have a trait of interest.

def The hypergeometric random variable X is the number of objects with the trait of interest
in the random sample. It takes the parameters (N, n, r ).

The pmf of a hypergeometric random variable is given by:

  
r N −r
x n−x
P(X = x) =  
N
n

r 
prop E [X ] = n
N
 r  N − r  N − n
prop V (X ) = n
N N N −1

6
Problem 1

An urn contains 4 white and 4 black balls. We randomly choose 4 balls. If 2 of them are white
and 2 are black, we stop. If not, we replace the balls in the urn and again randomly select 4
balls. This continues until exactly 2 of the 4 chosen are white. What is the probability that we
shall make exactly three selections?

Problem 2

An urn contains N white and M black balls. Balls are randomly selected, one at a time, until
a black one is obtained. If we assume that each ball selected is replaced before the next one
is drawn, what is the probability that (i) exactly n draws are needed (ii) at least k draws are
needed?

1
Problem 3

The probability that a machine produces a defective item is 0.02. Each item is checked as it is
produced. Assuming that these are independent trials, what is the probability that at least 100
items must be checked to find one that is defective?

2
Problem 1

It is known that disks produced by a certain company will be defective with probability 0.01
independently of each other. The company sells the disks in packages of 10 and offers a money-
back guarantee that at most 1 of the 10 disks is defective. (i) What proportion of packages is
returned? (ii) If someone buys three packages, what is the probability that exactly one of them
will be returned?

Problem 2 If the independent random variables X and Y are binomially distributed, respec-
tively with n = 3, p = 13 , and n = 5, p = 13 , write down the probability that X + Y ≥ 1.

Solution:
Let X ∼ Binomial(3, 13 ) and Y ∼ Binomial(5, 31 ). We want:

P(X + Y ≥ 1) = 1 − P(X + Y = 0).

Since X and Y are independent:

P(X + Y = 0) = P(X = 0)P(Y = 0).

For a binomial distribution X ∼ Binomial(n, p):
 
n 0
P(X = 0) = p (1 − p)n = (1 − p)n .
0
Thus:
 3
1 2 8
P(X = 0) = (1 − )3 = = .
3 3 27
 5
1 5 2 32
P(Y = 0) = (1 − ) = = .
3 3 243
Now,

1
 8 32 256
P(X + Y = 0) = × = .
27 243 6561
Finally:
256 6305
P(X + Y ≥ 1) = 1 − = .
6561 6561

2
Problem 1

If electricity power failures occur according to a Poisson distribution with an average of 3 failures
every twenty weeks, calculate the probability that there will not be more than one failure during
a particular week.

Problem 2

If the number of claims handled daily by an insurance company follows Poisson distribution
with mean 5, what is the probability that there will be 4 claims each in exactly 3 of the next 5
days ? Assume that the number of claims on different days is independent.

1
Problem 3

Suppose that the annual number of tropical cyclones that are formed off the U.S. Pacific coast
has a Poisson distribution with mean 15. What is the probability that a given year has at most
5 cyclones?