Contents

CHAPTER IV
JOINT PROBABILITY DISTRIBUTIONS

Department of Foundation Year

Institute of Technology of Cambodia

LIN Mongkolsery
[email protected]

2021-2022

PROBABILITY ITC 1 / 27
 Contents

Contents

1 Jointly Distributed Random Variables

2 Expected Values, Covariance, and Correlation

3 The Distribution of the Sample Mean

4 The Distribution of a Linear Combination

PROBABILITY ITC 1 / 27
 Jointly Distributed Random Variables

Contents

1 Jointly Distributed Random Variables

2 Expected Values, Covariance, and Correlation

3 The Distribution of the Sample Mean

4 The Distribution of a Linear Combination

PROBABILITY ITC 2 / 27
 Jointly Distributed Random Variables

Two Discrete Random Variables

1 Dx = the set of all possible values of rv X .
2 D = {(x, y ) : x ∈ Dx , y ∈ Dy }

Definition 1
Let X and Y be two drv’s defined on the sample space S. The probability
that X = x and Y = y is denoted by

p(x, y ) = P(X = x, Y = y ).

The function p(x, y ) is called the joint probability mass function (joint
pmf) of X and Y and has the following properties:
1 0 ≤ p(x, y ) ≤ 1.
X X
2 p(x, y ) = 1.
(x,y )∈D
X X
3 P[(X , Y ) ∈ A] = p(x, y ), where A ⊆ D.
PROBABILITY
(x,y )∈A ITC 3 / 27
 Jointly Distributed Random Variables

Two Discrete Random Variables

Example 2
Roll a pair of fair dice. For each of the 36 sample points with probability
1/36, let X denote the smaller and Y the larger outcome on the dice. The
joint pmf of X and Y is given by the probabilities

1, 1≤x =y ≤6

p(x, y ) = 362
 , 1 ≤ x < y ≤ 6.

36

PROBABILITY ITC 4 / 27
 Jointly Distributed Random Variables

Two Discrete Random Variables

Definition 3
The marginal probability mass function of X , denoted by pX (x), is
given by
X
pX (x) = p(x, y ) for each possible value x.
y ∈Dy

Similarly, the marginal probability mass function of Y is

X
pY (y ) = p(x, y ) for each possible value y .
x∈Dx

PROBABILITY ITC 5 / 27
 Jointly Distributed Random Variables

Two Discrete Random Variables

Example 4
Let the joint pmf of X and Y be defined by
x +y
p(x, y ) = , x = 1, 2, 3, y = 1, 2.
21
Then
2x + 3
pX (x) = , x = 1, 2, 3,
21
and
2+y
pY (y ) = , y = 1, 2.
7

PROBABILITY ITC 6 / 27
 Jointly Distributed Random Variables

Two Continuous Random Variables

Definition 5
Let X and Y be two crv’s. A joint probability density function (joint
pdf) for these two variables is a function f (x, y ) satisfying f (x, y ) ≥ 0 and
Z ∞ Z ∞
f (x, y )dxdy = 1. Then for any (measurable) set A ⊆ R2 ,
−∞ −∞
ZZ
P ((X , Y ) ∈ A) = f (x, y )dxdy .
A

In particular, if A = {(x, y ) : a ≤ x ≤ b, c ≤ y ≤ d}, then

Z b Z d
P ((X , Y ) ∈ A) = P(a ≤ X ≤ b, c ≤ Y ≤ d) = f (x, y )dxdy .
a c

PROBABILITY ITC 7 / 27
 Jointly Distributed Random Variables

Two Continuous Random Variables

Definition 6
The marginal probability density functions of X and Y , denoted by
fX (x) and fY (y ), respectively, are given by
Z ∞
fX (x) = f (x, y )dy for − ∞ < x < ∞
−∞
Z ∞
fY (y ) = f (x, y )dx for − ∞ < y < ∞
−∞

Example 7
Let X and Y have the joint pdf
4
f (x, y ) = (1 − xy ), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1.
3
Find the marginal pdfs of X and of Y .
PROBABILITY ITC 8 / 27
 Jointly Distributed Random Variables

Independent Random Variables

Definition 8
Two random variables X and Y are said to be independent if for every
pair of x and y values

p(x, y ) = pX (x).pY (y ) X , Y are discrete

or
f (x, y ) = fX (x).fY (y ) X , Y are continuous.
Otherwise, X and Y are said to be dependent.

PROBABILITY ITC 9 / 27
 Jointly Distributed Random Variables

More Than Two Random Variables

Definition 9
If X1 , X2 , . . . , Xn are all discrete rv’s, the joint pmf of the variables is the
function

p(x1 , x2 , . . . , xn ) = P(X1 = x1 , X2 = x2 , . . . , Xn = xn ).

If the variables are continuous, the joint pdf of X1 , . . . , Xn is the function

f (x1 , x2 , . . . , xn ) such that for any n intervals [a1 , b1 ], . . . , [an , bn ],
Z b1 Z bn
P(a1 ≤ x1 ≤ b1 ,. . . , an ≤ xn ≤ bn ) = ... f (x1 , . . . , xn )dxn . . . dx1 .
a1 an

PROBABILITY ITC 10 / 27
 Jointly Distributed Random Variables

More Than Two Random Variables

Example 10
Consider an experiment consisting of n independent and identical trials, in
which each trial can result in any one of r possible outcomes. Let
pi = P(outcome i on any particular trial), and define random variables by
Xi = the number of trials resulting in outcome i(i = 1, . . . , r ). Such an
experiment is called a multinomial experiment, and the joint pmf of
X1 , . . . , Xr is called the multinomial distribution. By using a counting
argument analogous to the one used in deriving the binomial distribution,
the joint pmf of X1 , . . . , Xr can be shown to be
n!

 p x1 . . . prxr , xi = 0, 1, 2, . . .
p(x1 , . . . , xr ) = (x1 !)(x2 !) . . . (xr )! 1
0, otherwise,


where x1 + · · · + xr = n.
PROBABILITY ITC 11 / 27
 Jointly Distributed Random Variables

Definition 11
The random variables X1 , X2 , . . . , Xn are said to be independent if for
every subset Xi1 , Xi2 , . . . , Xik of the variables (each pair, each triple, and so
on), the joint pmf or pdf of the subset is equal to the product of the
marginal pmf’s or pdf’s.

Example 12
An electronic device runs until one of its three components fails. The
lifetimes (in weeks), X1 , X2 , X3 , of these components are independent, and
each has the Weibull pdf
2x (− x ) 2
f (x) = e 5 , 0 < x < ∞.
25
The probability that the device stops running in the first three weeks is
Y3
equal to 1 − P(Xi > 3, i = 1, 2, 3) = 1 − P(Xi > 3) = 0.66.
i=1

PROBABILITY ITC 12 / 27
 Jointly Distributed Random Variables

Conditional Distributions

Definition 13
Let X and Y be two crv’s with joint pdf f (x, y ) and marginal pdf fX (x).
Then for any x for which fX (x) > 0, the conditional probability density
function of Y given that X = x is

f (x, y )
fY |X (y |x) = , −∞ < y < ∞.
fX (x)

If X and Y are discrete, replacing pdf’s by pmf’s in this definition gives

the conditional probability mass function of Y when X = x.

PROBABILITY ITC 13 / 27
 Jointly Distributed Random Variables

Example 14
Two components of a minicomputer have the following joint pdf for their
useful lifetimes X and Y :
(
xe −x(1+y ) , x ≥ 0 and y ≥ 0
f (x) =
0, otherwise.

1 What is the probability that the lifetime X of the first component

exceeds 3?
2 What are the marginal pdf’s of X and Y ? Are the two lifetimes
independent? Explain.
3 What is the probability that the lifetime of at least one component
exceeds 3?
4 What is the probability that the lifetime X exceeds 3, knowing that
the lifetime Y is 4?

PROBABILITY ITC 14 / 27
 Expected Values, Covariance, and Correlation

Contents

1 Jointly Distributed Random Variables

2 Expected Values, Covariance, and Correlation

3 The Distribution of the Sample Mean

4 The Distribution of a Linear Combination

PROBABILITY ITC 15 / 27
 Expected Values, Covariance, and Correlation

Expected Values, Covariance, and Correlation

Definition 15
Let X and Y be jointly distributed rv’s with pmf p(x, y ) or pdf f (x, y )
according to whether the variables are discrete or continuous. Then the
expected value of a function h(X , Y ), denoted by E [h(X , Y )] or µh(X ,Y ) ,
is given by
X X


 h(x, y )p(x, y ), X , Y are discrete
x y
E [h(X , Y )] = Z ∞ Z ∞
h(x, y )f (x, y )dxdy , X , Y are continuous



−∞ −∞

PROBABILITY ITC 16 / 27
 Expected Values, Covariance, and Correlation

Covariance

Note: µX = E (X )
Definition 16
The covariance between two rv’s X and Y is

Cov(X , Y ) = E [(X − µX )(Y − µY )]

X X


 (x − µX )(y − µY )p(x, y ), X , Y discrete
x y
= Z ∞Z ∞
(x − µX )(y − µY )p(x, y )dxdy , X , Y continuous



∞ −∞

Theorem 1
Cov(X , Y ) = E (XY ) − µX .µY .

PROBABILITY ITC 17 / 27
 Expected Values, Covariance, and Correlation

Correlation

Definition 17
The correlation coefficient of X and Y , denoted by Corr(X , Y ), ρX ,Y ,
or just ρ, is defined by
Cov(X , Y )
ρX ,Y = .
σX .σY

Theorem 2
1 If a and c are either both positive or both negative,

Corr(aX + b, cY + d) = Corr(X , Y ).
2 For any two rv’s X and Y , −1 ≤ Corr(X , Y ) ≤ 1.
3 If X and Y are independent, then ρ = 0.
4 ρ = 1 or −1 iff Y = aX + b for some numbers a and b with a 6= 0.

PROBABILITY ITC 18 / 27
 Expected Values, Covariance, and Correlation

Example 18
An instructor has given a short quiz consisting of two parts. For a
randomly selected student, let X = the number of points earned on the
first part and Y = the number of points earned on the second part.
Suppose that the joint pmf of X and Y is given the table
y
p(x, y ) 0 5 10 15
0 .02 .06 .02 .10
x 5 .04 .15 .20 .10
10 .01 .15 .14 .01

1 What is the total expected score E (X + Y )?

2 If the maximum of the two scores is recorded, what is the expected
recorded score?
3 Compute the covariance and correlation for X and Y .

PROBABILITY ITC 19 / 27
 The Distribution of the Sample Mean

Contents

1 Jointly Distributed Random Variables

2 Expected Values, Covariance, and Correlation

3 The Distribution of the Sample Mean

4 The Distribution of a Linear Combination

PROBABILITY ITC 20 / 27
 The Distribution of the Sample Mean

The Distribution of the Sample Mean

Definition 19
The rv’s X1 , X2 , . . . , Xn are said to form a (simple) random sample of
size n if
1 The Xi ’s are independent random variables.
2 Every Xi has the same probability distribution.

Theorem 3
Let X1 , X2 , . . . , Xn be a random sample from a distribution with mean
value µ and standard deviation σ and let X = (X1 + · · · + Xn )/n (sample
mean). Then
1 E (X ) = µX = µ
√
2 V (X ) = σX2 = σ 2 /n and σX = σ/ n
In addition, with T0 = X1 + · · · + Xn (the sample total),
√
E (T0 ) = nµ, V (T0 ) = nσ 2 , and σT0 = nσ
PROBABILITY ITC 21 / 27
 The Distribution of the Sample Mean

The Central Limit Theorem

Theorem 4
If X1 , X2 , . . . , Xn ∼ N(µ, σ 2 ), then

X ∼ N(µ, σ 2 /n).

Theorem 5 (The Central Limit Theorem (CLT))

Let X1 , X2 , . . . , Xn be a random sample from a distribution with mean µ
and variance σ 2 . Then if n is sufficiently large, X has approximately a
normal distribution with µX = µ and σX2 = σ 2 /n, and T0 also has
approximately a normal distribution with µT0 = nµ, σT2 0 = nσ 2 . The larger
the value of n, the better the approximation.

Rule of Thumb
If n > 30, the CLT can be applied.
PROBABILITY ITC 22 / 27
 The Distribution of the Sample Mean

The Central Limit Theorem

Example 20
The amount of a particular impurity in a batch of a certain chemical
product is a random variable with mean value 4.0g and standard deviation
1.5g . If 50 batches are independently prepared, what is the (approximate)
probability that the sample average amount of impurity X is between 3.5
and 3.8g ?

PROBABILITY ITC 23 / 27
 The Distribution of a Linear Combination

Contents

1 Jointly Distributed Random Variables

2 Expected Values, Covariance, and Correlation

3 The Distribution of the Sample Mean

4 The Distribution of a Linear Combination

PROBABILITY ITC 24 / 27
 The Distribution of a Linear Combination

The Distribution of a Linear Combination

Definition 21
Given a collection of n random variables X1 , . . . , Xn and n numerical
constants a1 , . . . , an , the rv
n
X
Y = a1 X1 + · · · + an Xn = ai Xi
i=1

is called the linear combination of the Xi ’s.

PROBABILITY ITC 25 / 27
 The Distribution of a Linear Combination

The Distribution of a Linear Combination

Theorem 6
Let X1 , X2 , . . . , Xn have mean values µ1 , . . . , µn , respectively, and
variances σ12 , . . . , σn2 , respectively.
1 Whether or not the Xi ’s are independent,
E (a1 X1 + · · · + an Xn ) = a1 E (X1 ) + · · · + an E (Xn ).
2 If X1 , . . . , Xn are independent,

V (a1 X1 + · · · + an Xn ) = a12 V (X1 ) + · · · + an2 V (Xn )

q
σa1 X1 +···+an Xn = a12 σ12 + · · · + an2 σn2 .

3 For any X1 , . . . , Xn ,
n X
X n
V (a1 X1 + · · · + an Xn ) = ai aj Cov(Xi , Xj ).
i=1 j=1
PROBABILITY ITC 26 / 27
 The Distribution of a Linear Combination

The Case of Normal Random Variables

Theorem 7
If X1 , X2 , . . . , Xn are independent, normally distributed rv’s (with possibly
different means and/or variances), then any linear combination of the Xi ’s
also has a normal distribution.

Example 22
A gas station sells three grades of gasoline: regular, extra, and super.
These are priced at $3.00, $3.20, and $3.40 per gallon, respectively. Let
X1 , X2 , and X3 denote the amounts of these grades purchased (gallons) on
a particular day. Suppose the Xi ’s are independent with
µ1 = 1000, µ2 = 500, µ3 = 300, σ1 = 100, σ2 = 80, and σ3 = 50. Suppose
that the revenue from sales is Y = 3.0X1 + 3.2X2 + 3.4X3 . Find
P(Y > 4500)?

PROBABILITY ITC 27 / 27