UW STAT 231 Fall 2019

试听课福利 1: Ultimate Formula Sheet

讲师: Bill Zhuo EZ 4.0

Disclaimer
This complementary study package is provided by Easy 4.0 Education Inc. and its affiliated
mentors. This study package seeks to support your study process and should be used as a
complement, NOT substitute to course material, lecture notes, problem sets, past tests and
other available resources.

We acknowledge that this package contains some materials provided by professors and
staff of the University of Waterloo, and the sources of these materials are cited in details
wherever they appear.

This package is distributed for free to students participating in Easy 4.0’s review seminars,
and are not for sale or other commercial uses whatsoever. We kindly ask you to refrain from
copying or selling in part or in whole any information provided in this package.

Thank you for choosing Easy 4.0. We sincerely wish you the best of luck in all of your
exams.

1-1
 Chapter 2: Statistical Models and Maximum Likelihood Estimation

Likelihood function for 𝜃 is

𝐿(𝜃) = 𝐿(𝜃; 𝑦) = 𝑃 (𝑌 = 𝑦; 𝜃) for 𝜃 ∈ Ω

Relative likelihood function is

𝐿(𝜃)
𝑅(𝜃) = , 𝜃∈Ω
𝐿(𝜃)̂

0 ≤ 𝑅(𝜃) ≤ 1 for all 𝜃 ∈ Ω and 𝑅(𝜃)̂ = 1.

Log-relative likelihood function is

𝐿(𝜃)
𝑟(𝜃) = log 𝑅(𝜃) = log
𝐿(𝜃)̂
= ℓ(𝜃) − ℓ(𝜃)̂ for 𝜃 ∈ Ω

Likelihood function for independent experiments:

Suppose we have two independent data set 𝑦1 and 𝑦2 , corresponding to two
independent random variables 𝑌1 and 𝑌2 . The likelihood function for 𝜃 based on
the data 𝑦1 and 𝑦2 is

𝐿(𝜃) = 𝐿1 (𝜃) × 𝐿2 (𝜃), 𝜃∈Ω

Likelihood function for multinomial models

𝑘
𝑛! 𝑦
𝐿(𝜃) = ∏ 𝜃𝑖 𝑖
𝑦1 !𝑦2 ! ⋯ 𝑦𝑘 ! 𝑖=1
The corresponding log-likelihood function is
𝑘
ℓ(𝜃) = ∑ 𝑦𝑖 log 𝜃𝑖
𝑖=1

Invariance property of MLE: If 𝜃 ̂ is the MLE of 𝜃, the 𝑔(𝜃)̂ is the MLE of 𝑔(𝜃) for
given function 𝑔(⋅).

2-2
 Chapter 4: Estimation

Suppose our assumed model 𝑌𝑖 ∼ 𝐺(𝜇, 𝜎), 𝑖 = 1, 2, … , 𝑛 is reasonable and the

observed data are 𝑦1 , 𝑦1 , … , 𝑦𝑛 . 𝜇̂ = 𝑦 ̄ is the maximum likelihood estimate of 𝜇.

1 𝑛 1 𝑛
𝜇̃ = 𝑌 ̄ = ∑ 𝑌𝑖 , 𝜎̃ 2 = ∑(𝑌𝑖 − 𝑌 ̄ )2
𝑛 𝑖=1 𝑛 𝑖=1

𝑛
1
𝑠=√ ∑(𝑦𝑖 − 𝑦)̄ 2
𝑛 − 1 𝑖=1

For Poisson data with unknown mean 𝜃 the point estimate of 𝜃 is

1 𝑛
𝜃 ̂ = 𝑦̄ = ∑𝑦
𝑛 𝑖=1 𝑖

Suppose 𝜃 is a scalar and that 𝐿(𝜃) = 𝐿(𝜃; 𝑦) is the likelihood function for 𝜃 based
on the observed data 𝑦. The relative likelihood function 𝑅(𝜃) is defined as

𝐿(𝜃)
𝑅(𝜃) = 𝑅(𝜃; 𝑦) = for 𝜃 ∈ Ω
𝐿(𝜃)̂

A 100𝑝% likelihood interval for 𝜃 is the set {𝜃 ∶ 𝑅(𝜃) ≥ 𝑝}.

• Values of 𝜃 inside a 10% likelihood interval are plausible values of 𝜃 in light of

the observed data.

• Values of 𝜃 inside a 50% likelihood interval are very plausible values of 𝜃 in

light of the observed data.

• Values of 𝜃 outside a 10% likelihood interval are implausible values of 𝜃 in

light of the observed data.

• Values of 𝜃 outside a 1% likelihood interval are very implausible values of 𝜃

in light of the observed data.

The log-relative likelihood function is given by

𝑟(𝜃) = log 𝑅(𝜃) = ℓ(𝜃) − ℓ(𝜃)̂ for 𝜃 ∈ Ω

2-3
For Binomial Distribution Maximizing 𝐿(𝜃), we find
𝑦
𝜃̂ =
𝑛
and
𝐿(𝜃) = 𝜃𝑦 (1 − 𝜃)𝑛−𝑦 for 0 < 𝜃 < 1.

The coverage probability for the interval estimator is

𝐶(𝜃) = 𝑃 (𝜃 ∈ [𝐿(𝑌 ), 𝑈 (𝑌 )]) = 𝑃 [𝐿(𝑌 ) ≤ 𝜃 ≤ 𝑈 (𝑌 )]

The 100𝑝% confidence interval for a parameter is an interval estimate [𝐿(𝑦), 𝑈 (𝑦)]
for which

𝑃 (𝜃 ∈ [𝐿(𝑌 ), 𝑈 (𝑌 )]) = 𝑃 [𝐿(𝑌 ) ≤ 𝜃 ≤ 𝑈 (𝑌 )] = 𝑝

Suppose 𝑌 = (𝑌1 , … , 𝑌𝑛 ) is a random sample from the 𝐺(𝜇, 𝜎0 ) distribution. The

pivotal quantity is

𝑌̄ − 𝜇
𝑄 = 𝑄(𝑌 ; 𝜇) = √ ∼ 𝐺(0, 1)
𝜎0 / 𝑛

The 100𝑝% confidence interval (Two sided confidence interval) for 𝜇 is of the form:

point estimate ± C × sd of the estimator.

If 𝑌 ∼ Binomial(𝑛, 𝜃), then by the Central Limit Theorem

𝑌 − 𝑛𝜃
√𝑛𝜃(1 − 𝜃)

has approximately a 𝐺(0, 1) distribution for large n.

The approximate 100𝑝% confidence interval for Binomial Distribution 𝜃 given by

̂ − 𝜃)̂
𝜃(1
𝜃 ̂ ± 𝐶√
𝑛

2-4
An approximate 100𝑝% confidence interval for a random sample from the
Poisson(𝜃):

𝜃̂
𝜃 ̂ ± 𝐶√
𝑛

An approximate 100𝑝% confidence interval for a random sample from the

Exponential(𝜃):

𝜃̂
𝜃̂± 𝐶 √
𝑛

A 100𝑝% likelihood interval is an approximate 100𝑞% where

𝑞 = 2𝑃 (𝑍 ≤ √−2 log 𝑝) − 1

and 𝑍 ∼ 𝑁 (0, 1).

If 𝑎 is a value such that

𝑝 = 2𝑃 (𝑍 ≤ 𝑎) − 1 where 𝑍 ∼ 𝑁 (0, 1)

then the likelihood interval

2
{𝜃 ∶ 𝑅(𝜃) ≥ 𝑒−𝑎 /2
}

is an approximate 100𝑝% confidence interval.

The Gamma function is defined as

∞
Γ(𝑎) = ∫ 𝑦𝑎−1 𝑒−𝑦 d𝑦, 𝑎>0
0

Γ(𝑎) = (𝑎 − 1)Γ(𝑎 − 1)
Γ(𝑎) = (𝑎 − 1)! for 𝑎 = 1, 2, …
√
Γ(1/2) = 𝜋

2-5
For 𝑘 > 2 the Chi-squared probability density funciton is
1
𝑓(𝑥; 𝑘) = 𝑥(𝑘/2)−1 𝑒−𝑥/2
2𝑘/2 Γ(𝑘/2)
for 𝑥 > 0 and 𝑘 = 1, 2, …

If 𝑋 ∼ 𝜒2 (𝑘)
𝐸(𝑋) = 𝑘, 𝑉 𝑎𝑟(𝑋) = 2𝑘.

Probabilities for the random variable 𝑊 ∼ 𝜒2 (1) :

√ √
𝑃 (𝑊 ≤ 𝜔) = 𝑃 (|𝑍| ≤ 𝜔) = 2𝑃 (𝑍 ≤ 𝜔) − 1

and
√ √
𝑃 (𝑊 > 𝜔) = 𝑃 (|𝑍| > 𝜔) = 2(1 − 𝑃 (𝑍 ≤ 𝜔)),

where 𝑍 ∼ 𝑁 (0, 1).

Probabilities for the random variable 𝑋 ∼ 𝜒2 (2) :

𝑃 (𝑋 ≤ 𝑥) = 1 − 𝑒−𝑥/2 and 𝑃 (𝑋 > 𝑥) = 1 − 𝑃 (𝑋 ≤ 𝑥) = 𝑒−𝑥/2

The likelihood ratio statistic

𝐿(𝜃)
Λ = Λ(𝜃; 𝜃)̃ = −2 log [ ] = 2ℓ(𝜃)̃ − 2ℓ(𝜃)
𝐿(𝜃)̃

A 100% confidence interval for 𝜇 of a Gaussian distribution with known 𝜎 has the
form:
point estimate ± c (table value) × sd(estimator).

Such an interval is often called a “two-sided, equal-tailed” confidence interval.

We find the value 𝑐 in the Normal table such that 𝑃 (−𝑐 ≤ 𝑍 ≤ 𝑐) = 𝑝 or
1+𝑝
𝑃 (𝑍 ≤ 𝑐) = 2 , with 𝑍 ∼ 𝐺(0, 1).
For a 90% confidence interval, c=1.645.
For a 95% confidence interval, c=1.960.
For a 99% confidence interval, c=2.576.

𝑌 ̄ −𝜇
𝑆/ 𝑛
√ ∼ 𝑡(𝑛 − 1).

2-6
Suppose 𝑌1 , 𝑌2 , … , 𝑌𝑛 is a random sample from a 𝐺(𝜇, 𝜎) distribution.
√
• A 100𝑝% confidence interval for 𝜇 if 𝜎 is unknown has the form 𝑦 ̄ ± 𝑐 ⋅ 𝑠/ 𝑛,
with 𝑐 such that 𝑃 (𝑇 ≤ 𝑐) = (1 + 𝑝)/2 and 𝑇 ∼ 𝑡(𝑛 − 1).

• A 100𝑝% confidence interval for 𝜎 is

2
(𝑛 − 1)𝑠 √ (𝑛 − 1)𝑠 2
[√ , ],
𝑏 𝑎

with 𝑎 and 𝑏 such that

1−𝑝 1+𝑝
𝑃 (𝑈 ≤ 𝑎) = , 𝑃 (𝑈 ≤ 𝑏) = and 𝑈 = (𝑛 − 1)𝑆 2 /𝜎2 ∼ 𝜒2 (𝑛 − 1).
2 2

• For a new value 𝑌 ∼ 𝐺(𝜇, 𝜎), a 100𝑝% prediction interval is given by

1 1
[𝑦 ̄ − 𝑐 ⋅ 𝑠√1 + , 𝑦 ̄ + 𝑐 ⋅ 𝑠 √1 + ] ,
𝑛 𝑛

with 𝑐 such that

1+𝑝
𝑃 (−𝑐 ≤ 𝑇 ≤ 𝑐) = 𝑝 or 𝑃 (𝑇 ≤ 𝑐) = where 𝑇 ∼ 𝑡(𝑛 − 1).
2

2-7
 p.f./p.d.f. Mean Variance m.g.f

Discrete p.f.

Binomial(𝑛, 𝑝) (𝑛
𝑦
)𝑝𝑦 𝑞𝑛−𝑦
𝑛𝑝 𝑛𝑝𝑞 (𝑝𝑒𝑡 + 𝑞)𝑛
0 < 𝑝 < 1, 𝑞 = 1 − 𝑝 𝑦 = 0, 1, 2, … , 𝑛

Bernoulli(𝑝) 𝑝𝑦 (1 − 𝑝)1−𝑦
𝑝 𝑝(1 − 𝑝) (𝑝𝑒𝑡 + 𝑞)
0 < 𝑝 < 1, 𝑞 = 1 − 𝑝 𝑦 = 0, 1

𝑘
Negative Binomial(𝑘, 𝑝) (𝑦+𝑘−1
𝑦
)𝑝𝑘 𝑞𝑦 𝑘𝑞 𝑘𝑞
𝑝
( 1−𝑞𝑒 𝑡)
𝑝 𝑝2
0 < 𝑝 < 1, 𝑞 = 1 − 𝑝 𝑦 = 0, 1, 2, … 𝑡 < − ln 𝑞

𝑝
Geometric(𝑝) 𝑝𝑞𝑦 𝑞 𝑞 1−𝑞𝑒𝑡
𝑝 𝑝2
0 < 𝑝 < 1, 𝑞 = 1 − 𝑝 𝑦 = 0, 1, 2, … 𝑡 < − ln 𝑞

(𝑦𝑟 )(𝑁−𝑟
𝑛−𝑦 )
Hypergeometric(𝑁, 𝑟, 𝑛)
(𝑁𝑛)
𝑛𝑟 𝑟
𝑛𝑁 (1 − 𝑟 𝑁−𝑛
) intractable
𝑁 𝑁 𝑁−1
𝑟 < 𝑁, 𝑛 < 𝑁 𝑦 = 0, 1, 2, … , min(𝑟, 𝑛)

𝑒−𝜃 𝜃𝑦
Poisson(𝜃) 𝑦! 𝑡 −1)
𝜃 𝜃 𝑒𝜃(𝑒
𝜃>0 𝑦 = 0, 1, …

𝑛! 𝑦 𝑦 𝑦
Multinomial(𝑛, 𝜃1 , … , 𝜃𝑘 ) 𝜃 1 𝜃2 2
𝑦1 !𝑦2 !…𝑦𝑘 ! 1
… 𝜃𝑘𝑘
𝑘 𝑘 (𝑛𝜃1 , … , 𝑛𝜃𝑘 ) Var(𝑌𝑖 ) = 𝑛𝜃𝑖 (1 − 𝜃𝑖 )
𝜃𝑖 > 0, ∑𝑖=1 𝜃𝑖 = 1 𝑦𝑖 = 0, 1, … ; ∑𝑖=1 𝑦𝑖 = 𝑛

Continuous p.d.f.

𝑒𝑏𝑡 −𝑒𝑎𝑡
1 𝑎+𝑏 (𝑏−𝑎)2 (𝑏−𝑎)𝑡
Uniform(𝑎, 𝑏) 𝑓(𝑦) = 𝑏−𝑎
, 𝑎<𝑦<𝑏 2 12
𝑡≠0

1
Exponential(𝜃)
𝑓(𝑦) = 1𝜃 𝑒−𝑦/𝜃 , 𝑦 > 0 𝜃 𝜃2 1−𝜃𝑡
𝜃>0 𝑡 < 1/𝜃
2 2
N(𝜇, 𝜎2 ) or G(𝜇, 𝜎2 ) 𝑓(𝑦) = √ 1 𝑒−(𝑦−𝜇) /(2𝜎 ) 2 𝑡2 /2
2𝜋𝜎 𝜇 𝜎2 𝑒𝑢𝑡+𝜎
−∞ < 𝜇 < ∞, 𝜎 > 0 −∞ < 𝑦 < ∞
1
𝑓(𝑦) = 2𝑘/2 Γ(𝑘/2)
𝑦(𝑘/2)−1 𝑒−𝑦/2 ,
Chi-squared(𝑘) (1 − 2𝑡)−𝑘/2
𝑦>0 𝑘 2𝑘
𝑘>0 ∞ 𝑡 < 1/2
where Γ(𝑎) = ∫0 𝑥𝑎−1 𝑒−𝑥 d𝑥
𝑦2 −(𝑘+1)/2
𝑓(𝑦) = 𝑐𝑘 (1 + 𝑘
) 𝑘
Student 𝑡 0 𝑘−2
−∞ < 𝑦 < ∞ where undefined
𝑘>0 √ if 𝑘 > 1 if 𝑘 > 2
𝑐𝑘 = Γ( 𝑘+1
2
)/ 𝑘𝜋Γ( 𝑘2 )

2-8
2-9
2-10
 UW EASY4.0 广告招商处
妖兽啦！老板又逼着比尔卓打广告啦！欢迎各位莅临我们周日 5
月 19 日晚 19：00-21:00 的试听课！

2-11