Likelihood Ratio Tests:

Let 𝑋 be a random variable having pdf 𝑓 𝑥; 𝜃1 , 𝜃2 , … , 𝜃𝑛 , 𝜃1 , 𝜃2 , … , 𝜃𝑛 ∈ Ω ⊆ 𝑅 𝑛 , Ω being

the parameter space. Let 𝑋1, 𝑋2, … , 𝑋𝑛 be a random sample of size 𝑛 from the distribution having pdf or

pmf 𝑓 𝑥; 𝜃1 , 𝜃2 , … , 𝜃𝑛 and let 𝜔 ⊂ 𝛺 we want to test the hypothesis 𝐻0 : 𝜃1 , 𝜃2 , … , 𝜃𝑛 𝜀 𝜔 against the

alternative 𝐻1 : 𝛳 𝜀 𝜔𝑐 , the function is

𝑛

𝐿 𝜔 = 𝑓 𝑥𝑖 ; 𝜃1 , 𝜃2 , … , 𝜃𝑛 𝑖𝑓 𝜃1 , 𝜃2 , … , 𝜃𝑛 ∈ ω
𝑖=1

And
𝑛

𝐿 Ω = 𝑓 𝑥𝑖 ; 𝜃1 , 𝜃2 , … , 𝜃𝑛 𝑖𝑓 𝜃1 , 𝜃2 , … , 𝜃𝑛 ∈ Ω
𝑖=1

Are as known as likelihood function.

For simplification, let 𝜃 𝑛 = 𝜃1 , 𝜃2 , … , 𝜃𝑛 , then

𝑛

𝐿 𝜔 = 𝑓 𝑥𝑖 ; 𝜃 𝑛 𝑖𝑓 𝜃 𝑛 ∈ ω
𝑖=1

And
𝑛

𝐿 Ω = 𝑓 𝑥𝑖 ; 𝜃 𝑛 𝑖𝑓 𝜃 𝑛 ∈ Ω
𝑖=1

Let 𝐿 𝜔 and 𝐿 Ω , respectively, be the maxima of 𝐿 𝜔 and 𝐿 Ω which are assume to exist the ratio

𝐿 𝜔
𝜆 𝑥 = 𝜆 𝑥1 , 𝑥2 , … , 𝑥𝑛 =
𝐿 Ω

Is known as likelihood Ratio.

Let 𝜆0 be a positive proper fraction i.e., 0 < 𝜆0 < 1. The likelihood ratio principle statistic that the null

hypothesis, 𝐻0 : 𝛳 𝑛 ∈ Ω is rejected ⇔ 𝜆 ≤ 𝜆0 . The function 𝜆 defines a random variable 𝜆(𝑋1 , 𝑋2 , … , 𝑋𝑛 )

And the level of significance denoted by 𝛼, is given by

𝛼 = 𝑃 𝜆(𝑋1 , 𝑋2 , … , 𝑋𝑛 ) ≤ 𝜆0 |𝐻0

Note: (i) LRT is used when uniformly most powerful test (UMPT) do not exist.
(ii) The principle leads to the same test when both 𝐻0 and 𝐻1 are simple, as provided by N-P lemma,

In this case Ω = 𝜃0 , 𝜃1 , Ω = 𝜃0 𝑜𝑟 𝜔 = 𝜃1

Example: Let 𝑋1, 𝑋2, … , 𝑋𝑛 be a random sample of size 𝑛 from𝑁 𝜃1 , 𝜃2 . Find LRT for testing

𝐻0 : 𝜃1 = 0, 𝜃2 > 0 against 𝐻1 : 𝜃1 ≠ 0, 𝜃2 > 0.

Solution: In this case, no UMP exist for testing 𝐻0 against 𝐻1 . In this case

Ω= 𝜃1 , 𝜃2 ; −∞ < 𝜃1 < ∞ , 𝜃0 > 0

ω= 𝜃1 , 𝜃2 ; 𝜃1 = 0 , 𝜃0 > 0

Let 𝑋1, 𝑋2, … , 𝑋𝑛 be a random sample of size 𝑛(> 1) from this distribution. Then
𝑛 𝑛
2 2
1 − 𝑥𝑖 − 𝜃1 −𝑛 − 𝑥𝑖 − 𝜃1
𝐿 Ω = exp = 2𝜋𝜃2 2 exp
2𝜋𝜃2 2𝜃2 2𝜃2
𝑖=1 𝑖=1

And
𝑛 𝑛
1 − 𝑥𝑖 2 −𝑛 − 𝑥𝑖 2
𝐿 ω = exp = 2𝜋𝜃2 2 exp
2𝜋𝜃2 2𝜃2 2𝜃2
𝑖=1 𝑖=1

𝑛
−𝑛 − 𝑥𝑖 2
𝐿 ω = 2𝜋𝜃2 2 exp
2𝜃2
𝑖=1

𝑛
𝜕𝑙𝑛𝐿 ω 𝑖=1 𝑥𝑖 2
𝐿 𝜔 is obtained by setting = 0, which gives 𝜃2 = and so
𝜕𝜃2 𝑛

−𝑛 −𝑛
𝑛 2 2 𝑛 2 𝑛 2 2
2𝜋 𝑖=1 𝑥𝑖 𝑛 𝑖=1 𝑥𝑖 2𝜋 𝑖=1 𝑥𝑖 𝑛
𝐿 𝜔 = exp − 𝑛 2
= exp −
𝑛 2 𝑖=1 𝑥𝑖 𝑛 2
𝑛
𝑛𝑒 −1 2
= (1)
2𝜋 𝑛𝑖=1 𝑥𝑖 2

𝜕𝑙𝑛𝐿 Ω 𝜕𝑙𝑛𝐿 Ω
Setting = 0 and = 0, we get
𝜕𝜃1 𝜕𝜃2

𝑛 𝑛 2
𝑖=1 𝑥1 𝑖=1 𝑥𝑖 − 𝑥
𝜃1 = = 𝑥 𝑎𝑛𝑑 𝜃2 = = 𝑠2
𝑛 𝑛
𝑛
𝑛𝑒 −1 2
𝐿 Ω = (2)
2𝜋 𝑛𝑖=1 𝑥𝑖 − 𝑥 2

So
 𝑛 𝑛
𝑛 2 2
𝑥
𝑖=1 𝑖 − 𝑥 𝑛𝑠 2 2
𝜆 𝑥1 , 𝑥2 , … , 𝑥𝑛 = 𝑛 2
=
𝑖=1 𝑥𝑖 𝑛𝑠 2 + 𝑛𝑥

𝑥2
If 𝑥 = 0, 𝜆 = 1 and in this case, the experiment tends to confirm 𝐻0 . But 𝑥 and 𝑠 2 decide considerably from

𝑥2
0, the experiment tends to neglate 𝐻0 . Now, greater the deviation of 𝑠 2 from 0, the smaller 𝜆 becomes i.e. if

𝜆 is used as a test critical region for testing 𝐻0 is a set defined by 0 ≤ 𝜆 ≤ 𝜆0 , where 0 < 𝜆0 < 1. Thus we

reject 𝐻0 if 𝜆 ≤ 𝜆0 . Now
−𝑛
𝑥2 2
𝜆 = 1+ 2 ≤ 𝜆0
𝑠
𝑛
𝑥2 2 1
⇒𝜆 = 1+ 2 ≥
𝑠 𝜆0

𝑥2 −2/𝑛
⇒ 𝜆 = 2 ≥ 𝜆0 −1
𝑠
−2
𝑛(𝑛 − 1)𝑥 2 𝑛
⇒𝜆= ≥ (𝑛 − 1) 𝜆 0 −1
𝑛𝑠 2

When 𝐻0 : 𝜃1 = 0 is true, 𝑡ℎ𝑒 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐,

𝑛 𝑋−0 𝑛𝑋
𝑇= = ~𝑡𝑛−1
𝑛𝑠 2 𝑛 2
𝑖=1 𝑋𝑖 − 𝑋
(𝑛 − 1)
(𝑛 − 1)

The level of significance is given by 𝛼 = 𝑃( 𝑇 ≥ 𝐶|𝐻0 )

Remark: For testing 𝐻0 : 𝜃0 , 𝜃2 > 0 against 𝐻1 : 𝜃1 ≠ 𝜃0 , 𝜃2 > 0, 𝑡ℎ𝑒 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐,

𝑛 𝑋 − 𝜃0 𝑛 𝑋 − 𝜃0
𝑇= = ~𝑡𝑛−1
𝑛𝑠 2 𝑛 2
𝑖=1𝑋𝑖 − 𝑋
(𝑛 − 1)
(𝑛 − 1)

Example: Let 𝑋1, 𝑋2, … , 𝑋𝑛 be a random sample from𝑁 𝜇, 𝜎 2 , where 𝜇 and 𝜎 2 are unknown. What is the

LRT of size 𝛼 for testing 𝐻0 : 𝜇 = 𝜇0 against 𝐻1 : 𝜇 ≠ 𝜇0 .

Solution: Here Ω = 𝜇, 𝜎 2 ; −∞ ≤ 𝜇 ≤ ∞ ,0 < 𝜎 2 < ∞ , ω = 𝜇, 𝜎02 ; 𝜇 = 𝜇0 , 0 < 𝜎 2 < ∞

 𝑛
𝑥 𝑖 −𝑥 2
If 𝜇, 𝜎 2 ∈ Ω, the observed maximum likelihood estimators are 𝜇 = 𝑥 and 𝜎 2 = 𝑖=1
. Thus
𝑛

𝑛
𝑛 2
1 −𝑛 𝑖=1 𝑥𝑖 − 𝑥 𝑛𝑒 −1 2
𝐿 Ω = 𝑛 𝑒𝑥𝑝 𝑛 =
𝑛 2 2 2 𝑖=1 𝑥𝑖 − 𝑥
2 2𝜋 𝑛𝑖=1 𝑥𝑖 − 𝑥 2
𝑖=1 𝑥𝑖 − 𝑥
2𝜋 𝑛
𝑛
𝑥 𝑖 −𝜇 0 2
If 𝜇, 𝜎 2 ∈ ω, the observed maximum likelihood estimators are 𝜇 = 𝜇0 and 𝜎 2 = 𝑖=1
. Thus
𝑛

𝑛
𝑛 2
1 −𝑛 𝑖=1 𝑥𝑖 − 𝜇0 𝑛𝑒 −1 2
𝐿 ω = 𝑛 𝑒𝑥𝑝 𝑛 =
𝑛 2 2 2 𝑖=1 𝑥𝑖 − 𝜇0
2 2𝜋 𝑛𝑖=1 𝑥𝑖 − 𝜇0 2
𝑖=1 𝑥𝑖 − 𝜇0
2𝜋
𝑛

The Likelihood Ratio Test is

𝑛
𝑛𝑒 −1 2 𝑛
𝑛 2 𝑛 2 2
𝐿 𝜔 2𝜋 𝑖=1 𝑥𝑖 − 𝜇0 𝑥
𝑖=1 𝑖 − 𝑥
𝜆= = 𝑛 = 𝑛 2
≤ 𝜆0
𝐿 Ω 𝑛𝑒 −1 2 𝑖=1 𝑥𝑖 − 𝜇0
2𝜋 𝑛𝑖=1 𝑥𝑖 − 𝑥 2

𝑛
𝑛 2 2
𝑥𝑖 − 𝑥
𝑖=1
⇒𝜆= 𝑛 2 2
≤ 𝜆0
𝑖=1 𝑥𝑖 − 𝑥 + 𝑛 𝑥𝑖 − 𝜇0
𝑛
2
1
⇒𝜆= ≤ 𝜆0
𝑛 𝑥 − 𝜇0 2
1+ 𝑛 𝑖 2
𝑖=1 𝑥𝑖 − 𝑥

Note that 𝜆 is close to 1 when 𝑥 is close to 𝜇0 and it is small when 𝑥 and 𝜇0 differ by a great deal.
2
1 𝑛
𝜆= ≤ 𝜆0
𝑛 𝑥 − 𝜇0 2
1+ 𝑛 𝑖 2
𝑖=1 𝑥𝑖 − 𝑥

−2
𝑛 𝑥𝑖 − 𝜇0 2
⇒𝜆 =1+ 𝑛 2
≥ 𝜆0𝑛
𝑖=1 𝑥𝑖 − 𝑥

−2
𝑛 𝑥𝑖 − 𝜇0 2 𝑛
⇒𝜆= 𝑛 2
≥ 𝜆 0 −1
𝑥
𝑖=1 𝑖 − 𝑥

−2
𝑛 𝑥𝑖 − 𝜇0 2 𝑛
⇒𝜆= 𝑛 2 ≥ (𝑛 − 1) 𝜆0 − 1
𝑖=1 𝑥𝑖 − 𝑥
(𝑛 − 1)
 𝑛
𝑛 𝑋 −𝜇 0 𝑖=1 𝑋 𝑖 −𝑋 2 2
When 𝐻0 is true ~𝑁(0,1)and ~𝜒𝑛−1 . Hence under 𝐻0
𝜎 𝜎2

𝑛 𝑋 − 𝜇0
𝜎 𝑛 𝑋 − 𝜇0 𝑋 − 𝜇0
𝑇= = = ~𝑡𝑛−1
𝑛 𝑛
𝑆
2 2
𝑖=1 𝑋𝑖 − 𝑋 𝑖=1 𝑋𝑖 − 𝑋 𝑛
2
𝜎 (𝑛 − 1) (𝑛 − 1)

According to LRT criterion, 𝐻0 is rejected if

−2
𝑇 2 ≥ (𝑛 − 1) 𝜆0𝑛 − 1

That is, we reject 𝐻0 : 𝜇 = 𝜇0 and accept 𝐻1 : 𝜇 ≠ 𝜇0 if 𝑇 ≥ 𝑡𝛼 (𝑛 − 1) and the critical region is given by
2

𝛼= 𝑥1 , 𝑥2 , … , 𝑥𝑛 : 𝑡 ≥ 𝑡𝛼 (𝑛 − 1)
2

(⋕⋕⋕)

Remark: (i) If the alternative hypothesis is 𝐻1 : 𝜇 > 𝜇0 , then the critical region would be

𝛼= 𝑥1 , 𝑥2 , … , 𝑥𝑛 : 𝑡 ≥ 𝑡𝛼 (𝑛 − 1)

(ii) If the alternative hypothesis is 𝐻1 : 𝜇 < 𝜇0 , then the critical region would be

𝛼= 𝑥1 , 𝑥2 , … , 𝑥𝑛 : 𝑡 ≤ −𝑡𝛼 (𝑛 − 1)

Example: Let 𝑋1, 𝑋2, … , 𝑋𝑛 be a random sample from𝑁 𝜇, 𝜎 2 population we wish to test 𝐻0 : 𝜎 2 = 𝜎02

against 𝐻1 : 𝜎 2 ≠ 𝜎02 (Two tailed LRT for variance of a normal population)

Solution: Here Ω = 𝜇, 𝜎 2 ; −∞ ≤ 𝜇 ≤ ∞ , 𝜎 2 > 0 , ω = 𝜇, 𝜎02 ; −∞ ≤ 𝜇 ≤ ∞ , 𝜎02 > 0

Since
2
2
1 𝑥−𝜇
𝑓 𝑥; 𝜇, 𝜎 = 𝑒𝑥𝑝 −
2𝜋𝜎 2 2𝜎 2

𝑛 2
1 𝑖=1 𝑥−𝜇 1 𝑛𝑆 2
𝐿 𝑋, 𝜇 = 𝑛 𝑒𝑥𝑝 − = 𝑛 𝑒𝑥𝑝 −
2𝜋𝜎 2 2 2𝜎 2 2𝜋𝜎 2 2 2𝜎 2

1 𝑛𝑆 2
𝐿 𝜔 = 𝑛 𝑒𝑥𝑝 − 2
2𝜋𝜎02 2 2𝜎0

1 𝑛𝑆 2 1 𝑛
𝐿 Ω = 𝑛 𝑒𝑥𝑝 − 2 = 𝑛 𝑒𝑥𝑝 −
2𝜋𝑆 2 2 2𝑆 2𝜋𝑆 2 2 2
So

1 𝑛𝑆 2
𝑛 𝑒𝑥𝑝 − 𝑛
𝐿 𝜔2𝜋𝜎02 2 2𝜎02 𝑆2 2 𝑛 𝑆2
𝜆= = = 𝑛 𝑒𝑥𝑝 − 1+ 2
𝐿 Ω 1 𝑛 2 𝜎0
𝑛 𝑒𝑥𝑝 − 2 𝜎02 2
2𝜋𝑆 2 2
𝑛
𝑥 𝑖 −𝑥 2 𝑛
Now, 𝑆 2 = 𝑖=1
⇒ 𝑛𝑆 2 = 𝑖=1 𝑥𝑖 − 𝑥 2
𝑛

𝑛
𝑥 𝑖 −𝑥 2
Thus 𝑖=1
~𝜒𝑛2−1
𝜎02

𝑛𝑆 2 2
⇒ ~𝜒𝑛−1
𝜎02

So
𝑛
2
𝜒𝑛−1 2 𝑛 𝜒𝑛2−1
𝜆= 𝑒𝑥𝑝 − 1+
𝑛 2 𝑛

Thus the region 0 < 𝜆 < 𝜆0 is equivalent to,

𝑛
𝜒𝑛2−1 2 𝑛 2
𝜒𝑛−1
𝑒𝑥𝑝 − 1 + < 𝜆0
𝑛 2 𝑛

2
2
𝑛 𝜒𝑛−1 𝑛 𝑛
⇒ 𝜒𝑛−1 2 𝑒𝑥𝑝 − < 𝜆0 𝑛 2 exp =𝐶
2 2

Since the alternative hypothesis is two sides so we have

𝛼 𝛼
𝑃 𝜒 2 > 𝜒12 = , 𝑃 𝜒 2 < 𝜒22 =
2 2
2 2
⇒ 𝜒12 = 𝜒𝑛−1, 2
𝛼 𝑎𝑛𝑑 𝜒2 = 𝜒
𝑛−1,1−
𝛼
2 2

Example: Find an 𝛼-level LRT of 𝐻0 : 𝑝 ≤ 𝑝0 against 𝐻1 : 𝑝 > 𝑝0 based on a random sample of size 1 from

𝐵(𝑛, 𝑝).

Solution: Consider the LRT

𝑛 𝑥
𝐿 𝜔 𝑝 1 − 𝑝 𝑛−𝑥 ; 𝑝 ≤ 𝑝0
𝜆 𝑥 = = 𝑥
𝐿 Ω 𝑛 𝑥
𝑝 1 − 𝑝 𝑛−𝑥 ; 0 ≤ 𝑝 ≤ 1
𝑥
𝑛 𝑥 𝑛−𝑥 𝑥
𝐿 𝑋; 𝑝 = 𝑝 1−𝑝 with of mle 𝑝 = 𝑝 = 𝑛
𝑥
 𝑛 𝑥 𝑛−𝑥
𝑥
𝐿 𝜔 = 𝑝 1 − 𝑝0 𝑖𝑓 𝑝0 <
𝑥 0 𝑛
𝑥 𝑥 𝑥 𝑛−𝑥 𝑥
𝑛
= 1− 𝑖𝑓 𝑝0 ≥
𝑥 𝑛 𝑛 𝑛
𝑥 𝑥 𝑥 𝑛−𝑥
𝑛
𝐿 Ω = 1−
𝑥 𝑛 𝑛

So

𝑝0𝑥 1 − 𝑝0 𝑛−𝑥 𝑥
𝜆 𝑥 = 𝑖𝑓 𝑝0 ≤
𝑥 𝑥 𝑥 𝑛−𝑥 𝑛
1 −
𝑛 𝑛
𝑥
=1 𝑖𝑓 𝑝0 ≥
𝑛

So

𝑥 𝑥
log 𝜆 𝑥 = 𝑥 log 𝑝0 + 𝑛 − 𝑥 log(1 − 𝑝0 ) − 𝑥 log − 𝑛 − 𝑥 log 1 −
𝑛 𝑛

𝜕 log 𝜆 𝑥 𝑥 𝑛 𝑥
= log 𝑝0 − log(1 − 𝑝0 ) − log +𝑥 − 1 − log 1 −
𝜕𝑥 𝑛 𝑥 𝑛

1 𝜕 log 𝜆 𝑥 𝑛𝑝0
⇒ = log 𝑥 − (𝑛 + 1)
𝜆 𝑥 𝜕𝑥 (1 − 𝑝0 )𝑥 1 − 𝑛

′
𝑛𝑝0 log 𝑒 − 𝑛+1
⇒𝜆 𝑥 =𝜆 𝑥 log 𝑥
(1 − 𝑝0 )𝑥 1 − 𝑛

Thus

𝜆′ 𝑥 ≤ 0 ⇔ 𝑥 > 𝑛𝑝0

and 𝜆′ 𝑥 > 0 ⇔ 𝑥 < 𝑛𝑝0

So 𝜆 𝑥 is decreasing function of 𝑥.

Thus 𝜆 𝑥 < 𝑎 ⇒ 𝑥 > 𝜆−1 𝑎 = 𝐶

The constant 𝐶 can be determined by size 𝛼 such that

𝛼 = 𝑃 𝑋 > 𝐶|𝐻0 = 𝑃 𝑋 > 𝐶|𝜃0

Thus the LRT rejects to 𝐻0 if 𝑥 > 𝐶

If such a 𝐶 does not exist we chose 𝑏 such that

𝑃 𝑋 > 𝑏|𝜃 ≤ 𝛼 and 𝑃 𝑋 > 𝑏 − 1 |𝜃 > 𝛼

(⋕⋕⋕)