P8120 Analysis of Categorical Data

Lecture 5: Inference for a Single Proportion (Part 2)

February 6th, 2025
Learning Objectives
 Test hypotheses about a single binomial proportion using approximate and exact methods

Practice Problems: Posted.

Lecture 4 Review

 The likelihood function expresses the probability of the observed (response) data expressed as a function of
the parameter.
o 𝐿𝐿(𝜋𝜋) = 𝑃𝑃 (𝑋𝑋 = 7 |𝜋𝜋) = �50
7
�𝜋𝜋 7 (1 − 𝜋𝜋) 50−7

 A range of possible values for a population parameter is given by a confidence interval.

Two methods to calculate confidence intervals
o Wald Confidence Interval (Approximate) used when 𝑛𝑛𝑝𝑝̂ ≥ 5 and 𝑛𝑛(1 − 𝑝𝑝̂ ) ≥ 5
o Exact Confidence Interval used when 𝑛𝑛𝑝𝑝̂ < 5 and 𝑛𝑛(1 − 𝑝𝑝̂ ) < 5
____________________________________________________________________________________

Hypothesis Testing
A hypothesis test is a method for using sample data to decide between two competing claims (hypotheses) about a
population parameter. If it were possible to carry out a census of the entire population then we would know
which of the two hypotheses are correct, but usually it is the case that we need to decide between the
two hypotheses using information from a sample.

The null hypothesis, H0, is usually chosen to represent “no change” or “no association” whereas the alternative
hypothesis, H1, usually specifies a change, difference, or association.

One Sided vs. Two-Sided Tests

Let 𝜋𝜋0 = null/hypothesized value for 𝜋𝜋. Recall:

𝜋𝜋 = true proportion (population)
One-sided 𝑝𝑝̂ = best guess of 𝜋𝜋 aka point estimate
𝐻𝐻0 : 𝜋𝜋 ≤ 𝜋𝜋0 𝜋𝜋0 = hypothesized value (known/given)
𝐻𝐻1 : 𝜋𝜋 > 𝜋𝜋0

𝐻𝐻0 : 𝜋𝜋 ≥ 𝜋𝜋0
𝐻𝐻1 : 𝜋𝜋 < 𝜋𝜋0

Two-sided
𝐻𝐻0 : 𝜋𝜋 = 𝜋𝜋0
𝐻𝐻1 : 𝜋𝜋 ≠ 𝜋𝜋0

P8120 Spring 2025 1

The null hypothesis H0 is rejected in favor of H1 only if sample evidence strongly suggests that H0 is false. If the sample
does not provide such evidence, then H0 is not rejected.

Therefore, the two possible outcomes of a hypothesis test are reject H0 or fail to reject H0

ASIDE: Do not accept the null hypothesis H0

• In a court case, defendants are guilty or not guilty; there is no verdict of “innocent”.
• In a statistical test, the null hypothesis is rejected or not rejected.
• If p>0.05, avoid “the drug was ineffective” or “there was no difference between groups”.
Instead, “we did not see evidence of a drug effect, or “there was no significant difference between groups.”

Template for Conducting a Hypothesis Test

(1) Explicitly define the population parameter of interest.

(2) State the null and alternative hypotheses.
(3) State the significance level of the test.
(4) State any necessary assumptions
(5) State the form of the test statistic and its null distribution.
(6) State the decision rule and compute the p-value.
(7) State your conclusions in the context of the problem.

P8120 Spring 2025 2

Recall the definition of p-value:

A p-value is the probability of observing a test statistic (data) as extreme as or more extreme
than that test statistic (data) that we observed, given that the null hypothesis is true.

Also recall that there are different errors that can be made when conducting a hypothesis test. Consider the table
below:

α =Type I error rate = P(reject H0 | H0 true)

β = Type II error rate = P(fail to reject H0 | H1 true)

1 −β = Power of the test = P(reject H0| H1 true)

There are TWO ways to perform a hypothesis test for a single proportion. We will review both in this lecture.

1) Score Test (Approximate)

o uses the normal approximation to the binomial distribution;
o Requirement: 𝑛𝑛𝜋𝜋0 ≥ 5 and 𝑛𝑛(1 − 𝜋𝜋0 ) ≥ 5

2) Exact Hypothesis Test

o uses the binomial distribution when a normal approximation is inappropriate;
o typically used when 𝑛𝑛𝜋𝜋0 < 5 and 𝑛𝑛(1 − 𝜋𝜋0 ) < 5

P8120 Spring 2025 3

Approach #1: Score Test for a Single Proportion

We want to test whether the true proportion of interest is different (larger or smaller) than some particular value
𝜋𝜋0 using our data (n = sample size, 𝑝𝑝̂ = MLE of 𝜋𝜋)

Two-sided
𝐻𝐻0 : 𝜋𝜋 = 𝜋𝜋0
𝐻𝐻1 : 𝜋𝜋 ≠ 𝜋𝜋0

If 𝑛𝑛𝜋𝜋0 ≥ 5 and 𝑛𝑛(1 − 𝜋𝜋0 ) ≥ 5,

then by the Central Limit Theorem, we know that:

𝜋𝜋 (1 − 𝜋𝜋)
𝑝𝑝̂ ~𝑁𝑁 �𝜋𝜋, �
𝑛𝑛

But under the null hypothesis 𝐻𝐻0 : 𝜋𝜋 = 𝜋𝜋0 , we would have:

𝜋𝜋0 (1 − 𝜋𝜋0 )
𝑝𝑝̂ ~𝑁𝑁 �𝜋𝜋0 , �
𝑛𝑛

P8120 Spring 2025 4

Example. Again, consider the acupuncture accrual example. Originally, investigators were interested in
determining if the true proportion of those who would refuse acupuncture therapy was 20% (H0, hypothesized
value). They observed that 7 of the 50 participants refused acupuncture therapy the first month and assumed that
the number of participants who refuse acupuncture treatment (X) follows a Bin(50, π) distribution. Is there
evidence that the true proportion of those who refused acupuncture treatment is different from 20%?

Let’s examine this question using the 7 steps:

(1) 𝜋𝜋 = true proportion of those who would refuse acupuncture treatment.

(2) H0:

H1:

(3) Set α = 0.05

(4) Assumptions

(5) Test Statistics and Null Distribution

𝑝𝑝̂ − 𝜋𝜋0
𝑧𝑧 = =
�𝜋𝜋0 (1 − 𝜋𝜋0 )
𝑛𝑛

(6) Decision

Approach 1 (Critical value approach)

Approach 2 (p-value approach)

(7) Conclusion
At the 5% level of significance, we have _________________ evidence to conclude that the true
proportion of those who refuse acupuncture treatment is different from 20%.

Model Answer for manuscript:

Fourteen percent of the sample refused acupuncture therapy (95% CI: 4.4% to 23.6%). There is
insufficient evidence to conclude that the true proportion of those who refuse acupuncture therapy is
different from 20% (p-value = 0.2888).
P8120 Spring 2025 5
Below is the SAS code and output for addressing the question above.

data acupuncture;
input refused;
cards;
7
43
;
run;

title 'Acupuncture example: Score test';

proc freq data = acupuncture order = data;
table refused / binomial(p=0.20) alpha = 0.05;
weight refused;
run;

P8120 Spring 2025 6

Example. Again, consider the acupuncture accrual example. What if we now wanted to test if the true proportion
of those who refused acupuncture therapy is less than 20%?

Notation Recap:
𝝅𝝅 The population proportion (parameter); often referred to as the “truth”.
�
𝒑𝒑 The sample proportion (statistic); computed from your sample data
𝝅𝝅𝟎𝟎 The null value for 𝝅𝝅. This is the value you want to compare against 𝝅𝝅.

P8120 Spring 2025 7

Approach #2: Exact Hypothesis Test for a Single Proportion

We want to test whether the true proportion of interest is different (larger or smaller) than some particular value 𝜋𝜋0
using our data (n = sample size, x = observed number of successes)

Null and Alternative Hypotheses:

Two-sided
𝐻𝐻0 : 𝜋𝜋 = 𝜋𝜋0
𝐻𝐻1 : 𝜋𝜋 ≠ 𝜋𝜋0

But now it may be the case that 𝑛𝑛𝜋𝜋0 < 5 and 𝑛𝑛(1 − 𝜋𝜋0 ) < 5. So, we cannot use Z and trust that it follows a
normal distribution. Instead, we work directly with the binomial distribution to calculate a p-value.

Recall: X = random variable corresponding to number of successes

X ~ Bin(n,𝜋𝜋)

P-value = Probability of observing our data or data more extreme if H0 is true

There are several suggestions for how to compute a p-value for an exact binomial test. Here is how SAS does it:

Computing Exact two-sided p-value:

1. Compute 𝐴𝐴 = 𝑃𝑃(𝑋𝑋 ≥ 𝑥𝑥 0 | 𝜋𝜋 = 𝜋𝜋0 ) and 𝐵𝐵 = 𝑃𝑃 (𝑋𝑋 ≤ 𝑥𝑥 0 | 𝜋𝜋 = 𝜋𝜋0 )
2. P-value = 2 ∗ min (𝐴𝐴, 𝐵𝐵)

𝑛𝑛
𝑃𝑃(𝑋𝑋 = 𝑥𝑥|𝜋𝜋 = 𝜋𝜋0 ) = � � 𝜋𝜋0𝑥𝑥 (1 − 𝜋𝜋0 ) 𝑛𝑛−𝑥𝑥
𝑥𝑥

P8120 Spring 2025 8

Example. Out of 6 participants in a pilot study, 2 refused acupuncture treatment. Test the claim that the true
proportion of participants who will refuse acupuncture treatment is different from 20%.

Let’s examine this claim using the 7 steps:

(1) 𝜋𝜋 = true proportion of those who would refuse acupuncture treatment

(2) H0:

H1:

(3) Set α = 0.05

(4) Assumptions

(5) Test Statistics and Null Distribution

𝐴𝐴 = 𝑃𝑃 (𝑋𝑋 ≥ 𝑥𝑥 0 | 𝜋𝜋 = 𝜋𝜋0 )

𝐵𝐵 = 𝑃𝑃(𝑋𝑋 ≤ 𝑥𝑥 0 | 𝜋𝜋 = 𝜋𝜋0 )

(6) Decision

P-value = 2 ∗ min (𝐴𝐴, 𝐵𝐵) =

(7) Conclusion
At the 5% level of significance, we have _________________ evidence to conclude that the true
proportion of those who refused acupuncture therapy is different from 20%.

Model Answer for manuscript:

33.3% of the sample refused acupuncture treatment (95% CI: 4.3% to 77.7%). There is insufficient
evidence to conclude that the true proportion that refuse is different from 20% (p-value = 0.6893).

P8120 Spring 2025 9

Below is the SAS code and output for addressing the question above.

data acupuncture_exact;
input refused;
cards;
2
4
;
run;

title 'Acupuncture example: Exact Test';

proc freq data = acupuncture_exact order = data;
table loss / binomial (p=0.20) alpha = 0.05;
exact binomial;
weight loss;
run;

P8120 Spring 2025 10