Power & Sample size:

An intro
Clinical Research Training Program

Ho Thi Nhan, MD, PhD

1
 Learning Objectives
• Understand power and sample size estimation
– Importance in both study design and analysis
• Example sample size calculation (with
Gpower)
– continuous, binary outcome
– one sample
– two sample comparison

2
 Recap
Truth
Decision No Difference Difference
No difference Right Type II error (β)
Difference Type I error (α) Right (1-β=power)
Type I error: null hypothesis (H0) is wrongly rejected.
e.g. conclude that the two drugs produced different effects when in fact there
was no difference between them

Type II error: null hypothesis (H0) is not rejected when it is in

4
 Factors affecting power
• Precision and variance of measurements
– More precise (narrower CI) => more sample size
• Effect size
– Larger => better power
– Clinically meaningful or practically important
difference
• Significance level (alpha): P(Type I error)
– Smaller => smaller power
• Type of statistical test
– E.g. parametric (t-test) > non-parametric
(Wilcoxon) 5
 When calculate power
• Definitely before the study
– Occasionally during: interim (if original calculation
are suspect)
– Sometimes after: negative results => calculate if
the study is underpowered
• Satisfactory power
– 80%, 90%

6
 Sample size
• How many subjects are needed to answer the
research question (to achieve desired power)

7
 CI, margin of error
=> Affect sample size
• CI: Point Estimate + Margin of Error
• Investigator must specify the desired margin
of error (E)
– not a statistical issue, but a clinical or a practical
one
– E.g. normal birth weight: E must be small
normal adult weight: E may be large

8
Sample size calculation examples

9
 One Sample, Continuous Outcome
• Estimate the mean (μ) of a continuous
outcome
• Sample size
= minimum number of subjects required to
ensure that the margin of error in the CI for μ
does not exceed E.
• σ = the standard deviation
• Z from standard normal distribution (e.g.
Z=1.96 for 95%CI)
10
 One Sample, Continuous Outcome
• E.g. SBP in children 3-5 yrs: 95%CI, E=5, σ=20

 Sample size=62 (round up)

11
 One sample, continuous outcome,
comparing to constant
• Compare mean of sample to known mean
• H0: μ = μ 0 and H1: μ ≠ μ 0 (μ 0 =known mean)

12
 One sample, continuous outcome,
comparing to constant
E.g. fasting blood glucose/coffee drinker
~80mg/dl, σ=20; known average in general
population =70mg/dl, α=0.05 two sided,
power=0.8

 ES=(80-70)/20=0.5

13
One sample, continuous outcome,
comparing to constant

14
 One Sample, binary Outcome
• To estimate the proportion (p)

• p ranges 0 to 1
• Need: approximate/anticipated value of p
• p(1-p) maximized if p=0.5 => most
conservative, largest, sample size

15
 One Sample, binary Outcome
• E.g. proportion of C-section at Vinmec: 95%CI,
E=5%, no prior info about % C-section => use
p=0.5

• Sample size=385

16
 One Sample, binary Outcome,
compare to constant
• Compare proportion of sample to known
proportion
• H0: p1=p0 ; H1: p1≠p0 (p0=known proportion)

17
 One Sample, binary Outcome,
compare to constant
E.g. p c-section =0.4 in Vinmec 2018; p general
VN = 0.25
α=0.05 two sided; power=0.8

=> ES=(0.4-0.25)/sqrt(0.4*0.6)

18
One Sample, binary Outcome,
compare to constant

19
 2 independent samples,
continuous outcome
• Estimate the difference in means between two
independent populations (t-test)
• H0: µ1=µ2 vs. H1: µ1≠µ2
ES= |µ1-µ2|/σ

20
 2 independent samples,
continuous outcome
E.g. RCT: new drug lower SBP 5mmHg vs. no
new drug; σ SBP=20; α=0.05 two sided;
power=0.8; => ES = 5/20 = 0.25

21
 2 Matched samples,
continuous outcome
• Test the mean difference in a continuous
outcome variable based on matched data
• H0: µd=0; H1: µd≠0
• ES= µd/σd

22
 2 Matched samples,
continuous outcome
E.g. SBP before and after 6 month programed
work out: µd=10mmHg, σd=20mmHg;
α=0.05 two sided; power=0.8

=> ES=10/20=0.5

23
2 Matched samples, continuous
outcome

24
 Two Independent Samples,
Binary Outcome
• Compare the proportions in two independent
populations
• H0: p1=p2; H1: p1≠p2
• ES=|p1-p2|/sqrt(p*(1-p))
p=(p1+p2)/2
,

25
 Two Independent Samples,
Binary Outcome
E.g. flu 25% in vaccinated students, 35% flu in
non vaccinated students
α=0.05 two sided; power=0.8

26
 References
• An introduction to power and sample size
estimation.
https://emj.bmj.com/content/20/5/453
• Power and Sample Size Determination.
http://sphweb.bumc.bu.edu/otlt/MPH-
Modules/BS/BS704_Power/BS704_Power_pri
nt.html
• Gpower 3.1:
http://www.psychologie.hhu.de/arbeitsgrupp
en/allgemeine-psychologie-und-
arbeitspsychologie/gpower.html 27
 Practice
• A clinical trial is planned to evaluate if stem cell
transplantation (SCT) increases endometrial thickness (measured
by mm) vs. standard treatment in patients with Asherman’s
syndrome.
• 1) Choose best option
• a) No power estimation needed if the trial aims to test only
the efficacy
• a) No power estimation needed if the trial aims to test only
the safety
• c) No sample size estimation needed if the trial is phase 1 or 2
• d) None of the above

28
 Practice
• A clinical trial is planned to evaluate if stem cell
transplantation (SCT) increases endometrial thickness (measured
by mm) vs. standard treatment in patients with Asherman’s
syndrome.
• 1) Choose best option
• a) No power estimation needed if the trial aims to test only
the efficacy
• a) No power estimation needed if the trial aims to test only
the safety
• c) No sample size estimation needed if the trial is phase 1 or 2
• d) None of the above

29
• 2) When power estimation for this trial
should be done?
• a) Shortly before completing the trial
• b) After having funding approval but before
trial implementation
• c) During planning/proposal phase
• d) No power estimation needed

30
• 2) When power estimation for this trial
should be done?
• a) Shortly before completing the trial
• b) After having funding approval but before
trial implementation
• c) During planning/proposal phase
• d) No power estimation needed

31
• 3) Sample size estimation should be based on
• a) Statistical test comparing the mean of a
continuous outcome in one sample vs. constant
• b) Statistical test comparing the means of a
continuous outcome in two independent samples
• c) Statistical test comparing the proportions of a
binary outcome in two independent samples
• d) Statistical test comparing the proportions of a
binary outcome in one sample vs. constant
32
• 3) Sample size estimation should be based on
• a) Statistical test comparing the mean of a
continuous outcome in one sample vs. constant
• b) Statistical test comparing the means of a
continuous outcome in two independent samples
• c) Statistical test comparing the proportions of a
binary outcome in two independent samples
• d) Statistical test comparing the proportions of a
binary outcome in one sample vs. constant
33
• If the main outcome of the above clinical trial is the rate of
on-going pregnancy instead of endometrial thickness,
• 4) Sample size estimation should be based on:
• a) Statistical test comparing the mean of a continuous
outcome in one sample vs. constant
• b) Statistical test comparing the means of a continuous
outcome in two independent samples
• c) Statistical test comparing the proportions of a binary
outcome in two independent samples
• d) Statistical test comparing the proportions of a binary
outcome in one sample vs. constant
34
• If the main outcome of the above clinical trial is the rate of
on-going pregnancy instead of endometrial thickness,
• 4) Sample size estimation should be based on:
• a) Statistical test comparing the mean of a continuous
outcome in one sample vs. constant
• b) Statistical test comparing the means of a continuous
outcome in two independent samples
• c) Statistical test comparing the proportions of a binary
outcome in two independent samples
• d) Statistical test comparing the proportions of a binary
outcome in one sample vs. constant
35
• 5) To estimate sample size, the investigator
needs to specify
• a) Assumption regarding the rate of on-going
pregnancy in each group
• b) Alpha level
• c) Desired power
• d) All of the above

36
• 5) To estimate sample size, the investigator
needs to specify
• a) Assumption regarding the rate of on-going
pregnancy in each group
• b) Alpha level
• c) Desired power
• d) All of the above

37
• An investigator would like to evaluate the change in
endometrial thickness (measured by mm) in patients with
Asherman’s syndrome before and after SCT.
• 6) Sample size estimation should be based on:
• a) Statistical test comparing the mean of a continuous
outcome in one sample vs. constant
• b) Statistical test comparing the means of a continuous
outcome in two independent samples
• c) Statistical test comparing the proportions of a binary
outcome in two independent samples
• d) Statistical test comparing the proportions of a binary
outcome in one sample vs. constant
38
• An investigator would like to evaluate the change in
endometrial thickness (measured by mm) in patients with
Asherman’s syndrome before and after SCT.
• 6) Sample size estimation should be based on:
• a) Statistical test comparing the mean of a continuous
outcome in one sample vs. constant
• b) Statistical test comparing the means of a continuous
outcome in two independent samples
• c) Statistical test comparing the proportions of a binary
outcome in two independent samples
• d) Statistical test comparing the proportions of a binary
outcome in one sample vs. constant
39
• An investigator would like to estimate the proportion of
Asherman’s syndrome patients having spontaneous pregnancy in
Hanoi.
• 7) Choose best option:
• a) No sample size estimation needed because there is no
comparison
• b) He should assume that the proportion of Asherman’s
syndrome patients having spontaneous pregnancy is 50%
• c) He should make the best assumption possible about the
proportion of Asherman’s syndrome patients having
spontaneous pregnancy in Hanoi based on literature
• d) None of the above
40
• An investigator would like to estimate the proportion of
Asherman’s syndrome patients having spontaneous pregnancy in
Hanoi.
• 7) Choose best option:
• a) No sample estimation needed because there is no
comparison
• b) He should assume that the proportion of Asherman’s
syndrome patients having spontaneous pregnancy is 50%
• c) He should make the best assumption possible about the
proportion of Asherman’s syndrome patients having
spontaneous pregnancy in Hanoi based on literature
• d) None of the above
41
• 8) To estimate required sample size, he should
specify
• a) Assumption about the proportion of Asherman’s
syndrome patients having spontaneous pregnancy in
Hanoi
• b) Margin of error
• c) Type of confidence interval (e.g. 99%, 95%, 90%...)
• d) All of the above

42
• 8) To estimate required sample size, he should
specify
• a) Assumption about the proportion of Asherman’s
syndrome patients having spontaneous pregnancy in
Hanoi
• b) Margin of error
• c) Type of confidence interval (e.g. 99%, 95%, 90%...)
• d) All of the above

43
• An investigator would like to estimate the mean of
endometrial thickness in receptive phase of healthy women
having health check at Vinmec.
• 9) To estimate sample size, he should specify
• a) Assumption about the mean of endometrial thickness in
receptive phase of healthy women
• b) Assumption about the standard deviation of endometrial
thickness in receptive phase of healthy women
• c) Both a and b
• d) Neither a nor b

44
• An investigator would like to estimate the mean of
endometrial thickness in receptive phase of healthy women
having health check at Vinmec.
• 9) To estimate sample size, he should specify
• a) Assumption about the margin of error for the mean of
endometrial thickness in receptive phase of healthy women
• b) Assumption about the standard deviation of endometrial
thickness in receptive phase of healthy women
• c) Both a and b
• d) Neither a nor b

45
• 10) To estimate sample size, he should
specify
• a) Margin of error
• b) Type of confidence interval (e.g. 99%, 95%,
90%...)
• c) Both a and b
• d) Neither a nor b

46
• 10) To estimate sample size, he should
specify
• a) Margin of error
• b) Type of confidence interval (e.g. 99%, 95%,
90%...)
• c) Both a and b
• d) Neither a nor b

47
 Practice 1
• Vinmec would like to do a study evaluating the
efficacy and safety of probiotic supplement for
preventing further episodes of acute respiratory
infection in children with recurrent respiratory
infection.
– Research question, hypotheses, aims/objectives
– Study population
– Study design
– Outcomes and outcome measurement
– Bias, confounding, … to be considered
– Sample size- power estimation
– Hypothesis testing approach, Analysis plan (statistical test)
48
 Practice 2
• Estimate sample size or power or precision of
the estimates for your study

49
Thank you!

50