PASS Sample Size Software NCSS.

com

Chapter 471

Confidence Intervals
for the Difference
Between Two Means
Introduction
This procedure calculates the sample size necessary to achieve a specified distance from the difference in sample
means to the confidence limit(s) at a stated confidence level for a confidence interval about the difference in
means when the underlying data distribution is normal.
Caution: This procedure assumes that the standard deviations of the future samples will be the same as the
standard deviations that are specified. If the standard deviation to be used in the procedure is estimated from a
previous sample or represents the population standard deviation, the Confidence Intervals for the Difference
between Two Means with Tolerance Probability procedure should be considered. That procedure controls the
probability that the distance from the difference in means to the confidence limits will be less than or equal to the
value specified.

Technical Details
There are two formulas for calculating a confidence interval for the difference between two population means.
The different formulas are based on whether the standard deviations are assumed to be equal or unequal.
For each of the cases below, let the means of the two populations be represented by µ1 and µ2 , and let the
standard deviations of the two populations be represented as σ1 and σ 2 .

Case 1 – Standard Deviations Assumed Equal

When σ1 = σ 2 = σ are unknown, the appropriate two-sided confidence interval for µ1 - µ2 is

1 1
X 1 − X 2 ± t1−α / 2 ,n1 + n2 −2 s p +
n1 n2
where

( n1 − 1) s12 + ( n2 − 1) s22
sp =
n1 + n2 − 2
Upper and lower one-sided confidence intervals can be obtained by replacing α/2 with α.

471-1
© NCSS, LLC. All Rights Reserved.
PASS Sample Size Software NCSS.com
Confidence Intervals for the Difference Between Two Means

The required sample size for a given precision, D, can be found by solving the following equation iteratively

1 1
D = t1−α / 2 ,n1 + n2 −2 s p +
n1 n2
This equation can be used to solve for D or n1 or n2 based on the values of the remaining parameters.

Case 2 – Standard Deviations Assumed Unequal

When σ1 ≠ σ 2 are unknown, the appropriate two-sided confidence interval for µ1 – µ2 is

s12 s22
X 1 − X 2 ± t1−α / 2 ,ν +
n1 n2
where
2
 s12 s22 
 + 
ν=  n1 n2 
s14 s24
+
n12 ( n1 − 1) n22 ( n2 − 1)
In this case t is an approximate t and the method is known as the Welch-Satterthwaite method. Upper and lower
one-sided confidence intervals can be obtained by replacing α/2 with α.
The required sample size for a given precision, D, can be found by solving the following equation iteratively

s12 s22
D = t1−α / 2 ,ν +
n1 n2
This equation can be used to solve for D or n1 or n2 based on the values of the remaining parameters.

Confidence Level
The confidence level, 1 – α, has the following interpretation. If thousands of samples of n1 and n2 items are drawn
from populations using simple random sampling and a confidence interval is calculated for each sample, the
proportion of those intervals that will include the true population mean difference is 1 – α.
Notice that is a long term statement about many, many samples.

471-2
© NCSS, LLC. All Rights Reserved.
PASS Sample Size Software NCSS.com
Confidence Intervals for the Difference Between Two Means

Procedure Options
This section describes the options that are specific to this procedure. These are located on the Design tab. For
more information about the options of other tabs, go to the Procedure Window chapter.

Design Tab
The Design tab contains most of the parameters and options that you will be concerned with.

Solve For
Solve For
This option specifies the parameter to be solved for from the other parameters.

One-Sided or Two-Sided Interval

Interval Type
Specify whether the interval to be used will be a one-sided or a two-sided confidence interval.

Confidence
Confidence Level (1 – Alpha)
The confidence level, 1 – α, has the following interpretation. If thousands of samples of n1 and n2 items are
drawn from populations using simple random sampling and a confidence interval is calculated for each sample,
the proportion of those intervals that will include the true population mean difference is 1 – α.
Often, the values 0.95 or 0.99 are used. You can enter single values or a range of values such as 0.90, 0.95 or 0.90
to 0.99 by 0.01.

Sample Size (When Solving for Sample Size)

Group Allocation
Select the option that describes the constraints on N1 or N2 or both.
The options are

• Equal (N1 = N2)

This selection is used when you wish to have equal sample sizes in each group. Since you are solving for both
sample sizes at once, no additional sample size parameters need to be entered.

• Enter N1, solve for N2

Select this option when you wish to fix N1 at some value (or values), and then solve only for N2. Please note
that for some values of N1, there may not be a value of N2 that is large enough to obtain the desired power.

• Enter N2, solve for N1

Select this option when you wish to fix N2 at some value (or values), and then solve only for N1. Please note
that for some values of N2, there may not be a value of N1 that is large enough to obtain the desired power.

• Enter R = N2/N1, solve for N1 and N2

For this choice, you set a value for the ratio of N2 to N1, and then PASS determines the needed N1 and N2,
with this ratio, to obtain the desired power. An equivalent representation of the ratio, R, is

471-3
© NCSS, LLC. All Rights Reserved.
PASS Sample Size Software NCSS.com
Confidence Intervals for the Difference Between Two Means

N2 = R * N1.

• Enter percentage in Group 1, solve for N1 and N2

For this choice, you set a value for the percentage of the total sample size that is in Group 1, and then PASS
determines the needed N1 and N2 with this percentage to obtain the desired power.
N1 (Sample Size, Group 1)
This option is displayed if Group Allocation = “Enter N1, solve for N2”
N1 is the number of items or individuals sampled from the Group 1 population.
N1 must be ≥ 2. You can enter a single value or a series of values.
N2 (Sample Size, Group 2)
This option is displayed if Group Allocation = “Enter N2, solve for N1”
N2 is the number of items or individuals sampled from the Group 2 population.
N2 must be ≥ 2. You can enter a single value or a series of values.
R (Group Sample Size Ratio)
This option is displayed only if Group Allocation = “Enter R = N2/N1, solve for N1 and N2.”
R is the ratio of N2 to N1. That is,
R = N2 / N1.
Use this value to fix the ratio of N2 to N1 while solving for N1 and N2. Only sample size combinations with this
ratio are considered.
N2 is related to N1 by the formula:
N2 = [R × N1],
where the value [Y] is the next integer ≥ Y.
For example, setting R = 2.0 results in a Group 2 sample size that is double the sample size in Group 1 (e.g., N1 =
10 and N2 = 20, or N1 = 50 and N2 = 100).
R must be greater than 0. If R < 1, then N2 will be less than N1; if R > 1, then N2 will be greater than N1. You can
enter a single or a series of values.
Percent in Group 1
This option is displayed only if Group Allocation = “Enter percentage in Group 1, solve for N1 and N2.”
Use this value to fix the percentage of the total sample size allocated to Group 1 while solving for N1 and N2.
Only sample size combinations with this Group 1 percentage are considered. Small variations from the specified
percentage may occur due to the discrete nature of sample sizes.
The Percent in Group 1 must be greater than 0 and less than 100. You can enter a single or a series of values.

Sample Size (When Not Solving for Sample Size)

Group Allocation
Select the option that describes how individuals in the study will be allocated to Group 1 and to Group 2.
The options are

• Equal (N1 = N2)

This selection is used when you wish to have equal sample sizes in each group. A single per group sample
size will be entered.

471-4
© NCSS, LLC. All Rights Reserved.
PASS Sample Size Software NCSS.com
Confidence Intervals for the Difference Between Two Means

• Enter N1 and N2 individually

This choice permits you to enter different values for N1 and N2.

• Enter N1 and R, where N2 = R * N1

Choose this option to specify a value (or values) for N1, and obtain N2 as a ratio (multiple) of N1.

• Enter total sample size and percentage in Group 1

Choose this option to specify a value (or values) for the total sample size (N), obtain N1 as a percentage of N,
and then N2 as N - N1.
Sample Size Per Group
This option is displayed only if Group Allocation = “Equal (N1 = N2).”
The Sample Size Per Group is the number of items or individuals sampled from each of the Group 1 and Group 2
populations. Since the sample sizes are the same in each group, this value is the value for N1, and also the value
for N2.
The Sample Size Per Group must be ≥ 2. You can enter a single value or a series of values.
N1 (Sample Size, Group 1)
This option is displayed if Group Allocation = “Enter N1 and N2 individually” or “Enter N1 and R, where N2 =
R * N1.”
N1 is the number of items or individuals sampled from the Group 1 population.
N1 must be ≥ 2. You can enter a single value or a series of values.
N2 (Sample Size, Group 2)
This option is displayed only if Group Allocation = “Enter N1 and N2 individually.”
N2 is the number of items or individuals sampled from the Group 2 population.
N2 must be ≥ 2. You can enter a single value or a series of values.
R (Group Sample Size Ratio)
This option is displayed only if Group Allocation = “Enter N1 and R, where N2 = R * N1.”
R is the ratio of N2 to N1. That is,
R = N2/N1
Use this value to obtain N2 as a multiple (or proportion) of N1.
N2 is calculated from N1 using the formula:
N2=[R x N1],
where the value [Y] is the next integer ≥ Y.
For example, setting R = 2.0 results in a Group 2 sample size that is double the sample size in Group 1.
R must be greater than 0. If R < 1, then N2 will be less than N1; if R > 1, then N2 will be greater than N1. You can
enter a single value or a series of values.
Total Sample Size (N)
This option is displayed only if Group Allocation = “Enter total sample size and percentage in Group 1.”
This is the total sample size, or the sum of the two group sample sizes. This value, along with the percentage of
the total sample size in Group 1, implicitly defines N1 and N2.

471-5
© NCSS, LLC. All Rights Reserved.
PASS Sample Size Software NCSS.com
Confidence Intervals for the Difference Between Two Means

The total sample size must be greater than one, but practically, must be greater than 3, since each group sample
size needs to be at least 2.
You can enter a single value or a series of values.
Percent in Group 1
This option is displayed only if Group Allocation = “Enter total sample size and percentage in Group 1.”
This value fixes the percentage of the total sample size allocated to Group 1. Small variations from the specified
percentage may occur due to the discrete nature of sample sizes.
The Percent in Group 1 must be greater than 0 and less than 100. You can enter a single value or a series of
values.

Precision
Distance from Mean Difference to Limit(s)
This is the distance from the confidence limit(s) to the difference in means. For two-sided intervals, it is also
known as the precision, half-width, or margin of error.
You can enter a single value or a list of values. The value(s) must be greater than zero.

Standard Deviations
S1 and S2 (Standard Deviations)
Enter an estimate of the standard deviation of group 1 or 2. The standard deviation must be a positive number.
Caution: The sample size estimates for this procedure assume that the standard deviation that is achieved when
the confidence interval is produced is the same as the standard deviation entered here.
Press the 'Standard Deviation Estimator' button to obtain help on estimating the standard deviation.
You can enter a range of values such as 1, 2, 3 or 1 to 10 by 1.
Standard Deviation Equality Assumption
Specify whether the standard deviations are assumed to be the same or different. The choice will determine which
of the two common confidence interval formulas for estimating the difference in population means will be used.

• Assume S1 and S2 are Unequal

When the standard deviations are assumed to be unequal, the variances are not pooled and an approximate
method is used for the confidence interval formula. This approximate method is sometimes called the Welch-
Satterthwaite method.

• Assume S1 and S2 are Equal

When the standard deviations are assumed to be equal, the pooled variance formula is used in the calculation
of the confidence interval. The degrees of freedom are N1 + N2 – 2.
Recommendation: Because the standard deviations of two populations are rarely equal, it is recommended
that the standard deviations are assumed to be unequal. The Welch-Satterthwaite confidence interval
calculation is generally accepted and commonly used.

471-6
© NCSS, LLC. All Rights Reserved.
PASS Sample Size Software NCSS.com
Confidence Intervals for the Difference Between Two Means

Example 1 – Calculating Sample Size

Suppose a study is planned in which the researcher wishes to construct a two-sided 95% confidence interval for
the difference between two population means such that the width of the interval is no wider than 20 units. The
confidence level is set at 0.95, but 0.99 is included for comparative purposes. The standard deviation estimates,
based on the range of data values, are 32 for Population 1 and 38 for Population 2. Instead of examining only the
interval half-width of 10, a series of half-widths from 5 to 15 will also be considered.
The goal is to determine the necessary sample size for each group.

Setup
This section presents the values of each of the parameters needed to run this example. First, from the PASS Home
window, load the Confidence Intervals for the Difference Between Two Means procedure window by
expanding Means, then Two Independent Means, then clicking on Confidence Interval, and then clicking on
Confidence Intervals for the Difference Between Two Means. You may then make the appropriate entries as
listed below, or open Example 1 by going to the File menu and choosing Open Example Template.
Option Value
Design Tab
Solve For ................................................ Sample Size
Interval Type ........................................... Two-Sided
Confidence Level .................................... 0.95 0.99
Group Allocation ..................................... Equal (N1 = N2)
Distance from Mean Diff to Limit(s) ........ 5 to 15 by 1
S1 (Standard Deviation Group 1) ........... 32
S2 (Standard Deviation Group 2) ........... 38
Std. Dev. Equality Assumption ............... Assume S1 and S2 are Unequal

Annotated Output
Click the Calculate button to perform the calculations and generate the following output.

Numeric Results
Numeric Results for Two-Sided Confidence Intervals for the Difference in Means
The standard deviations are assumed to be Unknown and Unequal.

Target Actual
Dist from Dist from
Confidence Mean Diff Mean Diff
Level N1 N2 N to Limits to Limits S1 S2
0.950 380 380 760 5.000 4.995 32.00 38.00
0.950 265 265 530 6.000 5.995 32.00 38.00
0.950 195 195 390 7.000 6.995 32.00 38.00
0.950 150 150 300 8.000 7.984 32.00 38.00
0.950 119 119 238 9.000 8.973 32.00 38.00
0.950 97 97 194 10.000 9.951 32.00 38.00
0.950 80 80 160 11.000 10.973 32.00 38.00
0.950 68 68 136 12.000 11.918 32.00 38.00
0.950 58 58 116 13.000 12.926 32.00 38.00
0.950 50 50 100 14.000 13.947 32.00 38.00
0.950 44 44 88 15.000 14.895 32.00 38.00
0.990 655 655 1310 5.000 5.000 32.00 38.00
0.990 455 455 910 6.000 5.999 32.00 38.00

(report continues)

471-7
© NCSS, LLC. All Rights Reserved.
PASS Sample Size Software NCSS.com
Confidence Intervals for the Difference Between Two Means

References
Ostle, B. and Malone, L.C. 1988. Statistics in Research. Iowa State University Press. Ames, Iowa.
Zar, Jerrold H. 1984. Biostatistical Analysis (Second Edition). Prentice-Hall. Englewood Cliffs, New Jersey.

Report Definitions
Confidence level is the proportion of confidence intervals (constructed with this same confidence level,
sample size, etc.) that would contain the true difference in population means.
N1 and N2 are the number of items sampled from each population.
N is the total sample size, N1 + N2.
Target Dist from Mean Diff to Limit is the value of the distance that is entered into the procedure.
Actual Dist from Mean Diff to Limit is the value of the distance that is obtained from the procedure.
S1 and S2 are the standard deviations upon which the distance from mean difference to limit calculations are
based.

Summary Statements
Group sample sizes of 380 and 380 produce a two-sided 95% confidence interval with a distance
from the difference in means to the limits that is equal to 4.995 when the estimated standard
deviations are 32.00 and 38.00.

This report shows the calculated sample size for each of the scenarios.

Plots Section

These plots show the sample size of each group versus the precision for the two confidence levels.

471-8
© NCSS, LLC. All Rights Reserved.
PASS Sample Size Software NCSS.com
Confidence Intervals for the Difference Between Two Means

Example 2 – Validation using Ostle and Malone

Ostle and Malone (1988) page 150 give an example of a precision calculation for a confidence interval for the
difference between two means when the confidence level is 95%, the two standard deviations are 6.2185 and
16.06767, and the sample sizes are 7 and 6. The precision is 13.433 (when df = 6.257, not 6).

Setup
This section presents the values of each of the parameters needed to run this example. First, from the PASS Home
window, load the Confidence Intervals for the Difference Between Two Means procedure window by
expanding Means, then Two Independent Means, then clicking on Confidence Interval, and then clicking on
Confidence Intervals for the Difference Between Two Means. You may then make the appropriate entries as
listed below, or open Example 2 by going to the File menu and choosing Open Example Template.
Option Value
Design Tab
Solve For ................................................ Sample Size
Interval Type ........................................... Two-Sided
Confidence Level .................................... 0.90
Group Allocation ..................................... Enter N2, solve for N1
N2 ........................................................... 6
Distance from Mean Diff to Limit(s) ........ 13.433
S1 (Standard Deviation Group 1) ........... 6.2185
S2 (Standard Deviation Group 2) ........... 16.06767
Std. Dev. Equality Assumption ............... Assume S1 and S2 are Unequal

Output
Click the Calculate button to perform the calculations and generate the following output.

Numeric Results
Numeric Results for Two-Sided Confidence Intervals for the Difference in Means
The standard deviations are assumed to be Unknown and Unequal.

Target Actual
Dist from Dist from
Confidence Mean Diff Mean Diff
Level N1 N2 N to Limits to Limits S1 S2
0.900 7 6 13 13.433 13.433 6.22 16.07

PASS also calculated the sample size in Group 1 to be 7.

471-9
© NCSS, LLC. All Rights Reserved.
PASS Sample Size Software NCSS.com
Confidence Intervals for the Difference Between Two Means

Example 3 – Validation using Zar

Zar (1984) page 132 gives an example of a precision calculation for a confidence interval for the difference
between two means when the confidence level is 95%, the pooled standard deviation estimate is 0.7206, and the
sample sizes are 6 and 7. The precision is 0.88.

Setup
This section presents the values of each of the parameters needed to run this example. First, from the PASS Home
window, load the Confidence Intervals for the Difference Between Two Means procedure window by
expanding Means, then Two Independent Means, then clicking on Confidence Interval, and then clicking on
Confidence Intervals for the Difference Between Two Means. You may then make the appropriate entries as
listed below, or open Example 3 by going to the File menu and choosing Open Example Template.
Option Value
Design Tab
Solve For ................................................ Distance from Mean Difference to Limit
Interval Type ........................................... Two-Sided
Confidence Level .................................... 0.95
Group Allocation ..................................... Enter N1 and N2 individually
N1 (Sample Size Group 1) ..................... 6
N2 (Sample Size Group 2) ..................... 7
S1 (Standard Deviation Group 1) ........... 0.7206
S2 (Standard Deviation Group 2) ........... S1
Std. Dev. Equality Assumption ............... Assume S1 and S2 are Equal

Output
Click the Calculate button to perform the calculations and generate the following output.

Numeric Results
Numeric Results for Two-Sided Confidence Intervals for the Difference in Means
The standard deviations are assumed to be Unknown and Equal.

Dist from
Confidence Mean Diff
Level N1 N2 N to Limits S1 S2
0.950 6 7 13 0.882 0.72 0.72

PASS also calculates the precision to be 0.88.

471-10
© NCSS, LLC. All Rights Reserved.