Effect Size Calculator:

A guide to using the spreadsheet

Robert Coe

Effect Size Calculator is a Microsoft Excel spreadsheet. It runs in version 5 or later (including
Office97). If you enter the mean, number of values and standard deviation for the two groups being
compared, it will calculate the 'Effect Size' for the difference between them, and show this
difference (and its 'confidence interval') on a graph. It will also calculate the standard 't-test' for
comparing two means to see whether the difference is statistically significant.
The spreadsheet consists of two sheets: 'Calculator', in which data are entered and values calculated,
and 'Graph', which plots the effect size estimate and its confidence intervals. Click on the tabs at
the bottom of the screen to alternate between them.
Calculator

This sheet is divided into three sections:

1. DATA ENTRY (columns A to G)
This section contains the cells in which data can be entered. Do not alter the contents of any cell
that is shaded. Type only in blank cells, otherwise important formulae may be lost.

Column Heading Description

A Outcome Measure Type a short label for each outcome measure entered. The
contents of cells A4 to A7 are used as the labels for the effect
size estimates in the Graph.
B Treatment group: Enter the mean for the treatment group.
mean
C Treatment group: n Enter the number of values in the treatment group.
D Treatment group: SD Enter the standard deviation of the values in the treatment
group.
E Control group: mean Enter the mean for the control group.
F Control group: n Enter the number of values in the control group.
G Control group: SD Enter the standard deviation of the values in the control
group.
2. RAW DIFFERENCE (columns H to M)
This section calculates the raw difference between the two groups and the 'pooled' standard
deviation, together with p-values and confidence limits for these.

Column Heading Description

H Pooled standard This is the pooled estimate of standard deviation from both
deviation groups, based on the assumption that any difference between
their SDs is only due to sampling variation.
I p-value for difference This is the 'p-value' for an F-test of whether their SDs are
in SDs close enough to differ only by chance. It is the probability
that a difference as big as this would have occurred if the
samples were drawn from the same population.
Conventionally, values less than 0.05 are taken to cast doubt
on this assumption.
J Mean Difference This is simply the difference between the two means. If the
outcome is measured on a familiar scale, this difference is
interpretable as the size of the effect.
K p-value for mean diff This is the 'p-value' for a standard T-test of whether the two
(2-tailed T-test) means are close enough to differ only by chance. It is the
probability that a difference as big as this would have
occurred if the samples were drawn from the same
population. Conventionally, values less than 0.05 are taken to
cast doubt on this assumption, ie if p < 0.05, the difference is
unlikely to have arisen by chance and is said to be
'statistically significant'.
L Confidence interval The confidence interval is an alternative way to indicate the
for difference: lower variability in estimates from small samples. The default
calculation here is a '95% confidence interval' (other
M Confidence interval percentages can be given by changing the value in cell W10).
for difference: upper If multiple samples of two groups of the same size as these,
taken from a population in which the true difference was the
value in column J, there would be variation in the differences
found. However, for every 100 samples taken, for 95 of them
(on average) the difference would be between the lower and
upper confidence limits. The confidence interval is usually
interpreted as a 'margin of uncertainty' around the estimate of
the difference between experimental and control groups.
3. STANDARDISED EFFECT SIZE (columns N to S)
This section calculates the difference between the two groups as a standardised Effect Size, corrects
it for bias and computes a confidence interval.

Column Heading Description

N Effect Size This is the difference between the two means (column J),
divided by the pooled estimate of standard deviation (column
H). It calibrates the difference between the experimental and
control groups (ie the effect of the intervention) in terms of
the standard deviation.
O Bias corrected The effect size estimate is slightly biased and is therefore
(Hedges) corrected using a factor provided by Hedges and Olkin
(1985). The correction factors are stored in column AA. This
corrected estimate is the one plotted in the Graph.
P Standard Error of E.S. This is a measure of the amount the effect size estimate would
estimate vary if you repeatedly took different samples.
Q Confidence interval See comments on column L, above. Again this is a 95%
for Effect Size: lower confidence interval; other values can be shown by changing
cell W10. Upper and lower confidence limits are also shown
R Confidence interval in the Graph.
for Effect Size: upper
S Effect Size based on In some cases it may not be appropriate to use a pooled
control group SD estimate of standard deviation, so the control group SD is
used.
The formulae to calculate these statistics have been pasted into ten rows of the spreadsheet (ie rows
4 to 13). If you want to calculate more than ten Effect Sizes, the formulae can be copied into
further rows. The easiest way to do this is to first select the cells containing the bottom row with
the formulae already in and the rows into which you want to paste them. For example, if you want
a further five rows, click in cell H13, drag to S18 and release. Then press CTRL + D (ie hold down
CTRL and press and release D).
Graph

The graph plots the Effect Size estimate (column O) and its confidence limits (columns Q and R).
It uses the text in column A as a label for each Effect Size.
By default, the graph plots four Effect Sizes, corresponding to the values in rows 4 to 7. To include
more (or fewer) Effect Sizes, move the pointer over one of the diamonds representing the Effect Size
estimate, and click. The text
=SERIES("Effect Size estimate",Calculator!$A$4:$A$7,Calculator!$O$4:$O$7,1)
will appear in the formula bar (towards the top of the screen). If you want only two Effect Sizes to
be shown, change both the '7's into '5's (so that the values in rows 4 to 5 will be plotted) and press
RETURN. Alternatively, to plot five Effect Sizes, replace the '7's with '8's. Then click on one of
the Upper confidence limit points and repeat the correction, and again for the Lower confidence limit.