Introduction to Analysis of Variance (ANOVA)

Back to Comparing Group Means

What inferential test should we use to determine if mean blood pressure differs between a group
receiving Medication A versus a group receiving Medication B?
What if there are three medications under investigation (Medications A, B, and C)?
Can we use a t test?

Purpose of Analysis of Variance (ANOVA)

Why do we use an ANOVA inferential test?
To determine if there is a difference between two or more population means
i.e., tests the null hypothesis that “all population means are equal”
H O : μ1=μ2=μ 3 … μk
Alternative hypothesis is “not all population means are equal”
What are some examples of when this null hypothesis would be of interest?

A Little Terminology
Each population can be thought of as a “condition or group.” For example:
IV = tutoring program
Levels:
no tutor
once/week
daily
We denote the total number of groups by “k” and each group by “ j.” Consider the tutoring
example above:
k =3for groups j=1 ,2 ,∧3
So, the mean for group “ j” is denoted by M j
e.g., the mean of the no tutor group is denoted as M 1∨M no tutor
and the number of individuals in condition “ j” is denoted as n j(e.g., if there are 5 people in the
no tutor group, n1 =5)
The total mean across all individuals is referred to as the grand mean, which is denoted as
M G∨M Grand
Statistical Hypotheses for ANOVA
Hypotheses:
H O : μ1=μ2=μ 3
The mean of group 1 equals the mean of group 2 equals the mean of group 3 in the population
H A :not ( μ1=μ2 =μ 3)
At least two of the group means are not equal in the population. For example, it may be any of
the following:
μ1 ≠ μ2 ≠ μ 3
μ1 ≠ μ2 =μ 3
μ1=μ 2 ≠ μ 3
The General Idea
• Consider a situation where there are three conditions (k =3, where j=1 ,2 , 3)
• If M 1, M 2, M 3 are very similar, then the evidence suggests H O is likely
true
• But if M 1, M 2, M 3 become more discrepant (further apart), there is
increasing evidence that H O is false
• So, our goal is to quantify how discrepant M 1, M 2, M 3 are

Quantifying Discrepancy Between Group Means

• We quantify how discrepant M 1, M 2, M 3 are by using the variance of the group means
(M 1, M 2, M 3), referred to as between-group variance
• the typical squared distance that group means M j lie from the grand mean M G
• If M 1, M 2, M 3 are equal, variance of M j will be zero (between-group variance = 0)
• But, as M 1, M 2, M 3 deviate from one another, their variance will increase

Evaluating H O Using the Variance of M j

Treatment Effect and Error
The variability between group means M j (between-group variance) is due to two sources
Treatment effect: actual differences between group population means μ j
systematic component
Error: random sampling error
random component
Thus, we have:

Variance ( M j ) =Variance ( μ j ) + Error Variance

Partitioning Variance

The F Ratio
• What have we done so far to determine if the observed differences were
more than that attributable to random error? __________
• To determine if the observed variance in M j is more than that attributable to
random error, we compute an F ratio

Variance of M j Treatment Effect + Error Variance

F= =
Error Variance Error Variance
Obtaining F
To obtain F, we need:
Variance of M j (which we can compute)
Error variance

Variance of M j Treatment Effect + Error Variance

F= =
Error Variance Error Variance

Obtaining F
Error variance is estimated by considering the variance of the scores within each condition
(within-group variance).
Why is within-group variance considered error variance?
Because all individuals in a given condition have the same treatment
Thus, differences between individuals in the same condition cannot be attributable to the
treatment
(Variance of scores within each condition (within-group variance) is an estimate of the error
variance
Distribution of F ratios
If the null hypothesis is true, the value of F will be around 1.00
Because F ratios are computed from two variances, they are always positive numbers (because
variance are squared deviations)
The table of F critical values is organized by two df:
df numerator (between group)
df denominator (within group)
Note on how do we calculate the grand mean from summary data?
• With equal group sizes (unweighted):

M 1 + M 2+ M 3
M Grand =
k
• With unequal group sizes (weighted):

n1 M 1 +n2 M 2 +n 3 M 3
M Grand =
n
• Note, they’re the same. For example:
20+ 30+40 5(20)+ 5(30)+ 5(40) 450
M Grand = =30= =
3 15 15

Exercises for SSBG

• If all M j are the same, what will SSBG equal? _______
• Can SSBG ever be less than zero? _______
• As M j get further apart, does SSBG increase or decrease? _______

Calculation of df BG
df BG is just the number of groups minus 1.
df BG=k−1
For example, if there are 3 groups (k =3), then df BG=3−1=2.

Calculation of MS BG
Once we have SSBG and df BG, we can compute MS BG =SS BG /df BG
In our previous example of three groups for which SSBG =180, we have:
SS 180
MS BG = BG = =90
df BG 2

Exercises for MS BG
If all M j are the same, what will MS BG equal? _______
Can MS BG ever be less than zero? _______
As M j get further apart, does MS BG increase or decrease? _______
Calculation of df WG
df WG is the sum of the df j for each condition, where df j=n j−1
or (n−k)
For example, if there are 3 conditions, each with 10 individuals, then df WG is:
df WG =( 10−1 )+ ( 10−1 ) + ( 10−1 ) =27
Or df WG =30−3=27
Calculation of MS WG
Once we have SSWG and df WG we can compute MS WG=SSWG /df WG
In our previous example of three conditions, each with n j=10, for which SSWG =330, we have:
SSWG 330
MS WG= = =12.22
df WG 27

Computing F
Once you have MS BG and MS WG , the F ratio is computed using
MS BG
F=
MSWG
Using our current example
90
F= =7.36
12.22
We now need to find the critical value for df BG=2 and df WG =27, which is _____ (α =.05)
F-table: http://www.socr.ucla.edu/Applets.dir/F_Table.html

An Example
A researcher is studying the effects of mode of presentation (lecture, book, video) on
understanding of a history lesson. The researcher randomly assigns 120 students to one of the
three conditions (40 per condition), and measures knowledge of the history topic.
The M and SS values for the three conditions are:
M lecture =20 SSlecture =1220
M book =19 SSbook =1190
M video=24 SSvideo =1080
Test the null of no effect using α =.05 .
An Example, cont.
Step 1: State the research hypothesis:
 
Step 2: Identify the inferential test and set a:

Step 3: State the statistical hypotheses:

HO :
HA:

Step 4: Calculate statistics from the sample:

Step 5: Make a decision

if F obtained > F critical, reject H O
if F obtained < F critical, fail to reject H O

Example Cont.
20+ 19+ 24
M Grand = =21
3
SSBG =nl ( M l− M G )2 +nb ( M b −M G )2+n v (M v −M G )2=40 (20−21)2 +40(19−21)2+ 40(24−21)2
¿ 40 ( 1 ) + 40 ( 4 ) + 40 ( 9 )
¿ 40+ 160+ 360=560
SS 560
MS BG = BG = =280
df BG 2
Example Cont.
SSWG =SS l +SS b +SS v =1220+ 1190+1080=3490
SS 3490
MS WG= WG = =29.83
df WG 39+39+39
MS BG 280
F= = =9.39
MSWG 29.93
F crit =3.09 reject null
Assumptions of ANOVA
The same as independent samples t test.
Homogeneity of variance (not a concern with equal group sizes)
Normality (ANOVA is robust to violation of this assumption)
Independence of observations

Post-Hoc Tests
If you fail to reject the null, then you conclude there are no differences between any of the group
means
What happens when we reject the null?
then we know there is a difference somewhere between the group means… we just don’t know
where
Post-hoc tests are pairwise comparisons conducted to determine which pairs of conditions are
significantly different

Spiraling Alpha
One approach for comparing each pair of conditions is to conduct an independent-samples t test
for each pair.
i.e., if there are three conditions (A, B, C), there would be three comparisons (AB, AC, BC).
Imagine how many comparisons there would be with four or more groups?
This causes a problem because the chance of making a Type I error across all comparisons will
be higher than our a priori specified alpha (e.g., .05).

Solutions
Bonferroni test: adjust the Type I error rate accordingly for each test so that it adds up to the
α
intended Type I error ( )
¿ tests
Tukey’s test
Scheffe’s test
In general, all three approaches will give similar results when the number of groups is relatively
small (~4 or fewer)
As number of conditions increases, Bonferroni and Scheffe become very conservative
hard to reject the null (i.e., low power)
As a result, Tukey’s approach is often preferred
Effect size for ANOVA
• Compute percentage of variance accounted for by the treatment conditions
• In published reports of ANOVA, this is usually called η2(pronounced eta-
squared)
• Same concept as R2 :proportion of variance in the DV explained by the
IV (treatment/condition)
SSbetween
η2 =
SStotal