Week 6

ANOVA
Schedule
• Mock Questions
• One-way ANOVA
• Two-way ANOVA
When we use one-way ANOVA?
We want to test….. More than 2 conditions
Anova
• Null hypothesis: the treatment conditions have
no effect on the participant’s scores

H 0 : 1 =  2 =  3
• H1: There is at least one mean difference
among the populations
• The treatment conditions do affect the scores
• The acceptable shorthand is “Not H0”
The F-Ratio: The Test Statistic for ANOVA

• The F-ratio can be calculated after we have

analyzed the total variability into two basic
components

• Denominator of the F-ratio is called the error term

(measures only random and unsystematic
variability)
 The Logic of Analysis of Variance
• Between-treatments variance
• Variability results from general differences between the treatment conditions
• Variance between treatments measures differences between sample means
• Within-treatments variance
• Variability within each sample
• Individual scores are not the same within each sample
• Analysis of Variance is a variability ratio
• Partitioning – separating total variance into its component parts

x1
x3
Variance Between
x2
x1
x3
Variance WITHIN
x2
 The F-Ratio: The Test Statistic for ANOVA

• The value obtained for the F-ratio determines whether any treatment effects exist;
two possibilities:
• Fail to reject H0
• No systematic treatment effects; the differences are entirely caused by random, unsystematic
factors
• the means are very close to overall mean or distribution melt together
𝑆𝑀𝐴𝐿𝐿
𝐿𝐴𝑅𝐺𝐸
• means are fairly close to overall mean and distributions overlap a bit; difficult to distinguish
𝑆𝑖𝑚𝑙𝑎𝑟
𝑆𝑖𝑚𝑙𝑎𝑟

• Reject H0
• The treatment does have an effect and causes systematic differences between samples
• At least one mean is an outlier and each distribution is narrow; distinct from each other.
𝐿𝐴𝑅𝐺𝐸
• 𝑆𝑀𝐴𝐿𝐿
 Reporting the Results of Analysis of Variance

• Treatment means and standard deviations are presented in text, table,

or graph
• Results of ANOVA are summarized, including
• F and df F(df between, df within) = .xxx
• p value
• η2
• E.g., F(2, 18) = .284, p > .05, η2 = 0.03
Example

• No AI vs automation only vs Virtual

team
Exercise
• Matcha vs coffee vs water
Two-way ANOVA
Using a Second Factor to Reduce Variance
Caused by Individual Differences (1 of 2)

• Participant characteristics (such as age and gender) may

vary considerably from one person to another
• Participant characteristics can influence the scores
obtained in a study
• Large variance tends to reduce the size of the t statistic or
F-ratio and, therefore, reduces the likelihood of finding
significant mean differences
Using a Second Factor to Reduce Variance
Caused by Individual Differences (2 of 2)

• Solution to the problem of high variance: use the specific variable

associated with consistent individual differences as a second factor
• Instead of one group in each treatment, the researcher divides the
participants into two separate groups within each treatment
• One factor consists of the two treatments and the second factor
is, for example, based on gender or on age (e.g.,
adolescents/adults)
Independent variable (e.g., Treatment) or
quasi-independent variable (e.g., gender)
Dependent variable
Categorical
Independent variable (e.g., Treatment) or
quasi-independent variable (e.g., gender) Dependent variable
Categorical

Independent variable (e.g., Treatment) or

quasi-independent variable (e.g., gender)
Categorical
 Training
Score

IQ Group (high vs low)

Training -> Score

IQ Group -> Score
Treatment * IQ Group – Score
An Overview of the Two-Factor, Independent-
Measures ANOVA (1 of 4)

• Complex analysis of variance

• Two independent variables are manipulated
(factorial ANOVA; only two-factor in this text)
• Both independent variables and quasi-independent variables may be
employed as factors in a two-factor ANOVA
• An independent variable (factor) is manipulated in an experiment
An Overview of the Two-Factor, Independent-
Measures ANOVA (2 of 4)

• A quasi-independent variable (factor) is not manipulated but defines

the groups of scores in a nonexperimental study
• The two-factor ANOVA allows us to examine three types of mean
differences within one analysis
An Overview of the Two-Factor, Independent-
Measures ANOVA (3 of 4)

• Factorial designs
• Consider more than one factor
• We will study two-factor designs only
• Also limited to situations with equal n’s in each group
• Joint impact of factors is considered
• Three hypotheses tested by three F-ratios
• Large F-ratio → greater treatment differences than would be
expected with no treatment effects
An Overview of the Two-Factor, Independent-
Measures ANOVA (4 of 4)

• The three F-ratios have the same basic structure

• Numerator measures treatment mean differences
• Denominator measures treatment mean differences when there is
no treatment effect

variance (differences) between treatments

F=
variance (differences) expected with no treatment effect
 Main Effects

• Mean differences among levels of one factor

• Differences are tested for statistical
significance
• Each factor (factor A and factor B) is
evaluated independently of the other
factor(s) in the study

H 0 :  A1 =  A2 H 0 :  B1 =  B2
H1 :  A1   A2 H1 :  B1   B2
Interactions
• The mean differences between individual
treatment conditions, or cells, are different from
what would be predicted from the overall main
effects of the factors

• H0: There is no interaction between factors A and

B
• H1: There is an interaction between factors A and
B
 Training
Score

IQ Group (high vs low)

Training -> Score

IQ Group -> Score
Treatment * IQ Group – Score
More about Interactions (1 of 2)

• Interaction can be defined as the unique effect of two factors working

together
• Alternative definitions (slightly different perspectives)
• If the two factors are independent, so one factor doesn’t influence
the effect of the other, then there is no interaction
• When the two factors are not independent, so the effect of one
factor depends on the other, then there is interaction, and this can
be seen in a graph
More about Interactions (2 of 2)

• Dependence of factors
• The effect of one factor depends on the level or value of the other
factor
• Unique combinations of the factors produce unique effects
• Interaction can be observed in a graph
• Nonparallel lines (lines that cross, converge or diverge) indicate
that interaction is occurring
• Is called the A × B interaction, or “ the A by B” interaction
Graph Showing
the Treatment
Means for No
Interaction
Graph for an
Interaction
Independence of Main Effects and Interactions

• Three distinct tests

• Main effect of factor A
• Main effect of factor B
• Interaction of A and B
• A separate F test is conducted for each
• Results of one are independent of the others

variance (mean differences ) between treatments

F=
variance (mean differences ) expected if there is no treatment effect
An Example of the Two-Factor ANOVA and
Effect Size

• The A-effect: the main effect of factor A typically evaluates the mean
differences between rows
• The B-effect: the main effect of factor B typically evaluates the mean
differences between columns
• The A × B interaction: evaluates mean differences between treatment
conditions that are not predicted from the overall main effects
An Example of the Two-Factor ANOVA and Effect
Size

• First stage
• Identical to independent samples ANOVA
• Compute SStotal, SSbetween treatments, and
SSwithin treatments
• Second stage
• Partition the SSbetween treatments into three separate components:
differences attributed to factor A, to factor B, and to any remaining
mean differences that define the interaction
Structure of
the Analysis
for a Two-
Factor
ANOVA
Example
• Ackerman and Goldsmith (2011)
• learning performance
• studied text from printed pages vs from a computer screen
• Self-regulated vs Fixed time control
 Factor B: Text Presentation Mode
Paper Computer
Screen
Self-regulated 11 4
8 4
9 8
10 5
Trow = 70
7 4
M=9 M=5
T = 45 T = 25
SS = 10 SS = 12 N = 20
Factor A G = 155
Fixed 10 10 ∑X2 = 1303
Time Control
7 6
10 10
6 10
7 9 Trow = 85
M=8 M=9
T = 40 T = 45
SS = 14 SS = 12
Tcol = 85 Tcol = 70
 Stage 1 of the Two-Factor Analysis
2


G
SS total = X −
2
N

SStotal = 1303 – 1552/20

= 1303 – 1201.25
= 101.75

dftotal = N – 1
= 20 – 1 = 19
 Stage 1 of the Two-Factor Analysis
SS within −treatments =  SSinside each treatment

SSwithin= 10 + 12 + 14 + 12
= 48

dfwithin = Σdfinside each treatment = 4 + 4 + 4 + 4

=16
Stage 1 of the Two-Factor Analysis
T 2 G2
SS between −treatments =  n
−
N
452 252 402 452 1552
SS between= + + + -
5 5 5 5 20
=405 + 125 + 320 + 405 -1201.25
=53.75

or SSbetween = SStotal – SSwithin

SSbetween = 101.75 – 48
= 53.75

dfbetween treatments = k – 1

= k – 1 = number of cells – 1 = 4 – 1 = 3
Stage 2 of the Two-Factor Analysis
• This stage determines the numerators for the three
F-ratios by partitioning SSbetween treatments

Tcol2 G 2
SS B = 
2 2

n
Trow G −
SS A = −
row N ncol N

SS AxB = SS between treatments − SS A − SS B

Stage 2 of the Two-Factor Analysis
• This stage determines the numerators for the three
F-ratios by partitioning SSbetween treatments
2
G2

Trow
SS A = −
nrow N
702 852 1552
SS A = + +
10 10 20
=490 + 722.5 – 1201.25
=11.25

df A = number of rows – 1
=2–1
=1
Stage 2 of the Two-Factor Analysis
Tcol2 G 2
SS B =  −
ncol N

852 702 1552

SS B = + +
10 10 20
=722.5 + 490 – 1201.25
=11.25

df B = number of rows – 1
=2–1
=1
 Stage 2 of the Two-Factor Analysis
SS AxB = SS between treatments − SS A − SS B

SS AxB = SS between – SS A – SS B
=53.75 – 11.25 – 11.25
=31.25

df AxB = df between – df A – df B
=3–1-1
=1

An easier alternative formulat = df A x df B = 1 x 1 = 1

Mean Squares and F-Ratios for the Two-
Factor ANOVA
SS A SS B SS AxB
MS A = MS B = MS AxB =
df A df B df AxB

SS within treatments
MS within treatments =
df within treatments
MS A MS B MS AxB
FA = FB = FAxB =
MS within MS within MS within
Mean Squares for the Two-Factor ANOVA
SS within treatments
MS within treatments =
df within treatments
MS within = 48/16 = 3

SS A SS B SS AxB
MS A = MS B = MS AxB =
df A df B df AxB

MS A = 11.25/1=11.25 MS B = 11.25/1=11.25 MS AxB = 31.25/1 = 31.25

F-Ratios for the Two-Factor ANOVA

MS A MS B MS AxB
FA = FB = FAxB =
MS within MS within MS within

FA = 11.25/3 = 3.75
FB = 11.25/3 = 3.75
FAxB = 31.25/3 = 10.42
Measuring Effect Size for the Two-Factor
ANOVA
• Compute a value for η2 (eta squared), the
percentage of variance explained by the treatment
effects
SS A SS A
 =
2
= = 11.25/11.25 + 48 = 0.190
− SS B − SS AxB SS A + SS within treatments
A
SS total

SS B SS B
 B2 = = = 11.25/11.25 + 48 = 0.190
SStotal − SS A − SS AxB SS B + SS within _ treatments

SS AxB SS AxB
 AxB
2
= = =31.25/31.25+48 = 0.394
SS total − SS A − SS B SS AxB + SS within treatments
Source Sum of Square df Mean Square F
(SS)
Between treatments 53.75 3
Factor A (time) 11.25 1 11.25 F(1,16) = 3.75
Factor B (mode) 11.25 1 11.25 F(1,16) = 3.75
AxB 31.25 1 31.25 F(1,16) = 10.42
Within Groups 48 16 3
Total 101.75 19

Summary table of a Two-Factor ANOVA

 Reporting the Results of
a Two-Factor ANOVA

• Report means and standard deviations (usually in a table or graph due

to the complexity of the design)
• Report the results of all three hypothesis tests (A and B main effects; A
× B interaction)
• For each test include F, df, p value, and η2
• E.g.,The two-factor analysis of variance showed no significant main
effect for time control, F(1, 16) = 3.75, p > .05, η2 = 0.190. However, the
interaction between factors was significant, F(1, 16) = 10.41, p < .01, η2
= 0.394
Interpreting the Results from a Two-Factor
ANOVA

• Focus on the overall pattern of results

• Significant interactions require particular attention because even if you
understand the main effects, interactions go beyond what main effects
alone can explain
• Extensive practice is typically required to be able to clearly articulate
results that include a significant interaction
 • The validity of the ANOVA depends on three

Assumptions assumptions common to other hypothesis

tests (t test and single-factor ANOVA)
• The observations within each sample
for the Two- must be independent of each other
• The populations from which the samples
Factor are selected must be normally
distributed
ANOVA • The populations from which the samples
are selected must have equal variances
(homogeneity of variance)
The ANOVA for
an
Independent-
Measures Two-
Factor Design
The End