Analysis of Variance

[email protected]
39GCUAX2OS

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 Learning Objectives
In this chapter, you learn:
• The basic concepts of experimental design
• How to use one-way analysis of variance to test for differences among
the means of several groups
[email protected]
39GCUAX2OS
• How to use two-way analysis of variance and interpret the interaction
effect
• How to perform multiple comparisons in a one-way analysis of variance
and a two-way analysis of variance

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 General ANOVA Setting
• Investigator controls one or more factors of interest
• Each factor contains two or more levels
• Levels can be numerical or categorical
[email protected]
• Different levels produce different groups
39GCUAX2OS
• Think of each group as a sample from a different
population
• Observe effects on the dependent variable
• Are the groups the same?
• Experimental design: the plan used to collect the data

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 Completely Randomized Design

• Experimental units (subjects) are assigned randomly to groups

• Subjects are assumed homogeneous
• Only one factor or independent variable
• With two or more levels
[email protected]
39GCUAX2OS

• Analyzed by one-factor analysis of variance (ANOVA)

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 One-Way Analysis of Variance
• Evaluate the difference among the means of three or
more groups
Examples: Number of accidents for 1st, 2nd, and 3rd shift
[email protected]
39GCUAX2OS Expected mileage for five brands of tires

• Assumptions
• Populations are normally distributed
• Populations have equal variances
• Samples are randomly and independently drawn

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 Hypotheses of One-Way ANOVA

• H0 : μ1 = μ2 = μ3 =  = μc
• All population means are equal
• i.e., no factor effect (no variation in means among groups)
[email protected]
39GCUAX2OS
•
H• 1At: Not
leastall
oneofpopulation
the population
mean ismeans are equal
different
• i.e., there is a factor effect
• Does not mean that all population means are different
(some pairs may be the same)

Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 One-Way ANOVA
H0 : μ1 = μ2 = μ3 =  = μc
H1 : Not all μ j are equal
The Null Hypothesis is True
[email protected]
All Means are the same:
39GCUAX2OS
(No Factor Effect)

μ1 = μ 2 = μ 3
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
 One-Way ANOVA
(continued)
H0 : μ1 = μ2 = μ3 =  = μc
H1 : Not all μ j are equal
The Null Hypothesis is NOT true
[email protected]
39GCUAX2OS
At least one of the means is different
(Factor Effect is present)

or

μ1 = μ2  μ3 μ1  μ2  μ3
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Partitioning the Variation
• Total variation can be split into two parts:

SST = SSA + SSW

[email protected]
39GCUAX2OS

SST = Total Sum of Squares

(Total variation)
SSA = Sum of Squares Among Groups
(Among-group variation)
SSW = Sum of Squares Within Groups
(Within-group variation)

This file is meant for personal use by [email protected] only.

Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Partitioning the Variation

SST = SSA + SSW

Total Variation = the aggregate variation of the individual

[email protected]
39GCUAX2OS data values across the various factor levels (SST)

Among-Group Variation = variation among the factor

sample means (SSA)

Within-Group Variation = variation that exists among

the data values within a particular factor level (SSW)

This file is meant for personal use by [email protected] only.

Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Partition of Total Variation
Total Variation (SST)

[email protected]
39GCUAX2OS

Variation Due to Variation Due to Random

= Factor (SSA) + Error (SSW)

This file is meant for personal use by [email protected] only.

Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Total Sum of Squares
SST = SSA + SSW
c nj

SST =  ( Xij − X) 2
[email protected]
39GCUAX2OS
Where: j=1 i=1

SST = Total sum of squares

c = number of groups or levels
nj = number of observations in group j
Xij = ith observation from group j
X = grand mean (mean of all data values)
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Total Variation
(continued)

2 2 2
SST = ( X 11 − X ) + ( X 12 − X ) +    + ( X cn − X )
c

[email protected]
39GCUAX2OS
Response, X

Group 1 Group 2 Group 3

This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Among-Group Variation
SST = SSA + SSW
c
SSA =  n j ( X j − X)2
[email protected]
39GCUAX2OS
j=1
Where:
SSA = Sum of squares among groups
c = number of groups
nj = sample size from group j
Xj = sample mean from group j
X = grand mean (mean of all data values)
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Among-Group Variation
(continued)
c
SSA =  n j ( X j − X)2
j=1

SSA
Variation Due to
MSA =
[email protected]
39GCUAX2OS
Differences Among Groups
c −1
Mean Square Among =
SSA/degrees of freedom

i j
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Among-Group Variation
(continued)

SSA = n1 (X1 − X) + n 2 (X 2 − X) +    + n c (X c − X)
2 2 2

[email protected]
39GCUAX2OS
Response, X

X3
X2 X
X1

Group 1 Group 2 Group 3

This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Within-Group Variation
SST = SSA + SSW
c nj

SSW =   ( Xij − X j ) 2

[email protected]
39GCUAX2OS
j=1 i=1
Where:
SSW = Sum of squares within groups
c = number of groups
nj = sample size from group j
Xj = sample mean from group j
Xij = ith observation in group j
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Within-Group Variation
(continued)

c nj

SSW =   ( Xij − X j )2
j=1 i=1
SSW
Summing the variation
MSW =
[email protected]
39GCUAX2OS
within each group and then
adding over all groups n−c
Mean Square Within =
SSW/degrees of freedom

μj
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Within-Group Variation
(continued)

SSW = (X11 − X1 ) + (X12 − X 2 ) +    + (Xcn c − Xc )

2 2 2

[email protected]
39GCUAX2OS Response, X

X3
X2
X1

Group 1 Group 2 Group 3

This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Obtaining the Mean Squares
The Mean Squares are obtained by dividing the various
sum of squares by their associated degrees of freedom

SSA Mean Square Among

MSA = (d.f. = c-1)
[email protected]
39GCUAX2OS c −1
SSW
MSW = Mean Square Within
n−c (d.f. = n-c)

SST
MST = Mean Square Total
n −1 (d.f. = n-1)
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 One-Way ANOVA Table

Source of Degrees of Sum Of Mean Square F

Variation Freedom Squares (Variance)

Among SSA FSTAT =

c-1 SSA MSA =
[email protected]
39GCUAX2OS Groups c-1
MSA
Within SSW
n-c SSW MSW = MSW
Groups n-c

Total n–1 SST

c = number of groups
n = sum of the sample sizes from all groups
df = degrees of freedom
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 One-Way ANOVA
F Test Statistic
H0: μ1= μ2 = … = μc
H1: At least two population means are different

• Test statistic
MSA
[email protected]
39GCUAX2OS FSTAT =
MSW
MSA is mean squares among groups
MSW is mean squares within groups

• Degrees of freedom
• df1 = c – 1 (c = number of groups)
• df2 = n – c (n = sum of sample sizes from all populations)

This file is meant for personal use by [email protected] only.

Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Interpreting One-Way ANOVA
F Statistic
• The F statistic is the ratio of the among estimate
of variance and the within estimate of variance
• The ratio must always be positive
• df1 = c -1 will typically be small
[email protected]
39GCUAX2OS • df2 = n - c will typically be large

Decision Rule:
◼ Reject H0 if FSTAT > Fα, 
otherwise do not reject
H0 0 Do not Reject H0
reject H0
This file is meant for personal use by [email protected] only.
Fα
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 One-Way ANOVA
F Test Example

You want to see if three Club 1 Club 2 Club 3

different golf clubs yield 254 234 200
different distances. You 263 218 222
randomly select five 241 235 197
[email protected]
39GCUAX2OS measurements from trials on an 237 227 206
automated driving machine for 251 216 204
each club. At the 0.05
significance level, is there a
difference in mean distance?

This file is meant for personal use by [email protected] only.

Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 One-Way ANOVA Example: Scatter
Plot
Distance
Club 1 Club 2 Club 3 270
254 234 200 260 •
263 218 222 ••
241 235 197
250 X1
[email protected]
240 •
39GCUAX2OS
237 227 206 • ••
251 216 204 230
• X
220 ••
X2 •
210
x1 = 249.2 x 2 = 226.0 x 3 = 205.8
•• X3
200 •
•
x = 227.0 190

1
This file is meant for personal use by [email protected] only.
2 3
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or Club
distribution
action. prohibited.
 One-Way ANOVA Example
Computations
Club 1 Club 2 Club 3 X1 = 249.2 n1 = 5
254 234 200 X2 = 226.0 n2 = 5
263 218 222
X3 = 205.8 n3 = 5
241 235 197
[email protected]
39GCUAX2OS 237 227 206 n = 15
X = 227.0
251 216 204 c=3
SSA = 5 (249.2 – 227)2 + 5 (226 – 227)2 + 5 (205.8 – 227)2 = 4716.4
SSW = (254 – 249.2)2 + (263 – 249.2)2 +…+ (204 – 205.8)2 = 1119.6

MSA = 4716.4 / (3-1) = 2358.2 2358.2

FSTAT = = 25.275
MSW = 1119.6 / (15-3) = 93.3 93.3
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 One-Way ANOVA Example Solution

H0: μ1 = μ2 = μ3 Test Statistic:

H1: μj not all equal
MSA 2358.2
 = 0.05 FSTAT = = = 25.275
MSW 93.3
[email protected]
df1= 2 df2 = 12
39GCUAX2OS

Critical Decision:
Value:
Reject H0 at  = 0.05
Fα = 3.89
 = .05 Conclusion:
There is evidence that
0 Do not Reject H 0
at least one μj differs
reject H0
FSTAT = 25.275 from the rest
Fα = 3.89
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 One-Way ANOVA
Excel Output

SUMMARY
Groups Count Sum Average Variance
Club 1 5 1246 249.2 108.2
Club 2 5 1130 226 77.5
[email protected]
39GCUAX2OS
Club 3 5 1029 205.8 94.2
ANOVA
Source of
SS df MS F P-value F crit
Variation
Between
4716.4 2 2358.2 25.275 0.0000 3.89
Groups
Within
1119.6 12 93.3
Groups
Total 5836.0 14
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 ANOVA Assumptions
• Randomness and Independence
• Select random samples from the c groups (or randomly
assign the levels)
[email protected]
39GCUAX2OS
• Normality
• The sample values for each group are from a normal
population
• Homogeneity of Variance
• All populations sampled from have the same variance
• Can be tested with Levene’s Test

This file is meant for personal use by [email protected] only.

Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Factorial Design:
Two-Way ANOVA

• Examines the effect of

• Two factors of interest on the dependent variable
• e.g., Percent carbonation and line speed on soft drink
bottling process
[email protected]
39GCUAX2OS • Interaction between the different levels of these
two factors
• e.g., Does the effect of one particular carbonation level
depend on which level the line speed is set?

This file is meant for personal use by [email protected] only.

Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Two-Way ANOVA
(continued)

• Assumptions

• Populations are normally distributed

[email protected]
39GCUAX2OS
• Populations have equal variances
• Independent random samples are drawn

This file is meant for personal use by [email protected] only.

Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Two-Way ANOVA
Sources of Variation
Two Factors of interest: A and B
r = number of levels of factor A
[email protected]
39GCUAX2OS
c = number of levels of factor B
n’ = number of replications for each cell
n = total number of observations in all cells
n = (r)(c)(n’)
Xijk = value of the kth observation of level i of
factor A and level j of factor B
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Two-Way ANOVA
Sources of Variation (continued)

SST = SSA + SSB + SSAB + SSE Degrees of

Freedom:
SSA r–1
Factor A Variation
[email protected]
39GCUAX2OS
SST SSB c–1
Factor B Variation
Total Variation
SSAB
Variation due to interaction (r – 1)(c – 1)
between A and B
n-1
SSE rc(n’ – 1)
Random variation (Error)
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Two-Way ANOVA Equations

Total Variation: r c n
SST =  ( Xijk − X) 2

i=1 j=1 k =1
[email protected]
39GCUAX2OS
Factor A Variation: r
SSA = cn  ( Xi.. − X)
 2

i=1

Factor B Variation: c
SSB = rn ( X. j. − X)2
j=1
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Two-Way ANOVA Equations
(continued)

Interaction Variation:
r c
[email protected]
39GCUAX2OS SSAB = n ( Xij. − Xi.. − X.j. + X)2
i =1 j=1

Sum of Squares Error:

r c n
SSE =  ( Xijk − Xij. ) 2

i =1 j =1 k =1

This file is meant for personal use by [email protected] only.

Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Two-Way ANOVA Equations
(continued)
r c n

where:  X
i=1 j=1 k =1
ijk

X= = Grand Mean
c n
rcn
 X
j=1 k =1
ijk
[email protected]
39GCUAX2OS Xi.. = = Mean of ith level of factor A (i = 1, 2, ..., r)
cn
r n

 X ijk
X. j. = i=1 k =1
= Mean of jth level of factor B (j = 1, 2, ..., c)
rn
n
Xijk
Xij. = 
r = number of levels of factor A
= Mean of cell ij
k =1 n
c = number of levels of factor B
n’ = number of replications in each cell
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Mean Square Calculations

SSA
MSA = Mean square factor A =
r −1

SSB
[email protected] MSB = Mean square factor B =
39GCUAX2OS
c −1

SSAB
MSAB = Mean square interactio n =
(r − 1)(c − 1)

SSE
MSE = Mean square error =
rc(n'−1)
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Two-Way ANOVA:
The F Test Statistics
F Test for Factor A Effect
H0: μ1..= μ2.. = μ3..= • • = µr..
MSA Reject H0 if
H1: Not all μi.. are equal FSTAT =
MSE FSTAT > Fα

[email protected]
39GCUAX2OS F Test for Factor B Effect
H0: μ.1. = μ.2. = μ.3.= • • = µ.c.
MSB Reject H0 if
H1: Not all μ.j. are equal FSTAT =
MSE FSTAT > Fα

F Test for Interaction Effect

H0: the interaction of A and B is
equal to zero
MSAB
H1: interaction of A and B is not FSTAT = Reject H0 if
MSE FSTAT > Fα
zero
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Two-Way ANOVA
Summary Table
Source of Sum of Degrees of Mean
F
Variation Squares Freedom Squares

MSA MSA
Factor A SSA r–1
[email protected]
= SSA /(r – 1) MSE
39GCUAX2OS
MSB MSB
Factor B SSB c–1
= SSB /(c – 1) MSE

AB MSAB MSAB
SSAB (r – 1)(c – 1)
(Interaction) = SSAB / (r – 1)(c – 1) MSE

MSE =
Error SSE rc(n’ – 1)
SSE/rc(n’ – 1)
Total SST n–1
This file is meant for personal use by [email protected] only.
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Features of Two-Way ANOVA
F Test
• Degrees of freedom always add up
• n-1 = rc(n’-1) + (r-1) + (c-1) + (r-1)(c-1)
• Total = error + factor A + factor B + interaction
[email protected]
39GCUAX2OS
• The denominators of the F Test are always the same
but the numerators are different
• The sums of squares always add up
• SST = SSE + SSA + SSB + SSAB
• Total = error + factor A + factor B + interaction

This file is meant for personal use by [email protected] only.

Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Examples:
Interaction vs. No Interaction
◼ Interaction is present:
• No interaction: line
segments are parallel some line segments
not parallel

[email protected]
39GCUAX2OS Factor B Level 1
Mean Response

Mean Response
Factor B Level 1
Factor B Level 3

Factor B Level 2
Factor B Level 2
Factor B Level 3

Factor A Levels Factor A Levels

This file is meant for personal use by [email protected] only.

Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
 Summary
In this chapter we discussed
• The one-way analysis of variance
• The logic of ANOVA
• ANOVA assumptions
[email protected]
39GCUAX2OS
• F test for difference in c means
• The two-way analysis of variance
• Examined effects of multiple factors
• Examined interaction between factors

This file is meant for personal use by [email protected] only.

Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.