Analysis Of Variance

by
Dr. A Vadivel
“
 ANOVA
• Analysis of variance, known by the acronym ANOVA, allows statistical comparison among
samples taken from many populations.

• In ANOVA, the comparison is typically the result of an experiment.

• For example, the management of a general merchandiser might be brainstorming ways of improving
sales of mobile electronics items. At Arlingtons, the management decided to try selling those items
in four different in-store locations and then observe what the sales would be in each of those
locations.

• The basis for an ANOVA experiment is called the factor, which in the Arlingtons scenario is in-
store location. The statistical use of the word “factor” complements the everyday usage, illustrated
by a question such as “How much of a factor is in-store location in determining mobile electronics
sales?”
 Cont’d…
• The actual different locations (in-aisle, special front, endcap, and expert counter) are the levels
of the factor.

• Levels of a factor are analogous to the categories of a categorical variable, but you call in-
store location a factor and not a categorical variable because the variable under study is mobile
electronics sales. Levels provide the basis of comparison by dividing the variable under study
into groups.

• In the Arlingtons scenario, the groups are the stores selling the mobile electronics items in
the mobile electronics aisle, the stores selling those items at the special front location, the stores
selling those items at the endcap location, and the stores selling those items at the expert counter.
 Cont’d..
When performing ANOVA analysis, among the types of experiments that can be conducted are:

• Completely randomized design: An experiment with only one factor.

• Factorial design: An experiment in which more than one factor is considered. This chapter
discusses two-way ANOVA that involves two factors as an example of this type of design.
(Arlingtons considering the effects of allowing mobile payments while also experimenting with in-
store location would be an example of a factorial design.)

• Randomized block design: An experiment in which the members of each group have been placed
in blocks either by being matched or subjected to repeated measurements aswas done with the two
populations of a paired t test.
 Cont’d…
• While ANOVA literally does analyze variation, the purpose of ANOVA is to reach conclusions about
possible differences among the means of each group, analogous to the hypothesis tests of the previous
chapter.

• Every ANOVA design uses samples that represent each group and subdivides the total variation observed
across all samples (all groups) toward the goal of analyzing possible differences among the means of each
group.

• How this subdivision, called partitioning, works is a function of the design being used, but total variation,
represented by the quantity sum of squares total (SST), will always be the starting point. As with other
statistical methods, ANOVA requires making assumptions about the populations that the groups represent..
The Completely Randomized Design: One-Way ANOVA

• The completely randomized design is the ANOVA method that analyzes a single factor.

• You execute this design using the statistical method one-way ANOVA.

• One-way ANOVA is a two-part process.

• You first determine if there is a significant difference among the group means. If you reject
the null hypothesis that there is no difference among the means, you continue with a second
method that seeks to identify the groups whose means are significantly different from the
other group means.
Analyzing Variation in One-Way ANOVA

• In one-way ANOVA, to analyze variation towards the goal of determining possible

differences among the group means, you partition the total variation into variation that
is due to differences among the groups and variation that is due to differences within
the groups .

• The within-group variation (SSW) measures random variation. The among-group

variation (SSA) measures differences from group to group. The symbol n represents
the number of values in all groups and the symbol c represents the number of groups.
 Cont’d…

Assuming that the c groups represent populations whose values are randomly and independently selected,
follow a normal distribution, and have equal variances, the null hypothesis of no differences in the population
means:
is tested against the alternative that not all the c population means are equal:
 Cont’d…

• To perform an ANOVA test of equality of population means, you subdivide the total variation in
the values into two parts—that which is due to variation among the groups and that which is due
to variation within the groups.

• The total variation is represented by the sum of squares total (SST). Because the population
means of the c groups are assumed to be equal under the null hypothesis, you compute the total
variation among all the values by summing the squared differences between each individual value
and the grand mean, . The grand mean is the mean of all the values in all the groups combined.
ANOVA
 Cont’d…
• You compute the among-group variation, usually called the sum of squares among groups (SSA),
by summing the squared differences between the sample mean of each group,, and the grand mean, ,
weighted by the sample size, , in each group. Equation (11.2) shows the computation of the among-
group variation.
 Cont’d…
The within-group variation, usually called the sum of squares within groups (SSW), measures the
difference between each value and the mean of its own group and sums the squares of these
differences over all groups. Equation (11.3) shows the computation of the within-group
variation.
 Cont’d…
• Because you are comparing c groups, there are c - 1 degrees of freedom associated with the sum of squares among
groups.

• Because each of the c groups contributes - 1 degrees of freedom, there are n - c degrees of freedom associated with
the sum of squares within groups. In addition, there are n - 1 degrees of freedom associated with the sum of squares
total because you are comparing each value, , to the grand mean, , based on all n values. If you divide each of these
sums of squares by its respective degrees of freedom, you have three variances, which in ANOVA are known as
mean squares: MSA (mean square among), MSW (mean square within), and MST (mean square total).
F-Test
F-Test
F-Test
F-Test
One-Way ANOVA F-Test Assumptions
One-Way ANOVA F-Test Assumptions: Contd…
Levene Test
The Turkey-Kramer Procedure
The Turkey-Kramer Procedure
The Turkey-Kramer Procedure
Example_01
Example_01( Cont’d…)
Example_01( Cont’d…)
The Factorial Design: Two- Way ANOVA

• In this section, the single-factor completely randomized design is extended to the two-factor
factorial design, in which two factors are simultaneously evaluated.

• Each factor is evaluated at two or more levels.

• For example, in the Arlingtons scenario on page 372, the company faces the business problem
of simultaneously evaluating four locations and the effectiveness of providing mobile payment
to determine which location should be used and whether mobile payment should be made
available. Although this section uses only two factors, you can extend factorial designs to three
or more factors.
 Cont’d…

To analyze data from a two-factor factorial design, you use two-way ANOVA. The following
definitions are needed to develop the two-way ANOVA procedure:

r = number of levels of factor A

c = number of levels of factor B
n = number of values (replicates) for each cell (combination of a particular level of factor A
and a particular level of factor B)
n = number of values in the entire experiment (where n = rcn’)
Xijk = value of the kth observation for level i of factor A and level j of factor B
 Cont’d…

To analyze data from a two-factor factorial design, you use two-way ANOVA. The following definitions are
needed to develop the two-way ANOVA procedure:
 Cont’d…
.
• Because of the complexity of these computations, you should only use computerized methods when
performing this analysis.

• However, to help explain the two-way ANOVA, the decomposition of the total variation is illustrated.
In this discussion, only cases in which there are an equal number of values (also called replicates)
(sample sizes n’) for each combination of the levels of factor A with those of factor B are considered.
 Factor and Interaction Effects

• There is an interaction between factors A and B if the effect of factor A is different for various
levels of factor B.

• Thus, when dividing the total variation into different sources of variation, you need to account
for a possible interaction effect, as well as for factor A, factor B, and random variation.

• To accomplish this, the total variation (SST) is subdivided into sum of squares due to factor A
(or SSA), sum of squares due to factor B (or SSB), sum of squares due to the interaction effect
of A and B (or SSAB), and sum of squares due to random variation (or SSE).
 Cont’d…

This decomposition of the total variation (SST) is displayed in Figure 11.9.

 Cont’d…

The sum of squares total (SST) represents the total variation among all the values around the grand mean.
Equation (11.7) shows the computation for total variation.

The sum of squares due to factor A (SSA) represents the differences among the various levels of
factor A and the grand mean. Equation (11.8) shows the computation for factor A variation.
 Cont’d…

The sum of squares due to factor B (SSB) represents the differences among the various levels of factor B and the
grand mean. Equation (11.9) shows the computation for factor B variation.

The sum of squares due to interaction (SSAB) represents the interacting effect of specific combinations of
factor A and factor B. Equation (11.10) shows the computation for interaction variation.
 Cont’d…

The sum of squares error (SSE) represents random variation—that is, the differences among the values
within each cell and the corresponding cell mean. Equation (11.11) shows the computation for random
variation.
 Cont’d…

• Because there are r levels of factor A, there are r - 1 degree of freedom associated with SSA.

• Similarly, because there are c levels of factor B, there are c - 1 degrees of freedom associated with SSB.
Because there are n’ replicates in each of the rc cells, there are rc (n’ – 1) degrees of freedom associated
with the SSE term. Carrying this further, there are n - 1 degrees of freedom associated with the sum of
squares total (SST) because you are comparing each value, X ijk, to the grand mean, X, based on all n
values.

• Therefore, because the degrees of freedom for each of the sources of variation must add to the degrees
of freedom for the total variation (SST), you can calculate the degrees of freedom for the interaction
component (SSAB) by subtraction. The degrees of freedom for interaction are (r – 1)(c – 1).
 Cont’d…
If you divide each sum of squares by its associated degrees of freedom, you have the four variances or
mean square terms (MSA, MSB, MSAB, and MSE). Equations (11.12a–d) give the mean square terms
needed for the two-way ANOVA table.
 Testing For Factor and Interaction Effects

There are three different tests to perform in a two-way ANOVA:

• A test of the hypothesis of no difference due to factor A
• A test of the hypothesis of no difference due to factor B
• A test of the hypothesis of no interaction of factors A and B

To test the hypothesis of no difference due to factor A:

against the alternative:

 Cont’d…

you use the FSTAT test statistic in Equation (11.13).

You reject the null hypothesis at the a level of significance if

where Fα is the upper-tail critical value from an F distribution with r - 1 and rc(n’ – 1) degrees of freedom.
To test the hypothesis of no difference due to factor B:
 ANOVA

against the alternative:

you use the FSTAT test statistic in Equation (11.14).

You reject the null hypothesis at the a level of significance if

 Cont’d…

where Fα is the upper-tail critical value from an F distribution with c - 1 and rc(n’ – 1) degrees of freedom.
To test the hypothesis of no interaction of factors A and B:
H0: The interaction of A and B is equal to zero
against the alternative: A
H1: The interaction of A and B is not equal to zero

you use the FSTAT test statistic in Equation (11.15).

You reject the null hypothesis at the a level of significance if

 Cont’d…

where Fα is the upper-tail critical value from an F distribution with (r – 1)(c – 1) and rc(n’ – 1) degrees of freedom.
Table 11.6 presents the entire two-way ANOVA table.
 Cont’d…

• To interpret the results, you start by testing whether there is an interaction effect between factor A (mobile
payments) and factor B (in-store locations).

• If the interaction effect is significant, further analysis will focus on this interaction. If the interaction effect
is not significant, you can focus on the main effects—the potential effect of permitting mobile payment
(factor A) and the potential differences in in-store locations (factor B).

• Using the 0.05 level of significance, to determine whether there is evidence of an interaction effect, you
reject the null hypothesis of no interaction between mobile payments and in-store locations if the computed
FSTAT statistic is greater than 2.9011, the upper-tail critical value from the F distribution, with 3 and 32
degrees of freedom.
 Cont’d…

Because FSTAT = 0.4100 < 2.9011 or the p-value = 0.7469 > 0.05, you do not reject H 0. You conclude that there
is insufficient evidence of an interaction effect between mobile payment and in-store location. You can now
focus on the main effects.
 Cont’d…

• Using the 0.05 level of significance and testing whether there is an effect due to mobile payment
options (yes or no) (factor A), you reject the null hypothesis if the computed F STAT test statistic is
greater than 4.1491, the upper-tail critical value from the F distribution with 1 and 32 degrees of
freedom .

• Because FSTAT = 62.7258 > 4.1491 or the p-value = 0.0000 < 0.05, you reject H 0. You conclude that
there is evidence of a difference in the mean sales when mobile payment methods are permitted as
compared to when they are not. Because the mean sales when mobile payment methods are permitted
is 31.523 and is 30.806 when they are not, you can conclude that permitting mobile payment methods
has led to an increase in mean sales.
 Cont’d…

Using the 0.05 level of significance and testing for a difference among the in-store locations (factor B), you reject
the null hypothesis of no difference if the computed F STAT test statistic is greater than 2.9011, the upper-tail critical
value from the F distribution with 3 degrees of freedom in the numerator and 32 degrees of freedom in the
denominator (see Figures 11.10 and 11.11). Because F STAT = 98.9631 > 2.9011 or the p-value = 0.0000 < 0.05, you
reject H0. You conclude that there is evidence of a difference in the mean sales among the four in-store locations.
Two-Way Classification
Cont’d…
Cont’d…
Cont’d…
Example_02
Example_02_ Solution
Example_02(Cont’d…)
Example_02(Cont’d…)
Example_02(Cont’d…)
Power of the Test

• The power of a hypothesis test is the probability that you correctly reject a false
null hypothesis.
• The power of the test is affected by the level of significance, the sample size, and
whether the test is one-tail or two-tail.