1

INSE 6220 -- Week 10

Advanced Statistical Approaches to Quality

• Experimental Design
• Design of Experiments in Process Improvement
• Factorial Experiments
• Factorial Design

Concordia University
 2

Introduction to Design of Experiments

• An experiment is a test or a series of tests
• Experiments are used widely in the engineering world
▪ Process characterization & optimization
▪ Evaluation of material properties
▪ Product design & development
▪ Component & system tolerance determination
• “All experiments are designed experiments, some are poorly designed, some are well-designed”
• Controllable factors are those input parameters that can
be modified in an experiment or process. For example, in
cooking rice, these factors include the quantity and quality
of the rice and the quantity of water used for boiling.

• Uncontrollable input factors are those parameters that

cannot be changed. In the rice-cooking example, this may
be the temperature in the kitchen

• Responses, or output measures, are the elements of the

process outcome that gage the desired effect. In the
cooking example, the taste and texture of the rice are the
responses.
 3

Experimental Design
➢ Objectives of an experiment
may include:

1) Determining which variables are

most influential on response y.
2) Determining where to set influential
x’s so that y is near nominal
requirement. (which level?)
3) Determining where to set influential
x’s so that variability in y is small.
4) Determining where to set influential
x’s so that effects of uncontrollable
variables z are minimized.
 4

Statistical Analysis
• A factorial experiment is one that investigates the effects of two or more independent variables
(factors) on a single dependent variable (response).
• By a factor, we mean a discrete variable used to classify experimental units, such as
temperature, time, or pressure that may be varied from trial to trial.
• Modeling real world phenomena often requires more than just one factor. ANOVA can be
extended to handle the two-factor factorial experiment. Let the two factors be denoted A and B,
with a levels of factor A and b levels of factor B. The experiment is replicated n times.

i = 1, 2,..., a

yijk =  +  i +  j + ( )ij +  ijk  j = 1, 2,..., b
k = 1, 2,..., n

 5

Statistical Analysis
Let yi .. denote the total of the observations at the i-th level of factor A, y. j . denote
the total of the observations at the j-th level of factor B, yij . denote the total of the
observations in the ij-th cell of Table in previous slide, and y... denote the grand
total of all the observations.

Define yi.. , y. j . , yij . , and y... as the corresponding row, column, cell, and grand
averages. That is,
b n
yi..
yi.. =  yijk yi.. = i = 1, 2,..., a
j =1 k =1 bn
a n y. j .
y. j . =  yijk y. j . = j = 1, 2,..., b
i =1 k =1 an
n yij .
yij . =  yijk yij . =
k =1 n
a b n
y...
y... =  yijk y... =
i =1 j =1 k =1 abn
 6

Hypotheses in Two-Way ANOVA

 7

Total Variability Decomposition

The analysis of variance decomposes the total corrected sum of squares
a b n
SST =  ( yijk − y... ) 2
i =1 j =1 k =1

as follows
a b n a b

 ( y
i =1 j =1 k =1
ijk − y... ) = bn ( yi.. − y... ) + an ( y. j . − y... ) 2
2

i =1
2

j =1
a b
+ n ( yij . − yi.. − y. j . + y... )2
i =1 j =1
a b n
+  ( yijk − yij . )2
i =1 j =1 k =1

Or symbolically
SST = SS A + SS B + SS AB + SS E
 8

Statistical Analysis
The corresponding degree of freedom decomposition is

abn −1 = (a −1) + (b −1) + (a −1)(b −1) + ab(n −1)

This decomposition is usually summarized in an analysis of variance table for a

two-factor factorial, fixed effects model, such as the one shown in the Table below:

Source of Sum of Degrees of

Variation Squares Freedom Mean Square F0
A SSA a-1 SS MSA
MS A = A F0 =
a −1 MSE
B SSB b-1 SS MSB
MS B = B F0 =
b −1 MSE
Interaction SSAB (a – 1)(b – 1) SS AB MSAB
MS AB = F0 =
(a − 1)( b − 1) MSE
Error SSE ab(n-1) SS E
MS E =
ab(n − 1)
Total SST abn - 1
 9

Statistical Analysis
The ANOVA is usually performed with computer software, although simple computing
formulas for the sums of squares may be obtained easily. The computing formulas for
these sums of squares follow.
a b
y...2n
SST =  y − 2
ijk
i =1 j =1 k =1 abn
• Main effects
yi2.. y...2
a
SS A =  −
i =1 bn abn
b y.2j .
y...2
SS B =  −
j =1 an abn

• Interaction a b yij2.
y...2
SS AB =  − − SS A − SS B
i =1 j =1 n abn

• Error SS E = SST − SS A − SS B − SS AB
 10

Example
Aircraft primer paints are applied to aluminum surfaces by two methods: dipping and
spraying. The purpose of the primer is to improve paint adhesion; some parts can be primed
using either application method. An engineer interested in learning whether three different
primers differ in their adhesion properties performed a factorial experiment to investigate the
effect of paint primer type and application method on paint adhesion. Three specimens were
painted with each primer using each application method, a finish paint was applied, and the
adhesion force was measured. The 18 runs from this experiment were run in random order.
The resulting data are shown in Table 12-1. The circled numbers in the cells are the cell
totals. The objective of the experiment was to determine which combination of primer paint
and application method produced the highest adhesion force. It would be desirable if at least
one of the primers produced high adhesion force regardless of application method, as this
method add some flexibility to the manufacturing process.
 11

Example (cont.)
The ANOVA described above may be applied to the aircraft primer
paint experiment. The sums of squares required are:
a b n
y...2
SST =  y − 2
ijk
i =1 j =1 k =1 abn
(89.8) 2
= (4.0) + (4.5) + ... + (5.0) −
2 2 2
= 10.72
18
a
yi2.. y...2
SSprimers =  −
i =1 bn abn
(28.7) 2 + (34.1) 2 + (27.0) 2 (89.8) 2
= − = 4.58
6 18
b y2
y...2
SSmethods =  −
. j.

j =1 an abn
2
(40.2) 2 + (49.6) 2 (89.8) 2
= − = 4.91
9 18
 12

a b yij2.
y...2
SSinteration =  − − SSprimers − SSmethods
i =1 j =1 n abn
(12.8) 2 + (15.9) 2 + (11.5) 2 + (15.9)2 + (18.2)2 + (15.5) 2 (89.8)2
= − − 4.58 − 4.91
3 18
= 0.24
and

SS E = SST − SS primers − SS methods − SSinteration

= 10.72 − 4.58 − 4.91 − 0.24
= 0.99

Sum of Degrees of
Source of Variation Squares Freedom Mean Square F0
Primer types 4.58 2 2.29 28.63
Application methods 4.91 1 4.91 61.38
Interaction 0.24 2 0.12 1.5
Error 0.99 12 0.08
Total 10.72 17
13
 14

Python code
#data from the adhesion force experiment
#FactorA = 3levels and FactorB = 2levels -> 3^2 = 9
df = pd.DataFrame({'AM': np.repeat(['Dipping', 'Spraying'], 9),
'PT': np.tile(np.repeat(['1', '2', '3'], 3), 2),
'force': [4.0, 4.5, 4.3,
5.6, 4.9, 5.4,
3.8, 3.7, 4.0,
5.4, 4.9, 5.6,
5.8, 6.1, 6.3,
5.5, 5.0, 5.0]})

import statsmodels.api as sm
from statsmodels.formula.api import ols
model = ols('force ~ C(PT) + C(AM) + C(PT):C(AM)', data=df).fit()
sm.stats.anova_lm(model, typ=2)
 15

Residual Analysis
Just as in the single-factor experiments, the residuals from a factorial experiment
play an important role in assessing model adequacy. The residuals from a two-
factor factorial are
eijk = yijk − yˆijk
= yijk − yij .

That is, the residuals are simply the difference between the observations and the
corresponding cell averages.

Table: Residuals for the Aircraft Primer Paint Experiment

Primer Type Application Method

Dipping Spraying
1 -0.26, 0.23, 0.03 0.10, -0.40, 0.30
2 0.30, -0.40, 0.10 -0.26, 0.03, 0.23
3 -0.03, -0.13, 0.16 0.34, -0.17, -0.17
 16

Residual Analysis
 17

Two-Way ANOVA with One Replicate

Two-way tests can also be analyzed on data with only one replicate (n=1) per
treatment combination. However, the interaction between the factors cannot be
tested. In other words, it is not possible to test the null hypothesis of no
interaction, but we can test the null hypotheses regarding main effects as long
as we assume no interaction. Two-way ANOVA table without replication is
shown in the following table:
The 2k Factorial Design
18

The 22 Design
The geometry of the 2 design is shown in Fig. 12-17a. Note that the design can be
2

represented geometrically as a square with the 2 = 4 runs forming the corners of the
2

square. Fig. 12-17b shows the 4 runs in a tabular format often called the test matrix.

“-” and “+” denote the low and

high levels of a factor,
respectively
Low and high are arbitrary
terms
Geometrically, the four runs
form the corners of a square
Factors can be quantitative or
qualitative, although their
treatment in the final model
The letters (1), a, b, and ab represent the totals of all will be different
n observations taken at these design points.
The 2k Factorial Design
19

The quantities in brackets in the above equations are called

contrasts. For example, the A contrast is
Contrast A = a + ab − b − (1)
The 2k Factorial Design
20

In these equations, the contrast coefficients are always either +1 or -1. A table of plus
and minus signs, can be used to determine the sign of each run for a particular contrast.

Let k be the number of factors. Then the effects and the sums of squares for
A, B, and AB are obtained as follows:
Contrast (Contrast) 2 2 k −2
Effect = SS = = n (Effect) 2
n2k −1 n2 k

[a + ab − b − (1)]2
SS A =
4n
[b + ab − a − (1)]2
Therefore the sums of squares for A, B, and AB are: SS B =
4n
[ab + (1) − a − b]2
SS AB =
4n
 21

Analysis of Variance for 2k Factorial Design

 22

Example (Router Experiment)

We can compute the factor effect estimates as follows:
1
A= [a + ab − b − (1)]
2n Contrast
1 133.1 Effect =
=
2(4)
[96.1 + 161.1 − 59.7 − 64.4] =
8
= 16.64 n2k −1
1
B = [b + ab − a − (1)]
2n (Contrast) 2 2 k −2
1 60.3 SS = k
= n (Effect) 2
= [59.7 + 161.1 − 96.1 − 64.4] = = 7.54 n2
2(4) 8
2 2 n
y...2
SSTotal =  yijk
1
AB = [ab + (1) − a − b]
2n
2
−
i =1 j =1 k =1 4n
1 69.7
= [161.1 + 64.4 − 96.1 − 59.7] = = 8.71
2(4) 8
 23

Example (cont.)

Source of Sum of Degrees of P-value

Variation Squares Freedom Mean Square F0
Bit Size (A) 1107.226 1 1107.226 185.25 1.17 x 10-8
Speed (B) 227.256 1 227.256 38.03 4.82 x 10-5
AB 303.631 1 303.631 50.80 1.20 x 10-5
Error 71.723 12 5.977
Total 1709.836 15
How to decide on significant or non-significant factors?  = 0.05  f ,1,2 k
( n −1)
= f0.05,1,12 = 4.7472
24
 25

Python code
% two-level experiment
n = 4; k = 2;
% data from the router experiment
data = np.array([[18.2, 18.9, 12.9, 14.4], [27.2, 24.0, 22.4, 22.5], [15.9, 14.5, 15.1, 14.2], [41.0, 43.9, 36.3, 39.9]])
M = np.array([[-1., -1., 1.], [ 1., -1., -1.], [-1., 1., -1.], [ 1., 1., 1.]]) #design matrix
R = np.tile(M, (n, 1)) #replicate of design matrix
g1 = R[:,0]; g2 = R[:,1] #factors A and B
y = data.flatten(order='F')
# Fit the model
formula = 'y ~ C(g1) + C(g2) + C(g1):C(g2)'
df = pd.DataFrame(np.stack((y, g1, g2),-1), columns=['y', 'g1', 'g2'])
model = ols(formula, df).fit()
# Peform analysis of variance on fitted linear model
table = anova_lm(model, typ=2)
print(table)

A B AB
 = 0.05  f ,1,2 k
( n −1)
= f0.05,1,12 = 4.7472 yˆ = y + x1 + x2 + x1 x2
2 2 2
 26

Regression Model and Residual Analysis

A B AB
yˆ = y + x1 + x2 + x1 x2
2 2 2
 16.64   7.54   8.71 
At x1 = −1 and x2 = −1: yˆ = 23.83 +   ( −1) +   ( −1) +   (−1)(−1) = 16.1
 2   2   2 

e1 = 18.2 − 16.1 = 2.1 e3 = 12.9 − 16.1 = −3.2

Residuals: 
e2 = 18.9 − 16.1 = 2.8 e4 = 14.4 − 16.1 = −1.7

Interaction Plot
 27

Residual Analysis
The data of normal probability plot appear fairly linear, suggesting that no reason to doubt the
normality assumption. Also, the residual plots against the fitted values as well as against the
factor levels of A and B exhibit random scatter around 0. Thus, the model assumptions are valid.
There is nothing unusual about the residual plots.
Factorial Experiments: 2k Design
28

Analysis Procedure for Factorial Designs

1. Estimate the factor effects 4. Analyze residuals
2. Form preliminary model 5. Refine model, if necessary
3. Test for significance of factor effects 6. Interpret results

The 2k Design for k>=3 factors

Cube plots are

often useful visual
displays of
experimental results
 29

A = y A+ − y A−
B = yB + − yB −
C = yC + − yC −
etc, etc, ...
Analysis done
via computer
 30
2k Design for k=3 factors

Contrast
Effect =
n2k −1

(Contrast) 2 2 k −2
SS = k
= n (Effect) 2
n2
• Except for column the identity I, every column has an equal number of + and – signs
• The sum of the product of signs in any two columns is zero
• Multiplying any column by column I leaves that column unchanged (identity element)
• The product of any two columns yields a column in the table: A  B = AB
AB  BC = AB 2C = AC
 31

The 2k Design for k>=3 factors

1
A = y A+ − y A− = [a + ab + ac + abc − b − c − bc − (1)]
4n
1
B = yB + − yB − = [b + ab + bc + abc − a − c − ac − (1)]
4n
1
C = yC + − yC − = [c + ac + bc + abc − a − b − ab − (1)]
4n

1
AB = [ab + (1) + abc + c − b − a − bc − ac]
4n
1
AC = [ac + (1) + abc + b − a − c − ab − bc]
4n
1
BC = [bc + (1) + abc + a − b − c − ab − ac]
4n

1
ABC = [abc − bc − ac + c − ab + b + a − (1)]
4n
 32

Example 1: An experiment was performed to investigate the surface finish of a metal

part. The experiment is a 23 factorial design in the factors feed rate (A), depth of cut (B),
and tool angle (C), with n=2 replicated. The Table below presents the observed surface-
finish data for this experiment, and the design is shown graphically in Figure 12-25.
Design Factors
Surface
Run A B C Finish Totals
1 (1) -1 -1 -1 9, 7 16
2 a 1 -1 -1 10, 12 22
3 b -1 1 -1 9, 11 20
4 ab 1 1 -1 12, 15 27
5 c -1 -1 1 11, 10 21
6 ac 1 -1 1 10, 13 23
7 bc -1 1 1 10, 8 18
8 abc 1 1 1 16, 14 30

Contrast A
A=
n 2k −1
1
=  a + ab + ac + abc − b − c − bc − (1)
(2)(23−1 )
1
=  22 + 27 + 23 + 30 − 20 − 21 − 18 − 16
(2)(4)
1
=  27  = 3.375
8
 33

The main effect may be estimated using the corresponding equations. The effect
of A, for example is 1
A =  a + ab + ac + abc − b − c − bc − (1) 
4n
1
=  22 + 27 + 23 + 30 − 20 − 21 − 18 − 16
4(2)
1
=  27  = 3.375
8

and the sum of squares for A is given by (Contrast A ) 2

SS A =
n 2k
(27) 2
= = 45.5625
2(8)
 34

ANOVA Table

How to decide on significant or non-significant factors?

 = 0.15  f ,1,8 = 2.5352

 35

From examining the magnitude of the effects, feed rate (A) is clearly dominant, followed by
depth of cut (B) and the AB interaction, although the interaction effect is relatively. Based
on the P-values in the Table below, it is clear that the feed rate (A) is highly significant.

Source of Sum of Degrees of

Variation Squares Freedom Mean Square F0
A 45.5625 1 45.5625 18.69
B 10.5625 1 10.5625 4.33
C 3.0625 1 3.0625 1.26
AB 7.5625 1 7.5625 3.10
AC 0.0625 1 0.0625 0.03
BC 1.5625 1 1.5625 0.64
ABC 5.5625 1 5.5625 2.08
Error 19.5000 8 2.4375
Total 92.9375 15
2 2 2 n 2
y....
SSTotal =  yijhk
2
−
i =1 j =1 h =1 k =1 8n
The three largest effects are A, B, and the AB interaction. Thus the regression
model used to obtain the predicted values is given by:
A B AB
Regression Equation: yˆ = y + x1 + x2 + x1 x2
2 2 2
 36

A B AB
y

abc

bc
b
ab

ac
c

(1) a