Improve: Introduction to DOE

All Rights Reserved, Juran Institute, Inc. 1 6507 SSGB.0910 v4

 Improve: Introduction to DOE

All Rights Reserved, Juran Institute, Inc. 2 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Design of Experiment: Overview

When Xs are operating parameters and you want to develop a mathematical model
or determine the best configuration of Xs, Design of Experiment, or DOE, is the most
efficient tool to use. DOE allows testing of multiple Xs to determine how they affect
the Y, either by themselves or in combination with other Xs.
This chapter will introduce DOE concepts, terminology, process steps, and
interpretation.
To illustrate the need for designed experiments, consider the case of a certain fellow
who decided he wanted to investigate the causes of intoxication. As the story goes,
he drank some whiskey and water on Monday and became highly inebriated.
The next day, he repeated the experiment holding all variables constant except one…
he decided to replace the whiskey with vodka.
As you may guess, the result was drunkenness. On the third day, he repeated the
experiment for the last time. On this trial he used gin in lieu of the whiskey and vodka.
This time it took him two days just to be able to gather enough of his faculties to
analyze the experimental results.
After recovering, he concluded that water causes intoxication. Why? Because it was
the common variable!

All Rights Reserved, Juran Institute, Inc. 3 6507 SSGB.0910 v4

 Improve: Introduction to DOE

All Rights Reserved, Juran Institute, Inc. 4 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Experimental Design

The formal plan for conducting an experiment is called the “experimental

design” (also the “experimental pattern”). It includes choices in response,
factors, levels, blocks, and treatments, as well as the use of certain tools called
planned grouping, randomization, repetition, and/or replication.
Typically the analysis of the results of an experiment is straightforward,
particularly if computer-based tools, such as MINITAB®, are available.
Because the best analysis in the world cannot rescue a poorly designed
experiment, it is the way in which you conduct the experiment that is key.

All Rights Reserved, Juran Institute, Inc. 5 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Example: One Factor at a Time

Suppose you are studying the gas mileage (in miles per gallon).
The two Xs you are studying are the vehicle timing setting and the gasoline
type. Traditionally, you might begin by holding the timing constant at T1 and
testing both gas types. Gas Type 1 gave the highest miles per gallon.
Since Gas Type 1 was the best, test it with the Timing 2. Now, you see that the
best results are obtained with Gas Type 1 and Timing Type 1.
What did you miss?

All Rights Reserved, Juran Institute, Inc. 6 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Example: One Factor at a Time (continued)

In the traditional approach, the combination of Gas Type 2 and Timing Type 2
was never tested. A full factorial designed experiment tests all combinations of
all factors. In this case, the traditional approach would not have given the best
solution.
An interaction exists between gas type and timing. The effect of timing on MPG
depends on the gas type.

All Rights Reserved, Juran Institute, Inc. 7 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Changing Many Things At Once

This may be the most commonly used method of experimentation in business

today. However, these experiments are often called “solutions.”
Every untested process change is actually an experiment because the results
are unknown.

All Rights Reserved, Juran Institute, Inc. 8 6507 SSGB.0910 v4

 Improve: Introduction to DOE

DOE Terminology

The Dependent Variable: The Y variable is often called the response variable.
The Independent Variable: The X variable is commonly known as a factor. In all cases, a
factor must be treated as a discrete variable. If the factor is continuous by nature, it must be
classified into levels—a high and a low level. If the factor is discrete, factor levels naturally
exist.
Factor: A factor (or input) is one of the controlled or uncontrolled variables whose influence
on a response (output) is being studied in the experiment. A factor may be quantitative, e.g.,
temperature in degrees, time in seconds. A factor may also be qualitative, e.g., different
machines, different operator, clean or not clean.
Level: The levels of a factor are the values of the factor being studied in the experiment.
Levels should be set wide enough apart so effects on the Y variable can be detected. Levels
are often referred to as “-1” and “+1.” For quantitative factors, each chosen value becomes
a level, e.g., if the experiment is to be conducted at two different temperatures, then the
factor of temperature has two “levels.” A qualitative factor such as cleanliness can have two
levels as well, particularly, clean vs. not clean.
k1 x k2 x k3 ... Factorial: Description of the basic design. The number of ks is the number
of factors. The value of each k is the number of levels of interest for that factor. Example: A 2
x 3 x 3 design indicates three input variables. One input has two levels, and the other two
each have three levels.

All Rights Reserved, Juran Institute, Inc. 9 6507 SSGB.0910 v4

 Improve: Introduction to DOE

DOE Terminology

Interaction Plots
Main Effect Plots

•
New
Cycle Time

•
Cycle Time

Old
Y
Y

X
X 1 2 3
New Old Number Of Associates
Type of Application
K-way Interaction: Interaction between
different K input variables.
Cycle Time

Main Effect: The change in the average

Y

response (output) observed during a

change from one level to another for a
single factor (input).
X
1 2 3 Interaction: The combined effect of two
Number of Associates factors observed over and above the main
(or singular) effect of each factor.

All Rights Reserved, Juran Institute, Inc. 10 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Types of Designed Experiments

Factorial designs are classified in three main types: Full factorial, fractional factorial, and
screening designs.
Factorial Designs are a family of designs that provide considerable flexibility in identification
and exploration of cause-effect relationships among process factors.
Full factorial designs provide more information, but this requires greater resources.
Studying more than 4 or 5 factors makes the number of experiments required unwieldy.
Fractional Designs provide a means of studying a higher number of factors with less
experimentation. Not as much information is obtained, but the resource requirement is
reduced.
Screening Designs are simply a subset of fractional designs that provide a means of
reducing the number of factors to a more manageable level. Screening experiments are used
to separate the vital few X variables from possible Xs at a very low cost.
A Nested Design is used when one (or more) factors cannot be combined with all the levels
of the other factors. For example, suppose you are evaluating a resin. The resin might come
from two different suppliers. Each supplier has multiple lots. Each lot has multiple samples
taken.
Mixture Designs are typically used in continuous processing operations that have
constraints built into the process. One such constraint might be, the sum of the input
components must equal a certain weight.

All Rights Reserved, Juran Institute, Inc. 11 6507 SSGB.0910 v4

 Improve: Introduction to DOE

DOE Terminology

Treatment: A single level assigned to a single factor during an experimental run, e.g.,
temperature at 250 degrees.
Test Run (Experimental Run): A single combination of factor levels that yields one or more
observations of the output variable.
Treatment Combination: An experimental run using a set of the specific levels of each input
variable. The number of treatment combinations in a full experiment is the product of the
number of levels for each factor. In the case of a 2 x 3 x 3 design, there will be 18 possible
treatment combinations in the experiment.
Repetition: Running several experimental runs consecutively using the same treatment
combination.
Replication: Replication automatically implies that you do NOT run several experimental
runs consecutively using the same treatment combination. Replication occurs when an
experimental treatment is set up and conducted more than once. If you collect two data
points at each treatment, you have two replications.
In general, plan on making between two and five replications for each treatment. A replication
is not two measurements of the same data point but a measurement of two data points under
the same treatment conditions. This is the variation from sources other than the changes in
factor levels.
Replicating an experiment allows you to estimate the residual or experimental error.
All Rights Reserved, Juran Institute, Inc. 12 6507 SSGB.0910 v4
 Improve: Introduction to DOE
Design of Experiment Method

Prepare:
Step 1: Define the problem.
Step 2: State the hypotheses.
Step 3: State the factors and levels of interest.
Step 4: Create an appropriate MINITAB® experimental datasheet.
Perform:
Step 5: Run the experiment and collect the data.
a. Select the appropriate sample size.
b. Randomize the experimental runs, and collect the data.
Analyze:
Step 6: Construct the ANOVA table for the full model and use the appropriate graphical tool
to evaluate the data.
Step 7: Rerun a Reduced Model by eliminating:
a. Effects with nonsignificant P-Values
b. Effects plotted low on effects Normal or Pareto chart
Step 8: Investigate the residuals plots to ensure model fit.
Step 9: Using the ANOVA table and appropriate graphical tool, investigate significant main
effects and interactions.
Step 10: Calculate the variation for the main effects and interactions left in the model.
a. State the mathematical model obtained.
Conclude:
Step 11: Translate the statistical conclusion into process terms.
a. Formulate conclusions and recommendations.
Step 12: Replicate optimum conditions.
a. Plan the next experiment or institutionalize the change.

All Rights Reserved, Juran Institute, Inc. 13 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Formulas and Definitions

y =+ +  +  y = Observed output

ji i j ji  = Average response
H o :  's = 0  =
i Effect because of blocking factor

H a : 's  0  = Effect because of treatment factor
i
H o :  's = 0  = Random error
j
H a :  's ≠ 0
j

All Rights Reserved, Juran Institute, Inc. 14 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Project Variables (Factors)

 Dependent Variable (Y) = Gas Mileage

– Y is called the dependent variable because its value depends upon
the level setting for X.
 Independent Variables (X)
– X is called the independent variable because its value is set
independently. X will sometimes be called an experimental factor.
• Independent Variable (T) = Tire Pressure
• Independent Variable (G) = Gas Octane
• Independent Variable (S) = Speed
 Levels of Interest:
– Level - represents the low value of the levels
– Level + represents the high value of the levels

All Rights Reserved, Juran Institute, Inc. 15 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Experimental Trials

 How many trials would be necessary if there were 4 variables, each with 2
levels? How many variables would there be before the cost became prohibitive?
 What would be a different type of experiment that would not require all possible
runs?
 The experimental trials table represents all possible combinations of the three
factor levels.

All Rights Reserved, Juran Institute, Inc. 16 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Randomize the Experimental Runs—Collect Data

Considerations
 Develop and execute SOPs for all factors not in the study.
 Prepare a data-collection plan.
 Communicate the plan.
– Data collectors
– Stakeholders
 Train data collectors.
 Complete trial runs, if necessary.
– Check understanding of the level settings.
– Verify factor levels and produce results.
– Ensure transitions between runs are doable in a reasonable time
frame.
 Run the Experiment.
– Note any unexpected events.

All Rights Reserved, Juran Institute, Inc. 17 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Replication

Replication is a repeat of all experimental trials to obtain additional data to

increase the degree of belief in the experimental results. Replications are used
when:
 The interactions are of critical importance.
 Data are tricky to collect. Replications will supply an extra data point
when data from an experimental trial is lost.
 You need to increase the degree of belief in the experiment results.
 You need to reduce the risk when implementing solutions.
 Replications enable you to estimate random error.

All Rights Reserved, Juran Institute, Inc. 18 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Graph the Data

Make sure that your data are sorted in run order and not standard order! Look
for trends resulting from lurking variables that might have interfered with the
experiment.
MINITAB®: Stat > Quality Tools > Run Chart
 Data are arranged as:
– Single column: Gas mileage
– Subgroup size: 1
 If you find a trend in your data associated with time, try to uncover the
source of this variation.
– If you determine its source, find a way to control or eliminate this
additional source of experimental variation.
– If you cannot determine the trend's source or cannot control it,
you may need to rerun the experiment in random order.
If each set of repetitions appears consistent; proceed with analysis.

All Rights Reserved, Juran Institute, Inc. 19 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Normal Probability Plot of the Standardized Effects

(response is Mileage, Alpha = .05)
99
Effect Ty pe
Not Significant
95 Significant

90 B F actor N ame
A Tire pressure
80 B O ctane
A C S peed
70
Percent

60
50
40
30
20 BC

10 C
Pareto Chart of the Standardized Effects
5 (response is Mileage, Alpha = .05)
2.31
1 F actor N ame
A Tire pressure
-25 -20 -15 -10 -5 0 5 10 C B O ctane
Standardized Effect C S peed
B

A
Term

BC

Speed, octane, tire pressure, AB

and the interaction of octane and ABC

speed are the significant factors. AC

0 5 10 15 20
Standardized Effect

All Rights Reserved, Juran Institute, Inc. 20 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Session Commands

MINITAB®: Stat > DOE > Factorial > Analyze Factorial Design
 Response: Mileage
 Terms: Selected Terms
– Tire pressure, octane, speed, octane * speed

All Rights Reserved, Juran Institute, Inc. 21 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Estimated Effects & Coefficients for Gas Mileage (coded units)

Analysis of Variance for Gas Mileage (coded units)

All Rights Reserved, Juran Institute, Inc. 22 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Investigate Residuals

Recall from regression that residuals are the difference between the actual Y
value and the Y value predicted by the regression equation.
Plot the residuals by selecting:
MINITAB®: Stat > DOE > Factorial > Analyze Factorial Design
 Graphs: Residual Plots
 Four in one
 Select
 Normal plot of residuals
 Residuals against fits
 Residuals against order
Residuals should:
 Be randomly and normally distributed about a mean of zero
 Not correlate with the predicted Y
 Not exhibit trends over time (if data is chronological)
Problems with the residuals would indicate the model is inadequate.

All Rights Reserved, Juran Institute, Inc. 23 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Session Commands

MINITAB®: Stat > DOE > Factorial > Factorial Plots

You will need to select the desired factorial plots. Each is set up separately. You
must indicate which factors to include in each plot.
 Include all terms initially, then rerun with terms with P-Values of
< 0.05 for prediction model.

All Rights Reserved, Juran Institute, Inc. 24 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Session Commands

MINITAB®: Stat > DOE > Factorial > Factorial Plots

All Rights Reserved, Juran Institute, Inc. 25 6507 SSGB.0910 v4

 Improve: Introduction to DOE

All Rights Reserved, Juran Institute, Inc. 26 6507 SSGB.0910 v4

 Improve: Introduction to DOE

All Rights Reserved, Juran Institute, Inc. 27 6507 SSGB.0910 v4

 Improve: Introduction to DOE

Final Thoughts

 If you have 10 weeks to perform a designed experiment, take 8 weeks to

PLAN it, 1 week to RUN it, and 1 week to ANALYZE it.
 You almost never run a single designed experiment, you end up running
several.

All Rights Reserved, Juran Institute, Inc. 28 6507 SSGB.0910 v4

 Improve: Introduction to DOE
FILE: IMPROVE-DOE MATL STRENGTH.MTW

 CLASS EXERCISE: Full Factorial DOE

A Six Sigma improvement team discovered during the Analyze Phase that four variables,
Material Type, Injection Pressure, Injection Temperature, and Cooling Temperature have a
significant effect on Material Strength. Now, as part of the Improve Phase they want to
optimize the levels of these variables to obtain the best Material Strength.

Response: Material Strength

Factors and Levels:
Material Type A (-1), Material Type B (1)
Injection Pressure (75, 150)
Injection Temperature (85, 100)
Cooling Temperature (25, 45)
Design: 24 Factorial
Procedure:
 Use the 12-step methodology to set up, run, and analyze the experiment.
– Look for Interactions, and Main Effects.
 Be prepared to state your results
and conclusions.

45 Minutes

All Rights Reserved, Juran Institute, Inc. 29 6507 SSGB.0910 v4

 Improve: Introduction to DOE

All Rights Reserved, Juran Institute, Inc. 30 6507 SSGB.0910 v4