DESIGN OF EXPERIMENTS (DOE)

HISTORY

Developed in the early 1920s by Sir Ronald Fisher at the Rothamsted Agricultural
Field Research Station in London, England.

His initial experiments were concerned with determining the effect of various
fertilizers on different plots of land.

The final condition of the crop was not only dependent on the fertilizer but also on a
number of other factors (such as underlying soil condition, moisture content of the
soil, etc.) of each of the respective plots.

Fisher used DOE which could differentiate the effect of fertilizer and the effect of
other factors.

Since then DOE has been widely accepted and applied in biological and agricultural
fields as well as industry.
Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 1 / 34
 Examples

Medicine:
Does salted drinking water affect blood pressure (BP) in mice?
1% NaCl water 14 days Measure BP
How does a company test a new vaccine/drug?

Industry:
How does a manager check the performance of machines or operators in a factory?

Rotate the machines, workers, shifts, days

Agriculture:
How does one find out the effect of fertilizer on crops?

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 2 / 34
 SOME DEFINITIONS IN DOE

Experiment: An investigation in which the investigator applies some treatments to

experimental units and then observes the effect of the treatments on the
experimental units by measuring one or more response variables.
Treatment Factors and their levels:
I Treatment factor is used to mean any substance or item whose effect on the data is to
be studied.
I The levels are the specific types or amounts of the treatment factor that will actually
be used in the experiment.
I For example, a treatment factor might be a drug or a chemical additive or temperature
or teaching method, etc. The levels of such treatment factors might be the different
amounts of the drug to be studied, different types of chemical additives to be
considered, selected temperature settings in the range of interest, different teaching
methods to be compared, etc.
I If the levels of a treatment factor are quantitative (i.e., can be measured), then they
are usually chosen to be equally spaced.
I For convenience, treatment factor levels can be coded.
I We will sometimes use the term treatment to mean a level of a treatment factor in a
single factor experiment, or to mean a treatment combination in a factorial experiment.
Variable: All the factors, their levels and all the measured traits (responses) are
variables. They are attributes, characteristics, or dimensions being measured or
studied.
Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 3 / 34
 SOME DEFINITIONS IN DOE

Cases:The objects being studied, also known as subjects or participants.

Independent or class variables: Factors applied to the experimental units are
independent variables. Other names include predictor variables, regressors,
controlled variables, manipulated variables, explanatory variables, or input variables.
Response or dependent variables: Measured or observed traits in the experiment.
The strength of the paper measured after treated with starch is a response (or
dependent) variable. Other names include response variable, the regressand, the
measured variable, the responding variable, the explained variable, the outcome
variable, the experimental variable or the output variable.
Grouping: a.k.a subject variable, a type of independent variable with naturally
occurring groups such as males and females.
Main effects: This is the simple effect of a factor on a dependent variable. It is the
effect of the factor alone averaged across the levels of other factors. Example: In the
starch experiment, it was found that on average, the starch increased the strength
(main effect) compared to the control (no starch).

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 4 / 34
 SOME DEFINITIONS IN DOE

Interaction effect: An interaction is the variation among the differences between

means for different levels of one factor over different levels of another factor.
Example: Lets say starch and latex both significantly increased the paper strength
(main effects). However, when starch and latex applied to the same experimental
unit, it was found that the increase in paper strength was even higher. The paper
got the benefits of both factors, plus a bonus.
Experimental (or Sampling) Units: They are the ”material” to which the levels of
the treatment factor(s) are applied. For example, in agriculture these would be
individual plots of land, in medicine they would be human or animal subjects, in
industry they might be batches of raw material, factory workers, etc. If an
experiment has to be run over a period of time, with the observations being
collected sequentially, then the times of day can also be regarded as experimental
units. Exptal units must be representative.

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 5 / 34
 SOME DEFINITIONS IN DOE

Population: A population is any entire collection of people, animals, plants from

which we may collect data. i.e. the entire collection of items which is the focus of
concern. It is the entire group we are interested in, which we wish to describe or
draw conclusions about. Example, we are interested in average GPA values of all
KNUST students. All the students registered in KNUST is a population. A
population can be of any size. The members of the population must share at least
one measurable feature.
Sample: A sample is a group of units selected from a larger group (the population).
By studying the sample it is hoped to draw valid conclusions about the larger group.
A sample is generally selected for study because the population is too large to study
in its entirety.

NOTE!!!The critical difference between a population and a sample, is that with a

population our interest is to identify its characteristics whereas with a sample, our
interest is to make inferences about the characteristics of the population from which
the sample was drawn.

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 6 / 34
 SOME DEFINITIONS IN DOE

Parameter: A parameter is a value, usually unknown, and which therefore has to be

estimated. It is used to represent a certain population characteristic. For example,
the population mean is a parameter that is often used to indicate the average value
of a quantity. Parameters are often assigned Greek letters (e.g. σ for standard
deviation).
Statistic: A statistic is a quantity that is calculated from a sample of data. It is a
measure of the items in a random sample. For example, the mean, a variance and a
standard deviation calculated from a sample are statistics. The only reason to ever
draw a random sample is to infer something about the population from which it
came. When we calculate a given statistic we only do so in order to estimate a
corresponding parameter of the population from which the sample was
drawn.Statistics are assigned Roman letters (e.g. s for standard deviation). The
value of a statistic will vary from sample to sample.

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 7 / 34
 SOME DEFINITIONS IN DOE

Estimate: An estimate is an indication of the value of an unknown quantity based

on observed data. Example, suppose we want to know the average height of
KNUST students (population). We can use an estimate of this population mean by
calculating the mean of a sample of students. Estimators of population parameters
are sometimes distinguished from the true value by using the symbol ’hat’. For
example, σ =true population standard deviation. σ̂ estimated (from a sample)
population standard deviation.
Sampling Distribution: The sampling distribution is the probability distribution or
probability density function of the statistic.
I The Sampling Distribution of a statistic is the set of values that we would obtain if we
drew an infinite number of random samples from a given population and calculated the
statistic on each sample.
I Example: Throwing enough darts at a dart board will soon form a pattern that would
then represent the distribution of the process of that particular dart thrower throwing
darts at that dart board from a fixed distance.
I Derivation of the sampling distribution is the first step in calculating a confidence
interval or carrying out a hypothesis test for a parameter. Example, suppose that
x1 , x2 , . . . , xn are a simple random sample from a normally distributed population with
expected value and known variance s2 . Then the sample mean x is normally
distributed with expected value µ and variance s2 /n.
Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 8 / 34
 Three Fundamental Experimental Design Concepts Attributed to R.A. Fisher

Randomization: random assignment of treatments to experimental units.

Blocking: grouping similar experimental units together and assigning different

treatments within such groups of experimental units. These are 2 examples:
I A metallurgist wants to improve the strength of a steel product. Four factors are being
considered for the experiment, which might have some impact on the strength. It is
decided to study each factor at 2-levels (i.e. a low setting and a high setting). An
eight trial experiment is chosen by the experimenter but only four trials are possible to
run for each day. Here each day can be treated as a separate block.
I An experiment in a chemical process requires 2 batches of raw materials for conducting
the entire experimental runs. In order to minimize the batch-to-batch variability, we
need to treat each batch of raw material as a noise factor. In other words, each batch
of raw materials would form a block.

Replication: applying a treatment independently to two or more experimental units.

I There is a difference between replication and repeated measurements. For example,
suppose four subjects are each assigned to a drug and a measurement is taken on each
subject. The result is four independent observations on the drug. This is replication.
I On the other hand, if one subject is assigned to a drug and then measured four times,
the measurements are not independent. We call them repeated measurements.

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 9 / 34
 ART OF EXPERIMENTATION

Experimenter’s Initial Questions

How many observations do I need to take?
Given my limited budget, how can I gain as much information as possible?
What is the main purpose of running this experiment?
What do I hope to be able to show?

Reasons for running an Experiment

to determine the principal causes of variation in a measured response,
to find the conditions that give rise to a maximum or minimum response,
to compare the responses achieved at different settings of controllable variables,
to obtain a mathematical model in order to predict future responses.

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 10 / 34
 Observational and Experimental Studies

Observations can be collected from observational studies as well as from

experiments, but only an experiment allows conclusions to be drawn about cause
and effect.
Research studies may often be classified as either observational or experimental,
although some are a mixture of the two.

In an observational study, data are collected without any attempt to manipulate or

influence the outcome. For example:
I Fish may be collected from three different regions of a lake, in order to compare their
weights over the three locations.
I Children from three different schools may be compared for their performance on an
achievement test.
I Households from three suburbs are surveyed to compare their incomes and political
opinions.
I People with a particular disease are studied, to see if the severity of their symptoms is
related to their age, smoking habits, diet or amount of exercise.

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 11 / 34
 Observational and Experimental Studies

In experiments usually some manipulation is attempted, in order to see if the

outcome is related to the factor being controlled. For example:
I Twenty plots of carrots are grown in a field. Each plot is randomly allocated to one of
five fertilizers, with four plots for each fertilizer. At the end of the experiment, the
carrots from each plot are weighed. The yield of carrots with different fertilizers is
being studied.
I Twenty children from a class are each randomly assigned to one of five different
teaching methods, four children to each method. After three weeks of teaching, each
child is tested for understanding of the material taught. The different teaching
methods are being compared.
I Eggs are incubated in the laboratory at chosen temperatures, with the outcome of
interest being the sex of the hatchling.
I Fifty subjects are assigned randomly to one of five different working conditions (quiet,
loud pop music, people talking, etc.) There are ten in each group. They are then
tested repeatedly to find each persons average response time to a particular type of
stimulus (e.g. hearing a fire alarm bell). The influence of background noise on
response time is being investigated.
I People with a certain disease are randomly allocated to three different drugs. The
drugs are being compared for their influence on the progress of the disease.

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 12 / 34
 Designing/ Planning an Experiment

The following steps summarize a very large number of decisions that need to be made at
each stage of the experimental planning process. The steps are not independent, and at
any stage, it may be necessary to go back and revise some of the decisions made at an
earlier stage.
Define the objectives of the experiment.
Identify all sources of variation, including:
I treatment factors and their levels,
I experimental units,
I blocking factors, noise factors, and covariates.
Choose a rule for assigning the experimental units to the treatments.
Specify the measurements to be made, the experimental procedure, and the
anticipated difficulties.
Run a pilot experiment.
Specify the model.
Outline the analysis.
Calculate the number of observations that need to be taken.
Review the above decisions. Revise, if necessary.
IMPORTANT !!! It’s only after all of these decisions have been made should the data be
collected and analyzed and a report written.
Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 13 / 34
 KEY POINTS TO NOTE

A designed experiment is a methodical way to conduct a series of trials by optimizing the

means at the disposal of the researcher: time and money. But this method is not a
substitute for actual experiments, and especially cannot replace human intelligence and
ingenuity. It is the experimenter that has to
Define the objective It essential to precisely quantify the objective of the study.
Pose the problem well A poorly posed problem cannot be saved by a designed
experiment.
Be imaginative and creative
Conducting a designed experiment is not only a technique. It helps with the
interpretation of the results, but it is the experimenter who is responsible for the
hypotheses and conclusions.
Choose the factors
If the experimenter forgets an important factor of the study, it is he who is
responsible for anomalies and erroneous conclusions.

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 14 / 34
 KEY POINTS TO NOTE

Define limits on the study domain

This is one of the big difficulties encountered by the experimenter. If the domain is
too small, the hoped-for responses may not be inside the domain, which means
additional trials. If the domain is too large, the postulated model will not serve well
in interpretation of the results.
Carefully conduct the experiments
Badly conceived experiments, uncontrolled factors, vague measurements, errors in
notation, location, or transcription cannot be repaired by an experimental design.
On the contrary: they require a great deal of care in the experiments themselves to
pull out all the information.
Choose the responses
This is an equally vital point for the quality of interpretation. All responses are not
equal. Some are not precise enough; others do not provide effects that are
sufficiently clear-cut to draw conclusions from. Finally, it’s necessary that the chosen
responses allow us to answer the question of the study.
In summary, the designed experiment is a useful and strong technique, but both
control of the experiment and the responsibility for the conclusions drawn remains
firmly in the hands of the experimenter.
Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 15 / 34
 STANDARD DESIGNS

Factorial Designs

2-level Factorial Designs

Each experimental variable (factor, k) has 2 levels

Each combination is tested
Number of combinations is 2k
−→ 2k factorial design.
n is a power of 2.

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 16 / 34
 STANDARD DESIGNS

23 factorial design

y = β0 + β1 x1 + β2 x2 + β3 x3 + β12 x1 x2 + β13 x1 x3 + β23 x2 x3 + β123 x1 x2 x3

x1 x2 x3
1 -1 -1 -1
2 +1 -1 -1
3 -1 +1 -1
4 +1 +1 -1
5 -1 -1 +1
6 +1 -1 +1
7 -1 +1 +1
8 +1 +1 +1

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 17 / 34
 STANDARD DESIGNS

23 factorial design cont’d

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 18 / 34
 STANDARD DESIGNS

2-level or 2k−q fractional factorial designs

Not every combination is tested

Used for screening purposes
Full model cannot be estimated
n is also a power of 2 but we have fewer observations
q is called the generator of the design

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 19 / 34
 STANDARD DESIGNS

2-level or 2k−q fractional factorial designs Cont’d

Let’s consider a full 23 factorial design

Test I x1 x2 x3 x1 x2 x1 x3 x2 x3 x1 x2 x3
1 +1 -1 -1 -1 +1 +1 +1 -1
2 +1 +1 -1 -1 -1 -1 +1 +1
3 +1 -1 +1 -1 -1 +1 -1 +1
4 +1 +1 +1 -1 +1 -1 -1 -1
5 +1 -1 -1 +1 +1 -1 -1 +1
6 +1 +1 -1 +1 -1 +1 -1 -1
7 +1 -1 +1 +1 -1 -1 +1 -1
8 +1 +1 +1 +1 +1 +1 +1 +1

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 20 / 34
 STANDARD DESIGNS

2-level or 2k−q fractional factorial designs Cont’d

We can introduce variable x4 through interaction x1 x2 x3

Test I x1 x2 x3 x4
1 +1 -1 -1 -1 -1
2 +1 +1 -1 -1 +1
3 +1 -1 +1 -1 +1
4 +1 +1 +1 -1 -1
5 +1 -1 -1 +1 +1
6 +1 +1 -1 +1 -1
7 +1 -1 +1 +1 -1
8 +1 +1 +1 +1 +1

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 21 / 34
 STANDARD DESIGNS

2-level or 2k−q fractional factorial designs Cont’d

...

x1 x2 x1 x3 x1 x4 x2 x3 x2 x4 x3 x4 x1 x2 x3 x1 x2 x4 x1 x3 x4 x2 x3 x4 x1 x2 x3 x4
+1 +1 +1 +1 +1 +1 -1 -1 -1 -1 +1
-1 -1 +1 +1 -1 -1 +1 -1 -1 +1 +1
-1 +1 -1 -1 +1 -1 +1 -1 +1 -1 +1
+1 -1 -1 -1 -1 +1 -1 -1 +1 +1 +1
+1 -1 -1 -1 -1 +1 +1 +1 -1 -1 +1
-1 +1 -1 -1 +1 -1 -1 +1 -1 +1 +1
-1 -1 +1 +1 -1 -1 -1 +1 +1 -1 +1
+1 +1 +1 +1 +1 +1 +1 +1 +1 +1 +1

We note that the following columns are the same:

I and x1 x2 x3 x4 ; x2 and x1 x3 x4 ; x3 and x1 x2 x4 ; x1 x2 and x3 x4 ; x1 x3 and x2 x4 ; x2 x3
and x1 x4 ; x1 x2 x3 and x4 .

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 22 / 34
 STANDARD DESIGNS

2-level or 2k−q fractional factorial designs Cont’d

Table: Resolution numbers of fractional factorial experiments

Resolution Main effects aliased with: 2-factor interactions aliased with:

III 2-factor interactions or higher Main effects and interactions
IV 3-factor interactions or higher 2-factor interactions or higher
V 4-factor interactions or higher 3-factor interactions or higher

Notation: e.g 23−1

III or 23−1
IV

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 23 / 34
 STANDARD DESIGNS

2-level or 2k−q fractional factorial designs Cont’d

23−1 Fractional Factorial Design

(+1 = x1 x2 x3 ) (−1 = x1 x2 x3 )

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 24 / 34
 STANDARD DESIGNS

Plackett-Burman Designs

Only meant for estimating main effect models

y = β0 + β1 x1 + β2 x2 + · · · + βk xk

n is a multiple of 4
They are very efficient screening designs

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 25 / 34
 STANDARD DESIGNS

8-point Plackett-Burmann Design for k=7

x1 x2 x3 x4 x5 x6 x7
1 +1 +1 +1 -1 +1 -1 -1
2 -1 +1 +1 +1 -1 +1 -1
3 -1 -1 +1 +1 +1 -1 +1
4 +1 -1 -1 +1 +1 +1 -1
5 -1 +1 -1 -1 +1 +1 +1
6 +1 -1 +1 -1 -1 +1 +1
7 +1 +1 -1 +1 -1 -1 +1
8 -1 -1 -1 -1 -1 -1 -1

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 26 / 34
 STANDARD DESIGNS

Two - Level Designs- Conclusion

all model parameters are estimated independently of each other

all these designs can be used for quantitative and qualitative variables

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 27 / 34
 STANDARD DESIGNS

Central Composite Design (CCD)

Box & Wilson (1951)

3 Components:
I 2k factorial (or 2k−q fractional factorial) design
I 2k axial or star points (at a distance α from center)
I at least one center point
F α = 1:
√ face-centered CCD
F α = k: Spherical CCD

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 28 / 34
 STANDARD DESIGNS

Central Composite Design Cont’d

CCD for α = 1.68 > 1

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 29 / 34
 CCD for α = 1.68 > 1 cont’d

run x1 x2 x3
1 -1 -1 -1
2 -1 -1 1
3 -1 1 -1
4 -1 1 1
5 1 -1 -1
6 1 -1 1
7 1 1 -1
8 1 1 1
9 -1.68 0 0
10 1.68 0 0
11 0 -1.68 0
12 0 1.68 0
13 0 0 -1.68
14 0 0 1.68
15 0 0 0
16 0 0 0

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 30 / 34
 STANDARD DESIGNS

Central Composite Design Cont’d

Face- Centered CCD i.e. α = 1

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 31 / 34
 STANDARD DESIGNS

Box- Behnken Design

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 32 / 34
 Box- Behnken Design Cont’d

run x1 x2 x3
1 -1 -1 0
2 -1 1 0
3 1 -1 0
4 1 1 0
5 0 -1 -1
6 0 -1 1
7 0 1 -1
8 0 1 1
9 -1 0 -1
10 1 0 -1
11 -1 0 1
12 1 0 1
13 0 0 0
14 0 0 0
15 0 0 0

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 33 / 34
 STANDARD DESIGNS

Latin Square designs

A Latin square is an n × n array filled with n different Latin letters, each occurring
exactly once in each row and exactly once in each column.

Dr. R.K. Avuglah (Dept. of Mathematics, KNUST) DOE Lecture September 3, 2012 34 / 34