Quality Control

Basic Experimental Design for Process

Improvement - Part 1

Dr.-Ing. Amr Nounou

What Is Experimental Design?

A designed experiment is a test or series of tests in

which purposeful changes are made to the input
variables of a process so that we may observe and
identify corresponding changes in the output response.

The process can be visualized as some combination of

machines, methods, and people that transforms an
input material into an output product.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
2
Dr.-Ing. Amr Nounou - Spring 2024-2025
What Is Experimental Design?

This output product has one or more observable

quality characteristics or responses.

Some of the process variables x1, x2, . . . , xp are

controllable, whereas others z1, z2, . . . , zq are
uncontrollable (although they may be controllable
for purposes of the test).

Sometimes these uncontrollable factors are

called noise factors.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
3
Dr.-Ing. Amr Nounou - Spring 2024-2025
What Is Experimental Design?
The objectives of the experiment may include

1. Determining which variables are most influential on the response, y.

2. Determining where to set the influential x’s so that y is near the

nominal requirement.

3. Determining where to set the influential x’s so that variability in y is

small.

4. Determining where to set the influential x’s so that the effects of the
uncontrollable variables z are minimized.

Thus, experimental design methods may be used either

in process development
or process troubleshooting (improve process performance)
or to obtain a process that is robust (insensitive to external sources of variability)

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
4
Dr.-Ing. Amr Nounou - Spring 2024-2025
 Phase diagram of the use Application of quality-engineering techniques and
of quality-engineering the systematic reduction of process variability
methods

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
5
Dr.-Ing. Amr Nounou - Spring 2024-2025
What Is Experimental Design?

Experimental design methods can also be very useful in establishing statistical control of
a process as it can be used to identify these influential process variables.

Experimental design is a critically important engineering tool for improving a

manufacturing process.

It also has extensive application in the development of new processes.

Application of these techniques early in process development can result in

1. Improved yield
2. Reduced variability and closer conformance to the nominal
3. Reduced development time
4. Reduced overall costs

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
6
Dr.-Ing. Amr Nounou - Spring 2024-2025
Examples of DOE - Characterizing a process
An engineer has applied SPC to a process for soldering
electronic components to printed circuit boards. He has
reduced the average number of defective solder joints
per board to around 1%. However, since the average
board contains over 2,000 solder joints, he would like to
reduce defect levels further.
DOE
That is, how much does the response variable (defects
The flow solder machine has several variables that can be per unit) change when each factor is changed, and does
controlled. They include: changing the factors together produce different results
1. Solder temperature than are obtained from individual factor adjustments?
2. Preheat temperature Sometimes we call this kind of factorial experiment a
3. Conveyor speed
4. Flux type
screening experiment or characterization experiment.
5. Flux specific gravity
6. Solder wave depth The information from this experiment is used to identify
7. Conveyor angle the critical process factors and their direction of
In addition, several others cannot be easily controlled during adjustment in addition to which factors should be more
routine manufacturing, although they could be controlled for carefully controlled to prevent high defect levels.
purposes of a test. They are:
1. Thickness of the printed circuit board
2. Types of components used on the board As a result, control charts to one or more process
3. Layout of the components on the board variables in addition to the u chart on process output
4. Operator could be applied.
5. Production rate

To accomplish this task he can design an experiment Over time, if the process is sufficiently improved,
that will enable him to estimate the magnitude and controlling process input variables can be emphasized
direction of the factor effects. instead of control charting the output.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
7
Dr.-Ing. Amr Nounou - Spring 2024-2025
Examples of DOE - Optimizing a process
In a characterization experiment, we determine which
process variables affect the response.

A logical next step is to optimize—that is, to determine

the region in the important factors that lead to the best
possible response.
DOE
The figure shows a view of the time-temperature region
Suppose we are interested in improving the yield of a from above and the lines of constant yield are connected
chemical process. Let’s say that we know from the to form response contours for 60%, 70%, … yield.
results of a characterization experiment that the two
most important process variables that influence yield To locate the optimum, it is necessary to perform an
are operating temperature and reaction time. experiment that varies time and temperature together.
This type of experiment is called a factorial experiment.
Contour plot of yield as
The responses observed at the four corners of the
a function of reaction
time and reaction square indicate that we should move in the general
temperature, illustrating direction of increased temperature and decreased
an optimization reaction time to increase yield.
experiment.
A few additional runs could be performed in this direction
to locate the region of maximum yield. Once we are in
the region of the optimum, a more elaborate experiment
could be performed to give a very precise estimate of the
The process currently runs at 155°F and 1.7 h of optimum operating condition. This type of experiment,
reaction time, producing yields around 75%. called a response surface experiment.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
8
Dr.-Ing. Amr Nounou - Spring 2024-2025
Examples of DOE - A product design example

Suppose that a group of engineers is

designing a door hinge for an automobile.

The quality characteristic of interest is the

check effort, or the holding ability of the door DOE
latch that prevents the door from swinging
closed when the vehicle is parked on a hill.

The check mechanism consists of a spring and The engineers build a prototype hinge
a roller. mechanism in which all these factors can be
varied over certain ranges.
The engineering team believes the check effort
is a function of the following factors: An experiment can be designed consisting of
1. Roller travel distance various combinations of the factor levels.
2. Spring height pivot to base
3. Horizontal distance from pivot to spring This will produce information concerning which
4. Free height of the reinforcement spring factors are most influential on latch check effort,
5. Free height of the main spring and through use of this information the design
can be improved.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
9
Dr.-Ing. Amr Nounou - Spring 2024-2025
Guidelines for Designing Experiments

Procedure for designing an experiment:

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
10
Dr.-Ing. Amr Nounou - Spring 2024-2025
 One-Factor-At-a-Time
(OFAT) Experiments

OFAT a really bad idea

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
11
Dr.-Ing. Amr Nounou - Spring 2024-2025
Factorial Experiments
When there are several factors of interest in an experiment, a factorial design should be used.

In such designs, factors are varied together.

Specifically, by a factorial experiment we mean that in each complete trial or replicate of the
experiment, all possible combinations of the levels of the factors are investigated.

Thus, if there are two factors A and B with a levels of factor A and b levels of factor B, then each
replicate contains all ab possible combinations.

The effect of a factor is defined as the change in response produced by a change in the level of the
factor. This is called a main effect because it refers to the primary factors in the study.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
12
Dr.-Ing. Amr Nounou - Spring 2024-2025
Factorial Experiments
For example, in Figure 13.5, in this factorial design, both the
factors A and B have two levels, denoted by “−” and “+.”
These two levels are called “low” and “high,” respectively.
The main effect of factor A is the difference between the average
response at the high level of A and the average response at the
low level of A, or

That is, changing factor A from the low level (−) to the high level
(+) causes an average response increase of 20 units.

Similarly, the main effect of B is

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
13
Dr.-Ing. Amr Nounou - Spring 2024-2025
Factorial Experiments

In some experiments, the difference in response between the

levels of one factor is not the same at all levels of the other factors.

When this occurs, there is an interaction between the factors.

For example, consider the data in Figure 13.6.

At the low level of factor B, the A effect is

and at the high level of factor B, the A effect is

Since the effect of A depends on the level chosen for factor B, there is
interaction between A and B.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
14
Dr.-Ing. Amr Nounou - Spring 2024-2025
Factorial Experiments
When an interaction is large, the corresponding main effects have
little meaning.
For example, by using the data in Figure 13.6, we find the main
effect of A as

and we would be tempted to conclude that there is no A effect.

However, when we examine the main effect of A at different levels of

factor B, we see that this is not the case.

The effect of factor A depends on the levels of factor B.

Thus, knowledge of the AB interaction is more useful than

knowledge of the main effect.

A significant interaction can mask the significance of main effects.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
15
Dr.-Ing. Amr Nounou - Spring 2024-2025
Factorial Experiments
The concept of interaction can be illustrated graphically.
Figure 13.7 plots the data in Figure 13.5 against the levels of A
for both levels of B.
Note that the B− and B+ lines are roughly parallel, indicating that
factors A and B do not interact.

Figure 13.8 plots the data in Figure 13.6.

In Figure 13.8, the B− and B+ lines are not parallel, indicating the
interaction between factors A and B.

Such graphical displays are often useful in presenting the results

of experiments.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
16
Dr.-Ing. Amr Nounou - Spring 2024-2025
Examples of DOE - The Aircraft Primer Paint Problem
Aircraft primer paints are applied to aluminum surfaces by two methods—dipping
and spraying.
The purpose of the primer is to improve paint adhesion; some parts can be primed
using either application method.
A team using the DMAIC approach has identified three different primers that can
be used with both application methods.
Three specimens were painted with each primer using each application method, a
finish paint was applied, and the adhesion force was measured.

DOE
The 18 runs from this experiment were run in random order.
The circled numbers in the cells are the cell totals.
The objective of the experiment was to determine which combination of primer
paint and application method produced the highest adhesion force.
It would be desirable if at least one of the primers produced high adhesion force
regardless of application method, as this would add some flexibility to the
manufacturing process.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
17
Dr.-Ing. Amr Nounou - Spring 2024-2025
The General Two-factor Factorial Design
The statistical model:

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
18
Dr.-Ing. Amr Nounou - Spring 2024-2025
The statistical model:

The observations from a two-factor factorial experiment may be described by the model

where μ is the overall mean effect, τi is the effect of the ith level of factor A, βj is the
effect of the jth level of factor B, (τβ)ij is the effect of the interaction between A and B,
and εijk is an NID(0, σ2) random error component.

We are interested in testing the hypotheses of no significant factor A effect,

no significant factor B effect,
and no significant AB interaction.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
19
Dr.-Ing. Amr Nounou - Spring 2024-2025
The statistical model:

Let
yi.. denote the total of the observations at the ith level of factor A,
y.j. denote the total of the observations at the jth level of factor B,
yij. denote the total of the observations in the ijth cell of the table,
and y... denote the grand total of all the observations.

Define ȳi.., ȳ.j., ȳij., and ȳ... as the corresponding row, column, cell, and grand averages— that is,

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
20
Dr.-Ing. Amr Nounou - Spring 2024-2025
Statistical Analysis: the ANOVA

This decomposition is usually

summarized in an analysis of variance
table.

To test for no row factor effects, no

column factor effects, and no interaction
effects, we would divide the
corresponding mean square by mean
square error.

Each of these ratios will follow an F

distribution, with numerator degrees of
freedom equal to the number of degrees
of freedom for the numerator mean
square and ab(n − 1) denominator
degrees of freedom, when the null
hypothesis of no factor effect is true.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
21
Dr.-Ing. Amr Nounou - Spring 2024-2025
Statistical Analysis: the ANOVA

We would reject the corresponding hypothesis

if the computed F exceeded the tabular value at an appropriate significance level,
or alternatively if the P-value were smaller than the specified significance level.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
22
Dr.-Ing. Amr Nounou - Spring 2024-2025
Computing the sums of squares (could be
done with software):
The ANOVA is usually performed with computer software, although simple computing formulas for
the sums of squares may be obtained easily. The computing formulas for these sums of squares
follow.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
23
Dr.-Ing. Amr Nounou - Spring 2024-2025
Examples of DOE - The Aircraft Primer Paint Problem

DOE

The ANOVA is summarized in the table.

The P-values in this table were obtained from
a calculator (they can also be found using the
Probability Distribution function in the Calc
menu in Minitab).

The P-values for both main effects are very small, indicating that the type of
primer used and the application method significantly affect adhesion force.
Since the P-value for the interaction effect F-ratio is relatively large, we would
conclude that there is no interaction between primer type and application method.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
24
Dr.-Ing. Amr Nounou - Spring 2024-2025
Examples of DOE - The Aircraft Primer Paint Problem

DOE

Source:
https://www.graphpad.com/quickcalcs/pvalue1.cfm
Retrieved: 08/01/2024

P-value = 1 – 0.72309 =
0.270691
Source:
https://www.graphpad.com/quickcalcs/pValue2/
Retrieved: 08/01/2024

Basic Experimental Design for

Process Improvement
25
Dr.-Ing. Amr Nounou - Spring 2024-2025
Examples of DOE - The Aircraft Primer Paint Problem
In practice, ANOVA computations are performed on a
computer using a statistics software package.
The table shows ANOVA from Minitab.
Note the similarity of this display to the previous
table.
Because the computer carries more decimal places
than we did in the manual calculations, the F-ratios in
the two tables are slightly different. The P-value for DOE
each F-ratio is called “significance level”, and when a
P-value is less than 0.001, Minitab reports it as 0.000.

A graph of the adhesion force cell averages {ȳij.}

versus the levels of primer type for each application
method is shown.
This interaction graph was constructed by Minitab.
The absence of interaction is evident in the
parallelism of the two lines.

Furthermore, since a large response indicates greater Interaction graph

adhesion force, we conclude that spraying is a Graph of average
superior application method and that primer type 2 is adhesion force versus
most effective. Therefore, if we wish to operate the primer types
process so as to attain maximum adhesion force, we
should use primer type 2 and spray all parts.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
26
Dr.-Ing. Amr Nounou - Spring 2024-2025
Examples of DOE - The Aircraft Primer Paint Problem

D
O
E
As an alternative to using
the P-values, we could
compare the computed F-
ratios to a 5% (say) upper
critical value of the F
distribution.

Since F0.05,2,12 = 3.89 and

F0.05,1,12 = 4.75, we
conclude that primer type
and application method
affect adhesion force.

Furthermore, since 1.5 <

F0.05,2,12, there is no
indication of interaction
between these factors.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
27
Dr.-Ing. Amr Nounou - Spring 2024-2025
Residual Analysis
The residuals from a factorial experiment play an important role in assessing model adequacy.

The residuals from a two- factor factorial are

That is, the residuals are simply the difference between the observations and the corresponding cell
averages.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
28
Dr.-Ing. Amr Nounou - Spring 2024-2025
➢ The normal probability plot of these
residuals is shown in Figure 13.13.
This plot has tails that do not fall
exactly along a straight line passing
through the center of the plot, Residual Analysis
indicating some small problems with
the normality assumption, but the
departure from normality is not
serious.

➢ Figures 13.14 & 13.15 plot the

residuals versus the levels of primer
types and application methods,
respectively. There is some indication
that primer type 3 results in slightly
lower variability in adhesion force
than the other two primers.

➢ The graph of residuals versus fitted

values in Figure 13.16 does not
reveal any unusual or diagnostic
pattern.

Basic Experimental Design for Source: Douglas C. Montgomery, Statistical Quality Control: A
Process Improvement Modern Introduction, 7th edition, John Wiley & Sons Inc., 2013.
29
Dr.-Ing. Amr Nounou - Spring 2024-2025
 Thank you for your attention ...
Basic Experimental Design for
Process Improvement
30
Dr.-Ing. Amr Nounou - Spring 2024-2025