Day 6:

Design For Quality and Reliability

Design For Six Sigma (DFSS)

& Robust Design
 What is six sigma?
• Developed by Motorola in 1980’s.
• Jack Welch of GE made this as strategic business management
philosophy, saving billions of dollars.
• It is an industry-neutral discipline that can be applied to any
industry sectors, currently it is being applied to 70 different
industry sectors which include IT, manufacturing, Healthcare,
Defence, Banking and finance.
• It is a highly disciplined management tool with a statistical,
analytical and data driven approach.
• It focuses on developing a process, eliminating the defects, and
consequently delivering quality products and services
consistently.
 How six sigma works?
• It works by:
• Being adopted by the whole company as its business
strategy;
• Creating an internal infrastructure within the company;
• Using metrics to measure processes and changes to
processes;
• Using scientific methods, changing the working culture
and introducing business process management.
Motorola six sigma level
 Six sigma & Quality?
• Taking a process to six sigma level ensures that
the quality of a product is maintained, with the
primary goal being increased profits.

• Quality is technically defined as the degree of

excellence of a product or service and
conformance to customer requirement
 Six sigma performance & Benefits?
• Sigma is a statistical metric that corresponds to dpm
(defectives per million)
• Benefits:
• Eliminates the root cause of problems and defects in a process
• Creates robust products and services
• Reduces process variations
• Minimise or eliminate waste
• Ensures customer satisfaction
• Provides process standardisation
• Reduces rework by getting it right first time
• Addresses the key business requirements
• Helps gain competitive advantage
• Achieve organisational goals
A process is a series of steps designed to produce a product and/or
service as required by the customer

X f(X) Y
INPUT PROCESS OUTPUT
Cause Feedback that Effect
suggest changes
Something put into to input Output is the final
a process to its product delivered to
operation to an internal or
achieve an output external customer
DMAIC Philosophy
Six sigma roadmap
 Issues with Six Sigma
• It does not address the customer values deep
enough
• DMAIC based projects could not solve flawed
design
• Non-value added process cannot be removed
 What is Design For Six Sigma (DFSS)?
• DFSS is a new development from six sigma
• It is methodology to enhance new product and service
development process
• It provides more systematic way to manage the
deliverable, resources and trade-offs
• It helps you to deliver better products and services that
your customers want and willing to pay for at six sigma
level.
Six sigma & DFSS opportunity
Moving from Reactive to Predictive Quality
DFSS Methodology
DMADOV - Define
DMADOV - Measure
DMADOV - Analyse
DMADOV - Design
DMADOV - Optimise
DMADOV - Verify
DMAIC to DFSS
 Advantages of DFSS
• Provide structure to development process
• Anticipate problems and avoid them
• Reduce life cycle cost
• Improve product quality, reliability and durability
• Cultural change
• Minimize design changes
• Improve communication between functions
DMAIC & DFSS Comparison
DFSS – 14 Steps
DFSS Tools
What is Robustness?
 What is robustness?
• In the context of Robust Design we formally define robustness as: the
lack of sensitivity in performance to variation in external influences –
so for example:
– Would we expect to get the same level of fuel consumption from a
car if any four people were travelling in it?
– Would we expect to see a difference in the timekeeping of a watch
as its battery ran down?
• In other words, Robustness is about achieving the required degree of
consistency, within a tolerable range of variation in whatever we (or
the Customer) is interested in
– This degree of consistency is the basis of a robustness requirement
Robustness is at the heart of Robust Design

Every design concept has its own "mountain"

 …so What is Robust Design?
• Robust Design gives us a means to:
– Systematically explore the design space
– Understand the parameters that drive design sensitivity
– Include a robustness criterion to the optimisation problem
– Quantify the expected behaviour of the population
…much earlier in the design process, so that our designs
consistently maintain a high level of conformance to the
design specification
 The Building Blocks of Robust Design
Define

Understand the design

Capture the design Generate concepts space and pick a
requirements and make a selection design point that
meets requirements
nominal
specification concept
design

Characterise Optimise Verify

Identify sources of Verify the design and

Optimise the design
variation and check monitor in-service
parameters
design sensitivity behaviour

robustness robust design

assessment optimum verification

These building blocks are sometimes referred to as "DCOV"…

 The Robust Design Approach:
Understanding Robustness
2. Identify the critical 3. Use a structured 1. Capture the critical
design parameters that experimental approach customer requirements
influence performance to explore the available
design space to define Target
the Nominal Design Performance
x1
Lower Upper
Limit Limit

5. Quantify that
x2
Nominal
expected OK
variation Design Not OK Not OK

xn 6. Model how the input 7. …to identify what

variation is transmitted variation in performance
through the design… we can expect…
4. Understand why the design
parameters might vary
around their nominal values: 8. …so we can predict with
- Manufacturing variation? quantified probability how
- Environmental variation? consistently the design will meet
- Customer usage? the customer requirements
 Example: A Reciprocating Pump

• Throughout the course we will

refer to this simple example that
everyone can relate to in order to
demonstrate the concepts and
techniques of Robust Design
Reciprocating Pump: Define Requirements
• Customer satisfaction is • For cleanliness our Critical to
affected by: Quality Characteristic (CTQ)
– Cost of purchase and use has been defined as pump
– Cleanliness flow rate with a target of 10±1
– Capacity L/min, with an associated
– Reliability/Durability probability of conformance
– Ease of use
– Noise level
– Special features
– Safety
 Reciprocating Pump:
Define the Transfer Function for the CTQ
• From first principles we define
flow rate as:

F =– (π R2 ×radius
R =×Piston L × 10-6 - B) × N
– L = Stroke length
– B = Back flow
– N = Motor speed
 Reciprocating Pump:
Define and Quantify the Sources of Variation
• For pump flow rate, one of the important sources of variation is
manufacturing
• The piston and cylinder are manufactured in-house; the valves
and motor are purchased; specifications as follows:
 Reciprocating Pump: Characterise
Robustness

• At the extremes of the expected manufacturing

variation, the flow rate could vary from 8.5 up to 11.5
l/min! All parts are "within spec." but what level of
problem will this give to the customer?
Reciprocating Pump: Optimise Robustness
• Your objectives are:
– Maximise Dpk (a value of 2 or more is considered excellent)
– Minimise Cost Ratio (a value of 1 represents baseline cost)
• Subject to the following constraints:

• Dpk is a robustness metric that is adversely influenced by off-target

performance, and by variability in performance; higher Dpk is better
Generating and Selecting Concepts
Tools for Generating Concepts
Functional Means Analysis

FMA is a matrix method to:

List brainstormed possible

solutions against desired
functionality

Link different possibilities

and combinations to create
different concepts

Solution Solution
Concept A Concept B
Tools for Generating Concepts
Functional Means Analysis

Step 1:
list the functions in a
matrix format
Tools for Generating Concepts
Functional Means Analysis

Step 2
Brainstorm possible
solutions to meet the
functionality
T3

Tools for Generating Concepts

Functional Means Analysis

X X X Step 3:
Eliminate the no-hopers
X X ? Investigate other options in
more detail where
X X necessary before decision
to eliminate or not.
Remaining options are
? X X feasible.
X X ?
T2

Tools for Generating Concepts

Functional Means Analysis

Step 4
Create the various
solutions
You can choose a single
means for a function for
more than one concept if
you wish

Solution Solution
Concept A Concept B
 Tools for Ranking Concepts – Pugh Matrix
• The Pugh Matrix is a simple method to compare
concepts
• It is a qualitative tool but can be highly effective
• The Pugh Matrix (simple or weighted) can be used as a
first pass selection tool (screening out the chaff from a
large number of alternatives), thereafter using (if need
be) concept selection tools with a higher resolution
• We will illustrate the weighted Pugh matrix, since the
un-weighted version is a trivial simplification
 Weighted Pugh Matrix

1. We must have a set of criteria

3
by which to judge alternative
concepts; these should be the
Customer Requirements
2. For Weighted Pugh we also
need their relative importance 1 2
– Determined using a tool such as
AHP – see later
3. In assessing the merits of a
concept, compare it against the
target for each requirement and
decide how well it will likely
meet that requirement...
 Weighted Pugh Matrix

• Calculate the overall strengths

and weaknesses of each concept
against the target requirements...
• Then make a decision: here, the
best concept is B - or a "hybrid"
of B and C that takes features for
performance and aesthetics from
C is possible?

(7.5×0.116) + (5.0×0.307) + (7.5×0.148) + (2.5×0.111) + (10.0×0.081) + (7.5×0.237) = 6.4

 Tools for Requirements Prioritisation:
AHP - The Analytical Hierarchy Process
• AHP is a numeric and structured approach to decision making.
It also provides a consistency index
• The mathematics behind the computation of the relative
importance is fairly involved, but tractable
• We can approximate it with Excel, but specialist software is
available for AHP (Qualica)
 We use paired comparison matrices to get weightings for
each lower-level requirement relative to each requirement in
the level above

One of the matrices looks like this:

We must complete the lower

half of the triangular matrix
according to the scale given
previously. We must repeat
this for each criterion!
 Paired Comparisons
• When performing the paired comparisons the phrasing and ordering of the
question is VERY important. This is often the most difficult part!
If we were comparing the customer requirements of aesthetics (ROW) and cost
(COLUMN) we could ask:

"How much more important to the customer is improved aesthetics

than reduced cost?"

• Or we could be more specific if there were targets x and y for improvement:

"How much more benefit to the customer would it be to improve the perceived
aesthetics by x% than to reduce cost by y%?"

• We use a standard scale based on importance for making these judgments that
accounts for cases when improved aesthetics is either more or less important than
reduced cost…
 Paired Comparisons: Standardised Scale
Intensity of
Definition Interpretation
Importance

1 Equally important Two requirements contribute equally to overall satisfaction

Experience and judgment slightly favour one requirement

3 Considerably more important
over another in achieving overall satisfaction

Experience and judgment strongly favour one requirement

5 Essentially more important
over another in achieving overall satisfaction

Experience and judgment very strongly favour one

7 Demonstrably more important
requirement over another in achieving overall satisfaction

Experience and judgment completely favour one

9 Absolutely more important
requirement over another in achieving overall satisfaction

Sometimes one needs to interpolate a compromise

2,4,6,8 For compromise between the above values judgment numerically because there is no good wording to
describe it

If requirement X has one of the above numbers assigned to The inverse meaning to the above: If comparing X with Y
Reciprocals of
it when compared with requirement Y, then Y has the then a reciprocal value would mean that X was dominated
above
reciprocal value when compared with X by Y; e.g. 1/5 means that X is strongly dominated by Y
Paired Comparisons: Results, Consistency Ratio
We also get a "Consistency Ratio";
thresholds for adequate consistency are:
< 0.05 for a 3x3 matrix
< 0.09 for a 4x4 matrix
< 0.10 for higher order matrices
T4

Paired Comparisons: Results, Consistency Ratio

• If the consistency value is significantly above the threshold
value, then check the numbers entered, but do not fudge
them to achieve consistency: inconsistency is to be expected
from humans and so should only be managed
• In severe cases it means that the scoring could have been
done by picking numbers from the scale at random! It is then
worth examining the rationale for the scores given in the
paired comparisons:
– Was a question phrased the wrong way around?
– Are the comparisons made too extreme? (nothing is ever
simply "black and white")
Paired Comparisons: Results, %Importance
We end up with a set of
%Importance values for the
requirements, from which we can
deduce relative importance, e.g.
Reliability is twice as important as
Performance, which in turn is
twice as important as Servicability!
 Exercise: AHP and Weighted Pugh

In teams of 2 or 3 use your judgment and the Weighted Pugh Matrix method to assess the relative
merits of the three design concepts given below, based on the criteria given and their relative
importance (determined by AHP Prioritization)
The overall objective is to choose the best concept for you to buy for naming, signing and dating
certificates of achievement over a 5 year period. (It is expected that over 5000 certificates will be
needed in this period. Writing MUST be neat and look attractive). The concepts are:
1. Quill pen (cost = £5.00)
2. A standard ball point pen (cost = £0.50)
3. A classic fountain pen (cost = £55.00)
The criteria, and sub-criteria, have been brainstormed to be:
● Cost of Ownership (purchase cost of pen(s), maintenance cost, cost of consumables)
● Quality (durability of pen, appearance of written word)
 Exercise: AHP and Weighted Pugh

• It is likely that several ball point pens will be required over the duration of the
task, which contributes to their overall purchase cost. Ball point pens are
durable, but tend to produce "blobs" after a lot of use
• Quill pens are even less durable – they tend to break and split after a while,
which contributes to their overall purchase cost. They also require a lot of
maintenance – they require regular sharpening and cleaning. They also use a
lot of ink and can tend to give inconsistent results if not kept in tip-top
condition
• Fountain pens use a moderate amount of ink, and also require cleaning after
each use – which entails flushing the ink chamber out with water. A fountain
pen should last a lifetime, if maintained properly, and give consistent results
 Exercise: AHP and Weighted Pugh
1. Open file requirements_prioritisation_and_concept_selection.qdb
2. Using AHP, gain an understanding of the requirements structure and the
necessary paired comparisons and them complete them (see overleaf for
the process to follow)
3. Observe the calculated importance and consistency indices for
requirements
4. Using Weighted Pugh, select the best concept and comment on how
much better it is than the others
 Process to follow for exercise if using Qualica (1)
• First we must rate the relative importance of the top-level criteria (requirements) to determine
their influence on the choice of design we make…
1. In the Browser, expand the Requirements Prioritization folder by clicking on the down arrow
2. Double-click on VOC AHP Prioritization to open the tab in the Graphics pane
3. Ensure that the top-level requirements, VOC are displayed by clicking on the left or right-hand arrows
underneath Group, as necessary
4. Fill in the single comparison between quality and cost in the single unshaded cell in the lower-left part of the
matrix, clicking once to select the cell, then once more to access the list of options to choose from
5. Click on the recalculate icon to update the Importance values

1
2 3

Question to ask:
"How much more important is better
quality than lower cost of ownership?"
If it is more important, choose a value >1;
if less important, choose a value <1 and if
equally important, choose 1.

4
 Process to follow for exercise if using Qualica (2)
• Continue rating the relative importance of the criteria by comparing the lower-level requirements
of cost of ownership to each other…
1. Switch group so that cost of ownership is displayed by clicking on the left or right-hand arrows underneath
Group, as necessary
2. Fill in the three comparisons between purchase cost, maintenance cost and cost of consumables in the three
unshaded cells in the lower-left part of the matrix, clicking once to select the cell, then once more to access the
list of options to choose from
3. Click on the recalculate icon to update the Importance values

Example question to ask:

"How much more important is lower
maintenance cost than lower purchase cost
in achieving lower cost of ownership?"
If it is more important, choose a value >1;
if less important, choose a value <1 and if
equally important, choose 1. Repeat for all
other comparisons.

2
 Process to follow for exercise if using Qualica (3)
• Complete rating the relative importance of the criteria by comparing the lower-level requirements
of quality to each other…
1. Switch groups so that quality is displayed by clicking on the left or right-hand arrows underneath Group, as
necessary
2. Fill in the single comparison between appearance of written word cost and durability of pen in the single
unshaded cell in the lower-left part of the matrix, clicking once to select the cell, then once more to access the
list of options to choose from
3. Click on the recalculate icon to update the Importance values

Question to ask:
"How much more important is better
appearance of written word than greater
durability of pen in achieving higher quality?"
If it is more important, choose a value >1;
if less important, choose a value <1 and if
2 equally important, choose 1.
 Process to follow for exercise if using Qualica (4)
• Now we must rate the ability of our design choices to fulfil each criteria…
1. In the Browser, expand the Concept Selection folder by clicking on the down arrow
2. Double-click on Pugh Matrix (VOC) to open the tab in the Graphics pane
3. Fill in the three columns for quill pen, ball-point pen and fountain pen, clicking once to select a cell, then once
more to access the list of options to choose from
4. Click on the recalculate icon to calculate the overall merits of the concepts
5. Evaluate the results of the comparison and decide a way forward

1
2 Example question to ask:
"In relation to the target set for the
requirement, how well do I expect this
concept to perform?" Choose between:
-- unacceptable
3 - borderline
O almost meets requirement
+ meets requirement
++ exceeds requirement
5
 Example final result:

Fountain pen in this case is the superior choice,

with ball-point pen clearly the worst choice…
Understanding Engineering Systems:
P-Diagrams
 What is a P-diagram?
• P-diagram = Parameter Diagram
• For any given system, component or sub-system, it lists, categorises and
collates inputs and outputs for further use:
– Inputs
• Noise factors
• Control factors
• Signal factors
– Outputs
• CTQ (Ideal function)
• Side effects
 The system will experience
The Generic P-diagram variation from these. Can it
cope? THESE ARE WHAT THE
SYSTEM HAS TO BE ROBUST
Noise factors TO

Noise Factors/Sources of Variation are all

How do we those identified input parameters to a design
interact with the whose values cannot be set directly but will
system to get the vary and will influence the performance and What is the key
desired level of robustness of a design output from the
output? system?

Signal factor System CTQ

What are the
Stimulus to the Our intended output or
potentially
system the ideal output
harmful
consequences of
operating the
What are the aspects of Control factors Side effects system?
the design or environment
that are within the scope of Input parameters to a design Noise factors cause these side
the Designer to set? whose values can be set effects. Outside Rolls-Royce, these
directly in order to influence the are called error states. They are
performance and robustness of what we get as well as what we
a design intended
P-diagrams can be used in many contexts

Defining the Rig test Defining the

Concept Systems
variables for design or Robustness variables for
generation/ Design:
Design of building an Assessment Design
selection Linking
Experiments Analysis Optimisation
(debugging) sub-systems
Chain together
 Noise Factors in more detail
• Noise Factors are parameters that cannot be, (or chosen not
to be), deterministically controlled by the designer in real life
• Noise Factors are simply those factors that will affect the
ability of the design parameters to achieve the required
output
• Thus, Noise Factors cause the CTQ to deviate from target,
affecting our ability to meet the customer CTQ consistently
• Noise factors are the causes of variation
 The Impact of Noise Variation
Nominal Design
All these variation
Unit to Unit sources impacting
Variation the design before
Manufacturing Variation
the product even
Unit to Unit Test /
Measurement
enters service . . .
Variation
Uncertainty
Measurement Variation

Deterioratio
n . . . and even
Change Over Time
External Customer more variation
Usage
Noise Profiles sources once in
Operator Usage service!
External Noise
Environmental Variation

Variation in Performance
P-Diagram – Example for Vehicle Braking
There may be other CTQs that you are
Noise factors interested in. In this context we are
focusing on the CTQ for which robustness
Road conditions is the issue.
Manufacturing variation It is recommended to have a p-diagram
Vehicle load variation linked to only one CTQ.
Pad deteroriation The other CTQs will not be forgotten!
(They may appear as constraints in the
robust optimisation phase, for example)

Signal factor System CTQ

Force on the Stopping
brake pedal for distance
time t

Control factors Side effects

Squealing – leading to...

Pad material
Brake dust – leading to...
Pad thickness
Heat- leading to...
Pad area
Vibration – leading to...
Disc material
Systems Engineering: Linked P-Diagrams
• During the breakdown of the system into
subsystems and functionality, links between
P-diagrams are established
• Remember: consider and write down the
consequences of the side effects.
– Consider these in Systems Design and assess
their impact qualitatively via Failure Modes
and Effects Analysis and quantitatively via
robustness assessment of linked subsystem.
 Exercise: P-diagram
• You have been tasked with performing a robustness assessment of a
petrol filler flap design. The Voice of the Customer indicates a
problem with inconsistency in the opening force:

• As a result of these investigations, the CTQ is determined to be

Opening Force = 5N±1N – how robust is the design to this spec.?
 The Filler Cap Design

Flexible Plastic latch

Finger Plastic flap
assembly: 2% glass
location assembly -
filled nylon -
painted
unpainted
 The Filler Cap Design: Scope

Flap and
hinge IN Jaw
SCOPE assembly
OUT OF IN SCOPE
SCOPE
A Sample of Geometric Parameter
Definitions
Create a P-diagram to capture sources of variation impacting on
design performance

• Consider "what" we have control over to achieve the target

value of the CTQ (control factors/design parameters)
– Make a list
• Consider each control factor in turn and ask “why might it
vary from its nominal value”. Recall the different types and
categories of noise
– Make a list
• Now ask "why else might the value of the CTQ vary?"
 3
What are the things we
cannot keep constant but
affect the output?

1
What is the user-applied
stimulus to the system?

4
What are the side
effects of
continually
opening and
closing the flap?
2 Try also to
identify the
What are the things which consequences.
affect pull force that we are at
liberty to change in the design
of the system
20 minutes
 Exercise Debrief
• What were the issues encountered during the
exercise?
• What might have made the outcome better?
• Did the generation of a P-diagram improve
your understanding of the system?
 Purpose of DoE
• In Robust Design, DoE is used in a number of ways:
– To systematically explore the design space available in order to
understand the key parameters ("control factors") that drive the
performance characteristic in which you are interested
– To "screen out" those factors that have negligible effects in order to
simplify the problem
– To detect and quantify interactions between control factors
– To determine a nominal design
– To determine robustness of a nominal design
– To build fast-running "surrogate models" of slow-running analysis
codes to make practical a comprehensive search of the design space
and use of robust optimisation techniques that require a large number
of iterations of the analysis codes
 Inherent Complexity
• Modern products are inherently complex systems:
– Many system variables
– Interactions between variables
– Wide operating conditions
– Sensitive regimes
• It is obvious and essential that to design and build the best product requires
knowledge of the complexity such that behaviour is predictable

Input (Xs) Process/

Product (Xs)
Output (Ys) CTQs

• We need to know Y = ƒ(X1, X2 … Xn)

 DoE Roadmap – Increasing Sophistication of Knowledge
Required from DoEs
Determine Y and Screening Detailed Study Experiments Confirming
potential Xs Experiments on reduced number of factors Experiments

Affinity and P- Diagrams

Fractional Factorial Full or Fractional Response Surface

or Placket-Burman Factorial methods
Design of with centre
Experiments points
 The Simplest DoE 2x2
• Consider an experiment where there are 2
Factors, A and B each having two experimental
levels (say -1 and +1)
• To cover all combinations we need four
experiments: Run Factor
A B
1 -1 -1
2 -1 +1
3 +1 -1
4 +1 +1
T5

Analysis 2x2 DoE

• Analysis Of Means
Run Factor Result
A B Average for
A-1 = (R1 + R2)/2
1 -1 -1 R1
2 -1 +1 R2 3 +1 -1 R3for
Average
A+1 = (R3 + R4)/2
4 +1 +1 R4
Factor A has been run twice Factor A has been run twice at
level +1 so we can calculate the We can compare
at level -1 so we can
average for this averages for A
calculate the average for this
irrespective of B
 MainExperiment
Effects Plots
al
averages

Output
Output

-1 +1 -1 +1
Factor A Factor B

The output (response) scale is the same on both Main Effects

plots allowing direct comparison. In this case, on average,
Factor A has a greater effect on the output than factor B
 Interaction Plots
No Interaction
Lines are parallel. Effects of
factors A and B on the
response are independent. Interaction
The change in output due to Lines are NOT parallel.
changing Factor A is the Effect of factors A and B
same for both settings of on the response are NOT
Factor B Independent

Factor B at
+1

Output
Output

Factor B at
Factor B at -1
-1 Factor B at
+1

-1 +1 -1 +1
Factor A Factor A
 Example 2x2 DoE
• The impact resistance carbon fibre structures is considered to be affected by
number of layers and lay-up angle.
A 2x2 experiment was set up and run with the following results:

DESIGN EXPERIMENT
Run Factor Factor Impact
A B Layers Angle Resistance
1 -1 -1 4 30 275
2 -1 +1 4 50 285
3 +1 -1 7 30 270
4 +1 +1 7 50 325
Main Effects Plots

30 30
Impact Resistance

Impact Resistance
0 0
29 29
0 0
28 28
0 0
27 27
0 0

3 5
4 No 7 Angl
0 0
Layers e
 Interaction Plots
Angle 30 4 Layers

Angle 50 32 7 Layers
32
0 0
31 31

Impact Resistance
0
Impact Resistance

0
30 30
0 0
29 29
0 0
28 28
0 0
27 27
0 0

4 7 30 50
No Angle
Layers
Do the two interaction plots say the same thing?
 Interpreting Interactions
Negligible A Main
Positive A Main
Effect Effect

Response Y
Response Y

Factor B
Factor B
- -
No No
+ +
Interactio Interactio
n n
- -
+ Factor + Factor
A A

Response Y
Response Y

Factor B
-

Factor B
- Small
Small
Interactio + Interactio +
n n
- -
+ Factor + Factor
A

Factor B
A -

Response Y
Factor B

-
Response Y

Large Large
Interactio + Interactio +
n n
- -
+ Factor + Factor
A A
 Beyond 2x2 DoE: “Full Factorials”
No. Factors Standard X1 X2 X3 X4 X5 …
Order
1 - - - - -
2 + - - - -
3 - + - - - There is a basic
K=2 4 + + - - - pattern for determining
5 - - + - - the experimental
6 + - + - -
array called the
7 - + + - -
K=3 8 + + + - - "design": E=LK
9 - - - + -
10 + - - + -
11 - + - + -
12 + + - + -
13 - - + + -
14 + - + + -
15 - + + + -
K=4 16 + + + + -
17 - - - - +
18 + - - - +
19 - + - - +
20 + + - - +
21 - - + - +
Let’s look at a 7 factor (A,B,C,D,E,F & G) full factorial design.
Total Number of combinations =
What effects are we able to detect if we were to run all these possible
combinations?

Likelihood of
Term Example Qty DoE Options
Occurrence
Constant - 1 - 1.

Main Effects A, B, C …
2.

Clarity Decreasing
2-Way Interactions AB, BC, …

3-Way Interactions ABC, ABD .. 3.

4-Way Interactions ABCD,…

4.
5-Way Interactions ABCDE, ..

6-Way Interactions ABCDEF, .. 5. FULL FACTORIAL – 128 RUNS

7-Way Interactions ABCDEFG

TOTAL 128
 Drawing Conclusions from DoE
• You should always remember why the experiments were being conducted
and draw conclusions accordingly
• Remember an experiment is only as good as the experimenter.
Conclusions can be erroneous if:
– The wrong factors are flexed
– The wrong design matrix was chosen
– The factors’ levels are inappropriate
• Check conclusions versus what you expected. DoE should confirm what
has already been proven, and should enlighten you about phenomena
that you weren’t aware of or didn’t fully appreciate
• If results are counter-intuitive check out your experimental strategy,
execution and analysis with an expert. If conclusions still hold true, then
you have learnt something new!
 Types of Surrogate Models
Used in Robust Design
• Simple linear surrogate models
• Quadratic surrogate models for simply-curved
unimodal response surfaces
• Radial Basis Functions (RBFs) for curved or
‘peaky’ response surfaces
 Simple Linear Response Surface
Response surface for the
surrogate model
2-levels for each factor can
be used to generate a linear
surrogate model.

The training dataset is used

to determine the coefficients
for a linear equation such as

Resistance = 356.67 – 24.17.Layers – 2.5.Angle + 0.75.Layers.Angle

The Surrogate Model

 Quadratic Response Surfaces
1
All 4 of these response surfaces can be
modelled by the quadratic equation:

2 y = □ + □x1 + □x2 + □x12 + □x22 + □x1x2

Would a quadratic surrogate model capture

the response shown below?
3

4
Surrogate Models: Isight example
 Summary: SURROGATE MODELS
• Linear models may be adequate in many cases (e.g. modelling
a localised region of the design space)
• Quadratic models have a broader application and have the
advantage of simplicity and explicitness
– Minitab has a range of response surface modelling design
arrays that have desirable statistical properties
– Isight can be used to fit a limited range of Polynomial
models (up to quartic models)
• RBF’s should be used when the underlying response has a
high degree of curvature and/or is multi-modal
– Isight can also be used to fit RBF models (Minitab cannot)
T1

The Building Blocks of Robust Design

Define

Understand the design

Capture the design Generate concepts space and pick a
requirements and make a selection design point that
meets requirements
nominal
specification concept
design

Characterise Optimise Verify

Identify sources of Verify the design and

Optimise the design
variation and check monitor in-service
parameters
design sensitivity behaviour

robustness robust design

assessment optimum verification
 Purpose for Robustness
Quantification
• The purpose of having robustness
quantification is to be able to quantitatively
compare one candidate design to another
• This may a relative comparison (e.g.
comparing standard deviations), or it may be
an absolute comparison based on the
expected probability of conformance to an
engineering CTQ specification (such as life)
 Robustness Metric: ΔY
• This approach is useful when:
– You don’t know the transfer function Y=ƒ(X)
– You don’t know the value for σX for the noise factors, or σX
doesn’t make sense
– You know roughly what size of variation in X to expect
• The result is simply:
– Is the change in Y bigger than I can accept when the Xs
change by their expected amounts?
– Which X has the largest influence on ΔY?
Exercise: Δy for the Reciprocating Pump
For the pump, flow rate depends on
swept volume, motor speed, back flow

One way
valve
Tubing Flow-down: transfer function:
F = (10-6 π x R2 x L - B) N

R = Piston radius (mm)

L = Stroke length (mm)
B = Back flow (litres/stroke)
N = Motor speed (strokes/min)
Piston Flow rate specification: 10+/-1 l/min.
 Exercise: Δy for the Reciprocating Pump
The noise variation we expect to see in each of the
design characteristics (Xs) is given by:
Make Buy
Piston Stroke Motor
Back flow B
radius R length L speed N
(mm) (mm) (litres/stroke) (rpm)
Nominal 40 40 0.002 50
Lower limit 39.7 39.4 0.00185 44
Upper limit 40.3 40.6 0.00215 56

The process you will go through in the exercise is

illustrated overleaf…
 Exercise: ΔY for the Reciprocating Pump
1 2
Create a set of Run experiments:
experiments: Calculate flow rate using
4-factor Resolution IV the ‘simulation code’ (Excel
½-fraction in this case for the pump)

3 4
Export the experimental Analyse DoE to find and
results for flow rates store ‘main effects’
into Minitab (ΔYs)

5 6
Take absolute values
and sum them to find Draw Conclusions!
total ΔY for the pump

The first step has been done for you…

 Exercise: ΔY for the Reciprocating Pump
• A 4-factor, Resolution IV (½ fraction), un-randomised DOE array was
created in Minitab in worksheet "pump DeltaY" in the data.mpj Minitab
project file
– Copy the design array from Minitab and paste into the YELLOW CELLS
in worksheet "DeltaY" in the Excel spreadsheet pump.xls
– Copy
YELLOWthe flow rate results from the BLUE CELLS in the worksheet and
CELLS
paste into column C9 "Flowrate" in Minitab
– Follow the instructions on the next slideBLUE
to perform
CELLS an Analysis of
Means, and hence obtain the ΔY values for each factor. These are
simply the factor main effects!
 Exercise: ΔY for the Reciprocating Pump
• Use Stat > DOE > Factorial > Analyze • Select [Storage…] and in the new dialog
Factorial Design… and select column C9 that appears, tick the box marked
flowrate as the response Effects under Model Information and
• Select [Terms…] and in the new dialog click [OK]
that appears, use the pulldown menu • Click [OK] to run the analysis – the
for Include terms in the model up effects will be stored in column C10 of
through order to "1" and click [OK]" the worksheet named "EFFE1
 Exercise: ΔY for the Reciprocating Pump
• Use Calc > Calculator… and enter DeltaY • Use Calc > Column Statistics… and enter
in the Store result in variable text field, DeltaY into the Input variable text field
type ABS('EFFE1') into the Expression and click on [OK] – the result will be
text field and click on [OK] – the results displayed in the Session window
will be stored in column C11 of the • Question: do we have a robustness
worksheet named "DeltaY" problem?
 Monte Carlo Analysis for
Rear Bearing Support Structure
 Transmission of Variation
(Monte Carlo Simulation)
A single combination of design parameters (a single run of
a simulation code) yields a single result

x1
y

y = ƒ(x1, x2, x3, ...)

x2

x3
Another sample gives another value of y

x1
y

y = ƒ(x1, x2, x3, ...)

x2

x3
In "simple" Monte Carlo sampling, we sample repeatedly from each noise
distribution at random according to their chosen probability distributions.

x1
y

y = ƒ(x1, x2, x3, ...)

x2

Use the sample

standard deviation of
x3 the y-values to
estimate σy
T2

Estimating the output distribution

x1
y

y = f(x1, x2, x3, ...)

x2

x3
 Monte Carlo Simulation of the Reciprocating Pump Flow Rate
in Isight

One way Tubing

Flow-down: transfer function:
valve
F = (10-6 π x R2 x L - B) N

R = Piston radius (mm)

L = Stroke length (mm)
B = Back flow (litres/stroke)
N = Motor speed (strokes/min)
Flow rate specification: 10+/-1 l/min
Piston

We have a transfer function y=f(x) – it’s a ‘fast code’ situation

so we can use Monte Carlo simulation directly.
AN INTRODUCTION TO
OPTIMISATION IN ROBUST DESIGN
T1

The Building Blocks of Robust Design

Define

Understand the design

Capture the design Generate concepts space and pick a
requirements and make a selection design point that
meets requirements
nominal
specification concept
design

Characterise Optimise Verify

Identify sources of Verify the design and

Optimise the design
variation and check monitor in-service
parameters
design sensitivity behaviour

robustness robust design

assessment optimum verification
 Roles in Optimisation
• Optimisation is very much a team activity if it is to be useful…
• The Designer will have overall responsibility for “defining the optimisation problem”:
– Identifying the key objectives to be achieved in the design problem, e.g. weight, cost, performance
– Balancing various constraints from multiple sources that bound the available design space, e.g.
manufacturing, interfaces with other components
– Turning the results of the optimisation in to Statistical Specifications for parameters, in conjunction
with Manufacturing and Modellers to ensure design intent is achieved
• Analysts (of perhaps many disciplines) will have responsibility for turning the problem
definition in to a solvable optimisation problem, choosing an appropriate optimisation
strategy and executing it
• Modellers will have responsibility for constructing the necessary geometric models that can
flex in a way that represents the specified design intent – and for robust optimisation, can
flex in a way that represents the sources of variation present too!
 What is Optimisation?
• Optimisation is about adaptively searching the design space in order to
locate the best design
– Points in the design space to be analyzed are not fixed in advance, as is the
case of Design of Experiments, but are chosen as the search progresses
• The optimisation process may necessitate visiting hundreds, if not
thousands of points in the design space in order to locate the best design
– Unless the analysis is fast running, high quality surrogate models* will be
required to calculate the robustness metrics
– If using surrogate models for optimisation, a validation run of the predicted
optimum using actual code should be completed
• Don’t expect exact agreement but a similar improvement should result
 The Elements of Optimisation
• An objective that it is desired to improve as much as possible:
– For a turbine disc, we might want to maximize the life or minimize the weight
• A set of variables that can be manipulated in order to affect the value of the
desired objective
– In the turbine disc problem, the variables might include the size and shape of certain
features, or the material properties
• A set of constraints that allow the variables to take on certain values but exclude
others
– In the turbine disc problem, it does not make sense to have features sized outside our
manufacturing capability, so we might constrain variables according to the
largest/smallest size that can be made
• Obviously, the parameterisation of analysis models needs to be able to deal with
all these elements
 Some more on Constraints in Optimisation
• In general there will be many other design requirements that
must be achieved in addition to the main objective
– Such requirements will be represented as constraints on design space, dividing
it in to regions of feasibility and infeasibility
• feasible regions of the design space are those in which any chosen design will satisfy
all design constraints
• infeasible regions of the design space are those in which any chosen design will fail
to satisfy one or more design constraints
• A simple example of a design problem posed as an
optimisation problem is:
“By varying disc bore width between 12.0mm and 18.0mm
and disc bore height between 10.5mm and 22.75mm,
minimise disc weight whilst ensuring that disc life exceeds
10,000 flights.”
 So what makes Robust Optimisation Different?
• For robust optimisation, the problem statement will involve a robustness
metric (possibly as an added objective):
– Minimisation of Δy – the change in y for a given change in x
– Minimisation of σy or σy2 – standard deviation or variance of the
output distribution
– Maximisation of Sigma Level, Pc or Dpk – relating to probability of
conformance to specification or process capability
• The metric must be calculated at every point visited through the design
space – this can be computationally very expensive!
T5

Parameter Design Principle

Y

At the new nominal for X1 Y

Change X2 is highly capable but off
target, so we change the
mean of another design
Spec Y

variable (say X2) that has a

relationship with Y such
that it can bring Y back on
target without affecting the
variance of Y

Tol X X1
Exploit non-linearity to re-specify
mean of X1 with same tolerance
T2

Y Tolerance Design Principle

LSL
Spec
USL If variance of Y is still
too large, reduce
variation of one or
more targeted design
parameters by
improving capability

Reduce X3
variation in X3
 Why not simply reduce every σx?
• Cost!
Changes in technology
(e.g. a drilling operation replaced by both
drilling and reaming) lead to discrete
changes in cost

• In general Manufacturing Cost = ƒ(σx1, σx2, σx3, ...)

• σy is more sensitive to some σx’s than others