Concept Evaluation

Dr. Ender Yıldırım

Middle East Technical University

Department of Mechanical Engineering

1
Concept Generation and Evaluation

(Source: Dieter)
2
 Concept Evaluation
• Evaluation involves;
• Comparison
• Decision making
• Absolute comparison: Concept is
compared to a set of requirements.
• Relative comparison: Concepts are
compared with each other.
• To make a valid comparison, concepts
must have the same level of abstraction.

3
 Concept Evaluation
EXAMPLE
Function: To move some object.
The force can be applied by various CONCEPTS
• a hydraulic piston
• a linear electric motor
• the impact of another object
• magnetic repulsion

• Different levels of abstraction!!!

• First two are refined mechanical components, last two are basic
physical principals

4
 Concept Evaluation

• Use decision matrices...

• ...to come up with the best concept
OR
• ...to identify concept that are worth refining.

5
 Pugh’s Concept Selection Method
(Basic Matrix Selection Method)
• Compare each concept to a reference concept.

• Determine the criterion to base the comparison.

• For each criterion, determine if the concept in question is

• better than,
• poorer than,
• or about the same as the reference concept.

6
Steps in Pugh’s concept selection method
STEP 1: Choose the criteria by which the concepts will be evaluated.
STEP 2: Formulate the decision matrix.
STEP 3: Clarify the design concepts.
STEP 4: Choose the datum concept.
STEP 5: Run the matrix.
STEP 6: Evaluate the ratings.
STEP 7: Establish a new datum and rerun the matrix.
STEP 8: Plan further work.
STEP 9: Secondary working session.
7
Steps in Pugh’s concept selection method
STEP 1: Choose the criteria by which the concepts will be evaluated.
• Design criteria related to the requirements

STEP 2: Formulate the decision matrix.

• Criteria are entered to the matrix as row headings.
• Concepts are the column headings of the matrix.
• Use same level of abstraction
• if sketch use sketches for all.
• If textual, use textual description for all concepts in the matrix.

8
Steps in Pugh’s concept selection method

STEP 3: Clarify the design concepts.

• Bring all members to a common level of understanding for
each concept.

STEP 4: Choose the datum concept.

• Choose an average concept.
• Note that a poor one will make all look good.
• Mark this column as DATUM.

9
Steps in Pugh’s concept selection method
STEP 5: Run the matrix.
• Concepts are compared with respect to the datum for each criterion.
• Three level scale:
• Better (+)
• Worse (-)
• About the same (S or 0).

STEP 6: Evaluate the ratings.

• Sum the ratings.
• Try to use the useful features of poorly rated concepts.
• Try to improve the highly rated concepts in regards to lowly rated
criterion.

10
 Steps in Pugh’s concept selection method

STEP 7: Establish a new datum and return the matrix

• Choose the best-rated concept as the DATUM.
• Eliminate the lowest rating concepts.

STEP 8: Plan further work.

STEP 9: Secondary working session.

• Over half of the concepts in the first session are eliminated.
• Objective: improve on the best existing.

• Continue this process until a final decision is reached.

11
 Pugh’s concept selection method
Concept Selection
• Concept generation and selection, viewed as alternating divergent and
convergent processes.
[Source: Dieter, Engineering Design, Fig 5.14, pp 186]

12
 Pugh’s concept selection method
EXAMPLE
• Concepts of the
CD jewel case
analyzed with a
Pugh’s concept
selection
method.

13
 Pugh’s concept selection method
EXAMPLE: Five concepts of the CD jewel case analyzed with a
Pugh’s concept selection method. (Dieter, Engineering Design , pp. 187)

Customer Requirements Selection Criteria

❑ Crack-resistant case ❑ Manufacturing cost
❑ Scratch-resistant case ❑ Easier opening
❑ Hinge doesn’t come apart ❑ Easier to remove leaflet
❑ Easier opening ❑ Easier to remove CD
❑ Easier to remove leaflet ❑ Hinge doesn’t come apart
❑ Easier Extraction of CD ❑ Stacking stability
❑ Fits hand better ❑ More secure locking
❑ More secure locking ❑ Fits hand better
❑ Recyclable plastic
❑ Cost

14
 Pugh’s concept selection method
EXAMPLE: Five concepts of the CD jewel case analyzed with a Pugh’s concept
selection method. (Dieter, Engineering Design, pp. 187)

15
 Pugh’s concept selection method
• EXAMPLE: The BikeE Rear Suspension Design Project

(Source: Ullman, The

Mechanical
Design Process)

16
 Pugh’s concept selection method
• EXAMPLE: The BikeE Rear Suspension Design Project

• Customer requirements • Selection Criteria

• Smooth ride on streets. • Smooth ride on streets.
• Eliminate shock from bumps. • Eliminate shock from bumps.
• No pogoing. • No pogoing.
• Easy to adjust for different
• Easy to adjust for different weights. weights.
• Easy to adjust for different heights. • Easy to adjust for ride harness.
• Easy to adjust for ride harness. • Good enviromental insensitivity.
• No noticable temperature effect. • Easy to maintain.
• No noticable dirt effect. • Looks like a suspension.
• No noticable water effect. • Easy to manufacture.
• Easy to maintain. • Latitude and sensitivity known.
• Looks like a suspension. • Understand critical parameters.
• Easy to manufacture.

17
 Pugh’s concept selection method
• EXAMPLE: The BikeE Rear Suspension Design Project

Conceptual configurations for

energy management system

• Air shock
• Spring coil and oil
• Elastomer

18
 Pugh’s concept selection method
• EXAMPLE: The BikeE Rear Suspension Design Project

(Source: Ullman, The

Mechanical
Design Process,
Table 8.3, pp.188)

19
 Improved selection methods
• In Pugh’s selection method, one concept may turn out to be
better than another concept.

• Do we know how apart they really are?

• We may actually need more quantitative results to make a

better selection.

• We need a way to rank ideas in a scaled manner.

20
 Measurement Scales
1. Nominal Scales
• Named categories (long, short, shiny, etc).
2. Ordinal Scale
• Items are ranked.
• Better or worse
• No information about how apart elements are
• Pugh’s method uses this scale.
3. Pairwise comparison
4. Interval scale
• Use a 1-10 scale to rank elements being compared.
5. Ratio Scale
• Interval scale with 0 reference ranking.

21
 Weighted Decision Matrix
• Rank the design criteria with a weighting factor.
• It is preferred to choose criteria that is measurable such that for each
concept a numerical value can be assigned against each criterion.
• Score each concept against each design criterion using a ratio
scale.
• Use 5 point (0-4) scale when information regarding criteria is not very
detailed.
• Use 11 point (0-10) scale when criterion information is complete.
• Multiply the scores for each concept for corresponding
weighting factor of the criteria being considered.
• Add up the weighted ratings of concepts.
• Declare the concept with highest score, winner.

22
 Weighted Decision Matrix
(Evaluation scheme for design concepts)
11-Point 5-Point
Description Description
Scale Scale
0 Totally useless solution
0 Inadequate
1 Very inadequate solution
2 Weak solution
1 Weak
3 Poor solution
4 Tolerable solution
5 Satisfactory solution 2 Satisfactory
6 Good solution with a few drawbacks
7 Good solution
3 Good
8 Very good solution
9 Excellent (exceeds the requirements)
4 Excellent
10 Ideal solution

23
 EXAMPLE - Weighted Decision Matrix
(Crane Hook)

Example: Design of a Steel Crane Hook

“...A heavy steel crane hook, for use in supporting ladles filled with
molten steel as they are transported through the steel mill, is
being designed...”

24
 EXAMPLE - Weighted Decision Matrix
(Crane Hook)
• Concept alternatives
1. Steel plates joined with welds.
2. Steel plates joined with rivets.
3. Monolithic Cast-steel hook.

• Selection of Evaluation Criteria

• Product design specifications as prime source for these criteria
• Material Cost: units - cents/lb
• Manufacturing Cost: units - US dollars
• Reparability: Evaluate based on Experience
• Durability: Evaluate based on Experience
• Relaibility: Evaluate based on Experience
• Time to produce: units – hours

25
 EXAMPLE - Weighted Decision Matrix
(Crane Hook)
• Construct a hierarchial objective tree.

Crane Hook

Cost Quality in service

Matl. Cost Mfg. Cost Reparability Time to

Durability Reliability
produce
C1=1.0

C1.1=0.6 C1.2=0.4

C1.1.1=0.3 C1.1.2=0.5 C1.1.3=0.2 C1.2.1=0.6 C1.2.2=0.3 C1.2.3=0.1

26
EXAMPLE - Weighted Decision Matrix
(Crane Hook)

27
 Analytic Hierarchy Process (AHP)

• A more structured way of constructing a Weighted Decision

Matrix.
• Well-suited for problems when design objectives have an hierarchical
structure.
• Helps with deciding on weights of each criteria.
• Uses pairwise comparison when finding weighting factors for
evaluation criteria.
• Uses pairwise comparison when finding ratings of concepts
against the design criteria.

28
 Saaty’s Fundamental scale for pairwise comparison

Intensity of
Definition Description
importance
Two activities contribute equally to
1 Equal importance
the objective
Judgment and experience slightly
3 Moderate importance
favor one activity over another
Judgment and experience strongly
5 Strong importance
favor one activity over another
An activity is favored very strongly
Very strong or demonstrated
7 over another; its dominance
importance
demonstrated in practice
The evidence favoring one activity
9 Extreme importance over another is of the highest
possible
These ratings are used to compromise
2,4,6,8
between the above values

29
 EXAMPLE for AHP
Example: Design of a Steel Crane Hook

Concept alternatives
1. Steel plates joined with welds.
2. Steel plates joined with rivets.
3. Monolithic Cast-steel hook.

• Selection of Evaluation Criteria

• Product design specifications as prime source for these criteria
• Material Cost: units - cents/lb
• Manufacturing Cost: units - US dollars
• Reparability: Evaluate based on Experience
• Durability: Evaluate based on Experience
• Relaibility: Evaluate based on Experience
• Time to produce: units – hours
30
 EXAMPLE for AHP
• Construct the square matrix to determine weighting factors (pairwise
comparison)

31
 EXAMPLE for AHP
• Normalize the weighting factors for the selection criteria.

32
 EXAMPLE for AHP
• Pairwise comparison to rank the design concepts with respect to durability
criteria. (do same comparison for other criteria).

33
 EXAMPLE for AHP
• Decision matrix for the steel crane hook obtained by the Analytic Hierarchy
Process (AHP).

34