5/30/2021

Note : These hand outs are used for GB Training of Henry Harwin
Management Academy and for the purpose of GB course reference 2

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 1
 5/30/2021

MEASUREMENT SYSTEM
ANALYSIS

Are My Measurements Consistent?

Applies to meters,
scales, calipers,
rulers.

Also applies to less

obvious situations,
such as consistency
of managers in
rating employees.

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 2
 5/30/2021

Sources of Variation:
The Mathematics
This is the total The real process variation
observed variation
s2total = s2process + s2 measurement system

This is the variation

due to measurement
system

Using a GR&R study, we will focus on Measurement System Variation

Measurement Process

● If the measurement system is poor then:

○ satisfactory parts may be being rejected and
○ unsatisfactory parts may be being accepted
○ a process may appear unsatisfactory when really the process is
all right, but the measurement system is at fault

● The belt should ask and answer:

○ is the observed variation due to my process, my measurement
system or both?
○ How big is the variation in my measurement system?

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 3
 5/30/2021

SOURCES OF VARIATION

Machines
P
Materials R
Methods
O
C
Measurement
E
Mother Nature S
Man
S

VARIATION TYPE

Common Versus Special Causes

Type Of Variation Definition Characteristics

No Undue Influence By Predictable

Common Cause But Uncontrollable
Any One Of The 6-Ms

Undue Influence
Unpredictable
Special Cause By At Least One Of But Controllable
The 6-Ms

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 4
 5/30/2021

OBSERVED VARIATION

Total Variability
(Observed variability)

Process A

s Observed
2
(Total)

Process B

Which Process is would you prefer ?

MEASUREMENT VARIATION

Measurement Total Variability

Variability (Observed variability)

Process A

s Meas.
2
System = s Observed
2
(Total)

Process B

Which Process is BETTER ?

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 5
 5/30/2021

PART VARIATION

Part Variability Measurement Total Variability

(Actual variability) Variability (Observed variability)

Process A

s 2Actual(Par t) + s Meas.
2
System = s Observed
2
(Total)

Process B

Which Process is BETTER ?

SOURCES OF VARIATION in MEASUREMENT

Machines M
E
Materials A
S
Methods U
R
Measurement E
M
E
Mother Nature N
T
Man

Measurement Systems are Most Neglected !!

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 6
 5/30/2021

Accuracy and Precision

● Accuracy : describes the differences between the measurement and
the actual value (average or location)
● Precision : describes the variation (spread) when the same part is
measured repeatedly with the same device (std deviation, variance or
spread)
● There are 4 combinations, imagine playing golf..

Which is best? which is worse?

How’s your golf?

COMPONENT OF MEASUREMENT ERROR

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 7
 5/30/2021

COMPONENT OF MEASUREMENT ERROR

Resolution? Can Change Be Detected ?

OK
Accuracy / Bias?
OK
Linearity?
OK
Stability?
OK
Precision (R&R)?

Each Component Of Measurement Error Can Contribute To Variation Causing Wrong Decisions To Be
Made

DEFINITION

• Resolution / Discrimination
– Capability to detect the smallest tolerable changes

• Inadequate Measurements Units

– Measurement units too large to detect variation present

• Guideline: “10 Bucket Rule”

– Increments in the measurement system should be one-tenth the
product specification or process variation

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 8
 5/30/2021

RESOLUTION

Poor Discrimination

Same process
1 2 3 4 5
output being
measured 1

Better Discrimination

1 2 3 4 5

1.3

ACCURACY / BIAS

Resolution? Can Change Be Detected ?

OK
Accuracy / Bias?
Measurements are “shifted” from
“true” value
OK
Linearity?
OK
Stability?
OK
Precision (R&R)?

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 9
 5/30/2021

ACCURACY / BIAS

Bias: is the difference between the observed average of repeat

measurements and a reference or “correct” value.
Correct
(Reference Standard)

Average
Value

TO IMPROVE ACCURACY
Calibrate When Needed / Scheduled
 Use Operations Instructions
 Review Specifications
 Review Software Logic
 Create Operational Definitions

LINEARITY
Linearity: is the difference in bias over a range of values, ie is the bias the same for
small parts as it is for larger ones

Resolution? Can Change Be Detected ?

OK
Measurements are “shifted” from
Accuracy / Bias?
“true” value
OK
Measurement is not “true” and/or
Linearity? consistent across the range of the
OK “gage”
Stability?
TO IMPROVE LINEARITY:
OK  Use Only In Restricted Range
Precision (R&R)?  Rebuild
 Use With Correction Factor / Table / Curve
 Sophisticated Study Required And Will Not Be Discussed In This Course

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 10
 5/30/2021

STABILITY
Stability: is the drift observed in measurements of a reference part over an
extended length of time

Resolution? Can Change Be Detected ?

OK
Measurements are “shifted” from
Accuracy / Bias?
“true” value
OK
Measurement is not “true” and/or
Linearity? consistent across the range of the
OK “gage”
Stability? Measurement drifts
OK
Precision (R&R)?

STABILITY

• Measurements remain constant and

predictable over time Master Value
(Reference Standard)
– for both mean and standard
deviation Time 1
• No drifts, sudden shifts, cycles, etc. Time 2
• Evaluated using control charts
To Improve Stability Act :
•Change / Adjust Components
•Establish “Life” Timeframe
•Use Control Charts
•Use / Update Current SOP

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 11
 5/30/2021

PRECISION

Resolution? Can Change Be Detected ?

OK
Measurements are “shifted” from
Accuracy / Bias?
“true” value
OK
Measurement is not “true” and/or
Linearity? consistent across the range of the
OK “gage”
Stability? Measurement drifts
OK
Precision (R&R)? Repeatability and Reproducibility

Precision Terms: Definitions

Repeatability: How well one person, can repeat the same
measurement, using the same equipment. Sometimes called “test-
retest error”. If repeatability is poor, look at the measurement tool
itself.
Reproducibility: How well multiple persons, using different test
equipment can get the same answer as each other. If reproducibility
is poor, look at training, standards, definitions, and differences in
measuring equipment.

We study these with Gage R&R Study

This is where we concentrate our Six Sigma activities

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 12
 5/30/2021

REPEATABILITY (A COMPONENT OF PRECISION)

• Variation That Occurs When Repeated Measurements Are Made Of The

Same Item Under Absolutely Identical Conditions
– Same:
– Operator
– Set-up
– Units
– Environmental Conditions
– Short-term

REPRODUCIBILITY (A COMPONENT OF PRECISION)

The Variation That Results When Different Conditions Are Used To Make The
Measurements
– Different:
– Operators
– Set-ups
– Test Units
– Environmental Conditions
– Locations
– Companies
– Long-term

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 13
 5/30/2021

EXAMPLE : Accuracy and Precision

We have two methods of measuring hardness of a rubber sample.
Method 1 is expensive and time consuming
Method 2 is cheap and quick
The “correct” answer is 5.0

Which method is more accurate?

Which method is more precise?
Which method would you prefer to use and why?
If you had to use method 2, what would you do?

Method 1

Method 2

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 14
 5/30/2021

Gage Repeatability
& Reproducibility

Topics

● The requirements for a good measuring system

● Variables MSA -The methods
● Sources of Variation -The Mathematics
● The Indices
○ precision to tolerance ratio
○ %R&R
● Tips on Sample Selection
● Minitab worked example
● Team Exercise

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 15
 5/30/2021

The Measurement System: Requirements

● Must be in Statistical Control

○ check by plotting graphs

● Variability must be small compared to product

specifications
○ calculate ratio called precision to tolerance ratio

● Variability must be small compared to process variation

○ calculate ratio called gage R&R %
● Measurement system discrimination is the ability to detect
small changes in the measured characteristic
○ calculate number of distinct categories

Variables MSA: Methods

● Method of study :
○ Designed Experiment
● Variables:
To be able to assess
○ Operator e.g. 3 operators
Reproducibility
○ Part e.g. 10 parts
● Replication / Trials: To be able to assess
○ do each measurement twice Repeatability
● Summary
○ operator x part x trial experiment
We don’t do MSA daily , but we do it frequently

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 16
 5/30/2021

Variables MSA: Methods

● Experimentation:
○ Minitab General Full Factorial
● Analysis:
○ Stat>Quality Tools>Gage Study>Gage R&R (crossed)
● Interpretation:
○ % precision to tolerance ratio
○ % gage R&R
○ Discrimination
○ % Contribution tree
○ Diagnostics - Graphical analysis

Sources of Variation:
The Mathematics

s2total = s2process + s2 measurement system

This is the variation

This is the total due to measurement
observed variation system

The real process variation

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 17
 5/30/2021

Sources of Variation:
The Mathematics
The total measurement system variation
can be partitioned into that due to
repeatability and that due to reproducibility

s2measurement system = s2repeatability + s2 reproducibility

This is the variation This is the variation

due to the due to reproducibility
measurement system
This is the variation
due to repeatability

Precision to Tolerance Ratio

5 . 15 * s MS 99% of my mes systrem variation should be less than 10%

P /T  Of the total tolerance, otherwise we start rejecting the

Good parts also and accepting the bad parts as well.
Tolerance

Calculates what percent of the tolerance is taken up by

measurement error
Shows how much of the customer specification is taken up by the
precision - the measurement system error
Sometimes called “tolerance theft”
Use P/T ratio to decide if the measuring system is good enough to
measure parts with respect to the specification
Note:
5.15 standard deviations accounts for 99% of MS variation The use of 5.15 is an industry standard

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 18
 5/30/2021

Ratio Rules: Telling Good from Bad

5.15* s Measurement System
P/T Ratio =
Tolerance
Over 30%
Danger
10 to 30%
Caution
Below 10%
Good

% Gage R & R (% GR&R)

s2total = s2process + s2 measurement system

s MS
% R& R   100
s T o ta l

Calculates what percent of the total variation is taken up

by measurement error

Shows how much of the total standard deviation is taken up by the

measurement system standard deviation
Use %R&R to decide if the measuring system is good enough to detect
changes in the process variation. If %R&R is large then improvements may
not be detected.

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 19
 5/30/2021

Ratio Rules: Detecting Process Change

s Measurement System
Gage R&R =
s Observed
Over 30%
Danger
15 to 30%
Caution
Below 15%
Good

% CONTRIBUTION

% Contribution =
s 2R & R *100
s Total
2

• Measurement System Variation (R & R) as a

percentage of Total Observed Process % Contribution
Variation
9%
• Includes both repeatability and reproducibility
1%

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 20
 5/30/2021

% STUDY VARIANCE

sR&R
% Study Variation = *100
s Total

• Looks At Standard Deviations Instead Of Variance

% Study Variation
• Measurement System Standard Deviation (R & R) As A
Percentage Of Total Observed Process Standard 30%
Deviation
10%
• Includes Both Repeatability And Reproducibility

% TOLERANCE
Precision to Tolerance P/T

5.15* s R&R
% Tolerance = *100
Tolerance

• Measurement error as a percent of tolerance

• Includes both repeatability and reproducibility
• 5.15 Study Variation = 99% % Tolerance

Acceptance 30%
Criteria
10%

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 21
 5/30/2021

DISTINCT CATEGORIES

s Process
2
Output
Number of Distinct Categories = 2*
s 2 R&R

• Number of divisions that the Measurement System can

Number of Distinct
accurately measure across the process variation Categories
• How well a measurement process can detect process
output variation--process shifts and improvement 5
• Less than 5 indicates Attribute conditions 10

ACCEPTABILITY SUMMARY

Tabular Method
Process
Control Product Number of
% Study Control Distinct
% Contribution Variation % Tolerance Categories

9% 30% 30% 5

1% 10% 10% 10

DESIRABLE TO HAVE ALL 4 INDICATORS SAY “GO”

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 22
 5/30/2021

MSA Diagnostics:
Understanding the Tables

PREPARE THE ANALYSIS : Example

Use the commands
> Stat > Quality Tools
> Gage R&R Study (Crossed)
Each process output
measured by each
operator

OR

> Gage R&R Study (Nested)

For “destructive tests”
where each process
output is measured
uniquely by each
operator

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 23
 5/30/2021

Two-Way ANOVA Table With Interaction

α to remove interaction term = 0.05
Source DF SS MS F P
Part 9 88.3619 9.81799 492.291 0.000
Operator 2 3.1673 1.58363 79.406 0.000
Part * Operator 18 0.3590 0.01994 0.434 0.974
Repeatability 60 2.7589 0.04598
Total 89 94.6471

Two-Way ANOVA Table Without Interaction

Source DF SS MS F P
Part 9 88.3619 9.81799 245.614 0.000
Operator 2 3.1673 1.58363 39.617 0.000
Repeatability 78 3.1179 0.03997
Total 89 94.6471

%Gage R&R = 27.86%

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 24
 5/30/2021

Minitab Example: The Graphs

We should not have a straight line

Components
By Part
of
Variation
GR&R <10%
There is a diff in averages of the
technicians

R Chart By Operator
Should be under control

Operator
Xbar Chart x
Part
This part of the variation is
hidden by means system
Interaction
variation. We cannot
differentiate between parts We want the same pattern when there
Lying in that band Is nothing special happening between
parts And operators

Attribute GR&R - Example

There are 3 Appraisers, 2-Replications and 20 invoices- Result is Pass or Fail
Also An Expert’s opinion is Considered as Standard. Find if the GR&R of the Invoice checking is Good ?

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 25
 5/30/2021

GR&R – Attribute : interpretation of results

MINITAB METHOD – DATA ANALYSIS .. Contd…

D a t e o f s t u d y : 1 /0 3 / 2 0 0 1
A s s e s s m e nt A gre e m e nt R e p o rt e d b y : J o s e
N a m e o f p ro d u c t: X Y Z R e p o r t
Mis c :

W ithin A p p ra is e r A p p ra is e r v s S ta n d a rd
Lower Variation
Appraiser Vs.
Lower Variation 10 0 1 00 Standard
Within Appraiser [ , ] 9 5 .0 % C I
P e rc e n t
90
90
Percent

Percent

80
80

70

70

Higher Variation Bob Sue To m

60

Bob S ue Tom

Within Appraiser A p p ra is e r A p p ra is e r
Higher Variation
Appraiser Vs.
Standard
Not Included If No
“Known”

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 26
 5/30/2021

GRR – Repeatability and Reproducibility

Precision

Repeatability
Reproducibility Precision+Bias

Misclassification Report : If ‘Pass’ is reported as ‘Fail’

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 27
 5/30/2021

GRR- Attribute – Summary Report

Proprietary/Gopalakrishna/Nirmala
Pharma Consultancy Services 28