Six sigma methodology

Variance Estimation and

Measurement System Analysis
Chapter 12 and 22
 Outline
I. Variance Estimation from ANOVA Results
II. Robust Operating Windows and ANOVA
III. Components of Measurement System
IV. Sources of Measurement Error
V. Types of Measurement System Studies
A. Gage Accuracy Study
B. Gage R&R Study
 I. Variance Estimation from ANOVA Results
Output Stamping Draw Depth;
Input: Tonnage

Is there a mean difference? Which groups are different?

 Analysis of Mean
don’t follow statistics blindly

• If compare tonnage 940-960 using one-way ANOVA 􀃆 no significant

mean difference
• If compare tonnage 910-920 using one-way ANOVA 􀃆 no significant
mean difference
Commission fees for different lending
companies
Variance Estimation in measurement
system
σ Total
variation

σ Common
σ Special cause
cause (inherent)

Material Product/process Gage

Batch Repeatability

?? Reproducibility
 Estimation of variance
• ANOVA provides an effective technique to
separate inherent variance and special cause
variance (i.e., group effect variation).
Sample Size and Degrees of Freedom (DF) from
One-Way ANOVA Table

– N= total # of samples = 73
– ni = sample size for each group
• Balanced Experiment (n is constant across groups)
– a= # of levels of factors (# groups) = 5
• Degrees of Freedom (DF)
– DF (total) = N – 1Example: 73 – 1 = 72
– DF (factor) = a –1Example: 5-1 = 4
– DF (error) = N – a  Example: 73 – 5 = 68
• DF (factor) + DF (error) = DF (total)
– SS Total

– SS Between Group (Factor)

- SS total = SS factor + SS error

 Example : One way anova ( single factor)

• SS Between Group – squared deviation of each group mean (for a factor)

from grand mean – allocated per group sample size
• SS total = variance(overall) x df(total)
– SS total ~ Numerator for the sample variance formula of all data
– Example: 863923 x 72 = 62202452
- SS factor = 3999085+358032+88555+8218615+108533
Mean Squared Error represents the inherent within group
variance (σ2 inherent).
 ANOVA
• ANOVA tests for mean differences by:
– convert the group means into a variance term (MS factor)
– use an F test to compare the MS factor (between group)
to the mean squared error (MSE) (within group).
• MS factor = SSfactor/DFfactor (variance term)
• MS error = SSerror/ DFerror
• F = MS factor / MS error
• a p-value can be estimated for the F-test value
What is Ho in this case ?
 Common Vs Special Cause
• Be Careful

Because the sum square based on means which are statistical estimates
And might be influenced by inherent cause (common cause )
 Factor and common cause variaiton
• Random Effects – factor levels are assumed to be
possible levels within a population of factor levels

• Thus, Inherent Variation (Within Group Variation)

Using ANOVA table
After estimating the factor variance effect, calculate
the % of contribution of each term.

% reduction = ( 411^2)/(411^2+852^2)=about 18%

ANOVA can be used in Process Capability to estimate
Pp and Cp

– Cp/Cpk – based on σinherent = Sqrt(MSE)

• Note: Minitab uses (CODED) a pooled standard
deviation method as opposed to the MSE so
within group calculation will differ slightly
– Pp/Ppk – based on σoverall = Sqrt(MStotal)
 Robust Operating Windows and ANOVA

• Robust Operating Window is a range of setting for an

input (X) in which the output is insensitive (i.e., no
effect on process mean or variance).
– Robust Windows – tolerances for inputs
• ANOVA provides a methodology to evaluate the
robustness of a process to various levels of a particular
factor (or treatment).
 Robust Analysis with ANOVA

Review
Output: Draw Depth
Input: Tonnage
• Spec: +/-1 mm
• ANOVA results
– Robust Range: 940-960
– Robust Range: 910-920
 Robust analysis Using ANOVA
• Which level of tonnage will you prefer to work
with? And Why
• – Two Robust (insignificant ranges)
• – Operating Tonnage from 940-960 yields a
part closer to the target value. ( 0 +/1 mm)
 KEY lessons from ANOVA
• ANOVA results decompose total observed variance into within
(inherent) + factor effect (special cause)
• Capable of interpreting an ANOVA table and calculate σwithin and
σfactor from output results.
• Important Points:
– Identify group(s) that yield the more desirable output (sometimes the
one that is different may be the desired outcome!)
– Evaluate results using % contribution to total variation to estimate
amount of potential reduction you can achieve by eliminating factor
(group) effects.
– Evaluate results using process capability or DPM to determine
practical significance of a factor (group) effect.
Components of measurement systems
• Quality measurement systems
– Manufacturing Gage:
• Attribute screens: go-no go, pass/fail
• Variable: calipers, probe, tape measure, coordinate
measurement machines
• Not limited to manufacturing > service companies
appraisal form (1-5 scale)  different manager may
rank employee differently
– Important
• Reject good parts and accept bad parts
• Operator may not be consistent every time!
 Gage R&R
• Tools (equipment)
• Operators
• Parts

– Should not be limited to those. Generally, in

manufacturing these are major sources of quality
measurement system
 Sources of Measurement Error
• Systematic Variations
1. accuracy: i.e. improper calibration of equipment
2. reproducibility: i.e. different operators using same
equipment with different techniques
• Random Variations
3. repeatability: i.e. inconsistent part locating
• Periodic Variations
4. stability: i.e. wear, deterioration, environmental
conditions
 Accuracy
• Accuracy is the difference between the true
average and the observed average
Reproducibility (Operator effect)
 Repeatability is a Measure Within
Gauge of Inherent Variation

• It Occurs when one operator uses the same gage to measure

the same part several times.
• You should always have Repeatability Variation
 Stability Measures the Different
Averages with Time Effect
• Difference in the average of at least 2 sets of
measurements obtained with a gage as a
result of time.
 Gage Accuracy
• Measure a master (reference) part using a more precise
instrument.
• Repeatedly measure master part with gage.

What do you think: Is the measurement gage accurate? Is it stable?

 Gauge RR Study
Identify: # parts, # trials per part and # operators.
• Obtain tolerance width for each feature.
– Tolerance Width = USL – LSL
• Common Applications (parts x trials x operators):
– 5 or 10 parts
– 2 or 3 trials
– 2 or 3 operators
• Example: 5x3x2  Two operators will measure
each of 5 parts three
times.
Gauge RR Sheet
Car Door Example to Conduct Gauge
R&R
 Steps Testing Range Stability Using
Control Chart
**1st Step : test range measurements out-control
(Ri > UCLR)
– Bad estimate of sigma repeatability occurs if
individual range measurements are out-of-control
** 2nd Step (when all measurements are in-control
(stable)):
– compute % Repeatability, % Reproducibility, and %
Gage R&R.
Example : car door
 Example : first step stability

Rp 0.23
R-Bar 0.10
X-Dif 0.32
UCLr 0.33
LCLr 0.00
Max Range 0.20

Range Stable? Yes

 Gauge R&R calculations
Most software will automatically compute:
– % Repeatability (% Equipment Variation)
– % Reproducibility (% Appraiser Variation)
– % R&R (% Repeatability and Reproducibility)
• We typically compute these value as:
– % of the tolerance width, and/or
– % of total variation in the parts measured.
 % Repeatability Calculations
• K = 5.15 or 6 (# of std deviations to use to estimate variation)
• Repeatability  Requires R-bar (average range between trials)

• EV= equipment variance ( rules measure 0.1 cm-micrometer 0.001cm)

K=6 (# of sigma’s) - represents 99.73% of values if normal distribution
K=5.15 (# of sigma’s) - represents 99% of values if normal distribution

http://www.qualitydigest.com/inside/twitter-ed/problems-gauge-rr-studies.html
 % Reproducibility Calculations
Appraiser Variation

K=6 (# of sigma’s) - represents 99.73% of values if normal distribution

K=5.15 (# of sigma’s) - represents 99% of values if normal distribution
Sigma R&R Calculations
 Obtain the % Gage R&R Ratio
Acceptability is determined by comparing gage
variation to either:
– Tolerance Width - % R&R – Tolerance

– Total Process Variation (TV) - % R&R – TV

 Repeatability and Reproducibility Ratios

• % Equipment Variation - Tolerance:

• % Appraiser Variation – Tolerance

• Note: may compute ratios vs. Total Variation Observed

General Criteria for Gage R&R Calculations

Measures: R&R % (Tolerance) or R&R% - TV

if R&R% is:
< 10% ~ “Green” measurement system is acceptable
10-30% ~ “Yellow” measurement system may be
acceptable based upon importance of application,
gage cost, cost of repairs ..
> 30% ~ “Red” measurement system needs
improvement
General Criteria for Repeatability (EV)
• Typically, manufacturers also have a
requirement for
EV (repeatability)
• % EV – Tolerance < 18% OR,
• % EV – Total Variation < 18%
 General Criteria for Gage R&R
Calculations

% R&R (tolerance) = 11%

– Does this measurement system meets R&R criteria
Relative to the tolerance?

% R&R – TV = 45% from the example e

– % R&R – TV fail to 30% criteria

– However, the variation in the parts for the study is quite
small.
– 5.15σTV = 0.26 which is approx ¼ tolerance width of 1 mm
 Minitab
Under Stat >> Quality Tools
• Types of Gage R&R Studies in Minitab
– Crossed design (when each part is measured
multiple times by each operator)
use either ANOVA or Average/Range Method
– Nested design (when each part is measured by
only one operator, such as in destructive testing)
destructive testing has different measure
characteristic, i.e. Crash testing.
Gauge R&R
 Example
• 1) check for stability of the gauge:

• 2) Calculate equipment variation (EV)

– EV = 0.03/d2= 0.03/1.693 *5.15= 0.09125
– σ repeatability = 0.03/1.693 = 0.01772
– Note that d2 depends on trial number
 Example
• 3) calculate the appraiser variation
compute differences of averages per part using same operator with per trails.
X diff  0.02

 0.09122 
AV   0.02*3.65      0.067
2

 10 
Note: K 2 depends on number of operators
0.067
 reproduce   .013
5.15
R & R  0.09122  0.067 2  0.113
R & R 0.113
 R&R    0.02197
5.15 5.15
 Example continue
• 4) part variation
R part  0.11 Part average for both operators {0.04 0.05  0.02 0.08  0.03}
Rp= 0.08- (-.03) =.11
PV  0.11* K3  0.11* 2.08  0.2288
Note: K 3 depends on number of parts
TV  0.1132  0.22882  0.25518
0.25518
 TV   0.04955
5.15
 Criteria Checking
 repeat  0.01772  repreduce  0.013  R & R  0.02197
EV  0.09125 AV =0.06695 R&R= 0.11314

5.15*0.02197
% R&R -Tolerance = *100%  11.3%  10  30%" Yellow"
0.5  (.5)
0.09125
% Equip  Tol  *100%  9.125%  18%" Acceptable "
1
0.06695
%Operator  Tol  *100%  6.7%  18%" Acceptable "
1
 How much variation is due to
Measurement system ?
 repeat  0.01772  repreduce  0.013  R & R  0.02197
EV  0.09125 AV =0.06695 R&R= 0.11314

5.15*0.02197
% R&R -Tolerance = *100%  11.3%  10  30%" Yellow"
0.5  (.5)
0.01772
% Equip  TV  *100%  35%  30%" Not Acceptable "
0.04995
0.013
%Operator  TV  *100%  26%  30%" Acceptable "
.04995
 Industry Debate: % TV versus %
Tolerance
– Opinion 1: purpose of gage is to separate gage variation
from product variation.
**Thus, use % TV requirement
– Opinion 2: purpose of gage is to measure relative to a
tolerance.
** Thus, use % Tolerance
• Others’ Opinion:
– Measure Both.
– If fail % TV, make sure that you have taken parts with
sufficient variation relative to the tolerance range.
 Attribute R&R
A. Attribute Ordinal Data: Typical Rating Scales:
(1 ~ 5 or 1 ~ 10)
B. Attribute Nominal Data: Unordered Rating
Scales
• Comparisons Available in Both Cases:
1. Operator to Master: % Rating Match to Master
– Goal “% Match > 90%” or “Upper CI > 90%”
2. Operator to Operator: % Match by All Operators
– Test Systematic Bias versus Master
 Example
• Objective is to test the ability of different purchasing
buyers to
rate suppliers consistently.
• Suppliers are rated based on various criteria:
– Quality Level
– Price Competitiveness
– On-time Delivery
– Technical know-how
• Create a master data set of overall ratings for suppliers
by an experienced purchasing committee (“master
rating”).
 Example
3 different buyers with 50 supplier information
packages.
• Buyers assign a rating for each package.
– Rating scale: low of 1 (unacceptable) to high of 10
(excellent)
• Assessment is based on:
– Buyers rating vs Master rating
– Between Buyers rating
Example