INTERNATIONAL ISO

STANDARD 17258

First edition
2015-01-15

Statistical methods — Six Sigma

— Basic criteria underlying
benchmarking for Six Sigma in
organisations
Méthodes statistiques — Six Sigma — Critères fondamentaux d’une
évaluation comparative Six Sigma pour les organisations
iTeh STANDARD PREVIEW
(standards.iteh.ai)
ISO 17258:2015
https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
50cbfc8ec59f/iso-17258-2015

Reference number
ISO 17258:2015(E)

© ISO 2015
ISO 17258:2015(E)


iTeh STANDARD PREVIEW

(standards.iteh.ai)
ISO 17258:2015
https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
50cbfc8ec59f/iso-17258-2015

COPYRIGHT PROTECTED DOCUMENT

© ISO 2015
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail [email protected]
Web www.iso.org
Published in Switzerland

ii  © ISO 2015 – All rights reserved

 ISO 17258:2015(E)


Contents Page

Foreword...........................................................................................................................................................................................................................................v
Introduction................................................................................................................................................................................................................................. vi
1 Scope.................................................................................................................................................................................................................................. 1
2 Normative references....................................................................................................................................................................................... 1
3 Terms and definitions...................................................................................................................................................................................... 1
4 Symbols and abbreviations........................................................................................................................................................................ 1
5 General considerations on benchmarking................................................................................................................................. 2
5.1 Objectives and framework............................................................................................................................................................. 2
5.1.1 Objective................................................................................................................................................................................... 2
5.1.2 Framework............................................................................................................................................................................. 2
6 Benchmark.................................................................................................................................................................................................................. 2
6.1 Overview and requirements for the criteria.................................................................................................................. 3
6.1.1 Types and description of criteria........................................................................................................................ 3
6.1.2 Requirements for criteria.......................................................................................................................................... 3
6.2 Overview and requirements for the measures............................................................................................................. 3
6.2.1 Scope of measures........................................................................................................................................................... 3
6.2.2 Generic measures............................................................................................................................................................. 4
6.2.3 Industry specific measures...................................................................................................................................... 8
6.2.4 iTeh STANDARD PREVIEW
Summary of measures.................................................................................................................................................. 9
7 Benchmarking.......................................................................................................................................................................................................10
7.1 Objective establishment (standards.iteh.ai)
step.................................................................................................................................................... 10
7.2 Measurement step.............................................................................................................................................................................. 10
7.2.1 Data collection.................................................................................................................................................................
ISO 17258:2015 10
7.2.2 Data arrangement......................................................................................................................................................... 11
https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
7.2.3 Data quality validation .............................................................................................................................................. 11
50cbfc8ec59f/iso-17258-2015
7.3 Controlling the quality of the measurement results............................................................................................. 11
7.3.1 Precision............................................................................................................................................................................... 11
7.3.2 Consistency......................................................................................................................................................................... 11
7.3.3 Up-to-date............................................................................................................................................................................ 11
7.4 Comparison step.................................................................................................................................................................................. 11
7.5 Internal benchmarking.................................................................................................................................................................. 11
7.5.1 Historical benchmarking........................................................................................................................................ 12
7.5.2 Functional benchmarking...................................................................................................................................... 12
7.5.3 I/O benchmarking........................................................................................................................................................ 12
7.6 External benchmarking................................................................................................................................................................. 12
7.7 Supply chain benchmarking...................................................................................................................................................... 12
7.7.1 Benchmarking in an organization................................................................................................................... 12
7.7.2 Benchmarking in a supply chain...................................................................................................................... 13
7.8 Six Sigma project selection......................................................................................................................................................... 13
8 Interface with other types of benchmarking........................................................................................................................13
Annex A (informative) Three dimensions of benchmarking.....................................................................................................15
Annex B (normative) Table of criteria (generic)...................................................................................................................................16
Annex C (informative) Table of criteria with examples of results........................................................................................18
Annex D (normative) Table of criteria for the food processing industry.....................................................................20
Annex E (normative) Table of criteria for the automotive industry..................................................................................22
Annex F (normative) Table of criteria for the retail banking industry...........................................................................24
Annex G (normative) Table of criteria for clothing and leather industry....................................................................27

© ISO 2015 – All rights reserved  iii

ISO 17258:2015(E)


Annex H (normative) Table of criteria for the telecoms industry........................................................................................29

Annex I (normative) Table of criteria for the hotel business....................................................................................................31
Annex J (normative) Table of criteria for the retail business...................................................................................................33
Annex K (informative) Example from the poultry industry.......................................................................................................35
Bibliography.............................................................................................................................................................................................................................. 37

iTeh STANDARD PREVIEW

(standards.iteh.ai)
ISO 17258:2015
https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
50cbfc8ec59f/iso-17258-2015

iv  © ISO 2015 – All rights reserved

 ISO 17258:2015(E)


Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of any
patent rights identified during the development of the document will be in the Introduction and/or on
the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity
assessment, as well as information about ISO’s adherence to the WTO principles in the Technical Barriers
iTeh STANDARD PREVIEW
to Trade (TBT), see the following URL: Foreword — Supplementary information.
(standards.iteh.ai)
The committee responsible for this document is ISO/TC 69, Statistical methods, Subcommittee SC 7, Six Sigma.
ISO 17258:2015
https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
50cbfc8ec59f/iso-17258-2015

© ISO 2015 – All rights reserved  v

ISO 17258:2015(E)


Introduction
Benchmarking is frequently used in various domains in connection with business activities. The Six
Sigma methodology requires an evaluation step using a benchmarking process. In other words, a
method for the comparison of levels of quality, performance, and productivity with the state-of-the-art
is required. This International Standard establishes what to compare and develops a methodology to
conduct a correct comparison between an organization’s levels of quality, performance, and productivity.
The numbers given by the benchmarking can be integrated into any improvement programme to
quantify any progress. They can also be used by other assessment processes in the organization such as
regulation compliancy or financial performance evaluation.
Benchmarking is the whole process of collecting and processing data and information and comparing
the results. The benchmark is the reference point for comparison.
The main point of this benchmarking methodology, based on transparency and the universal principle
of evaluation, is to give confidence to its calculating procedures and the results, so that comparisons
between organizations are accepted by all parties.
Benchmark and Six Sigma’s principles
The Sigma measure is a number ranging from, typically, near zero to 10 or more. The value six has
traditionally been considered “world class” (that can be approximated by 3,4 defects per million
opportunities — see ISO 13053-1 5.3) for mechanical and electronic industries. The criticality of defects
within each industry typically defines the Sigma level required in order to be a “world class” benchmark
quality level for that application. So, according to the different sectors and markets, the level of “world
iTeh STANDARD PREVIEW
class” can be different.
(standards.iteh.ai)
The Sigma level is an estimate of the proportion of defects (typically expressed in defects per million). A
“world class” Sigma level is the Sigma level that is considered essential to consistently deliver excellence
of product and service. ISO 17258:2015
https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
This benchmarking method is applicable to 50cbfc8ec59f/iso-17258-2015
all types of sectors, to all type of processes, to organizations
of all sizes, and to all methodologies for improvement, in association with DMAIC, or issues relating to
Design for Six Sigma (DFSS).
Criteria and defects
The Sigma level is based on the ratio of estimated (or observed or predicted) number of defects to the
number of opportunities according to the specifications and the variability of the process (for example,
one “defect” in a million deliveries).
A defect is something that a customer or a user cannot accept or it might have a negative impact on
performance.
Two types of customer are identified:
— the end user or consumer (Business to Consumer) and
— the professional (Business to Business).
The consumer has some needs but these needs are often implicit. Product requirements that address
these needs are not numerous and they can be summarized by the following:
a) to be safe in its intended usage (security, safety);
b) to do what it is supposed to do (functional, conformity);
c) to be available in the expected shape and not to break in its intended usage (availability, ease of use,
reliability);
d) to not do any harm to persons (ethical) or environment (pollution control).

vi  © ISO 2015 – All rights reserved

 ISO 17258:2015(E)


These four criteria cover most of the consumer’s needs for all sectors and on all continents.
A customer will have other criteria but all of these are likely to be summarized by the four generic
criteria described above.
Supply chain
A supply chain is the whole supplier/transformer link from the raw material to the final product or the
service for the consumer. Each sector has its supply chain organization.
Examples:
— Petrol sector — from the offshore extracting (extractor) unit to the gasoline retailer for consumers.
— Food sector — from the fields and orchards (raw material) to the consumer.
— Automotive sector — from the steel and glass supplier to the car manufacturer.
— Cosmetic sector — from the molecule to the perfume or beauty cream.
The level of quality and performance delivered to the consumer is the “total” of all quality and
performance levels of the different transformers along the supply chain.
This benchmarking methodology aims to give a comparison of levels between upstream and downstream
transformers (chain efficiency benchmarking) or between transformers at the same step (competitive
benchmarking).
iTeh STANDARD PREVIEW
Producer B
(standards.iteh.ai)
Extractor Supplier A Main Producer Retailer Consumer
ISO 17258:2015
https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
Producer C
50cbfc8ec59f/iso-17258-2015 Chain ef iciency benchmarking
Competitive benchmarking

Product or service transformation

Figure 1 — Supply Chain Cycle

© ISO 2015 – All rights reserved  vii

 iTeh STANDARD PREVIEW
(standards.iteh.ai)
ISO 17258:2015
https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
50cbfc8ec59f/iso-17258-2015
INTERNATIONAL STANDARD ISO 17258:2015(E)

Statistical methods — Six Sigma — Basic criteria

underlying benchmarking for Six Sigma in organisations

1 Scope
This International Standard describes a methodology for establishing the level of quality, performance,
and productivity of processes, products, and services according to Six Sigma principles. It is applicable
to all sectors (industries, services, administration, etc.) and to all types of organizations, whether it is
already involved in an improvement programme such as Six Sigma, Lean, or not. In particular, it can be
used to initiate a Six Sigma programme by providing a selection of improvement projects.
NOTE The focus of this methodology is on criteria, measures, measurement process, and comparison process.
The results can then be used to identify good practices of benchmarking.

2 Normative references
The following documents, in whole or in part, are normatively referenced in this document and are
indispensable for its application. For dated references, only the edition cited applies. For undated
references, the latest edition of the referenced document (including any amendments) applies.
iTeh STANDARD PREVIEW
ISO 13053-1, Quantitative methods in process improvement — Six Sigma — Part 1: DMAIC methodology
(standards.iteh.ai)
3 Terms and definitions
ISO 17258:2015
3.1 https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
benchmark 50cbfc8ec59f/iso-17258-2015
reference point against which comparisons can be made
3.2
benchmarking
activity of comparing objects or practices of interest to each other or against a benchmark to evaluate
criteria (or characteristic)
3.3
benchmarking method
logical sequence of general steps to describe the process of quantitatively comparing one or more
attributes against a reference attribute with respect to a specified scale

4 Symbols and abbreviations

A measure of total “space”

Alost measure of total lost “space”

C number (count)

Cair number of breaches of threshold of air pollution in a specified time

Cinput input defects level

Coutput output defects level

Cprocess process or activity defects level

© ISO 2015 – All rights reserved  1

ISO 17258:2015(E)


Cwater number of breaches of threshold of water pollution in a specified time

E category of user — experienced

F category of user — first time

k organization index

N total number

NE number of experienced users

NF number of first time users

t duration of time of study

t1 duration of time during which a product is returned

tlost measure of lost time

tstart measure of time required for starting or understanding a product

X measure (continuous scale)

5 General considerations on benchmarking

iTeh STANDARD PREVIEW
Benchmarking is the method for collecting and analysing the data, and from these results, for comparing
(standards.iteh.ai)
the quality, performance, and productivity level between two or more organizations.

5.1 Objectives and framework ISO 17258:2015

https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
The following paragraphs describe what an organization is recommended to do in order to deploy
50cbfc8ec59f/iso-17258-2015
benchmarking and to obtain benchmark results.

5.1.1 Objective

The organization should decide upon the objective of the benchmarking before starting any measurement
or comparison activities.
NOTE Examples of objectives are to improve processes, to compare to competition, to evaluate the gap for
performance improvement.

5.1.2 Framework

The framework encompasses parts that are interrelated. This methodology is built with two main parts:
— benchmark;
— benchmarking.
In the objective’s description of benchmarking, an organization should detail these two parts.

6 Benchmark
This clause describes the overview and requirements to determine some reference points for comparison
(see 3.1.1 and 3.1.2). The points to be detailed are
— the criteria and
— measures of the criteria.

2  © ISO 2015 – All rights reserved

 ISO 17258:2015(E)


6.1 Overview and requirements for the criteria

Benchmarking is required to be applied to the criteria. This methodology defines 10 main criteria. If
one of the criteria is irrelevant, the organization will explicitly give an explanation for excluding it from
benchmarking. Two optional criteria are proposed, making a total of 12. This International Standard
focuses on 10 main and two optional criteria clustered into three types. The main criteria are also
named generic criteria.

6.1.1 Types and description of criteria

To have a universal benchmark, 10 main criteria are defined and they are clustered into two types. A
third type is for optional criteria (see Table 1).

Table 1 — Benchmarking criteria

Number Type Title

1 Type 1: generic quality Compliance
2 Ethical behaviour
3 Security/Safety
4 Returns (customer listening)
5 Availability
6 One-time delivery
7 iTeh STANDARD PREVIEW On-time to market
8 (standards.iteh.ai)
Type 2: generic efficiency Productivity
9 Raw material efficiency
10 ISO 17258:2015 Energy efficiency
https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
11 Type 3: Option
50cbfc8ec59f/iso-17258-2015 Ease of use
12 Pollution control

These main criteria are divided into sub-criteria. Annex B gives the list of the sub-criteria. The main
criteria describe the basic requirement of the consumer such as security or on-time delivery. The sub-
criteria give a practical way to obtain the measure. A main criterion can have from one to eight sub-
criteria. In case of several sub-criteria, each one gives a measure and the main criteria can have up to
eight resulting measures (8-uple).
EXAMPLE Compliance has four sub-criteria: regulation, functional, contractual, and company policies. A
result of measures would be (100 %, 95 %, 82 %, and 95 %) and if the company has no policy, then it would be
(100 %, 95 %, 82 %, and not applicable).

6.1.2 Requirements for criteria

The organization should describe its policy and implementation of the 10 generic criteria with all its
sub-criteria (total of 30). Annex B gives the description of all main criteria and sub-criteria. If one of the
sub-criteria has no practice (e.g. no ethical charter or no charity programme in the organization), it shall
be explicitly mentioned and excluded from the benchmarking.

6.2 Overview and requirements for the measures

This sub-clause gives an overview and the requirements for the measures.

6.2.1 Scope of measures

The measures are performed on the main processes of the organization (see Introduction).

© ISO 2015 – All rights reserved  3

ISO 17258:2015(E)


The inputs of the organization level process are the inputs of the organization. The outputs of the
organization level process are the outputs (or deliveries) of the organization. The main process is a bundle
of processes and each process is a set of activities. Each process can belong to a type of process such as
functional process (e.g. purchasing, sales, finance), support process (IT process, HR), or management
process (project management, accounting management).
For each benchmarking, the organization should describe which processes are in the scope.

6.2.2 Generic measures

This is a requirement to obtain measures for the generic criteria.

6.2.2.1 Measure for compliance

Compliance is related to the application of legal regulations, business commitments, and organizational
policies. The organization will describe the regulations, the business agreements, and the policies it
applies. The measure is described in Table 2.

Table 2 — Compliance

Information need Compliance on regulation commitments and policies

of an organization
Unit of measure Count number (compliance requirements, compliance
requirements not fulfilled)
Measurement methodiTeh STANDARD PREVIEW
Count total number of compliance requirements (N)
(standards.iteh.ai)
Count number of compliance requirements not ful-
filled (C)
Data type Integer ISO 17258:2015
Measure (N, C, (N - C)/N) (%)
https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
50cbfc8ec59f/iso-17258-2015
Function Divide total number of requirements (N) minus num-
ber of requirements not fulfilled (C) by total number
of requirements (N)

Note For example, counting the number of non-conformities and the total number of requirements of
ISO 9001:2008 standard will give this measure.

Counting the number of security practices not fulfilled and the total number of security practices that
are mandatory will also give another measure.

6.2.2.2 Measure for ethical behaviour

Ethical behaviour is related to all of the social responsibility activities of an organization. The organization
should establish the list of its ethical “good practices” (GP). The measure is described in Table 3.

Table 3 — Ethical behaviour

Information need Ethical good practices (GP)

Unit of measure Count number (ethical good practices, ethical good
practices not fulfilled)
Measurement method Count total number of ethical GP to be applied (N)
Count the number of ethical GP not fulfilled (C)
Data type Integer
Measure (N, C, (N - C)/N) (%)

4  © ISO 2015 – All rights reserved

 ISO 17258:2015(E)


Table 3 (continued)
Function Divide the total number of ethical GP (N) to be applied
minus the number of ethical GP not fulfilled (C) by the
total number of ethical GP to be applied (N)

Note For example, do not throw your old computer but send it to a school or association.

6.2.2.3 Measure for security/safety

This measures what the consumer is expecting for its security (or safety). A breach of security (or
safety) is an event that jeopardizes the vital functions of an organism or an organization. The measure
is described in Table 4.

Table 4 — Security/Safety

Information need Security (or safety) for an organism or an organiza-

tion
Unit of measure Count number (breaches of security or safety)
Measurement method Count total number of breaches of security (or safety)
(C) over a specified period of time t
Data type Integer
Measure (C, t, C/t)
Function iTeh STANDARD PREVIEW
Number of breaches of security over a specified
period of time (week, month, year, etc.)
(standards.iteh.ai)
6.2.2.4 Measure for returns
ISO 17258:2015
A return is an explicit action issued from a dissatisfied customer (problem report, complaint, product
https://standards.iteh.ai/catalog/standards/sist/ce4aec4a-e667-4fb8-a30c-
return, a request for return). The measure is described in Table 5.
50cbfc8ec59f/iso-17258-2015

Table 5 — Returns

Information need Returns from consumers

Unit of measure Count number identified returns (complaints, prod-
uct return, report, withdrawal of goods, etc.)
Measurement method Count number of returns (complaints, product
return) (C) by specified period of time (t)
Count number of units sold (U)
Data type Integer
Measure (C, t, U, C/U)
Function Divide the total number of returns by specified
period of time (week, month, year) by the number of
units sold

6.2.2.5 Measure for availability

This International Standard records availability with two measures:

— proportion of time (i.e. useful time/total time);
— proportion of space (i.e. space where usable/total space).
The measure is described in Table 6.

© ISO 2015 – All rights reserved  5